2024-09-10T07:34:57.893 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T07:34:57.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T07:34:58.000 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880 branch: wip-rishabh-testing-20240909.165402-squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} email: null first_in_suite: false job_id: '7898880' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Scrub error on inode sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBsjShZzrJ9nUmQM0wSvp60Y3LYSGNE24FkJgUZNEdqh6oSBLPzAajR2Zapv8oCY+pobHX8OAYncnUrQy+aOyRA= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIUC1eFtiky9yQkM0WFqNVkbYCF+QIVb5hACucowxO46ffXSP0EaZc9QqZnlxtG6JQT4DAfQghxJpKFYah4ClE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T07:34:58.000 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T07:34:58.001 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T07:34:58.001 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T07:34:58.002 INFO:teuthology.task.internal:Checking packages... 2024-09-10T07:34:58.024 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T07:34:58.024 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T07:34:58.024 INFO:teuthology.packaging:ref: None 2024-09-10T07:34:58.024 INFO:teuthology.packaging:tag: None 2024-09-10T07:34:58.024 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:34:58.024 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:34:58.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:34:58.175 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T07:34:58.177 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T07:34:58.185 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T07:34:58.185 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T07:34:58.193 INFO:teuthology.task.internal:Saving configuration 2024-09-10T07:34:58.210 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T07:34:58.218 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T07:34:58.242 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:30:00.232499', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBsjShZzrJ9nUmQM0wSvp60Y3LYSGNE24FkJgUZNEdqh6oSBLPzAajR2Zapv8oCY+pobHX8OAYncnUrQy+aOyRA='} 2024-09-10T07:34:58.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:30:00.234387', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIUC1eFtiky9yQkM0WFqNVkbYCF+QIVb5hACucowxO46ffXSP0EaZc9QqZnlxtG6JQT4DAfQghxJpKFYah4ClE='} 2024-09-10T07:34:58.267 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T07:34:58.275 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T07:34:58.275 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T07:34:58.275 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T07:34:58.330 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efc06eefbe0>, signals=[15]) 2024-09-10T07:34:58.330 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T07:34:58.338 INFO:teuthology.task.internal:Opening connections... 2024-09-10T07:34:58.338 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:34:58.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:34:58.415 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:34:58.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:34:58.493 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T07:34:58.501 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-10T07:34:58.519 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-10T07:34:58.520 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:34:58.578 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:34:58.579 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:34:58.579 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:34:58.579 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:34:58.579 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-10T07:34:58.598 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-10T07:34:58.616 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-10T07:34:58.616 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-10T07:34:58.673 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-10T07:34:58.673 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-10T07:34:58.673 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-10T07:34:58.673 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:34:58.673 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:34:58.674 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:34:58.674 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-10T07:34:58.698 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T07:34:58.708 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T07:34:58.764 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T07:34:58.765 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:34:58.768 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:34:58.785 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T07:34:58.792 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T07:34:58.792 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:34:58.825 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:34:58.932 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T07:34:58.982 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T07:34:58.997 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T07:34:58.998 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T07:34:58.998 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T07:34:58.998 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:34:58.999 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:34:59.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:34:59.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T07:34:59.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:34:59.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T07:34:59.211 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:34:59.211 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T07:34:59.212 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-10T07:34:59.229 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-10T07:34:59.229 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:34:59.230 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T07:34:59.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum upgrade -y linux-firmware 2024-09-10T07:34:59.237 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:34:59.238 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T07:34:59.238 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-10T07:34:59.258 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-503.el9.x86_64 2024-09-10T07:34:59.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:34:59.259 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T07:34:59.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum upgrade -y linux-firmware 2024-09-10T07:35:01.543 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-10T07:35:01.906 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.2 MB 00:00 2024-09-10T07:35:09.984 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 20 MB 00:07 2024-09-10T07:35:10.947 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-10T07:35:15.459 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-10T07:35:16.723 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-10T07:35:16.852 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 19 kB 00:00 2024-09-10T07:35:17.622 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T07:35:18.173 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-10T07:35:19.096 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-10T07:35:23.061 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T07:35:24.683 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T07:35:25.206 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T07:35:25.207 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T07:35:25.207 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T07:35:25.309 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T07:35:25.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:35:25.458 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T07:35:25.459 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T07:35:25.461 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi070.front.sepia.ceph.com to complete... 2024-09-10T07:35:26.943 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-10T07:35:26.944 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-10T07:35:26.944 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-10T07:35:27.058 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:35:27.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:35:27.199 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T07:35:27.199 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T07:35:27.201 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2024-09-10T07:35:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10 07:35:35 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T07:35:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:FINISHED --2024-09-10 07:35:35-- 2024-09-10T07:35:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:Total wall clock time: 9.6s 2024-09-10T07:35:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:Downloaded: 1 files, 343M in 9.3s (36.9 MB/s) 2024-09-10T07:35:35.066 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T07:35:35.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T07:35:37.178 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying... ######################################## 2024-09-10T07:35:37.331 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing... ######################################## 2024-09-10T07:35:37.447 INFO:teuthology.orchestra.run.smithi070.stdout:Updating / installing... 2024-09-10T07:35:37.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10 07:35:37 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T07:35:37.655 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2024-09-10 07:35:37-- 2024-09-10T07:35:37.655 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 10s 2024-09-10T07:35:37.655 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 343M in 10s (34.3 MB/s) 2024-09-10T07:35:37.657 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:35:37.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T07:35:39.874 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying... ######################################## 2024-09-10T07:35:40.006 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing... ######################################## 2024-09-10T07:35:40.113 INFO:teuthology.orchestra.run.smithi003.stdout:Updating / installing... 2024-09-10T07:35:45.432 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T07:35:48.421 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T07:36:25.320 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T07:36:25.320 DEBUG:teuthology.orchestra.run.smithi070:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T07:36:25.420 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.420 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.420 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.420 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T07:36:25.421 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T07:36:25.422 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T07:36:25.423 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T07:36:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T07:36:25.425 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T07:36:25.426 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T07:36:25.427 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T07:36:25.428 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T07:36:25.429 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T07:36:25.430 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T07:36:25.431 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T07:36:25.432 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T07:36:25.433 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T07:36:25.434 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T07:36:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T07:36:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T07:36:25.437 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T07:36:25.438 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T07:36:25.439 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T07:36:25.440 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T07:36:25.441 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T07:36:25.442 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T07:36:25.443 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T07:36:25.444 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T07:36:25.445 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T07:36:25.446 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T07:36:25.447 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T07:36:25.448 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T07:36:25.449 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T07:36:25.450 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T07:36:25.451 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T07:36:25.452 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T07:36:25.453 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T07:36:25.454 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T07:36:25.455 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T07:36:25.456 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T07:36:25.457 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T07:36:25.458 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T07:36:25.459 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T07:36:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T07:36:25.461 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T07:36:25.462 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T07:36:25.463 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T07:36:25.464 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T07:36:25.465 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T07:36:25.466 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T07:36:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T07:36:25.468 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T07:36:25.469 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T07:36:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T07:36:25.471 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T07:36:25.472 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T07:36:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T07:36:25.474 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T07:36:25.475 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T07:36:25.476 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T07:36:25.477 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T07:36:25.478 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T07:36:25.479 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T07:36:25.480 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T07:36:25.481 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T07:36:25.482 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T07:36:25.483 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T07:36:25.484 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T07:36:25.485 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T07:36:25.486 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T07:36:25.487 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T07:36:25.488 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T07:36:25.489 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T07:36:25.490 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T07:36:25.491 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T07:36:25.492 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T07:36:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T07:36:25.494 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T07:36:25.495 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T07:36:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T07:36:25.497 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T07:36:25.498 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T07:36:25.499 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T07:36:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T07:36:25.501 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T07:36:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T07:36:25.503 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T07:36:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T07:36:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T07:36:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T07:36:25.507 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T07:36:25.508 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T07:36:25.509 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T07:36:25.510 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T07:36:25.511 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T07:36:25.512 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T07:36:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T07:36:25.514 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T07:36:25.515 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T07:36:25.516 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T07:36:25.517 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T07:36:25.518 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T07:36:25.519 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T07:36:25.520 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T07:36:25.521 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T07:36:25.522 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T07:36:25.523 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T07:36:25.524 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T07:36:25.525 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T07:36:25.526 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T07:36:25.527 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T07:36:25.528 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T07:36:25.529 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T07:36:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T07:36:25.531 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T07:36:25.532 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T07:36:25.533 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T07:36:25.534 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T07:36:25.535 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T07:36:25.536 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T07:36:25.537 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T07:36:25.538 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T07:36:25.539 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T07:36:25.540 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T07:36:25.540 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T07:36:25.540 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T07:36:25.540 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T07:36:25.541 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T07:36:25.542 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T07:36:25.543 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T07:36:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T07:36:25.545 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T07:36:25.546 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T07:36:25.547 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T07:36:25.548 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T07:36:25.549 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T07:36:25.550 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T07:36:25.551 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T07:36:25.552 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T07:36:25.553 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T07:36:25.554 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T07:36:25.555 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T07:36:25.556 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T07:36:25.557 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T07:36:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T07:36:25.559 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T07:36:25.560 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T07:36:25.561 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T07:36:25.562 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T07:36:25.563 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T07:36:25.564 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T07:36:25.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T07:36:25.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T07:36:25.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:36:25.571 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.571 DEBUG:teuthology.orchestra.run.smithi070:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T07:36:25.595 INFO:teuthology.orchestra.run.smithi070.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T07:36:25.595 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:36:25.595 INFO:teuthology.orchestra.run.smithi070.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:36:25.595 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2024-09-10T07:36:25.595 INFO:teuthology.orchestra.run.smithi070.stdout:for file in vmlinuz System.map config; do 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: fi 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:done 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T07:36:25.596 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:36:25.597 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.597 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2024-09-10T07:36:25.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-10T07:36:25.668 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T07:36:25.669 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:36:25.670 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T07:36:25.671 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T07:36:25.671 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:25.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T07:36:26.234 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-10T07:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T07:36:27.532 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-10T07:36:27.533 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-10T07:36:27.549 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.rYBnBHIFpC 2024-09-10T07:36:27.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rYBnBHIFpC 2024-09-10T07:36:27.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.rYBnBHIFpC 2024-09-10T07:36:27.753 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.rYBnBHIFpC is 6KB 2024-09-10T07:36:27.801 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.rYBnBHIFpC 2024-09-10T07:36:27.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T07:36:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T07:36:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T07:36:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T07:36:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T07:36:27.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-10T07:36:28.232 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T07:36:28.233 DEBUG:teuthology.orchestra.run.smithi003:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T07:36:28.324 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T07:36:28.325 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T07:36:28.326 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T07:36:28.327 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T07:36:28.328 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T07:36:28.329 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T07:36:28.330 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T07:36:28.331 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T07:36:28.332 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T07:36:28.333 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T07:36:28.334 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T07:36:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T07:36:28.336 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T07:36:28.337 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T07:36:28.338 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T07:36:28.339 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T07:36:28.340 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T07:36:28.341 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T07:36:28.342 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T07:36:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T07:36:28.344 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T07:36:28.345 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T07:36:28.346 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T07:36:28.347 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T07:36:28.348 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T07:36:28.349 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T07:36:28.350 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T07:36:28.351 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T07:36:28.352 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T07:36:28.353 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T07:36:28.354 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T07:36:28.355 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T07:36:28.356 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T07:36:28.357 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T07:36:28.358 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T07:36:28.359 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T07:36:28.360 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T07:36:28.361 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T07:36:28.362 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T07:36:28.363 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T07:36:28.364 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T07:36:28.365 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T07:36:28.366 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T07:36:28.367 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T07:36:28.368 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T07:36:28.369 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T07:36:28.370 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T07:36:28.371 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T07:36:28.372 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T07:36:28.373 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T07:36:28.374 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T07:36:28.375 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T07:36:28.376 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T07:36:28.377 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T07:36:28.378 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T07:36:28.379 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T07:36:28.380 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T07:36:28.381 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T07:36:28.382 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T07:36:28.383 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T07:36:28.384 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T07:36:28.385 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T07:36:28.386 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T07:36:28.387 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T07:36:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T07:36:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T07:36:28.390 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T07:36:28.391 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T07:36:28.392 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T07:36:28.393 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T07:36:28.394 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T07:36:28.395 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T07:36:28.396 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T07:36:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T07:36:28.398 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T07:36:28.399 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T07:36:28.400 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T07:36:28.401 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T07:36:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T07:36:28.403 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T07:36:28.404 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T07:36:28.405 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T07:36:28.406 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T07:36:28.407 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T07:36:28.408 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T07:36:28.409 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T07:36:28.410 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T07:36:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T07:36:28.412 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T07:36:28.413 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T07:36:28.414 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T07:36:28.415 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T07:36:28.416 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T07:36:28.417 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T07:36:28.418 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T07:36:28.419 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T07:36:28.420 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T07:36:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T07:36:28.422 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T07:36:28.423 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T07:36:28.424 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T07:36:28.425 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T07:36:28.426 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T07:36:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T07:36:28.428 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T07:36:28.429 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T07:36:28.430 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T07:36:28.431 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T07:36:28.432 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T07:36:28.433 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T07:36:28.434 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T07:36:28.435 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T07:36:28.436 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T07:36:28.437 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T07:36:28.438 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T07:36:28.439 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T07:36:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T07:36:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T07:36:28.442 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T07:36:28.443 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T07:36:28.444 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T07:36:28.445 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T07:36:28.446 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T07:36:28.447 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T07:36:28.448 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T07:36:28.449 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T07:36:28.450 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T07:36:28.451 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T07:36:28.452 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T07:36:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T07:36:28.454 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T07:36:28.455 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T07:36:28.456 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T07:36:28.457 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T07:36:28.458 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T07:36:28.459 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T07:36:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T07:36:28.461 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T07:36:28.462 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T07:36:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T07:36:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T07:36:28.465 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T07:36:28.466 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T07:36:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T07:36:28.468 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T07:36:28.469 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T07:36:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T07:36:28.471 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T07:36:28.472 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T07:36:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T07:36:28.474 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T07:36:28.475 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T07:36:28.476 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:36:28.479 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.480 DEBUG:teuthology.orchestra.run.smithi003:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T07:36:28.505 INFO:teuthology.orchestra.run.smithi003.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T07:36:28.506 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:36:28.506 INFO:teuthology.orchestra.run.smithi003.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:36:28.506 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-09-10T07:36:28.506 INFO:teuthology.orchestra.run.smithi003.stdout:for file in vmlinuz System.map config; do 2024-09-10T07:36:28.506 INFO:teuthology.orchestra.run.smithi003.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout: fi 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout:done 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:28.507 INFO:teuthology.orchestra.run.smithi003.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.508 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-09-10T07:36:28.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-09-10T07:36:28.588 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-09-10T07:36:28.588 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-09-10T07:36:28.588 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-09-10T07:36:28.588 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 82.el9 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8274670 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T07:36:28.589 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:36:28.590 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T07:36:28.591 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T07:36:28.591 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:28.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T07:36:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-10T07:36:30.663 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T07:36:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-10T07:36:30.702 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-10T07:36:30.758 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.kQ7eARmqaj 2024-09-10T07:36:30.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kQ7eARmqaj 2024-09-10T07:36:30.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.kQ7eARmqaj 2024-09-10T07:36:30.972 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.kQ7eARmqaj is 6KB 2024-09-10T07:36:31.022 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.kQ7eARmqaj 2024-09-10T07:36:31.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T07:36:31.099 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T07:36:31.099 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T07:36:31.100 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T07:36:31.100 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T07:36:31.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:36:31.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-10T07:36:58.073 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T07:36:58.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:36:58.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:37:01.261 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T07:37:01.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:37:01.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:37:16.605 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-10T07:37:19.677 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-10T07:37:25.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:37:25.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:37:28.669 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-10T07:37:28.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:37:28.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:37:40.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:37:40.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:38:12.581 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-10T07:38:13.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:38:13.117 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T07:38:13.117 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-10T07:38:13.151 INFO:teuthology.orchestra.run.smithi070.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:38:13.152 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:38:13.152 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T07:38:13.152 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T07:38:14.153 DEBUG:teuthology.parallel:result is None 2024-09-10T07:38:28.680 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T07:38:40.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:38:40.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:38:41.070 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-10T07:38:41.546 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:38:41.546 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T07:38:41.547 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-10T07:38:41.580 INFO:teuthology.orchestra.run.smithi003.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:38:41.581 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:38:41.581 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T07:38:41.581 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T07:38:42.583 DEBUG:teuthology.parallel:result is None 2024-09-10T07:38:42.583 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T07:38:42.593 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T07:38:42.593 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:38:42.597 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:38:42.638 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T07:38:42.643 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T07:38:42.651 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T07:38:42.652 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:38:42.676 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:38:42.734 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T07:38:42.740 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T07:38:42.740 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:38:42.744 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:38:42.835 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:38:42.868 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:38:42.886 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:38:42.919 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:38:42.922 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T07:38:42.929 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T07:38:42.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:38:42.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:38:42.990 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T07:38:42.998 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T07:38:42.999 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:38:43.003 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:38:43.037 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:38:43.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:38:43.220 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:38:43.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:38:43.353 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:38:43.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:38:43.408 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:38:43.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:38:43.556 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:38:43.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:38:43.689 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:38:43.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:38:43.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-10T07:38:43.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-10T07:38:43.825 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:38:43.904 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:38:44.274 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T07:38:44.282 INFO:teuthology.task.internal:Starting timer... 2024-09-10T07:38:44.282 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T07:38:44.292 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T07:38:44.300 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-10T07:38:44.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-09-10T07:38:44.387 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-09-10T07:38:44.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-10T07:38:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-10T07:38:44.492 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T07:38:44.492 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-09-10T07:38:44.546 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-09-10T07:38:44.546 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-10T07:38:44.590 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-10T07:38:44.590 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive'} 2024-09-10T07:38:44.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:38:44.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:38:44.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:38:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:38:44.706 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T07:38:44.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:38:44.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:38:44.838 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T07:38:44.850 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T07:38:44.853 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T07:38:44.997 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T07:38:45.008 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T07:38:45.009 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi070.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T07:43:27.718 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com')] 2024-09-10T07:43:27.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:43:27.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:43:27.812 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-10T07:43:27.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-10T07:43:27.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:43:27.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:43:28.012 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-10T07:43:28.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T07:43:28.115 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T07:43:28.125 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T07:43:28.126 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:43:28.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:43:28.130 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:43:28.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:43:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:43:28.215 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:43:28.219 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:43:28.245 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:43:28.270 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-09-10T07:43:28.294 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-10T07:43:28.295 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-09-10T07:43:28.318 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-10T07:43:28.324 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:43:28.348 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:43:28.353 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:43:28.377 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:43:28.456 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:43:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:43:28.463 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-10T07:43:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.468 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:43:28.474 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:43:28.474 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-10T07:43:28.474 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.475 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.475 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.475 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:43:28.476 INFO:teuthology.run_tasks:Running task install... 2024-09-10T07:43:28.483 DEBUG:teuthology.task.install:project ceph 2024-09-10T07:43:28.483 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T07:43:28.483 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T07:43:28.483 INFO:teuthology.task.install:Using flavor: default 2024-09-10T07:43:28.488 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T07:43:28.489 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T07:43:28.489 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:43:28.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:43:28.491 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:43:28.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:43:28.642 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:43:28.642 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:43:28.648 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:43:28.648 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:43:28.756 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:43:28.756 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:43:28.756 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:43:28.763 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:43:28.763 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:43:28.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:43:28.814 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:43:28.814 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:43:28.819 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:43:28.819 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:43:28.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:43:28.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:43:28.992 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:43:28.994 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:43:29.048 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-09-10T07:43:29.049 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-10T07:43:29.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-09-10T07:43:29.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-10T07:43:29.330 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-10T07:43:29.331 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-09-10T07:43:29.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:43:29.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:43:30.035 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-10T07:43:30.060 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-09-10T07:43:30.319 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:43:30.352 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-10T07:43:30.587 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T07:43:30.636 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-10T07:43:31.377 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T07:43:32.420 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T07:43:36.170 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-10T07:43:37.795 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 20 MB 00:04 2024-09-10T07:43:42.346 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-10T07:43:43.745 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T07:43:43.896 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-10T07:43:45.075 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T07:43:46.114 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T07:43:47.027 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T07:43:51.623 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-09-10T07:43:53.307 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-10T07:43:54.221 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:43:54.221 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:43:54.236 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:43:54.237 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:43:54.238 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:43:54.239 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:43:54.241 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:43:54.243 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:43:54.245 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:43:54.315 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-10T07:43:54.334 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-10T07:43:54.334 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-10T07:43:54.334 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:43:54.335 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:43:54.336 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:43:54.337 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:43:54.338 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:43:54.338 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:43:54.338 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:43:54.338 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:43:54.338 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:43:54.339 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:43:54.340 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:43:54.341 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:43:54.342 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:43:54.343 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:43:54.344 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:43:54.345 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:43:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:43:54.347 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:43:54.348 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout:Install 148 Packages 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 8 Packages 2024-09-10T07:43:54.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:43:54.350 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 226 M 2024-09-10T07:43:54.350 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-10T07:43:55.449 INFO:teuthology.orchestra.run.smithi003.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-10T07:43:55.624 INFO:teuthology.orchestra.run.smithi003.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 5.0 MB/s | 897 kB 00:00 2024-09-10T07:43:55.708 INFO:teuthology.orchestra.run.smithi003.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T07:43:55.788 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:43:55.789 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:43:55.802 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:43:55.803 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:43:55.804 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:43:55.805 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:43:55.807 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:43:55.810 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:43:55.811 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:43:55.816 INFO:teuthology.orchestra.run.smithi003.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T07:43:55.876 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:43:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:43:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:43:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:43:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:43:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:43:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:43:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:43:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:43:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:43:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:43:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:43:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout:Install 148 Packages 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 8 Packages 2024-09-10T07:43:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:43:55.917 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 226 M 2024-09-10T07:43:55.917 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T07:43:55.925 INFO:teuthology.orchestra.run.smithi003.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-10T07:43:55.993 INFO:teuthology.orchestra.run.smithi003.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-10T07:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 38 kB/s | 6.5 kB 00:00 2024-09-10T07:43:56.551 INFO:teuthology.orchestra.run.smithi003.stdout:(7/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 17 MB/s | 21 MB 00:01 2024-09-10T07:43:56.567 INFO:teuthology.orchestra.run.smithi070.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 5.0 MB/s | 897 kB 00:00 2024-09-10T07:43:56.634 INFO:teuthology.orchestra.run.smithi003.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-10T07:43:56.642 INFO:teuthology.orchestra.run.smithi070.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.9 MB/s | 145 kB 00:00 2024-09-10T07:43:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 432 kB/s | 25 kB 00:00 2024-09-10T07:43:56.801 INFO:teuthology.orchestra.run.smithi070.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 9.3 MB/s | 5.4 MB 00:00 2024-09-10T07:43:57.076 INFO:teuthology.orchestra.run.smithi070.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 5.5 MB/s | 2.4 MB 00:00 2024-09-10T07:43:57.343 INFO:teuthology.orchestra.run.smithi003.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 12 MB/s | 17 MB 00:01 2024-09-10T07:43:57.385 INFO:teuthology.orchestra.run.smithi070.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 2.6 MB/s | 1.5 MB 00:00 2024-09-10T07:43:57.510 INFO:teuthology.orchestra.run.smithi003.stdout:(11/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 11 MB/s | 11 MB 00:00 2024-09-10T07:43:57.543 INFO:teuthology.orchestra.run.smithi003.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.1 MB/s | 228 kB 00:00 2024-09-10T07:43:57.551 INFO:teuthology.orchestra.run.smithi070.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 9.9 MB/s | 4.7 MB 00:00 2024-09-10T07:43:57.560 INFO:teuthology.orchestra.run.smithi003.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 541 kB/s | 27 kB 00:00 2024-09-10T07:43:57.611 INFO:teuthology.orchestra.run.smithi003.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 475 kB/s | 32 kB 00:00 2024-09-10T07:43:57.644 INFO:teuthology.orchestra.run.smithi003.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-10T07:43:57.689 INFO:teuthology.orchestra.run.smithi003.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T07:43:57.720 INFO:teuthology.orchestra.run.smithi003.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 1.6 MB/s | 127 kB 00:00 2024-09-10T07:43:57.828 INFO:teuthology.orchestra.run.smithi003.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 3.5 MB/s | 506 kB 00:00 2024-09-10T07:43:57.896 INFO:teuthology.orchestra.run.smithi003.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 675 kB/s | 45 kB 00:00 2024-09-10T07:43:57.971 INFO:teuthology.orchestra.run.smithi003.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.8 MB/s | 136 kB 00:00 2024-09-10T07:43:58.061 INFO:teuthology.orchestra.run.smithi070.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 12 MB/s | 21 MB 00:01 2024-09-10T07:43:58.096 INFO:teuthology.orchestra.run.smithi003.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-10T07:43:58.119 INFO:teuthology.orchestra.run.smithi070.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 433 kB/s | 25 kB 00:00 2024-09-10T07:43:58.121 INFO:teuthology.orchestra.run.smithi003.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.1 MB/s | 161 kB 00:00 2024-09-10T07:43:58.155 INFO:teuthology.orchestra.run.smithi003.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.4 MB/s | 320 kB 00:00 2024-09-10T07:43:58.213 INFO:teuthology.orchestra.run.smithi003.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.2 MB/s | 301 kB 00:00 2024-09-10T07:43:58.239 INFO:teuthology.orchestra.run.smithi003.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.2 MB/s | 99 kB 00:00 2024-09-10T07:43:58.289 INFO:teuthology.orchestra.run.smithi003.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-10T07:43:58.409 INFO:teuthology.orchestra.run.smithi003.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.4 MB/s | 170 kB 00:00 2024-09-10T07:43:58.515 INFO:teuthology.orchestra.run.smithi003.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-10T07:43:58.540 INFO:teuthology.orchestra.run.smithi003.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 244 kB/s | 32 kB 00:00 2024-09-10T07:43:58.565 INFO:teuthology.orchestra.run.smithi003.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T07:43:58.636 INFO:teuthology.orchestra.run.smithi070.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 9.8 MB/s | 11 MB 00:01 2024-09-10T07:43:58.745 INFO:teuthology.orchestra.run.smithi070.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.1 MB/s | 228 kB 00:00 2024-09-10T07:43:58.853 INFO:teuthology.orchestra.run.smithi070.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 249 kB/s | 27 kB 00:00 2024-09-10T07:43:58.962 INFO:teuthology.orchestra.run.smithi070.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 293 kB/s | 32 kB 00:00 2024-09-10T07:43:58.991 INFO:teuthology.orchestra.run.smithi003.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 7.9 MB/s | 3.6 MB 00:00 2024-09-10T07:43:59.121 INFO:teuthology.orchestra.run.smithi070.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 4.6 MB/s | 751 kB 00:00 2024-09-10T07:43:59.124 INFO:teuthology.orchestra.run.smithi003.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-10T07:43:59.158 INFO:teuthology.orchestra.run.smithi003.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.5 MB/s | 249 kB 00:00 2024-09-10T07:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.4 MB/s | 164 kB 00:00 2024-09-10T07:43:59.231 INFO:teuthology.orchestra.run.smithi003.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 462 kB/s | 49 kB 00:00 2024-09-10T07:43:59.258 INFO:teuthology.orchestra.run.smithi003.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 164 kB/s | 16 kB 00:00 2024-09-10T07:43:59.271 INFO:teuthology.orchestra.run.smithi070.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.5 MB/s | 127 kB 00:00 2024-09-10T07:43:59.308 INFO:teuthology.orchestra.run.smithi003.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.6 MB/s | 285 kB 00:00 2024-09-10T07:43:59.379 INFO:teuthology.orchestra.run.smithi070.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 4.6 MB/s | 506 kB 00:00 2024-09-10T07:43:59.393 INFO:teuthology.orchestra.run.smithi003.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 5.6 MB/s | 766 kB 00:00 2024-09-10T07:43:59.418 INFO:teuthology.orchestra.run.smithi003.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 168 kB/s | 18 kB 00:00 2024-09-10T07:43:59.797 INFO:teuthology.orchestra.run.smithi070.stdout:(18/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 7.0 MB/s | 17 MB 00:02 2024-09-10T07:43:59.838 INFO:teuthology.orchestra.run.smithi070.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 1.1 MB/s | 45 kB 00:00 2024-09-10T07:43:59.931 INFO:teuthology.orchestra.run.smithi070.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 9.7 MB/s | 5.3 MB 00:00 2024-09-10T07:43:59.956 INFO:teuthology.orchestra.run.smithi070.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.1 MB/s | 136 kB 00:00 2024-09-10T07:43:59.981 INFO:teuthology.orchestra.run.smithi070.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.1 MB/s | 161 kB 00:00 2024-09-10T07:44:00.006 INFO:teuthology.orchestra.run.smithi070.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 6.3 MB/s | 320 kB 00:00 2024-09-10T07:44:00.074 INFO:teuthology.orchestra.run.smithi070.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.2 MB/s | 301 kB 00:00 2024-09-10T07:44:00.080 INFO:teuthology.orchestra.run.smithi003.stdout:(39/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 14 MB/s | 49 MB 00:03 2024-09-10T07:44:00.099 INFO:teuthology.orchestra.run.smithi070.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.1 MB/s | 99 kB 00:00 2024-09-10T07:44:00.110 INFO:teuthology.orchestra.run.smithi003.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-09-10T07:44:00.124 INFO:teuthology.orchestra.run.smithi070.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T07:44:00.175 INFO:teuthology.orchestra.run.smithi070.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T07:44:00.233 INFO:teuthology.orchestra.run.smithi070.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 550 kB/s | 32 kB 00:00 2024-09-10T07:44:00.308 INFO:teuthology.orchestra.run.smithi070.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-10T07:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.4 MB/s | 146 kB 00:00 2024-09-10T07:44:00.544 INFO:teuthology.orchestra.run.smithi003.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 585 kB/s | 253 kB 00:00 2024-09-10T07:44:00.584 INFO:teuthology.orchestra.run.smithi070.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 13 MB/s | 3.6 MB 00:00 2024-09-10T07:44:00.602 INFO:teuthology.orchestra.run.smithi003.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 677 kB/s | 801 kB 00:01 2024-09-10T07:44:00.642 INFO:teuthology.orchestra.run.smithi070.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 4.2 MB/s | 249 kB 00:00 2024-09-10T07:44:00.710 INFO:teuthology.orchestra.run.smithi070.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 740 kB/s | 49 kB 00:00 2024-09-10T07:44:00.711 INFO:teuthology.orchestra.run.smithi003.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-09-10T07:44:00.752 INFO:teuthology.orchestra.run.smithi003.stdout:(44/156): libquadmath-11.5.0-2.el9.x86_64.rpm 285 kB/s | 191 kB 00:00 2024-09-10T07:44:00.776 INFO:teuthology.orchestra.run.smithi070.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 248 kB/s | 16 kB 00:00 2024-09-10T07:44:00.819 INFO:teuthology.orchestra.run.smithi003.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-10T07:44:00.885 INFO:teuthology.orchestra.run.smithi070.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-10T07:44:00.911 INFO:teuthology.orchestra.run.smithi070.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 2.1 MB/s | 285 kB 00:00 2024-09-10T07:44:00.928 INFO:teuthology.orchestra.run.smithi003.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-10T07:44:01.087 INFO:teuthology.orchestra.run.smithi003.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-09-10T07:44:01.112 INFO:teuthology.orchestra.run.smithi003.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 348 kB/s | 125 kB 00:00 2024-09-10T07:44:01.171 INFO:teuthology.orchestra.run.smithi003.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-09-10T07:44:01.204 INFO:teuthology.orchestra.run.smithi003.stdout:(50/156): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-09-10T07:44:01.230 INFO:teuthology.orchestra.run.smithi003.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-10T07:44:01.255 INFO:teuthology.orchestra.run.smithi003.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 751 kB/s | 107 kB 00:00 2024-09-10T07:44:01.297 INFO:teuthology.orchestra.run.smithi003.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-10T07:44:01.330 INFO:teuthology.orchestra.run.smithi003.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 552 kB/s | 18 kB 00:00 2024-09-10T07:44:01.364 INFO:teuthology.orchestra.run.smithi003.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-09-10T07:44:01.378 INFO:teuthology.orchestra.run.smithi070.stdout:(37/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 15 MB/s | 49 MB 00:03 2024-09-10T07:44:01.380 INFO:teuthology.orchestra.run.smithi003.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T07:44:01.403 INFO:teuthology.orchestra.run.smithi070.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 37 kB/s | 18 kB 00:00 2024-09-10T07:44:01.406 INFO:teuthology.orchestra.run.smithi003.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-09-10T07:44:01.423 INFO:teuthology.orchestra.run.smithi003.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-10T07:44:01.437 INFO:teuthology.orchestra.run.smithi070.stdout:(39/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 1.4 MB/s | 766 kB 00:00 2024-09-10T07:44:01.456 INFO:teuthology.orchestra.run.smithi003.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-10T07:44:01.498 INFO:teuthology.orchestra.run.smithi003.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-10T07:44:01.523 INFO:teuthology.orchestra.run.smithi003.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-10T07:44:01.549 INFO:teuthology.orchestra.run.smithi003.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-10T07:44:01.553 INFO:teuthology.orchestra.run.smithi070.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-10T07:44:01.577 INFO:teuthology.orchestra.run.smithi003.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T07:44:01.599 INFO:teuthology.orchestra.run.smithi003.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T07:44:01.620 INFO:teuthology.orchestra.run.smithi070.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-10T07:44:01.625 INFO:teuthology.orchestra.run.smithi003.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-10T07:44:01.654 INFO:teuthology.orchestra.run.smithi070.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-10T07:44:01.696 INFO:teuthology.orchestra.run.smithi070.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-10T07:44:01.699 INFO:teuthology.orchestra.run.smithi003.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-09-10T07:44:01.721 INFO:teuthology.orchestra.run.smithi070.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-10T07:44:01.746 INFO:teuthology.orchestra.run.smithi070.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T07:44:01.772 INFO:teuthology.orchestra.run.smithi070.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T07:44:01.775 INFO:teuthology.orchestra.run.smithi003.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T07:44:01.814 INFO:teuthology.orchestra.run.smithi070.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-09-10T07:44:01.839 INFO:teuthology.orchestra.run.smithi070.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-10T07:44:01.859 INFO:teuthology.orchestra.run.smithi003.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-10T07:44:01.926 INFO:teuthology.orchestra.run.smithi003.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-10T07:44:01.948 INFO:teuthology.orchestra.run.smithi070.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-10T07:44:01.951 INFO:teuthology.orchestra.run.smithi003.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T07:44:01.973 INFO:teuthology.orchestra.run.smithi070.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 801 kB/s | 107 kB 00:00 2024-09-10T07:44:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-10T07:44:01.999 INFO:teuthology.orchestra.run.smithi070.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-10T07:44:02.027 INFO:teuthology.orchestra.run.smithi003.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-10T07:44:02.040 INFO:teuthology.orchestra.run.smithi070.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-10T07:44:02.052 INFO:teuthology.orchestra.run.smithi003.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T07:44:02.065 INFO:teuthology.orchestra.run.smithi070.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-10T07:44:02.100 INFO:teuthology.orchestra.run.smithi070.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-10T07:44:02.119 INFO:teuthology.orchestra.run.smithi003.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 919 kB/s | 61 kB 00:00 2024-09-10T07:44:02.125 INFO:teuthology.orchestra.run.smithi070.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-10T07:44:02.150 INFO:teuthology.orchestra.run.smithi070.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-10T07:44:02.175 INFO:teuthology.orchestra.run.smithi070.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T07:44:02.200 INFO:teuthology.orchestra.run.smithi070.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-10T07:44:02.211 INFO:teuthology.orchestra.run.smithi003.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-10T07:44:02.236 INFO:teuthology.orchestra.run.smithi003.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T07:44:02.251 INFO:teuthology.orchestra.run.smithi070.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-10T07:44:02.261 INFO:teuthology.orchestra.run.smithi003.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-10T07:44:02.284 INFO:teuthology.orchestra.run.smithi070.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-10T07:44:02.286 INFO:teuthology.orchestra.run.smithi003.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-10T07:44:02.309 INFO:teuthology.orchestra.run.smithi070.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T07:44:02.311 INFO:teuthology.orchestra.run.smithi003.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T07:44:02.335 INFO:teuthology.orchestra.run.smithi070.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-10T07:44:02.337 INFO:teuthology.orchestra.run.smithi003.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-10T07:44:02.360 INFO:teuthology.orchestra.run.smithi070.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T07:44:02.370 INFO:teuthology.orchestra.run.smithi003.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-10T07:44:02.385 INFO:teuthology.orchestra.run.smithi070.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T07:44:02.411 INFO:teuthology.orchestra.run.smithi070.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-10T07:44:02.454 INFO:teuthology.orchestra.run.smithi003.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-10T07:44:02.470 INFO:teuthology.orchestra.run.smithi070.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-09-10T07:44:02.513 INFO:teuthology.orchestra.run.smithi003.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-10T07:44:02.538 INFO:teuthology.orchestra.run.smithi003.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T07:44:02.545 INFO:teuthology.orchestra.run.smithi070.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T07:44:02.563 INFO:teuthology.orchestra.run.smithi003.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T07:44:02.588 INFO:teuthology.orchestra.run.smithi003.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-10T07:44:02.614 INFO:teuthology.orchestra.run.smithi003.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T07:44:02.629 INFO:teuthology.orchestra.run.smithi070.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-10T07:44:02.639 INFO:teuthology.orchestra.run.smithi003.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T07:44:02.664 INFO:teuthology.orchestra.run.smithi003.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T07:44:02.690 INFO:teuthology.orchestra.run.smithi003.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T07:44:02.704 INFO:teuthology.orchestra.run.smithi070.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-10T07:44:02.715 INFO:teuthology.orchestra.run.smithi003.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T07:44:02.729 INFO:teuthology.orchestra.run.smithi070.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-10T07:44:02.757 INFO:teuthology.orchestra.run.smithi003.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T07:44:02.796 INFO:teuthology.orchestra.run.smithi070.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.5 MB/s | 910 kB 00:00 2024-09-10T07:44:02.832 INFO:teuthology.orchestra.run.smithi003.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-10T07:44:02.846 INFO:teuthology.orchestra.run.smithi070.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-10T07:44:02.857 INFO:teuthology.orchestra.run.smithi003.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-10T07:44:02.871 INFO:teuthology.orchestra.run.smithi070.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-10T07:44:02.914 INFO:teuthology.orchestra.run.smithi070.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T07:44:02.944 INFO:teuthology.orchestra.run.smithi003.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-09-10T07:44:02.966 INFO:teuthology.orchestra.run.smithi003.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-09-10T07:44:02.991 INFO:teuthology.orchestra.run.smithi003.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2024-09-10T07:44:03.014 INFO:teuthology.orchestra.run.smithi070.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-10T07:44:03.017 INFO:teuthology.orchestra.run.smithi003.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T07:44:03.039 INFO:teuthology.orchestra.run.smithi070.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-10T07:44:03.042 INFO:teuthology.orchestra.run.smithi003.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:44:03.059 INFO:teuthology.orchestra.run.smithi003.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-10T07:44:03.064 INFO:teuthology.orchestra.run.smithi070.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-10T07:44:03.076 INFO:teuthology.orchestra.run.smithi003.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 64 kB 00:00 2024-09-10T07:44:03.090 INFO:teuthology.orchestra.run.smithi070.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-10T07:44:03.093 INFO:teuthology.orchestra.run.smithi003.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-10T07:44:03.115 INFO:teuthology.orchestra.run.smithi070.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-10T07:44:03.140 INFO:teuthology.orchestra.run.smithi070.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T07:44:03.165 INFO:teuthology.orchestra.run.smithi070.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T07:44:03.266 INFO:teuthology.orchestra.run.smithi070.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-10T07:44:03.291 INFO:teuthology.orchestra.run.smithi070.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-10T07:44:03.317 INFO:teuthology.orchestra.run.smithi070.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T07:44:03.342 INFO:teuthology.orchestra.run.smithi070.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T07:44:03.367 INFO:teuthology.orchestra.run.smithi070.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-10T07:44:03.368 INFO:teuthology.orchestra.run.smithi003.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-10T07:44:03.393 INFO:teuthology.orchestra.run.smithi003.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-09-10T07:44:03.429 INFO:teuthology.orchestra.run.smithi070.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.0 MB/s | 269 kB 00:00 2024-09-10T07:44:03.452 INFO:teuthology.orchestra.run.smithi003.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-10T07:44:03.459 INFO:teuthology.orchestra.run.smithi070.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-10T07:44:03.477 INFO:teuthology.orchestra.run.smithi003.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-10T07:44:03.484 INFO:teuthology.orchestra.run.smithi070.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.0 MB/s | 279 kB 00:00 2024-09-10T07:44:03.509 INFO:teuthology.orchestra.run.smithi070.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T07:44:03.511 INFO:teuthology.orchestra.run.smithi003.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-10T07:44:03.535 INFO:teuthology.orchestra.run.smithi070.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T07:44:03.536 INFO:teuthology.orchestra.run.smithi003.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T07:44:03.569 INFO:teuthology.orchestra.run.smithi003.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-10T07:44:03.577 INFO:teuthology.orchestra.run.smithi070.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T07:44:03.595 INFO:teuthology.orchestra.run.smithi003.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-09-10T07:44:03.637 INFO:teuthology.orchestra.run.smithi003.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.8 MB/s | 151 kB 00:00 2024-09-10T07:44:03.661 INFO:teuthology.orchestra.run.smithi070.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.8 MB/s | 5.1 MB 00:00 2024-09-10T07:44:03.670 INFO:teuthology.orchestra.run.smithi003.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-09-10T07:44:03.694 INFO:teuthology.orchestra.run.smithi070.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-10T07:44:03.720 INFO:teuthology.orchestra.run.smithi070.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T07:44:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-09-10T07:44:03.745 INFO:teuthology.orchestra.run.smithi070.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-10T07:44:03.770 INFO:teuthology.orchestra.run.smithi070.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-10T07:44:03.779 INFO:teuthology.orchestra.run.smithi003.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-09-10T07:44:03.795 INFO:teuthology.orchestra.run.smithi070.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T07:44:03.804 INFO:teuthology.orchestra.run.smithi003.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T07:44:03.821 INFO:teuthology.orchestra.run.smithi070.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:44:03.830 INFO:teuthology.orchestra.run.smithi003.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-10T07:44:03.855 INFO:teuthology.orchestra.run.smithi003.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-10T07:44:03.880 INFO:teuthology.orchestra.run.smithi003.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T07:44:03.905 INFO:teuthology.orchestra.run.smithi070.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 882 kB/s | 96 kB 00:00 2024-09-10T07:44:03.914 INFO:teuthology.orchestra.run.smithi003.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T07:44:03.939 INFO:teuthology.orchestra.run.smithi003.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T07:44:03.963 INFO:teuthology.orchestra.run.smithi070.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 448 kB/s | 64 kB 00:00 2024-09-10T07:44:03.964 INFO:teuthology.orchestra.run.smithi003.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T07:44:03.988 INFO:teuthology.orchestra.run.smithi070.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-09-10T07:44:03.998 INFO:teuthology.orchestra.run.smithi003.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T07:44:04.023 INFO:teuthology.orchestra.run.smithi003.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T07:44:04.090 INFO:teuthology.orchestra.run.smithi003.stdout:(124/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-09-10T07:44:04.115 INFO:teuthology.orchestra.run.smithi003.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-10T07:44:04.169 INFO:teuthology.orchestra.run.smithi003.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 369 kB/s | 19 kB 00:00 2024-09-10T07:44:04.199 INFO:teuthology.orchestra.run.smithi003.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-09-10T07:44:04.222 INFO:teuthology.orchestra.run.smithi070.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-10T07:44:04.224 INFO:teuthology.orchestra.run.smithi003.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 355 kB/s | 20 kB 00:00 2024-09-10T07:44:04.249 INFO:teuthology.orchestra.run.smithi003.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T07:44:04.275 INFO:teuthology.orchestra.run.smithi003.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T07:44:04.281 INFO:teuthology.orchestra.run.smithi070.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T07:44:04.300 INFO:teuthology.orchestra.run.smithi003.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T07:44:04.314 INFO:teuthology.orchestra.run.smithi070.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T07:44:04.340 INFO:teuthology.orchestra.run.smithi070.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-10T07:44:04.342 INFO:teuthology.orchestra.run.smithi003.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.3 MB/s | 272 kB 00:00 2024-09-10T07:44:04.368 INFO:teuthology.orchestra.run.smithi003.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-10T07:44:04.401 INFO:teuthology.orchestra.run.smithi003.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-10T07:44:04.406 INFO:teuthology.orchestra.run.smithi070.stdout:(107/156): lua-devel-5.4.4-4.el9.x86_64.rpm 50 kB/s | 22 kB 00:00 2024-09-10T07:44:04.427 INFO:teuthology.orchestra.run.smithi003.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T07:44:04.452 INFO:teuthology.orchestra.run.smithi003.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T07:44:04.477 INFO:teuthology.orchestra.run.smithi003.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T07:44:04.499 INFO:teuthology.orchestra.run.smithi070.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-10T07:44:04.502 INFO:teuthology.orchestra.run.smithi003.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T07:44:04.515 INFO:teuthology.orchestra.run.smithi070.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 3.3 MB/s | 49 kB 00:00 2024-09-10T07:44:04.528 INFO:teuthology.orchestra.run.smithi003.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T07:44:04.532 INFO:teuthology.orchestra.run.smithi070.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-09-10T07:44:04.566 INFO:teuthology.orchestra.run.smithi070.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T07:44:04.578 INFO:teuthology.orchestra.run.smithi003.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T07:44:04.591 INFO:teuthology.orchestra.run.smithi070.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-10T07:44:04.620 INFO:teuthology.orchestra.run.smithi003.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-10T07:44:04.633 INFO:teuthology.orchestra.run.smithi070.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-10T07:44:04.645 INFO:teuthology.orchestra.run.smithi003.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 635 kB/s | 74 kB 00:00 2024-09-10T07:44:04.658 INFO:teuthology.orchestra.run.smithi070.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-10T07:44:04.670 INFO:teuthology.orchestra.run.smithi003.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T07:44:04.684 INFO:teuthology.orchestra.run.smithi070.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-09-10T07:44:04.696 INFO:teuthology.orchestra.run.smithi003.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-10T07:44:04.709 INFO:teuthology.orchestra.run.smithi070.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-10T07:44:04.721 INFO:teuthology.orchestra.run.smithi003.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-09-10T07:44:04.734 INFO:teuthology.orchestra.run.smithi070.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-10T07:44:04.754 INFO:teuthology.orchestra.run.smithi003.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-10T07:44:04.759 INFO:teuthology.orchestra.run.smithi070.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T07:44:04.780 INFO:teuthology.orchestra.run.smithi003.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T07:44:04.793 INFO:teuthology.orchestra.run.smithi070.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T07:44:04.818 INFO:teuthology.orchestra.run.smithi070.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T07:44:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.2 MB/s | 220 kB 00:00 2024-09-10T07:44:04.878 INFO:teuthology.orchestra.run.smithi070.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T07:44:04.888 INFO:teuthology.orchestra.run.smithi003.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-10T07:44:04.903 INFO:teuthology.orchestra.run.smithi070.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T07:44:04.937 INFO:teuthology.orchestra.run.smithi070.stdout:(124/156): protobuf-compiler-3.14.0-13.el9.x86_ 911 kB/s | 863 kB 00:00 2024-09-10T07:44:04.970 INFO:teuthology.orchestra.run.smithi070.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-09-10T07:44:04.995 INFO:teuthology.orchestra.run.smithi070.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 840 kB/s | 19 kB 00:00 2024-09-10T07:44:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-09-10T07:44:05.049 INFO:teuthology.orchestra.run.smithi070.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T07:44:05.071 INFO:teuthology.orchestra.run.smithi070.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T07:44:05.080 INFO:teuthology.orchestra.run.smithi003.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-10T07:44:05.097 INFO:teuthology.orchestra.run.smithi070.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T07:44:05.139 INFO:teuthology.orchestra.run.smithi070.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-10T07:44:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 7.4 MB/s | 272 kB 00:00 2024-09-10T07:44:05.200 INFO:teuthology.orchestra.run.smithi070.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-10T07:44:05.225 INFO:teuthology.orchestra.run.smithi070.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T07:44:05.242 INFO:teuthology.orchestra.run.smithi070.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-10T07:44:05.276 INFO:teuthology.orchestra.run.smithi070.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 4.0 MB/s | 1.0 MB 00:00 2024-09-10T07:44:05.293 INFO:teuthology.orchestra.run.smithi070.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T07:44:05.335 INFO:teuthology.orchestra.run.smithi070.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-09-10T07:44:05.361 INFO:teuthology.orchestra.run.smithi070.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 882 kB/s | 59 kB 00:00 2024-09-10T07:44:05.386 INFO:teuthology.orchestra.run.smithi070.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-10T07:44:05.419 INFO:teuthology.orchestra.run.smithi070.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-10T07:44:05.445 INFO:teuthology.orchestra.run.smithi070.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-10T07:44:05.470 INFO:teuthology.orchestra.run.smithi070.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T07:44:05.496 INFO:teuthology.orchestra.run.smithi070.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T07:44:05.514 INFO:teuthology.orchestra.run.smithi003.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 4.3 MB/s | 3.2 MB 00:00 2024-09-10T07:44:05.521 INFO:teuthology.orchestra.run.smithi070.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-10T07:44:05.554 INFO:teuthology.orchestra.run.smithi070.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T07:44:05.588 INFO:teuthology.orchestra.run.smithi070.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T07:44:05.639 INFO:teuthology.orchestra.run.smithi070.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T07:44:05.772 INFO:teuthology.orchestra.run.smithi070.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-09-10T07:44:05.805 INFO:teuthology.orchestra.run.smithi003.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-10T07:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-10T07:44:05.923 INFO:teuthology.orchestra.run.smithi070.stdout:(151/156): librados2-19.1.1-305.g1cf819b7.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-10T07:44:05.948 INFO:teuthology.orchestra.run.smithi070.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 444 kB/s | 41 kB 00:00 2024-09-10T07:44:05.973 INFO:teuthology.orchestra.run.smithi070.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-10T07:44:06.024 INFO:teuthology.orchestra.run.smithi070.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-10T07:44:06.082 INFO:teuthology.orchestra.run.smithi070.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-10T07:44:06.140 INFO:teuthology.orchestra.run.smithi070.stdout:(156/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 5.7 MB/s | 3.2 MB 00:00 2024-09-10T07:44:06.146 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:44:06.146 INFO:teuthology.orchestra.run.smithi070.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-10T07:44:06.147 INFO:teuthology.orchestra.run.smithi003.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 729 kB/s | 249 kB 00:00 2024-09-10T07:44:06.273 INFO:teuthology.orchestra.run.smithi003.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 1.7 MB/s | 2.3 MB 00:01 2024-09-10T07:44:06.611 INFO:teuthology.orchestra.run.smithi003.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 1.4 MB 00:00 2024-09-10T07:44:06.881 INFO:teuthology.orchestra.run.smithi003.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 1.0 MB/s | 1.8 MB 00:01 2024-09-10T07:44:07.098 INFO:teuthology.orchestra.run.smithi003.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 2.4 MB 00:00 2024-09-10T07:44:07.105 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:44:07.105 INFO:teuthology.orchestra.run.smithi003.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-10T07:44:08.189 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T07:44:08.372 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T07:44:08.372 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T07:44:09.221 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-10T07:44:09.421 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-10T07:44:09.422 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-10T07:44:10.736 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T07:44:10.745 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T07:44:11.875 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-10T07:44:11.875 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-10T07:44:17.242 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T07:44:17.403 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:44:17.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:44:17.751 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:44:17.788 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:44:17.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:44:17.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:44:18.302 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:44:18.566 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:44:18.606 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:44:18.718 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:44:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:44:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-10T07:44:18.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:44:19.057 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:44:19.085 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:44:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:44:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:44:19.196 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:44:19.228 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:44:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:44:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:44:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:44:19.340 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:44:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:44:19.370 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:44:19.419 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:44:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:44:19.786 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:44:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:44:19.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:44:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:44:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:44:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:44:20.178 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:44:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:44:20.306 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:44:20.429 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:44:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:44:20.659 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:44:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:44:20.722 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:44:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:44:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:44:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:44:20.994 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:44:21.036 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:44:21.070 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:44:21.080 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:44:21.168 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:44:21.232 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:44:21.260 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:44:21.271 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:44:21.326 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:44:21.365 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:44:21.424 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:44:21.459 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:44:21.511 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:44:21.567 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:44:21.595 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:21.603 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:44:21.649 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:44:21.729 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:44:21.738 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:21.765 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:44:21.809 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:44:21.837 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:44:21.934 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:44:21.997 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:44:22.035 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:44:22.088 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:44:22.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:44:22.246 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:44:22.308 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:44:22.322 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:44:22.367 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:44:22.411 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:44:22.524 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:44:22.590 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:44:22.627 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:44:22.683 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:44:22.720 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:44:22.754 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:44:22.851 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:44:22.925 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:44:22.938 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:44:22.958 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:44:22.996 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:44:23.080 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:44:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:44:23.180 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:44:23.208 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:44:23.304 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:44:23.361 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:44:23.443 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:44:23.455 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:44:23.488 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:44:23.531 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:44:23.570 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:44:23.622 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:44:23.628 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:44:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:44:23.804 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:44:23.873 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:44:23.940 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:44:24.160 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:44:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:44:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:44:24.296 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:44:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:44:24.465 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:44:24.516 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:44:24.683 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:44:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:44:24.725 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:44:24.769 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:44:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:44:24.807 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:44:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:44:24.861 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:44:24.872 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:44:25.037 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:44:25.060 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:44:25.127 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:44:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:44:25.196 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:44:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:44:25.266 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:44:25.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:44:25.310 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:44:25.319 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:44:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:44:25.358 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:44:25.397 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:44:25.407 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:44:25.435 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:44:25.463 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:44:25.469 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:44:25.502 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:44:25.509 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:44:25.546 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:44:25.559 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:44:25.578 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:44:25.677 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:44:25.729 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:44:25.737 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:44:25.765 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:44:25.783 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:44:25.860 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:44:25.878 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:44:25.944 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:44:25.991 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:44:26.033 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:44:26.083 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:44:26.113 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:44:26.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:44:26.274 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:44:26.452 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:44:26.491 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:44:26.528 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:44:26.565 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:44:26.611 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:44:26.676 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:44:26.712 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:44:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:44:26.780 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:44:26.824 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:44:26.854 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:44:26.856 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:44:26.908 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:44:26.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:44:26.959 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:44:27.015 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:44:27.055 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:44:27.076 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:44:27.108 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:44:27.161 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:44:27.204 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:44:27.258 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:44:27.264 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:44:27.294 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:44:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:44:27.566 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:44:27.667 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:44:28.205 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:44:28.291 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:44:28.324 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:44:28.444 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:44:28.499 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:44:28.552 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:44:28.602 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:44:28.651 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:44:28.692 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:44:28.942 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:44:28.983 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:44:29.089 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:44:29.946 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:44:30.092 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:44:30.138 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:44:30.230 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:44:30.266 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:44:30.299 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:44:30.637 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:44:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:44:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:44:30.866 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:44:30.915 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:44:31.441 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:44:31.480 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:44:31.604 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:44:31.632 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:44:31.741 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:44:31.836 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:44:31.889 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:44:31.923 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:44:32.540 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:44:32.575 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:44:32.716 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:44:32.756 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:44:32.811 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:44:33.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:44:33.403 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:44:33.519 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:44:33.546 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:44:34.324 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:34.364 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:34.989 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:35.042 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:44:35.201 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:44:35.201 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:44:35.201 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:44:35.201 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:35.235 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:44:36.213 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:36.250 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:36.912 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:44:36.962 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:44:37.084 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:44:37.084 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:44:37.084 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:44:37.085 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:37.130 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:44:47.817 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-10T07:44:47.818 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:44:48.056 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:44:48.057 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:48.057 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:44:48.057 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:44:49.725 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:44:49.725 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:49.725 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:44:49.725 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:44:49.725 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:44:49.726 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:44:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:44:50.026 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:50.026 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:44:50.026 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:44:50.026 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:44:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:50.248 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-09-10T07:44:50.249 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:50.472 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:44:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:44:50.508 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:44:50.508 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:50.508 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:44:50.508 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:50.551 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:44:50.552 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:50.552 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:44:50.552 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:44:50.552 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:44:50.552 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:50.676 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:44:50.735 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:44:50.800 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:44:51.023 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:44:51.072 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:44:51.166 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:44:51.273 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:44:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:44:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:44:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:44:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:44:52.160 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:44:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:44:52.179 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:44:52.220 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:44:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:44:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:44:52.237 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:44:52.237 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:44:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:52.304 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:44:52.334 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:44:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:44:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:44:52.552 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:52.553 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:44:52.553 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:44:52.553 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:44:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:44:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:44:53.111 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:53.111 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:44:53.111 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:44:53.111 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:44:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:44:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:44:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:44:53.547 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:44:53.595 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:44:53.639 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:44:53.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:44:53.797 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:44:53.907 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:44:54.081 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:44:54.149 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:44:54.187 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:44:54.212 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:44:54.668 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:44:54.711 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:44:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:44:54.812 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:44:54.849 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:44:54.853 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:44:54.933 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:44:54.967 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:44:55.018 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:44:55.047 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:44:55.098 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:44:55.098 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:55.099 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:44:55.099 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:44:55.099 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:44:55.099 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:55.156 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:44:55.183 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:44:56.260 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:44:56.302 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:44:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:44:56.434 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:56.434 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:44:56.435 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:44:56.435 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:44:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:44:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:44:56.534 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:56.534 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:44:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:56.593 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:44:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:44:56.644 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:56.644 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:44:56.645 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:44:56.645 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:44:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:56.977 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:44:57.028 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:44:57.029 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:57.029 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:44:57.029 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:44:57.029 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:44:57.029 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:44:57.453 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:44:57.486 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:44:57.630 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:44:57.808 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:44:57.838 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:44:57.891 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:57.947 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:44:57.974 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:44:58.085 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:44:58.156 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:44:59.227 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:44:59.257 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:44:59.310 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:44:59.311 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:59.311 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:44:59.311 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:44:59.311 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:44:59.311 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:59.366 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:44:59.417 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:44:59.418 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:59.418 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:44:59.418 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:59.475 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:44:59.528 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:44:59.528 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:59.528 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:44:59.528 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:44:59.528 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:44:59.529 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:44:59.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:44:59.937 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:44:59.937 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:44:59.937 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:44:59.938 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:44:59.938 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:44:59.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:01.974 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:45:02.015 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:45:02.058 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:45:02.088 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:45:02.123 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:45:02.351 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:45:02.403 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:45:02.433 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:45:02.464 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:45:03.305 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:03.415 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:03.415 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:04.647 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:45:04.771 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:45:04.825 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:45:04.936 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:45:04.995 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:45:05.024 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:45:05.176 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:45:05.176 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:45:05.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:45:05.255 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:45:05.286 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:45:05.310 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:45:05.326 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:05.326 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:05.338 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:45:05.379 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:45:05.414 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:45:05.661 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:45:05.664 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:45:05.737 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:45:05.738 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:05.774 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:45:05.806 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:45:05.849 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:45:05.940 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:05.940 INFO:teuthology.orchestra.run.smithi070.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T07:45:05.941 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:06.573 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:06.622 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:06.664 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:45:08.042 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:45:08.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:45:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:45:08.358 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:45:08.396 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:45:08.421 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:45:08.466 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:45:08.466 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:45:08.599 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:45:08.599 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:45:08.708 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:45:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:45:08.748 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:08.748 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:09.001 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:45:09.002 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:45:09.102 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:45:09.179 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:09.373 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:09.373 INFO:teuthology.orchestra.run.smithi003.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T07:45:09.374 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:11.861 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:45:11.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:45:11.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:45:11.864 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:45:11.865 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:45:11.887 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:45:11.888 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:45:11.889 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:45:11.892 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:45:11.893 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:45:11.894 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:45:11.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:45:11.896 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:45:11.897 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:45:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:45:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:45:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:45:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:45:11.898 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:45:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:13.974 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:13.974 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:45:13.975 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.976 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:13.977 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:45:13.978 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:45:13.979 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:45:13.980 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:45:13.981 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:45:13.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:45:13.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.984 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:45:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:45:13.986 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:45:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:45:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:45:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:45:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:45:13.987 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:13.988 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T07:45:14.597 DEBUG:teuthology.parallel:result is None 2024-09-10T07:45:15.220 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:45:15.220 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:45:15.220 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:45:15.221 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:45:15.222 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:45:15.234 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:45:15.235 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:45:15.236 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:45:15.237 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:45:15.238 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:45:15.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:45:15.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:45:15.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:45:15.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:45:15.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:45:17.635 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:17.636 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.637 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:45:17.638 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.639 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:45:17.640 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:45:17.641 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:45:17.642 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:45:17.643 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:45:17.644 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:45:17.645 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:45:17.646 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:45:17.647 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:45:17.648 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:45:17.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:17.648 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-10T07:45:18.300 DEBUG:teuthology.parallel:result is None 2024-09-10T07:45:18.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:45:18.449 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:45:18.495 INFO:teuthology.orchestra.run.smithi003.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:45:18.495 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:45:18.496 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:45:18.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:45:18.642 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:45:18.692 INFO:teuthology.orchestra.run.smithi070.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:45:18.692 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:45:18.692 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:45:18.694 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T07:45:18.694 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:18.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:45:18.760 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:18.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:45:18.822 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T07:45:18.822 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:18.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:45:18.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:45:18.978 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:18.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:45:19.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:45:19.134 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T07:45:19.134 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:19.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:45:19.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:45:19.293 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:19.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:45:19.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:45:19.447 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T07:45:19.447 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:19.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:45:19.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:45:19.602 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:19.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:45:19.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:45:19.766 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T07:45:19.849 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T07:45:19.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-09-10T07:45:19.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-09-10T07:45:19.912 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T07:45:19.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T07:45:19.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T07:45:19.972 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T07:45:19.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T07:45:19.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T07:45:20.039 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T07:45:20.039 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T07:45:20.039 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T07:45:20.039 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:20.039 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T07:45:20.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:20.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:20.318 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:20.318 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T07:45:20.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:20.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:20.583 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T07:45:20.583 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T07:45:20.583 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880', 'branch': 'wip-rishabh-testing-20240909.165402-squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc}', 'email': None, 'first_in_suite': False, 'job_id': '7898880', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9893, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'sleep_before_teardown': 0, 'subset': '197/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240909.165402-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBsjShZzrJ9nUmQM0wSvp60Y3LYSGNE24FkJgUZNEdqh6oSBLPzAajR2Zapv8oCY+pobHX8OAYncnUrQy+aOyRA=', 'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFIUC1eFtiky9yQkM0WFqNVkbYCF+QIVb5hACucowxO46ffXSP0EaZc9QqZnlxtG6JQT4DAfQghxJpKFYah4ClE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_06:58:10', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T07:45:20.584 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T07:45:20.588 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T07:45:20.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T07:45:20.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T07:45:20.725 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:20.725 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:45:20.760 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:45:20.761 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-10T07:45:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:45:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:20.836 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-10 07:45:09.740687275 +0000 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-10 07:43:02.446581139 +0000 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-10 07:43:02.446581139 +0000 2024-09-10T07:45:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-10 07:43:02.446581139 +0000 2024-09-10T07:45:20.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:45:20.933 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-10T07:45:20.933 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-10T07:45:20.933 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000397684 s, 1.3 MB/s 2024-09-10T07:45:20.936 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:45:20.974 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-10 07:45:09.741687262 +0000 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-10 07:43:02.910575147 +0000 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-10 07:43:02.910575147 +0000 2024-09-10T07:45:21.053 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-10 07:43:02.910575147 +0000 2024-09-10T07:45:21.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:45:21.150 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-10T07:45:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-10T07:45:21.151 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000327058 s, 1.6 MB/s 2024-09-10T07:45:21.153 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:45:21.190 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-10T07:45:21.264 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:45:21.264 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-10 07:45:09.741687262 +0000 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-10 07:43:03.392568923 +0000 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-10 07:43:03.392568923 +0000 2024-09-10T07:45:21.265 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-10 07:43:03.392568923 +0000 2024-09-10T07:45:21.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:45:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-10T07:45:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-10T07:45:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000250517 s, 2.0 MB/s 2024-09-10T07:45:21.360 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:45:21.396 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-10T07:45:21.471 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-10 07:45:09.742687249 +0000 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-10 07:43:03.875562686 +0000 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-10 07:43:03.875562686 +0000 2024-09-10T07:45:21.472 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-10 07:43:03.875562686 +0000 2024-09-10T07:45:21.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:45:21.564 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-10T07:45:21.565 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-10T07:45:21.565 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000299498 s, 1.7 MB/s 2024-09-10T07:45:21.567 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:45:21.603 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:21.603 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:21.603 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:45:21.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:45:21.643 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 637 Links: 1 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 07:45:06.299762315 +0000 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 07:42:59.074410826 +0000 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 07:42:59.074410826 +0000 2024-09-10T07:45:21.722 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 07:42:59.074410826 +0000 2024-09-10T07:45:21.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:45:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T07:45:21.818 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T07:45:21.818 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00016629 s, 3.1 MB/s 2024-09-10T07:45:21.820 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:45:21.859 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-10T07:45:21.934 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 07:45:06.300762285 +0000 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 07:42:59.541396574 +0000 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 07:42:59.541396574 +0000 2024-09-10T07:45:21.935 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 07:42:59.541396574 +0000 2024-09-10T07:45:21.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:45:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T07:45:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T07:45:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000178487 s, 2.9 MB/s 2024-09-10T07:45:22.030 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:45:22.066 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-10T07:45:22.141 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 675 Links: 1 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 07:45:06.300762285 +0000 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 07:42:59.984383055 +0000 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 07:42:59.984383055 +0000 2024-09-10T07:45:22.142 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 07:42:59.984383055 +0000 2024-09-10T07:45:22.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:45:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T07:45:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T07:45:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000171561 s, 3.0 MB/s 2024-09-10T07:45:22.237 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:45:22.273 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-10T07:45:22.348 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:45:22.348 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:45:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 07:45:06.301762255 +0000 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 07:43:00.417369840 +0000 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 07:43:00.417369840 +0000 2024-09-10T07:45:22.349 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 07:43:00.417369840 +0000 2024-09-10T07:45:22.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:45:22.440 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T07:45:22.441 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T07:45:22.441 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000185594 s, 2.8 MB/s 2024-09-10T07:45:22.443 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:45:22.480 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:22.480 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T07:45:22.480 INFO:tasks.ceph:Generating config... 2024-09-10T07:45:22.482 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-10T07:45:22.482 INFO:tasks.ceph:[client] debug client = 20 2024-09-10T07:45:22.482 INFO:tasks.ceph:[client] debug ms = 1 2024-09-10T07:45:22.482 INFO:tasks.ceph:[client] pid file = 2024-09-10T07:45:22.482 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-10T07:45:22.483 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-10T07:45:22.483 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-10T07:45:22.483 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T07:45:22.484 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T07:45:22.485 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T07:45:22.485 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T07:45:22.485 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-10T07:45:22.485 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-10T07:45:22.485 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T07:45:22.485 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T07:45:22.485 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T07:45:22.485 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T07:45:22.486 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T07:45:22.487 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T07:45:22.487 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-10T07:45:22.487 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T07:45:22.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T07:45:22.580 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T07:45:22.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T07:45:22.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T07:45:22.726 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.70'), ('mon.c', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]')] 2024-09-10T07:45:22.727 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-10T07:45:22.728 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:22.728 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T07:45:22.763 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.70 --addv c '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:22.881 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:22.881 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 46339922-75e0-4945-8bb6-1acaef196937 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 46339922-75e0-4945-8bb6-1acaef196937 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-09-10T07:45:22.883117+0000 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-10T07:45:22.883117+0000 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 17 (quincy) 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.b 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-09-10T07:45:22.882 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T07:45:22.884 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T07:45:22.919 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 46339922-75e0-4945-8bb6-1acaef196937... 2024-09-10T07:45:22.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T07:45:22.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T07:45:23.057 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-10T07:45:23.057 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.058 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-10T07:45:23.059 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-10T07:45:23.060 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 46339922-75e0-4945-8bb6-1acaef196937 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.061 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.062 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-10T07:45:23.063 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-10T07:45:23.064 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-10T07:45:23.065 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.066 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-10T07:45:23.067 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-10T07:45:23.068 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-10T07:45:23.069 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-09-10T07:45:23.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-09-10T07:45:23.071 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-09-10T07:45:23.072 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.3,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on skipped message = False 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 46339922-75e0-4945-8bb6-1acaef196937 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-09-10T07:45:23.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug trim objects = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-09-10T07:45:23.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bdev async discard = True 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bdev enable discard = True 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore allocator = bitmap 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 1/20 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 1/20 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-10T07:45:23.075 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 4/10 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-10T07:45:23.076 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-09-10T07:45:23.077 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.078 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: mon down mkfs grace = 300 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on pool no app = False 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = "" 2024-09-10T07:45:23.079 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2024-09-10T07:45:23.080 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2024-09-10T07:45:23.081 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2024-09-10T07:45:23.081 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-09-10T07:45:23.081 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-09-10T07:45:23.081 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-09-10T07:45:23.086 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T07:45:23.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T07:45:23.185 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T07:45:23.185 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:23.185 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T07:45:23.220 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:23.221 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T07:45:23.294 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:45:23.295 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:23.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T07:45:23.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T07:45:23.412 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:45:23.412 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:23.487 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:45:23.487 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:23.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T07:45:23.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T07:45:23.566 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:45:23.566 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:23.640 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T07:45:23.640 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T07:45:23.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T07:45:23.744 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T07:45:23.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T07:45:23.861 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T07:45:23.868 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T07:45:23.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T07:45:23.973 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T07:45:23.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T07:45:24.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T07:45:24.138 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T07:45:24.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T07:45:24.198 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T07:45:24.304 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T07:45:24.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T07:45:24.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T07:45:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T07:45:24.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T07:45:24.528 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T07:45:24.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T07:45:24.612 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T07:45:24.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T07:45:24.721 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T07:45:24.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T07:45:24.829 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T07:45:24.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T07:45:24.938 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T07:45:24.965 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T07:45:24.965 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T07:45:24.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T07:45:25.021 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:25.021 INFO:tasks.ceph:role: osd.0 2024-09-10T07:45:25.021 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:45:25.022 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T07:45:25.084 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:25.084 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:25.084 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:25.084 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:25.084 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:25.085 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:25.085 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:25.085 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:25.085 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:25.085 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:25.483 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-10T07:45:25.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-10T07:45:25.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T07:45:25.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T07:45:25.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T07:45:25.736 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:25.736 INFO:tasks.ceph:role: osd.1 2024-09-10T07:45:25.736 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:45:25.736 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:25.799 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:25.800 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:25.800 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:26.157 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-10T07:45:26.161 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-10T07:45:26.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T07:45:26.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T07:45:26.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T07:45:26.364 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:26.364 INFO:tasks.ceph:role: osd.2 2024-09-10T07:45:26.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:45:26.365 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:26.428 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:26.429 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:26.429 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:26.429 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:26.429 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:26.816 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-10T07:45:26.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-10T07:45:26.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T07:45:26.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T07:45:26.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T07:45:27.034 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:27.034 INFO:tasks.ceph:role: osd.3 2024-09-10T07:45:27.034 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:45:27.034 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:27.098 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:27.518 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-10T07:45:27.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-10T07:45:27.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T07:45:27.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T07:45:27.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:27.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:27.831+0000 7f76d7464640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T07:45:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:27.833+0000 7f76d7464640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T07:45:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:27.833+0000 7f76d7464640 -1 bdev(0x55b68a0d1800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:27.833+0000 7f76d7464640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T07:45:31.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T07:45:31.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:32.026+0000 7ffb87c0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T07:45:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:32.026+0000 7ffb87c0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T07:45:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:32.026+0000 7ffb87c0e640 -1 bdev(0x559025299800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:32.027+0000 7ffb87c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T07:45:36.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T07:45:36.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:36.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:36.198+0000 7f2bd6e14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T07:45:36.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:36.198+0000 7f2bd6e14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T07:45:36.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:36.198+0000 7f2bd6e14640 -1 bdev(0x559b0bf4d800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:36.199+0000 7f2bd6e14640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T07:45:40.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T07:45:40.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:40.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:40.365+0000 7fa59b62c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T07:45:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:40.366+0000 7fa59b62c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T07:45:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:40.366+0000 7fa59b62c640 -1 bdev(0x55b0b3abd800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:40.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:45:40.366+0000 7fa59b62c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T07:45:44.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T07:45:44.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T07:45:44.498 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:44.498 INFO:tasks.ceph:role: osd.4 2024-09-10T07:45:44.498 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:45:44.498 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T07:45:44.555 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:44.556 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:44.589 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-10T07:45:44.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-10T07:45:44.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T07:45:44.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T07:45:44.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T07:45:44.792 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:44.792 INFO:tasks.ceph:role: osd.5 2024-09-10T07:45:44.792 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:45:44.792 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T07:45:44.847 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:44.848 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:44.881 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-10T07:45:44.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-10T07:45:44.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T07:45:44.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T07:45:45.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T07:45:45.090 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:45.090 INFO:tasks.ceph:role: osd.6 2024-09-10T07:45:45.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:45:45.090 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T07:45:45.146 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:45.146 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:45.146 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:45.146 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:45.146 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:45.147 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:45.147 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:45.147 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:45.147 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:45.147 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:45.179 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-10T07:45:45.184 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-10T07:45:45.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T07:45:45.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T07:45:45.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T07:45:45.384 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T07:45:45.384 INFO:tasks.ceph:role: osd.7 2024-09-10T07:45:45.384 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:45:45.384 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T07:45:45.442 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T07:45:45.442 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T07:45:45.443 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T07:45:45.476 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-10T07:45:45.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-10T07:45:45.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T07:45:45.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T07:45:45.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:45.741+0000 7fd2be418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T07:45:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:45.742+0000 7fd2be418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T07:45:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:45.742+0000 7fd2be418640 -1 bdev(0x564a90369800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:45.742+0000 7fd2be418640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T07:45:49.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T07:45:49.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:49.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:49.911+0000 7f34dd554640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T07:45:49.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:49.911+0000 7f34dd554640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T07:45:49.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:49.911+0000 7f34dd554640 -1 bdev(0x560aff3af800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:49.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:49.912+0000 7f34dd554640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T07:45:50.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:50.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:45:53.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T07:45:53.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:54.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:54.068+0000 7f5cf3b51640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T07:45:54.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:54.069+0000 7f5cf3b51640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T07:45:54.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:54.069+0000 7f5cf3b51640 -1 bdev(0x556f4e463800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:54.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:54.069+0000 7f5cf3b51640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T07:45:57.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T07:45:57.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:45:57.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:57.997+0000 7f711420d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T07:45:57.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:57.997+0000 7f711420d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T07:45:57.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:57.997+0000 7f711420d640 -1 bdev(0x56156ab73800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T07:45:57.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:45:57.998+0000 7f711420d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T07:46:01.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T07:46:02.053 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T07:46:02.053 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T07:46:02.117 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T07:46:02.216 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T07:46:02.314 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T07:46:02.411 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T07:46:02.507 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T07:46:02.604 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:02.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T07:46:02.701 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:02.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T07:46:02.762 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:02.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T07:46:02.860 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:02.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T07:46:02.956 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:02.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T07:46:03.051 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:03.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T07:46:03.146 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:03.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T07:46:03.243 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:03.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T07:46:03.337 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:03.338 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T07:46:03.374 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:03.374 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T07:46:03.451 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:03.451 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T07:46:03.485 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:03.486 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T07:46:03.560 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T07:46:03.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T07:46:03.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T07:46:03.613 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ== 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg== 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg== 2024-09-10T07:46:03.614 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA== 2024-09-10T07:46:03.615 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw== 2024-09-10T07:46:03.616 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2024-09-10T07:46:03.617 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA== 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ== 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg== 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg== 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-09-10T07:46:03.653 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg== 2024-09-10T07:46:03.654 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[client.2] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw== 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:[client.3] 2024-09-10T07:46:03.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA== 2024-09-10T07:46:03.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T07:46:03.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T07:46:03.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:03.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:03.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:03.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:03.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:03.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.191 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T07:46:04.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T07:46:04.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:04.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:04.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:04.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T07:46:04.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:04.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:05.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:05.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T07:46:05.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.310 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T07:46:05.511 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T07:46:05.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T07:46:05.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T07:46:05.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T07:46:05.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T07:46:05.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T07:46:05.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T07:46:05.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T07:46:06.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T07:46:06.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T07:46:06.168 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:46:06.172 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T07:46:06.207 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T07:46:06.207 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T07:46:06.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T07:46:06.249 INFO:tasks.ceph.mon.a:Started 2024-09-10T07:46:06.249 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T07:46:06.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T07:46:06.253 INFO:tasks.ceph.mon.b:Started 2024-09-10T07:46:06.253 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T07:46:06.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T07:46:06.257 INFO:tasks.ceph.mon.c:Started 2024-09-10T07:46:06.257 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T07:46:06.258 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T07:46:06.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T07:46:06.261 INFO:tasks.ceph.mgr.y:Started 2024-09-10T07:46:06.261 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T07:46:06.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T07:46:06.265 INFO:tasks.ceph.mgr.x:Started 2024-09-10T07:46:06.265 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T07:46:06.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T07:46:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.422+0000 7fd58d600640 1 Processor -- start 2024-09-10T07:46:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.422+0000 7fd58d600640 1 -- start start 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.423+0000 7fd58d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.423+0000 7fd58d600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 0x7fd5880d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.423+0000 7fd58d600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5881620f0 con 0x7fd5880cef90 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.423+0000 7fd58d600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd588072140 con 0x7fd5880b76c0 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.423+0000 7fd58d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5880d4200 con 0x7fd588161d10 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 0x7fd5880d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:52532/0 (socket says 172.21.15.3:52532) 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 -- 172.21.15.3:0/587442258 learned_addr learned my addr 172.21.15.3:0/587442258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 msgr2=0x7fd5880d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 msgr2=0x7fd5880d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586000640 1 -- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 msgr2=0x7fd5880d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586000640 1 -- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 msgr2=0x7fd5880d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586a00640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586000640 1 --2- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 0x7fd5880d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:46:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.424+0000 7fd586000640 1 --2- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 0x7fd5880d3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:46:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.438+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334486021 0 0) 0x7fd5881620f0 con 0x7fd5880cef90 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.438+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd56c003520 con 0x7fd5880cef90 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2754821079 0 0) 0x7fd56c003520 con 0x7fd5880cef90 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 msgr2=0x7fd5880d3b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd5880d3b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 msgr2=0x7fd5880d3280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 --2- 172.21.15.3:0/587442258 >> v2:172.21.15.70:3300/0 conn(0x7fd5880b76c0 0x7fd5880d3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd588072390 con 0x7fd5880cef90 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fd5800019f0 con 0x7fd5880cef90 2024-09-10T07:46:06.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 >> v1:172.21.15.3:6789/0 conn(0x7fd5880cef90 legacy=0x7fd588072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd56c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c004030 0x7fd56c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd56c004d40 0x7fd56c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd588072140 con 0x7fd56c004d40 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880d4200 con 0x7fd56c004030 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd56c00bbd0 con 0x7fd588161d10 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd587400640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd56c004d40 0x7fd56c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd586a00640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd56c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.439+0000 7fd586000640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c004030 0x7fd56c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.440+0000 7fd587400640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 msgr2=0x7fd56c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.440+0000 7fd587400640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd588161d10 0x7fd56c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.440+0000 7fd587400640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c004030 msgr2=0x7fd56c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.440+0000 7fd587400640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c004030 0x7fd56c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.440+0000 7fd587400640 1 -- 172.21.15.3:0/587442258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c003520 con 0x7fd56c004d40 2024-09-10T07:46:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.441+0000 7fd587400640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd56c004d40 0x7fd56c0051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd58000c4c0 tx=0x7fd56c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d275cfab208bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.441+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580033070 con 0x7fd56c004d40 2024-09-10T07:46:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.441+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58002f660 con 0x7fd56c004d40 2024-09-10T07:46:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.441+0000 7fd585600640 1 -- 172.21.15.3:0/587442258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58000c0e0 con 0x7fd56c004d40 2024-09-10T07:46:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 -- 172.21.15.3:0/587442258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd56c004d40 msgr2=0x7fd56c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 --2- 172.21.15.3:0/587442258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd56c004d40 0x7fd56c0051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd58000c4c0 tx=0x7fd56c005830 comp rx=0 tx=0).stop 2024-09-10T07:46:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd587400640 1 -- 172.21.15.3:0/587442258 reap_dead start 2024-09-10T07:46:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 -- 172.21.15.3:0/587442258 shutdown_connections 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 -- 172.21.15.3:0/587442258 >> 172.21.15.3:0/587442258 conn(0x7fd5880d4730 msgr2=0x7fd5880d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 -- 172.21.15.3:0/587442258 shutdown_connections 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.442+0000 7fd58d600640 1 -- 172.21.15.3:0/587442258 wait complete. 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.443+0000 7fd58d600640 1 Processor -- start 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.444+0000 7fd58d600640 1 -- start start 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.444+0000 7fd58d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 0x7fd58815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.444+0000 7fd58d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.444+0000 7fd58d600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5880c9170 0x7fd5880cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd58d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880b7b00 con 0x7fd588154d50 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd58d600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd588072950 con 0x7fd5880c9170 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd58d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5880d3b20 con 0x7fd56c004d40 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44752/0 (socket says 172.21.15.3:44752) 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 -- 172.21.15.3:0/1823302151 learned_addr learned my addr 172.21.15.3:0/1823302151 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586a00640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 0x7fd58815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 -- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 msgr2=0x7fd58815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 0x7fd58815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 -- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5880c9170 msgr2=0x7fd5880cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5880c9170 0x7fd5880cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 -- 172.21.15.3:0/1823302151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880c9610 con 0x7fd588154d50 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586a00640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 0x7fd58815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.445+0000 7fd586000640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5740108d0 tx=0x7fd574010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57401a020 con 0x7fd588154d50 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57400e940 con 0x7fd588154d50 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57400ed10 con 0x7fd588154d50 2024-09-10T07:46:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5880cba50 con 0x7fd588154d50 2024-09-10T07:46:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881551d0 con 0x7fd588154d50 2024-09-10T07:46:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd57401d020 con 0x7fd588154d50 2024-09-10T07:46:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.446+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5740148c0 con 0x7fd588154d50 2024-09-10T07:46:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.447+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5880d3f00 con 0x7fd588154d50 2024-09-10T07:46:06.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.450+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd574024020 con 0x7fd588154d50 2024-09-10T07:46:06.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.517+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd5880c8000 con 0x7fd588154d50 2024-09-10T07:46:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.519+0000 7fd57ea00640 1 -- 172.21.15.3:0/1823302151 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd574014ac0 con 0x7fd588154d50 2024-09-10T07:46:06.519 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 msgr2=0x7fd5880c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5740108d0 tx=0x7fd574010d90 comp rx=0 tx=0).stop 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 shutdown_connections 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5880c9170 0x7fd5880cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd588154d50 0x7fd5880c8c30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 --2- 172.21.15.3:0/1823302151 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd56c004d40 0x7fd58815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 >> 172.21.15.3:0/1823302151 conn(0x7fd5880d4730 msgr2=0x7fd5880b9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 shutdown_connections 2024-09-10T07:46:06.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:06.520+0000 7fd58d600640 1 -- 172.21.15.3:0/1823302151 wait complete. 2024-09-10T07:46:06.534 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T07:46:06.535 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T07:46:06.535 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:06.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T07:46:06.591 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:06.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T07:46:06.688 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:06.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T07:46:06.718 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:06.718+0000 7f45ebd8a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:46:06.723 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:06.724+0000 7fe841a99040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:46:06.765 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:06.765+0000 7f45ebd8a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:46:06.774 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:06.775+0000 7fe841a99040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:46:06.779 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:06.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T07:46:06.875 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:06.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T07:46:06.933 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:06.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T07:46:06.940 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:06.940+0000 7f45ebd8a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:46:06.954 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:06.955+0000 7fe841a99040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:46:06.996 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:06.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T07:46:07.003 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.003+0000 7f45ebd8a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:46:07.016 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.016+0000 7fe841a99040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:46:07.058 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:07.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T07:46:07.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 37828b5f-b518-4707-8c43-fd1331a1b122 0 2024-09-10T07:46:07.200 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.199+0000 7f45ebd8a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:46:07.202 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.202+0000 7fe841a99040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:46:07.332 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.332+0000 7f45ebd8a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:46:07.333 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.334+0000 7fe841a99040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:46:07.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.346+0000 7f11d2c00640 1 Processor -- start 2024-09-10T07:46:07.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.346+0000 7f11d2c00640 1 -- start start 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d2c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11cc15f2c0 0x7f11cc169320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d2c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc15f7e0 con 0x7f11cc06c230 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d2c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc15fe10 con 0x7f11cc148290 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d2c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0c9850 con 0x7f11cc15f2c0 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d1800640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11cc148290 0x7f11cc06bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34874/0 (socket says 172.21.15.70:34874) 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d1800640 1 -- 172.21.15.70:0/3847226336 learned_addr learned my addr 172.21.15.70:0/3847226336 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-10T07:46:07.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11d2200640 1 --2- 172.21.15.70:0/3847226336 >> v2:172.21.15.3:3300/0 conn(0x7f11cc15f2c0 0x7f11cc169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127827591 0 0) 0x7f11cc15fe10 con 0x7f11cc148290 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.348+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11b4003520 con 0x7f11cc148290 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.349+0000 7f11d2200640 1 -- 172.21.15.70:0/3847226336 >> v1:172.21.15.70:6789/0 conn(0x7f11cc148290 legacy=0x7f11cc06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.349+0000 7f11d2200640 1 -- 172.21.15.70:0/3847226336 >> v1:172.21.15.3:6789/0 conn(0x7f11cc06c230 legacy=0x7f11cc15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.349+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429177315 0 0) 0x7f11cc15f7e0 con 0x7f11cc06c230 2024-09-10T07:46:07.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.349+0000 7f11d2200640 1 -- 172.21.15.70:0/3847226336 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc1600f0 con 0x7f11cc15f2c0 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2200640 1 --2- 172.21.15.70:0/3847226336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc15f2c0 0x7f11cc169320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d90 tx=0x7f11c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e4e81bdef53e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007740 con 0x7f11cc15f2c0 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c40078e0 con 0x7f11cc15f2c0 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11cbe00640 1 -- 172.21.15.70:0/3847226336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007c90 con 0x7f11cc15f2c0 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2c00640 1 -- 172.21.15.70:0/3847226336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc15f2c0 msgr2=0x7f11cc169320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2c00640 1 --2- 172.21.15.70:0/3847226336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc15f2c0 0x7f11cc169320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d90 tx=0x7f11c402a660 comp rx=0 tx=0).stop 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2c00640 1 -- 172.21.15.70:0/3847226336 shutdown_connections 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2c00640 1 --2- 172.21.15.70:0/3847226336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc15f2c0 0x7f11cc169320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.350+0000 7f11d2c00640 1 -- 172.21.15.70:0/3847226336 >> 172.21.15.70:0/3847226336 conn(0x7f11cc0d4580 msgr2=0x7f11cc0d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.351+0000 7f11d2c00640 1 -- 172.21.15.70:0/3847226336 shutdown_connections 2024-09-10T07:46:07.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.351+0000 7f11d2c00640 1 -- 172.21.15.70:0/3847226336 wait complete. 2024-09-10T07:46:07.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.352+0000 7f11d2c00640 1 Processor -- start 2024-09-10T07:46:07.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 -- start start 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc06c230 0x7f11cc144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 0x7f11cc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 0x7f11cc0b9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc161900 con 0x7f11cc06c230 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc15fac0 con 0x7f11cc148290 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.353+0000 7f11d2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc161a80 con 0x7f11cc15f2c0 2024-09-10T07:46:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d2200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 0x7f11cc0b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 0x7f11cc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d2200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 0x7f11cc0b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60924/0 (socket says 172.21.15.70:60924) 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc06c230 0x7f11cc144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d2200640 1 -- 172.21.15.70:0/1844341898 learned_addr learned my addr 172.21.15.70:0/1844341898 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 -- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 msgr2=0x7f11cc0b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 0x7f11cc0b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 -- 172.21.15.70:0/1844341898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc06c230 msgr2=0x7f11cc144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc06c230 0x7f11cc144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 -- 172.21.15.70:0/1844341898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0b3b60 con 0x7f11cc148290 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.354+0000 7f11d0e00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 0x7f11cc159bd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c450 tx=0x7f11b800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8017070 con 0x7f11cc148290 2024-09-10T07:46:07.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b80035f0 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80039e0 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc0b3e40 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc0b4400 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11b801a070 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.355+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f11b8012740 con 0x7f11cc148290 2024-09-10T07:46:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.356+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc15ebb0 con 0x7f11cc148290 2024-09-10T07:46:07.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.361+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f11b8021020 con 0x7f11cc148290 2024-09-10T07:46:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.433+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37828b5f-b518-4707-8c43-fd1331a1b122", "id": 0} v 0) -- 0x7f11cc0b4c00 con 0x7f11cc148290 2024-09-10T07:46:07.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.437+0000 7f11c9600640 1 -- 172.21.15.70:0/1844341898 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37828b5f-b518-4707-8c43-fd1331a1b122", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f11b8003d50 con 0x7f11cc148290 2024-09-10T07:46:07.437 INFO:teuthology.orchestra.run.smithi070.stdout:0 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 msgr2=0x7f11cc159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 0x7f11cc159bd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c450 tx=0x7f11b800c910 comp rx=0 tx=0).stop 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 shutdown_connections 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15f2c0 0x7f11cc0b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc148290 0x7f11cc159bd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 --2- 172.21.15.70:0/1844341898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc06c230 0x7f11cc144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.438+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 >> 172.21.15.70:0/1844341898 conn(0x7f11cc0d4580 msgr2=0x7f11cc0d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.439+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 shutdown_connections 2024-09-10T07:46:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.439+0000 7f11d2c00640 1 -- 172.21.15.70:0/1844341898 wait complete. 2024-09-10T07:46:07.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new c36cec6f-bb73-44cd-8a1e-1da4e53fe991 1 2024-09-10T07:46:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.602+0000 7fd3ed000640 1 Processor -- start 2024-09-10T07:46:07.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.602+0000 7fd3ed000640 1 -- start start 2024-09-10T07:46:07.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3ed000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3e806c230 0x7fd3e815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3ed000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e815f520 con 0x7fd3e8148290 2024-09-10T07:46:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3ed000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e815f7e0 con 0x7fd3e815f0f0 2024-09-10T07:46:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3ed000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e80c9850 con 0x7fd3e806c230 2024-09-10T07:46:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3e7e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd3e815f0f0 0x7fd3e81608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34882/0 (socket says 172.21.15.70:34882) 2024-09-10T07:46:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3e7e00640 1 -- 172.21.15.70:0/1830380970 learned_addr learned my addr 172.21.15.70:0/1830380970 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3e6a00640 1 --2- 172.21.15.70:0/1830380970 >> v2:172.21.15.3:3300/0 conn(0x7fd3e806c230 0x7fd3e815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3e6000640 1 -- 172.21.15.70:0/1830380970 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376631878 0 0) 0x7fd3e815f7e0 con 0x7fd3e815f0f0 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.604+0000 7fd3e6000640 1 -- 172.21.15.70:0/1830380970 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3cc003520 con 0x7fd3e815f0f0 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.605+0000 7fd3e6a00640 1 -- 172.21.15.70:0/1830380970 >> v1:172.21.15.70:6789/0 conn(0x7fd3e815f0f0 legacy=0x7fd3e81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.605+0000 7fd3e6a00640 1 -- 172.21.15.70:0/1830380970 >> v1:172.21.15.3:6789/0 conn(0x7fd3e8148290 legacy=0x7fd3e806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.605+0000 7fd3e6a00640 1 -- 172.21.15.70:0/1830380970 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e815fe10 con 0x7fd3e806c230 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.606+0000 7fd3e6a00640 1 --2- 172.21.15.70:0/1830380970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e815ebb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca496d128c0d5ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.606+0000 7fd3e6000640 1 -- 172.21.15.70:0/1830380970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4037070 con 0x7fd3e806c230 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.606+0000 7fd3e6000640 1 -- 172.21.15.70:0/1830380970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d4032440 con 0x7fd3e806c230 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.606+0000 7fd3e6000640 1 -- 172.21.15.70:0/1830380970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40327f0 con 0x7fd3e806c230 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 -- 172.21.15.70:0/1830380970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 msgr2=0x7fd3e815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 --2- 172.21.15.70:0/1830380970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e815ebb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 -- 172.21.15.70:0/1830380970 shutdown_connections 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 --2- 172.21.15.70:0/1830380970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e815ebb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 -- 172.21.15.70:0/1830380970 >> 172.21.15.70:0/1830380970 conn(0x7fd3e80d4580 msgr2=0x7fd3e80d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 -- 172.21.15.70:0/1830380970 shutdown_connections 2024-09-10T07:46:07.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.607+0000 7fd3ed000640 1 -- 172.21.15.70:0/1830380970 wait complete. 2024-09-10T07:46:07.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.608+0000 7fd3ed000640 1 Processor -- start 2024-09-10T07:46:07.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.608+0000 7fd3ed000640 1 -- start start 2024-09-10T07:46:07.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e8157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e815f0f0 0x7fd3e815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e815fac0 con 0x7fd3e806c230 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81603d0 con 0x7fd3e815f0f0 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3ed000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e8161000 con 0x7fd3e8148290 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3e7e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e815f0f0 0x7fd3e815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.609+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e8157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60934/0 (socket says 172.21.15.70:60934) 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 -- 172.21.15.70:0/2668092991 learned_addr learned my addr 172.21.15.70:0/2668092991 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 -- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e815f0f0 msgr2=0x7fd3e815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e815f0f0 0x7fd3e815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 -- 172.21.15.70:0/2668092991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 msgr2=0x7fd3e8157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e8157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.610+0000 7fd3e6a00640 1 -- 172.21.15.70:0/2668092991 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e80b9a40 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3e6a00640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4037050 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d4038040 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e80b9d20 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004470 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81371a0 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3d403c6d0 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.611+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd3d4003d30 con 0x7fd3e8148290 2024-09-10T07:46:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.612+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e815ebb0 con 0x7fd3e8148290 2024-09-10T07:46:07.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.615+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd3d403cba0 con 0x7fd3e8148290 2024-09-10T07:46:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.686+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c36cec6f-bb73-44cd-8a1e-1da4e53fe991", "id": 1} v 0) -- 0x7fd3e80d3490 con 0x7fd3e8148290 2024-09-10T07:46:07.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.693+0000 7fd3df400640 1 -- 172.21.15.70:0/2668092991 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c36cec6f-bb73-44cd-8a1e-1da4e53fe991", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd3d403cd80 con 0x7fd3e8148290 2024-09-10T07:46:07.693 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 msgr2=0x7fd3e80b8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40097a0 tx=0x7fd3d40046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 shutdown_connections 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e815f0f0 0x7fd3e815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3e8148290 0x7fd3e80b8ff0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 --2- 172.21.15.70:0/2668092991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e806c230 0x7fd3e8157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 >> 172.21.15.70:0/2668092991 conn(0x7fd3e80d4580 msgr2=0x7fd3e806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 shutdown_connections 2024-09-10T07:46:07.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.694+0000 7fd3ed000640 1 -- 172.21.15.70:0/2668092991 wait complete. 2024-09-10T07:46:07.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 072ff2e0-964b-4d8a-b818-99256d7ed638 2 2024-09-10T07:46:07.713 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.713+0000 7fe841a99040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:46:07.727 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.728+0000 7f45ebd8a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:46:07.779 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.779+0000 7fe841a99040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:46:07.800 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.800+0000 7f45ebd8a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:46:07.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.859+0000 7f11cbe00640 1 Processor -- start 2024-09-10T07:46:07.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.859+0000 7f11cbe00640 1 -- start start 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11cc0d10b0 0x7f11cc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11cc0d14b0 0x7f11cc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0d1c70 con 0x7f11cc0d10b0 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0d1990 con 0x7f11cc0d14b0 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.860+0000 7f11cbe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc0cffc0 con 0x7f11cc15cde0 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60940/0 (socket says 172.21.15.70:60940) 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 -- 172.21.15.70:0/2158720553 learned_addr learned my addr 172.21.15.70:0/2158720553 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11cb400640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.70:3300/0 conn(0x7f11cc0d14b0 0x7f11cc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11c3e00640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.3:3300/0 conn(0x7f11cc0d10b0 0x7f11cc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 -- 172.21.15.70:0/2158720553 >> v2:172.21.15.70:3300/0 conn(0x7f11cc0d14b0 msgr2=0x7f11cc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.70:3300/0 conn(0x7f11cc0d14b0 0x7f11cc0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 -- 172.21.15.70:0/2158720553 >> v2:172.21.15.3:3300/0 conn(0x7f11cc0d10b0 msgr2=0x7f11cc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.3:3300/0 conn(0x7f11cc0d10b0 0x7f11cc0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.861+0000 7f11caa00640 1 -- 172.21.15.70:0/2158720553 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0d02a0 con 0x7f11cc15cde0 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.862+0000 7f11caa00640 1 --2- 172.21.15.70:0/2158720553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11cc072a50 tx=0x7f11bc02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f0b8865e048b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.862+0000 7f11ca000640 1 -- 172.21.15.70:0/2158720553 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11cc15cde0 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.862+0000 7f11ca000640 1 -- 172.21.15.70:0/2158720553 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc032440 con 0x7f11cc15cde0 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.862+0000 7f11ca000640 1 -- 172.21.15.70:0/2158720553 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc032810 con 0x7f11cc15cde0 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 -- 172.21.15.70:0/2158720553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 msgr2=0x7f11cc0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 --2- 172.21.15.70:0/2158720553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11cc072a50 tx=0x7f11bc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 -- 172.21.15.70:0/2158720553 shutdown_connections 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.70:3300/0 conn(0x7f11cc0d14b0 0x7f11cc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 --2- 172.21.15.70:0/2158720553 >> v2:172.21.15.3:3300/0 conn(0x7f11cc0d10b0 0x7f11cc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 --2- 172.21.15.70:0/2158720553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc15cde0 0x7f11cc0d0a80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 -- 172.21.15.70:0/2158720553 >> 172.21.15.70:0/2158720553 conn(0x7f11cc06f210 msgr2=0x7f11cc06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 -- 172.21.15.70:0/2158720553 shutdown_connections 2024-09-10T07:46:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.863+0000 7f11cbe00640 1 -- 172.21.15.70:0/2158720553 wait complete. 2024-09-10T07:46:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.865+0000 7f11cbe00640 1 Processor -- start 2024-09-10T07:46:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.865+0000 7f11cbe00640 1 -- start start 2024-09-10T07:46:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc0d14b0 0x7f11cc14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 0x7f11cc150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc160040 con 0x7f11cc0d14b0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc15fec0 con 0x7f11cc15cde0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cbe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc1601c0 con 0x7f11cc0d10b0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cb400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 0x7f11cc150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60950/0 (socket says 172.21.15.70:60950) 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 -- 172.21.15.70:0/3579740906 learned_addr learned my addr 172.21.15.70:0/3579740906 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11c3e00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc0d14b0 0x7f11cc14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 -- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 msgr2=0x7f11cc150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 0x7f11cc150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 -- 172.21.15.70:0/3579740906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc0d14b0 msgr2=0x7f11cc14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc0d14b0 0x7f11cc14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11caa00640 1 -- 172.21.15.70:0/3579740906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0b63a0 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.866+0000 7f11cb400640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 0x7f11cc150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11caa00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a220 tx=0x7f11bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc041020 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc03d620 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037040 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc14f790 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc14fd50 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.867+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11bc044020 con 0x7f11cc0d10b0 2024-09-10T07:46:07.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.868+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f11bc03dda0 con 0x7f11cc0d10b0 2024-09-10T07:46:07.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.868+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc0d0a80 con 0x7f11cc0d10b0 2024-09-10T07:46:07.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.873+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f11bc04b030 con 0x7f11cc0d10b0 2024-09-10T07:46:07.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.945+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "072ff2e0-964b-4d8a-b818-99256d7ed638", "id": 2} v 0) -- 0x7f11cc15fbb0 con 0x7f11cc0d10b0 2024-09-10T07:46:07.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.951+0000 7f11c3400640 1 -- 172.21.15.70:0/3579740906 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "072ff2e0-964b-4d8a-b818-99256d7ed638", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f11cc0b63a0 con 0x7f11cc0d10b0 2024-09-10T07:46:07.951 INFO:teuthology.orchestra.run.smithi070.stdout:2 2024-09-10T07:46:07.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 msgr2=0x7f11cc0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:07.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a220 tx=0x7f11bc0046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 shutdown_connections 2024-09-10T07:46:07.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11cc15cde0 0x7f11cc150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11cc0d14b0 0x7f11cc14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 --2- 172.21.15.70:0/3579740906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11cc0d10b0 0x7f11cc0b5ca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 >> 172.21.15.70:0/3579740906 conn(0x7f11cc06f210 msgr2=0x7f11cc06bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 shutdown_connections 2024-09-10T07:46:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:07.952+0000 7f11cbe00640 1 -- 172.21.15.70:0/3579740906 wait complete. 2024-09-10T07:46:07.955 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:07.955+0000 7fe841a99040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:46:07.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 08b5f076-289a-4f6c-a220-42a739ef896c 3 2024-09-10T07:46:07.983 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:07.983+0000 7f45ebd8a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:46:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.132+0000 7f94c2a00640 1 Processor -- start 2024-09-10T07:46:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.133+0000 7f94c2a00640 1 -- start start 2024-09-10T07:46:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.134+0000 7f94c2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94bc161d10 0x7f94bc0d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.134+0000 7f94c2a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 0x7f94bc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c2a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94bc1620f0 con 0x7f94bc0b76c0 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c2a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f94bc072140 con 0x7f94bc161d10 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c2a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f94bc0b7b30 con 0x7f94bc0cf6a0 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c0a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 0x7f94bc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94bc161d10 0x7f94bc0d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c0a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 0x7f94bc0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52386/0 (socket says 172.21.15.70:52386) 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94b3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94bc0b76c0 0x7f94bc0cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.70:58980/0 (socket says 172.21.15.70:58980) 2024-09-10T07:46:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94c0a00640 1 -- 172.21.15.70:0/1948472925 learned_addr learned my addr 172.21.15.70:0/1948472925 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bb400640 1 -- 172.21.15.70:0/1948472925 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121287879 0 0) 0x7f94bc1620f0 con 0x7f94bc0b76c0 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bbe00640 1 -- 172.21.15.70:0/1948472925 >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 msgr2=0x7f94bc0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bbe00640 1 --2- 172.21.15.70:0/1948472925 >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 0x7f94bc0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bbe00640 1 -- 172.21.15.70:0/1948472925 >> v1:172.21.15.3:6789/0 conn(0x7f94bc0b76c0 legacy=0x7f94bc0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.135+0000 7f94bbe00640 1 -- 172.21.15.70:0/1948472925 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc0d4200 con 0x7f94bc161d10 2024-09-10T07:46:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.136+0000 7f94bbe00640 1 --2- 172.21.15.70:0/1948472925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc161d10 0x7f94bc0d3b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94bc08d1f0 tx=0x7f94ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f91f972d2d85647 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94bb400640 1 -- 172.21.15.70:0/1948472925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac037070 con 0x7f94bc161d10 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94bb400640 1 -- 172.21.15.70:0/1948472925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ac032440 con 0x7f94bc161d10 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94bb400640 1 -- 172.21.15.70:0/1948472925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac02ae60 con 0x7f94bc161d10 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 -- 172.21.15.70:0/1948472925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc161d10 msgr2=0x7f94bc0d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1948472925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc161d10 0x7f94bc0d3b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94bc08d1f0 tx=0x7f94ac02a110 comp rx=0 tx=0).stop 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 -- 172.21.15.70:0/1948472925 shutdown_connections 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1948472925 >> v2:172.21.15.70:3300/0 conn(0x7f94bc0cf6a0 0x7f94bc0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1948472925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc161d10 0x7f94bc0d3b20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 -- 172.21.15.70:0/1948472925 >> 172.21.15.70:0/1948472925 conn(0x7f94bc0d4730 msgr2=0x7f94bc0d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 -- 172.21.15.70:0/1948472925 shutdown_connections 2024-09-10T07:46:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.137+0000 7f94c2a00640 1 -- 172.21.15.70:0/1948472925 wait complete. 2024-09-10T07:46:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.139+0000 7f94c2a00640 1 Processor -- start 2024-09-10T07:46:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.139+0000 7f94c2a00640 1 -- start start 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 0x7f94bc155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc072950 con 0x7f94bc0b76c0 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc072670 con 0x7f94bc0cf6a0 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c2a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f94bc072e70 con 0x7f94bc161d10 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c0a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c0a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60962/0 (socket says 172.21.15.70:60962) 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94c0a00640 1 -- 172.21.15.70:0/1657241412 learned_addr learned my addr 172.21.15.70:0/1657241412 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52398/0 (socket says 172.21.15.70:52398) 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.140+0000 7f94bbe00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 0x7f94bc155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 -- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 msgr2=0x7f94bc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 -- 172.21.15.70:0/1657241412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 msgr2=0x7f94bc155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 0x7f94bc155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 -- 172.21.15.70:0/1657241412 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc156430 con 0x7f94bc161d10 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94b3e00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94bbe00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 0x7f94bc155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c0a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009d90 tx=0x7f94b400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b400cd20 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b4003990 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4003d60 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc158640 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.141+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc158c00 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.142+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f94b4012050 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.142+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f94b40044c0 con 0x7f94bc161d10 2024-09-10T07:46:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.142+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94bc158e00 con 0x7f94bc161d10 2024-09-10T07:46:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.145+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f94bc156430 con 0x7f94bc161d10 2024-09-10T07:46:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.217+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "08b5f076-289a-4f6c-a220-42a739ef896c", "id": 3} v 0) -- 0x7f94bc0d4d20 con 0x7f94bc161d10 2024-09-10T07:46:08.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.223+0000 7f94b8c00640 1 -- 172.21.15.70:0/1657241412 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "08b5f076-289a-4f6c-a220-42a739ef896c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f94b4010040 con 0x7f94bc161d10 2024-09-10T07:46:08.223 INFO:teuthology.orchestra.run.smithi070.stdout:3 2024-09-10T07:46:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 msgr2=0x7f94bc0cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009d90 tx=0x7f94b400c490 comp rx=0 tx=0).stop 2024-09-10T07:46:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 shutdown_connections 2024-09-10T07:46:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94bc161d10 0x7f94bc0cdde0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94bc0cf6a0 0x7f94bc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 --2- 172.21.15.70:0/1657241412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94bc0b76c0 0x7f94bc155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 >> 172.21.15.70:0/1657241412 conn(0x7f94bc0d4730 msgr2=0x7f94bc0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 shutdown_connections 2024-09-10T07:46:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.224+0000 7f94c2a00640 1 -- 172.21.15.70:0/1657241412 wait complete. 2024-09-10T07:46:08.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 5c0cd914-4953-4bf1-88d0-e198e9de98c9 4 2024-09-10T07:46:08.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.387+0000 7f8e0ca00640 1 Processor -- start 2024-09-10T07:46:08.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.387+0000 7f8e0ca00640 1 -- start start 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8e0ca00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e0806bb20 0x7f8e080b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8e0ca00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0806d720 con 0x7f8e0815e520 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8e0ca00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0806da00 con 0x7f8e0815e900 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8e0ca00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8e08073b50 con 0x7f8e0806bb20 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8e06800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e0806bb20 0x7f8e080b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8dfd400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8e0815e900 0x7f8e08070c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34898/0 (socket says 172.21.15.70:34898) 2024-09-10T07:46:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.388+0000 7f8dfd400640 1 -- 172.21.15.70:0/313561851 learned_addr learned my addr 172.21.15.70:0/313561851 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-10T07:46:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.389+0000 7f8e06800640 1 -- 172.21.15.70:0/313561851 >> v1:172.21.15.70:6789/0 conn(0x7f8e0815e900 legacy=0x7f8e08070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.389+0000 7f8e06800640 1 -- 172.21.15.70:0/313561851 >> v1:172.21.15.3:6789/0 conn(0x7f8e0815e520 legacy=0x7f8e080b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.389+0000 7f8e06800640 1 -- 172.21.15.70:0/313561851 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e08073e30 con 0x7f8e0806bb20 2024-09-10T07:46:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.389+0000 7f8e05400640 1 -- 172.21.15.70:0/313561851 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985192534 0 0) 0x7f8e0806da00 con 0x7f8e0815e900 2024-09-10T07:46:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e06800640 1 --2- 172.21.15.70:0/313561851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0806bb20 0x7f8e080b4e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009d90 tx=0x7f8e0002a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62557888a9645e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e05400640 1 -- 172.21.15.70:0/313561851 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00007740 con 0x7f8e0806bb20 2024-09-10T07:46:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e05400640 1 -- 172.21.15.70:0/313561851 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e000078e0 con 0x7f8e0806bb20 2024-09-10T07:46:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e05400640 1 -- 172.21.15.70:0/313561851 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00007cb0 con 0x7f8e0806bb20 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e0ca00640 1 -- 172.21.15.70:0/313561851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0806bb20 msgr2=0x7f8e080b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.390+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/313561851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0806bb20 0x7f8e080b4e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009d90 tx=0x7f8e0002a560 comp rx=0 tx=0).stop 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.391+0000 7f8e0ca00640 1 -- 172.21.15.70:0/313561851 shutdown_connections 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.391+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/313561851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0806bb20 0x7f8e080b4e90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.391+0000 7f8e0ca00640 1 -- 172.21.15.70:0/313561851 >> 172.21.15.70:0/313561851 conn(0x7f8e080b4330 msgr2=0x7f8e080b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.391+0000 7f8e0ca00640 1 -- 172.21.15.70:0/313561851 shutdown_connections 2024-09-10T07:46:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.391+0000 7f8e0ca00640 1 -- 172.21.15.70:0/313561851 wait complete. 2024-09-10T07:46:08.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.392+0000 7f8e0ca00640 1 Processor -- start 2024-09-10T07:46:08.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.392+0000 7f8e0ca00640 1 -- start start 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 0x7f8e080ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 0x7f8e0814b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 0x7f8e08157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080b7840 con 0x7f8e0815e520 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080b76c0 con 0x7f8e0815e900 2024-09-10T07:46:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.393+0000 7f8e0ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8e080b79c0 con 0x7f8e0806bb20 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8e05e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 0x7f8e080ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 0x7f8e0814b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8e05e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 0x7f8e080ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:60966/0 (socket says 172.21.15.70:60966) 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8e05e00640 1 -- 172.21.15.70:0/3644968411 learned_addr learned my addr 172.21.15.70:0/3644968411 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8e06800640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 0x7f8e08157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 -- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 msgr2=0x7f8e080ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 0x7f8e080ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 -- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 msgr2=0x7f8e08157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 0x7f8e08157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 -- 172.21.15.70:0/3644968411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0814b5f0 con 0x7f8e0815e520 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8e06800640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 0x7f8e08157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.394+0000 7f8dfd400640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 0x7f8e0814b0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c3e0 tx=0x7f8df000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00074e0 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df0007680 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e08161b20 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0004a20 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e08159340 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8df0011070 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.395+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8df001d900 con 0x7f8e0815e520 2024-09-10T07:46:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.396+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e08063610 con 0x7f8e0815e520 2024-09-10T07:46:08.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.398+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8df0004c80 con 0x7f8e0815e520 2024-09-10T07:46:08.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.474+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5c0cd914-4953-4bf1-88d0-e198e9de98c9", "id": 4} v 0) -- 0x7f8e080b4e90 con 0x7f8e0815e520 2024-09-10T07:46:08.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.478+0000 7f8dfea00640 1 -- 172.21.15.70:0/3644968411 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c0cd914-4953-4bf1-88d0-e198e9de98c9", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8df001d3f0 con 0x7f8e0815e520 2024-09-10T07:46:08.477 INFO:teuthology.orchestra.run.smithi070.stdout:4 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 msgr2=0x7f8e0814b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 0x7f8e0814b0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c3e0 tx=0x7f8df000c8a0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 shutdown_connections 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0815e900 0x7f8e08157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0815e520 0x7f8e0814b0b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 --2- 172.21.15.70:0/3644968411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0806bb20 0x7f8e080ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 >> 172.21.15.70:0/3644968411 conn(0x7f8e080b4330 msgr2=0x7f8e0815bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 shutdown_connections 2024-09-10T07:46:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.479+0000 7f8e0ca00640 1 -- 172.21.15.70:0/3644968411 wait complete. 2024-09-10T07:46:08.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 976356b7-2f04-4f78-9e66-2ab3d912dd7b 5 2024-09-10T07:46:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.646+0000 7f703ac00640 1 Processor -- start 2024-09-10T07:46:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.646+0000 7f703ac00640 1 -- start start 2024-09-10T07:46:08.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.647+0000 7f703ac00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7034161d10 0x7f70340d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.647+0000 7f703ac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.647+0000 7f703ac00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70341620f0 con 0x7f70340b76c0 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.647+0000 7f703ac00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7034072140 con 0x7f7034161d10 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.647+0000 7f703ac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70340b7b30 con 0x7f70340cf6a0 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7038c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7034161d10 0x7f70340d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f702be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f70340b76c0 0x7f70340cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.70:58986/0 (socket says 172.21.15.70:58986) 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f702be00640 1 -- 172.21.15.70:0/3366296001 learned_addr learned my addr 172.21.15.70:0/3366296001 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-10T07:46:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 --2- 172.21.15.70:0/3366296001 >> v2:172.21.15.3:3300/0 conn(0x7f7034161d10 0x7f70340d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.70:48198/0 (socket says 172.21.15.70:48198) 2024-09-10T07:46:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 -- 172.21.15.70:0/3366296001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 msgr2=0x7f70340d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 --2- 172.21.15.70:0/3366296001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 -- 172.21.15.70:0/3366296001 >> v1:172.21.15.3:6789/0 conn(0x7f70340b76c0 legacy=0x7f70340cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033400640 1 -- 172.21.15.70:0/3366296001 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618922104 0 0) 0x7f70341620f0 con 0x7f70340b76c0 2024-09-10T07:46:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.648+0000 7f7033e00640 1 -- 172.21.15.70:0/3366296001 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340d4200 con 0x7f7034161d10 2024-09-10T07:46:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.649+0000 7f7033e00640 1 --2- 172.21.15.70:0/3366296001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70340d3b20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70240097a0 tx=0x7f702402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bc58937dc5d19b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f7033400640 1 -- 172.21.15.70:0/3366296001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024037070 con 0x7f7034161d10 2024-09-10T07:46:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f7033400640 1 -- 172.21.15.70:0/3366296001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702402ae50 con 0x7f7034161d10 2024-09-10T07:46:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f7033400640 1 -- 172.21.15.70:0/3366296001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024032660 con 0x7f7034161d10 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 -- 172.21.15.70:0/3366296001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 msgr2=0x7f70340d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 --2- 172.21.15.70:0/3366296001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70340d3b20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70240097a0 tx=0x7f702402a100 comp rx=0 tx=0).stop 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 -- 172.21.15.70:0/3366296001 shutdown_connections 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 --2- 172.21.15.70:0/3366296001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 --2- 172.21.15.70:0/3366296001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70340d3b20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 -- 172.21.15.70:0/3366296001 >> 172.21.15.70:0/3366296001 conn(0x7f70340d4730 msgr2=0x7f70340d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 -- 172.21.15.70:0/3366296001 shutdown_connections 2024-09-10T07:46:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.650+0000 7f703ac00640 1 -- 172.21.15.70:0/3366296001 wait complete. 2024-09-10T07:46:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.652+0000 7f703ac00640 1 Processor -- start 2024-09-10T07:46:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.652+0000 7f703ac00640 1 -- start start 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70341f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7034072950 con 0x7f7034161d10 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7034072670 con 0x7f70340b76c0 2024-09-10T07:46:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f703ac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7034072e70 con 0x7f70340cf6a0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f7033e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f702be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f7033e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52420/0 (socket says 172.21.15.70:52420) 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f7033e00640 1 -- 172.21.15.70:0/3807261471 learned_addr learned my addr 172.21.15.70:0/3807261471 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.653+0000 7f7038c00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70341f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 -- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 msgr2=0x7f70340cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 -- 172.21.15.70:0/3807261471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 msgr2=0x7f70341f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70341f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 -- 172.21.15.70:0/3807261471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70341576a0 con 0x7f70340b76c0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7033e00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7024035040 tx=0x7f70240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024041070 con 0x7f70340b76c0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702403c460 con 0x7f70340b76c0 2024-09-10T07:46:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024037040 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.654+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7034157980 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.655+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7034157f40 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.655+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f702403a070 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.655+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7024045400 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.656+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70340d3b20 con 0x7f70340b76c0 2024-09-10T07:46:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.658+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f702403ca80 con 0x7f70340b76c0 2024-09-10T07:46:08.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.730+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "976356b7-2f04-4f78-9e66-2ab3d912dd7b", "id": 5} v 0) -- 0x7f70340b8080 con 0x7f70340b76c0 2024-09-10T07:46:08.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.736+0000 7f7030c00640 1 -- 172.21.15.70:0/3807261471 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "976356b7-2f04-4f78-9e66-2ab3d912dd7b", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f702403cc60 con 0x7f70340b76c0 2024-09-10T07:46:08.736 INFO:teuthology.orchestra.run.smithi070.stdout:5 2024-09-10T07:46:08.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 msgr2=0x7f70341f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7024035040 tx=0x7f70240046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 shutdown_connections 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034161d10 0x7f70341f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70340cf6a0 0x7f70340cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 --2- 172.21.15.70:0/3807261471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70340b76c0 0x7f70341f4680 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 >> 172.21.15.70:0/3807261471 conn(0x7f70340d4730 msgr2=0x7f70340d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 shutdown_connections 2024-09-10T07:46:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.737+0000 7f703ac00640 1 -- 172.21.15.70:0/3807261471 wait complete. 2024-09-10T07:46:08.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 0c1ed442-a009-4274-9150-7c91a33f76f0 6 2024-09-10T07:46:08.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.901+0000 7fc6ea800640 1 Processor -- start 2024-09-10T07:46:08.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.901+0000 7fc6ea800640 1 -- start start 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 0x7fc6e40d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 0x7fc6e40cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e40cf940 con 0x7fc6e40cef90 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e4072140 con 0x7fc6e40b76c0 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6ea800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e41620f0 con 0x7fc6e4161d10 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6e2a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6e3400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 0x7fc6e40d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6e2a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52434/0 (socket says 172.21.15.70:52434) 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.902+0000 7fc6e2a00640 1 -- 172.21.15.70:0/3289362736 learned_addr learned my addr 172.21.15.70:0/3289362736 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3e00640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 0x7fc6e40cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3400640 1 -- 172.21.15.70:0/3289362736 >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 msgr2=0x7fc6e40d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3400640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3400640 1 -- 172.21.15.70:0/3289362736 >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 msgr2=0x7fc6e40cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:08.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3400640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 0x7fc6e40cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3400640 1 -- 172.21.15.70:0/3289362736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e40d4200 con 0x7fc6e4161d10 2024-09-10T07:46:08.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e2a00640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:08.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.903+0000 7fc6e3e00640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 0x7fc6e40cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.904+0000 7fc6e3400640 1 --2- 172.21.15.70:0/3289362736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 0x7fc6e40d3b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40b7320 tx=0x7fc6d402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ebdcd53c2f80d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.904+0000 7fc6e2000640 1 -- 172.21.15.70:0/3289362736 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4037070 con 0x7fc6e4161d10 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.904+0000 7fc6e2000640 1 -- 172.21.15.70:0/3289362736 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d402add0 con 0x7fc6e4161d10 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.904+0000 7fc6e2000640 1 -- 172.21.15.70:0/3289362736 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4032660 con 0x7fc6e4161d10 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 -- 172.21.15.70:0/3289362736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 msgr2=0x7fc6e40d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 --2- 172.21.15.70:0/3289362736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 0x7fc6e40d3b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40b7320 tx=0x7fc6d402a080 comp rx=0 tx=0).stop 2024-09-10T07:46:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 -- 172.21.15.70:0/3289362736 shutdown_connections 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.3:3300/0 conn(0x7fc6e40cef90 0x7fc6e40cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 --2- 172.21.15.70:0/3289362736 >> v2:172.21.15.70:3300/0 conn(0x7fc6e40b76c0 0x7fc6e40d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 --2- 172.21.15.70:0/3289362736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e4161d10 0x7fc6e40d3b20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 -- 172.21.15.70:0/3289362736 >> 172.21.15.70:0/3289362736 conn(0x7fc6e40d4730 msgr2=0x7fc6e40d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 -- 172.21.15.70:0/3289362736 shutdown_connections 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.905+0000 7fc6ea800640 1 -- 172.21.15.70:0/3289362736 wait complete. 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.907+0000 7fc6ea800640 1 Processor -- start 2024-09-10T07:46:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.907+0000 7fc6ea800640 1 -- start start 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.907+0000 7fc6ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 0x7fc6e4160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.907+0000 7fc6ea800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e40cef90 0x7fc6e41581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.907+0000 7fc6ea800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6ea800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4072950 con 0x7fc6e40b76c0 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6ea800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4072670 con 0x7fc6e4161d10 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6ea800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e40b7b00 con 0x7fc6e40cef90 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 0x7fc6e4160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52446/0 (socket says 172.21.15.70:52446) 2024-09-10T07:46:08.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 -- 172.21.15.70:0/1850982456 learned_addr learned my addr 172.21.15.70:0/1850982456 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e2a00640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e40cef90 0x7fc6e41581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 -- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e40cef90 msgr2=0x7fc6e41581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e40cef90 0x7fc6e41581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 -- 172.21.15.70:0/1850982456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 msgr2=0x7fc6e4160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 0x7fc6e4160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 -- 172.21.15.70:0/1850982456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e415f170 con 0x7fc6e4161d10 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3400640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 0x7fc6e4160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.908+0000 7fc6e3e00640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc007ef0 tx=0x7fc6dc00d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc01a070 con 0x7fc6e4161d10 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc003b90 con 0x7fc6e4161d10 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e415f450 con 0x7fc6e4161d10 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00de20 con 0x7fc6e4161d10 2024-09-10T07:46:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e415fa10 con 0x7fc6e4161d10 2024-09-10T07:46:08.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc6dc01d070 con 0x7fc6e4161d10 2024-09-10T07:46:08.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.909+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fc6dc004620 con 0x7fc6e4161d10 2024-09-10T07:46:08.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.910+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e40cf400 con 0x7fc6e4161d10 2024-09-10T07:46:08.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.912+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc6dc024020 con 0x7fc6e4161d10 2024-09-10T07:46:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.985+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0c1ed442-a009-4274-9150-7c91a33f76f0", "id": 6} v 0) -- 0x7fc6e40b53a0 con 0x7fc6e4161d10 2024-09-10T07:46:08.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.989+0000 7fc6db400640 1 -- 172.21.15.70:0/1850982456 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c1ed442-a009-4274-9150-7c91a33f76f0", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fc6dc010070 con 0x7fc6e4161d10 2024-09-10T07:46:08.989 INFO:teuthology.orchestra.run.smithi070.stdout:6 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 msgr2=0x7fc6e4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc007ef0 tx=0x7fc6dc00d4a0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 shutdown_connections 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e4161d10 0x7fc6e4158720 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6e40cef90 0x7fc6e41581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 --2- 172.21.15.70:0/1850982456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e40b76c0 0x7fc6e4160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:08.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 >> 172.21.15.70:0/1850982456 conn(0x7fc6e40d4730 msgr2=0x7fc6e4157640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:08.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 shutdown_connections 2024-09-10T07:46:08.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:08.990+0000 7fc6ea800640 1 -- 172.21.15.70:0/1850982456 wait complete. 2024-09-10T07:46:09.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new a7f357dd-1260-444b-ac01-702f14cab909 7 2024-09-10T07:46:09.105 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.106+0000 7fe841a99040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:46:09.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.154+0000 7fae6d200640 1 Processor -- start 2024-09-10T07:46:09.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.154+0000 7fae6d200640 1 -- start start 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae6d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 0x7fae680d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae6d200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae680d4f00 con 0x7fae680cfe90 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae6d200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae680d3de0 con 0x7fae68161b70 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae6d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae68161f50 con 0x7fae680d45a0 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae66a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 0x7fae680d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae67400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae68161b70 0x7fae680d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34900/0 (socket says 172.21.15.70:34900) 2024-09-10T07:46:09.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.155+0000 7fae67400640 1 -- 172.21.15.70:0/3485472169 learned_addr learned my addr 172.21.15.70:0/3485472169 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-10T07:46:09.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.156+0000 7fae66a00640 1 -- 172.21.15.70:0/3485472169 >> v1:172.21.15.70:6789/0 conn(0x7fae68161b70 legacy=0x7fae680d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.156+0000 7fae66a00640 1 -- 172.21.15.70:0/3485472169 >> v1:172.21.15.3:6789/0 conn(0x7fae680cfe90 legacy=0x7fae680d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.156+0000 7fae66a00640 1 -- 172.21.15.70:0/3485472169 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae680d40c0 con 0x7fae680d45a0 2024-09-10T07:46:09.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.157+0000 7fae66a00640 1 --2- 172.21.15.70:0/3485472169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 0x7fae680d49c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae540097d0 tx=0x7fae5402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c34dd7c74c7bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.157+0000 7fae66000640 1 -- 172.21.15.70:0/3485472169 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54037070 con 0x7fae680d45a0 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.157+0000 7fae66000640 1 -- 172.21.15.70:0/3485472169 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae54032440 con 0x7fae680d45a0 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.157+0000 7fae66000640 1 -- 172.21.15.70:0/3485472169 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54032810 con 0x7fae680d45a0 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 -- 172.21.15.70:0/3485472169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 msgr2=0x7fae680d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 --2- 172.21.15.70:0/3485472169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 0x7fae680d49c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae540097d0 tx=0x7fae5402a200 comp rx=0 tx=0).stop 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 -- 172.21.15.70:0/3485472169 shutdown_connections 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 --2- 172.21.15.70:0/3485472169 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae680d45a0 0x7fae680d49c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 -- 172.21.15.70:0/3485472169 >> 172.21.15.70:0/3485472169 conn(0x7fae68062630 msgr2=0x7fae68062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 -- 172.21.15.70:0/3485472169 shutdown_connections 2024-09-10T07:46:09.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.158+0000 7fae6d200640 1 -- 172.21.15.70:0/3485472169 wait complete. 2024-09-10T07:46:09.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.159+0000 7fae6d200640 1 Processor -- start 2024-09-10T07:46:09.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.159+0000 7fae6d200640 1 -- start start 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 0x7fae6815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 0x7fae6815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae680c9fc0 con 0x7fae680d45a0 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae680cf970 con 0x7fae680cfe90 2024-09-10T07:46:09.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.160+0000 7fae6d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae680d3a30 con 0x7fae68161b70 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52460/0 (socket says 172.21.15.70:52460) 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67400640 1 -- 172.21.15.70:0/2559376254 learned_addr learned my addr 172.21.15.70:0/2559376254 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 0x7fae6815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67e00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 0x7fae6815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 -- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 msgr2=0x7fae6815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 0x7fae6815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 -- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 msgr2=0x7fae680b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 -- 172.21.15.70:0/2559376254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6814d3f0 con 0x7fae680d45a0 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67e00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 0x7fae6815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae67400640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:09.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.161+0000 7fae66a00640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 0x7fae6815b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae54031000 tx=0x7fae540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54041020 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae54030990 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54032e60 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae681fbd60 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681fc2e0 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae54044070 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.162+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fae54030d80 con 0x7fae680d45a0 2024-09-10T07:46:09.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.163+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae680cf2d0 con 0x7fae680d45a0 2024-09-10T07:46:09.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.166+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fae5404b020 con 0x7fae680d45a0 2024-09-10T07:46:09.170 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.170+0000 7fe841a99040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:46:09.214 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:09.214+0000 7f45ebd8a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:46:09.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.255+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7f357dd-1260-444b-ac01-702f14cab909", "id": 7} v 0) -- 0x7fae680b99a0 con 0x7fae680d45a0 2024-09-10T07:46:09.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.258+0000 7fae5f400640 1 -- 172.21.15.70:0/2559376254 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7f357dd-1260-444b-ac01-702f14cab909", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fae54046b70 con 0x7fae680d45a0 2024-09-10T07:46:09.259 INFO:teuthology.orchestra.run.smithi070.stdout:7 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 msgr2=0x7fae6815b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 0x7fae6815b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae54031000 tx=0x7fae540046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 shutdown_connections 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae68161b70 0x7fae6815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae680d45a0 0x7fae6815b410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 --2- 172.21.15.70:0/2559376254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae680cfe90 0x7fae680b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 >> 172.21.15.70:0/2559376254 conn(0x7fae68062630 msgr2=0x7fae6815cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:09.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 shutdown_connections 2024-09-10T07:46:09.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:09.260+0000 7fae6d200640 1 -- 172.21.15.70:0/2559376254 wait complete. 2024-09-10T07:46:09.274 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T07:46:09.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T07:46:09.278 INFO:tasks.ceph.osd.0:Started 2024-09-10T07:46:09.278 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T07:46:09.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T07:46:09.280 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:09.280+0000 7f45ebd8a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:46:09.283 INFO:tasks.ceph.osd.1:Started 2024-09-10T07:46:09.283 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T07:46:09.283 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T07:46:09.287 INFO:tasks.ceph.osd.2:Started 2024-09-10T07:46:09.287 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T07:46:09.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T07:46:09.291 INFO:tasks.ceph.osd.3:Started 2024-09-10T07:46:09.292 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T07:46:09.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T07:46:09.297 INFO:tasks.ceph.osd.4:Started 2024-09-10T07:46:09.297 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T07:46:09.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T07:46:09.303 INFO:tasks.ceph.osd.5:Started 2024-09-10T07:46:09.303 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T07:46:09.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T07:46:09.307 INFO:tasks.ceph.osd.6:Started 2024-09-10T07:46:09.307 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T07:46:09.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T07:46:09.311 INFO:tasks.ceph.osd.7:Started 2024-09-10T07:46:09.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:09.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.522+0000 7fbe8b400640 1 Processor -- start 2024-09-10T07:46:09.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.522+0000 7fbe8b400640 1 -- start start 2024-09-10T07:46:09.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe8b400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe840b6a70 0x7fbe840d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840d1510 0x7fbe840d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe8b400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe840d1d90 con 0x7fbe840b6690 2024-09-10T07:46:09.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe8b400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbe840cf540 con 0x7fbe840b6a70 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe8b400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe840ceef0 con 0x7fbe840d1510 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe83e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe840b6a70 0x7fbe840d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe88a00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbe840b6690 0x7fbe840cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe88a00640 1 -- 172.21.15.3:0/1481521627 learned_addr learned my addr 172.21.15.3:0/1481521627 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.523+0000 7fbe89400640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840d1510 0x7fbe840d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.524+0000 7fbe83400640 1 -- 172.21.15.3:0/1481521627 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810431038 0 0) 0x7fbe840d1d90 con 0x7fbe840b6690 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.524+0000 7fbe83e00640 1 -- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840d1510 msgr2=0x7fbe840d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.524+0000 7fbe83e00640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840d1510 0x7fbe840d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.524+0000 7fbe83e00640 1 -- 172.21.15.3:0/1481521627 >> v1:172.21.15.70:6789/0 conn(0x7fbe840b6690 legacy=0x7fbe840cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.524+0000 7fbe83e00640 1 -- 172.21.15.3:0/1481521627 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe840cf1d0 con 0x7fbe840b6a70 2024-09-10T07:46:09.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.525+0000 7fbe83e00640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840d0fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe700097a0 tx=0x7fbe7002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89bddc13f8b3b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:09.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.525+0000 7fbe83400640 1 -- 172.21.15.3:0/1481521627 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70037070 con 0x7fbe840b6a70 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.525+0000 7fbe83400640 1 -- 172.21.15.3:0/1481521627 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe70032440 con 0x7fbe840b6a70 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.525+0000 7fbe83400640 1 -- 172.21.15.3:0/1481521627 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70032810 con 0x7fbe840b6a70 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 -- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 msgr2=0x7fbe840d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840d0fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe700097a0 tx=0x7fbe7002a180 comp rx=0 tx=0).stop 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 -- 172.21.15.3:0/1481521627 shutdown_connections 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840d1510 0x7fbe840d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 --2- 172.21.15.3:0/1481521627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840d0fd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 -- 172.21.15.3:0/1481521627 >> 172.21.15.3:0/1481521627 conn(0x7fbe84072560 msgr2=0x7fbe84072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 -- 172.21.15.3:0/1481521627 shutdown_connections 2024-09-10T07:46:09.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.526+0000 7fbe8b400640 1 -- 172.21.15.3:0/1481521627 wait complete. 2024-09-10T07:46:09.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.527+0000 7fbe8b400640 1 Processor -- start 2024-09-10T07:46:09.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.528+0000 7fbe8b400640 1 -- start start 2024-09-10T07:46:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.528+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 0x7fbe84151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.528+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 0x7fbe840b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe8b400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe840d1b00 con 0x7fbe840d1510 2024-09-10T07:46:09.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe8b400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbe840d1980 con 0x7fbe840b6a70 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe8b400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8415c530 con 0x7fbe840b6690 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe88a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 0x7fbe84151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe89400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 0x7fbe840b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46142/0 (socket says 172.21.15.3:46142) 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 -- 172.21.15.3:0/3762802858 learned_addr learned my addr 172.21.15.3:0/3762802858 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe88a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 0x7fbe84151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:52622/0 (socket says 172.21.15.3:52622) 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe89400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 0x7fbe840b9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44782/0 (socket says 172.21.15.3:44782) 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 -- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 msgr2=0x7fbe84151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 0x7fbe84151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 -- 172.21.15.3:0/3762802858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 msgr2=0x7fbe840b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 0x7fbe840b9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.529+0000 7fbe83e00640 1 -- 172.21.15.3:0/3762802858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe840cd6a0 con 0x7fbe840b6a70 2024-09-10T07:46:09.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe83e00640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe70035040 tx=0x7fbe700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70037050 con 0x7fbe840b6a70 2024-09-10T07:46:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe70038040 con 0x7fbe840b6a70 2024-09-10T07:46:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe840cd3c0 con 0x7fbe840b6a70 2024-09-10T07:46:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004490 con 0x7fbe840b6a70 2024-09-10T07:46:09.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe840cbbe0 con 0x7fbe840b6a70 2024-09-10T07:46:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.530+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe7003c6d0 con 0x7fbe840b6a70 2024-09-10T07:46:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.531+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbe7003eb20 con 0x7fbe840b6a70 2024-09-10T07:46:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.531+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe840d0fd0 con 0x7fbe840b6a70 2024-09-10T07:46:09.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.534+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbe7003edb0 con 0x7fbe840b6a70 2024-09-10T07:46:09.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.597+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe84069140 con 0x7fbe840b6a70 2024-09-10T07:46:09.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.598+0000 7fbe80c00640 1 -- 172.21.15.3:0/3762802858 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbe7003c9c0 con 0x7fbe840b6a70 2024-09-10T07:46:09.597 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:09.256869+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 msgr2=0x7fbe840cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe70035040 tx=0x7fbe700046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 shutdown_connections 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe840d1510 0x7fbe840b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe840b6a70 0x7fbe840cc900 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 --2- 172.21.15.3:0/3762802858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe840b6690 0x7fbe84151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:09.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 >> 172.21.15.3:0/3762802858 conn(0x7fbe84072560 msgr2=0x7fbe8415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:09.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 shutdown_connections 2024-09-10T07:46:09.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:09.599+0000 7fbe8b400640 1 -- 172.21.15.3:0/3762802858 wait complete. 2024-09-10T07:46:09.620 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T07:46:09.620 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T07:46:09.696 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.696+0000 7fe841a99040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:46:09.770 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:09.771+0000 7f45ebd8a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:46:09.799 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.800+0000 7fe841a99040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:46:09.869 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:09.869+0000 7f45ebd8a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:46:09.925 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.925+0000 7fe841a99040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:46:09.981 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:09.982+0000 7fe841a99040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:46:09.990 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:09.990+0000 7f45ebd8a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:46:10.046 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.047+0000 7f45ebd8a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:46:10.115 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.116+0000 7fe841a99040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:46:10.173 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.174+0000 7f45ebd8a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:46:10.180 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.180+0000 7fe841a99040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:46:10.235 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.235+0000 7fe841a99040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:46:10.236 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.236+0000 7f45ebd8a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:46:10.290 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.290+0000 7f45ebd8a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:46:10.368 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.368+0000 7fe841a99040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:46:10.418 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.418+0000 7f45ebd8a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:46:10.425 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:10.429 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.428+0000 7fe841a99040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:46:10.478 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.478+0000 7f45ebd8a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:46:10.492 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.492+0000 7fe841a99040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:46:10.538 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.538+0000 7f45ebd8a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:46:10.562 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.563+0000 7f45a3a00640 1 Processor -- start 2024-09-10T07:46:10.563 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.563+0000 7f45a3a00640 1 -- start start 2024-09-10T07:46:10.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a3a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f459c0ceef0 0x7f459c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:10.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a3a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c06c530 0x7f459c0d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:10.565 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a3a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459c0b80a0 con 0x7f459c15e9a0 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a3a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f459c15ed80 con 0x7f459c0ceef0 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a3a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f459c06c990 con 0x7f459c06c530 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f459be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f459c0ceef0 0x7f459c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f459be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f459c0ceef0 0x7f459c06bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44788/0 (socket says 172.21.15.3:44788) 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f459be00640 1 -- 172.21.15.3:0/1345233751 learned_addr learned my addr 172.21.15.3:0/1345233751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.565+0000 7f45a1a00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c06c530 0x7f459c0d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.566+0000 7f459be00640 1 -- 172.21.15.3:0/1345233751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c06c530 msgr2=0x7f459c0d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.566+0000 7f459be00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c06c530 0x7f459c0d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.566+0000 7f459be00640 1 -- 172.21.15.3:0/1345233751 >> v1:172.21.15.70:6789/0 conn(0x7f459c15e9a0 legacy=0x7f459c0b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.566+0000 7f459be00640 1 -- 172.21.15.3:0/1345233751 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0cf940 con 0x7f459c0ceef0 2024-09-10T07:46:10.566 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.566+0000 7f459b400640 1 -- 172.21.15.3:0/1345233751 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449729454 0 0) 0x7f4590001110 con 0x7f459c15e9a0 2024-09-10T07:46:10.567 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.567+0000 7f459be00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c0ceef0 0x7f459c06bff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45880098a0 tx=0x7f458802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457088a9287749ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:10.567 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.567+0000 7f459b400640 1 -- 172.21.15.3:0/1345233751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588037070 con 0x7f459c0ceef0 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.567+0000 7f459b400640 1 -- 172.21.15.3:0/1345233751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4588032440 con 0x7f459c0ceef0 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.567+0000 7f459b400640 1 -- 172.21.15.3:0/1345233751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588032810 con 0x7f459c0ceef0 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 -- 172.21.15.3:0/1345233751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c0ceef0 msgr2=0x7f459c06bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c0ceef0 0x7f459c06bff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45880098a0 tx=0x7f458802a200 comp rx=0 tx=0).stop 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 -- 172.21.15.3:0/1345233751 shutdown_connections 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c06c530 0x7f459c0d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 --2- 172.21.15.3:0/1345233751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c0ceef0 0x7f459c06bff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 -- 172.21.15.3:0/1345233751 >> 172.21.15.3:0/1345233751 conn(0x7f459c0d4690 msgr2=0x7f459c0d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 -- 172.21.15.3:0/1345233751 shutdown_connections 2024-09-10T07:46:10.568 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.568+0000 7f45a3a00640 1 -- 172.21.15.3:0/1345233751 wait complete. 2024-09-10T07:46:10.569 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.569+0000 7f45a3a00640 1 Processor -- start 2024-09-10T07:46:10.569 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 -- start start 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f459c0ceef0 0x7f459c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c15e9a0 0x7f459c0b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0d2da0 con 0x7f459c06c530 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0cf6f0 con 0x7f459c0ceef0 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.570+0000 7f45a3a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0d3080 con 0x7f459c15e9a0 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44800/0 (socket says 172.21.15.3:44800) 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 -- 172.21.15.3:0/587365971 learned_addr learned my addr 172.21.15.3:0/587365971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:10.572 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1a00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c15e9a0 0x7f459c0b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f459be00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f459c0ceef0 0x7f459c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 -- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c15e9a0 msgr2=0x7f459c0b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c15e9a0 0x7f459c0b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 -- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f459c0ceef0 msgr2=0x7f459c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f459c0ceef0 0x7f459c0cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 -- 172.21.15.3:0/587365971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c15e060 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.571+0000 7f45a1000640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f459000c3c0 tx=0x7f459000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590017070 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45900035f0 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900039e0 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c0cc7b0 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c0ccd70 con 0x7f459c06c530 2024-09-10T07:46:10.573 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f459001a070 con 0x7f459c06c530 2024-09-10T07:46:10.574 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.572+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4590003b80 con 0x7f459c06c530 2024-09-10T07:46:10.574 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.573+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c0cff00 con 0x7f459c06c530 2024-09-10T07:46:10.575 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.576+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4590021020 con 0x7f459c06c530 2024-09-10T07:46:10.577 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.577+0000 7fe841a99040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:46:10.620 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.620+0000 7f45ebd8a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:46:10.632 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.633+0000 7fe841a99040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:46:10.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.640+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f459c0cf4d0 con 0x7f459c06c530 2024-09-10T07:46:10.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.640+0000 7f4598c00640 1 -- 172.21.15.3:0/587365971 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f459c15e060 con 0x7f459c06c530 2024-09-10T07:46:10.640 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-10T07:46:10.641 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:09.256869+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 msgr2=0x7f459c15d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f459000c3c0 tx=0x7f459000c880 comp rx=0 tx=0).stop 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 shutdown_connections 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f459c15e9a0 0x7f459c0b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f459c0ceef0 0x7f459c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 --2- 172.21.15.3:0/587365971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459c06c530 0x7f459c15d770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.641+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 >> 172.21.15.3:0/587365971 conn(0x7f459c0d4690 msgr2=0x7f459c161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.642+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 shutdown_connections 2024-09-10T07:46:10.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:10.642+0000 7f45a3a00640 1 -- 172.21.15.3:0/587365971 wait complete. 2024-09-10T07:46:10.653 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T07:46:10.674 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.675+0000 7f45ebd8a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:46:10.687 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:10.688+0000 7fe841a99040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:46:10.726 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:10.726+0000 7f45ebd8a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:46:11.293 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-09-10T07:46:11.293+0000 7fe0bf82c640 -1 Falling back to public interface 2024-09-10T07:46:11.361 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-09-10T07:46:11.362+0000 7feafc3fd640 -1 Falling back to public interface 2024-09-10T07:46:11.373 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-10T07:46:11.374+0000 7f212600d640 -1 Falling back to public interface 2024-09-10T07:46:11.381 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-10T07:46:11.382+0000 7f9e616d6640 -1 Falling back to public interface 2024-09-10T07:46:11.385 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-10T07:46:11.386+0000 7fbdba80d640 -1 Falling back to public interface 2024-09-10T07:46:11.395 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-10T07:46:11.395+0000 7f591c80d640 -1 Falling back to public interface 2024-09-10T07:46:11.398 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-09-10T07:46:11.399+0000 7fb359e64640 -1 Falling back to public interface 2024-09-10T07:46:11.405 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-09-10T07:46:11.405+0000 7fe3e8d34640 -1 Falling back to public interface 2024-09-10T07:46:11.472 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:11.472+0000 7fe841a99040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:46:11.485 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-10T07:46:11.485+0000 7f45ebd8a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:46:12.492 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:12.492+0000 7fe7e1e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T07:46:12.492 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-10T07:46:12.493+0000 7fe7e1e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T07:46:15.669 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-09-10T07:46:15.669+0000 7fe0bf82c640 -1 osd.4 0 log_to_monitors true 2024-09-10T07:46:15.775 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-09-10T07:46:15.776+0000 7feafc3fd640 -1 osd.7 0 log_to_monitors true 2024-09-10T07:46:15.859 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-09-10T07:46:15.859+0000 7fe3e8d34640 -1 osd.5 0 log_to_monitors true 2024-09-10T07:46:15.976 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-10T07:46:15.976+0000 7f212600d640 -1 osd.1 0 log_to_monitors true 2024-09-10T07:46:16.073 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-09-10T07:46:16.073+0000 7fb359e64640 -1 osd.6 0 log_to_monitors true 2024-09-10T07:46:16.093 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-10T07:46:16.093+0000 7f9e616d6640 -1 osd.0 0 log_to_monitors true 2024-09-10T07:46:16.122 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-10T07:46:16.123+0000 7f591c80d640 -1 osd.2 0 log_to_monitors true 2024-09-10T07:46:16.149 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-10T07:46:16.149+0000 7fbdba80d640 -1 osd.3 0 log_to_monitors true 2024-09-10T07:46:17.460 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:17.607 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.607+0000 7ff312c00640 1 Processor -- start 2024-09-10T07:46:17.607 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.608+0000 7ff312c00640 1 -- start start 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff312c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c0cd6b0 0x7ff30c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff312c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0b5b20 0x7ff30c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff312c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c0cdfd0 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff312c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c156e60 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff312c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c15db10 con 0x7ff30c0b5b20 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff30be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c0cd6b0 0x7ff30c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff30be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff30c0cd6b0 0x7ff30c0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37792/0 (socket says 172.21.15.3:37792) 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff30be00640 1 -- 172.21.15.3:0/1601386991 learned_addr learned my addr 172.21.15.3:0/1601386991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:17.609 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.609+0000 7ff303e00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0b5b20 0x7ff30c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.610 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.610+0000 7ff30be00640 1 -- 172.21.15.3:0/1601386991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0b5b20 msgr2=0x7ff30c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:17.610 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.610+0000 7ff30be00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0b5b20 0x7ff30c0b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.610 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.610+0000 7ff30be00640 1 -- 172.21.15.3:0/1601386991 >> v1:172.21.15.3:6789/0 conn(0x7ff30c0cf3b0 legacy=0x7ff30c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.610+0000 7ff30b400640 1 -- 172.21.15.3:0/1601386991 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515794631 0 0) 0x7ff30c0cdfd0 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.610+0000 7ff30be00640 1 -- 172.21.15.3:0/1601386991 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c15ca60 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff30be00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cd6b0 0x7ff30c0cda90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098a0 tx=0x7ff2fc02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8976da238e97f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff30b400640 1 -- 172.21.15.3:0/1601386991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc037070 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff30b400640 1 -- 172.21.15.3:0/1601386991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc032440 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.611 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff30b400640 1 -- 172.21.15.3:0/1601386991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc032810 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff312c00640 1 -- 172.21.15.3:0/1601386991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cd6b0 msgr2=0x7ff30c0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.611+0000 7ff312c00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cd6b0 0x7ff30c0cda90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098a0 tx=0x7ff2fc02a290 comp rx=0 tx=0).stop 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 -- 172.21.15.3:0/1601386991 shutdown_connections 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0b5b20 0x7ff30c0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 --2- 172.21.15.3:0/1601386991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cd6b0 0x7ff30c0cda90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 -- 172.21.15.3:0/1601386991 >> 172.21.15.3:0/1601386991 conn(0x7ff30c06c240 msgr2=0x7ff30c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 -- 172.21.15.3:0/1601386991 shutdown_connections 2024-09-10T07:46:17.612 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.612+0000 7ff312c00640 1 -- 172.21.15.3:0/1601386991 wait complete. 2024-09-10T07:46:17.613 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.613+0000 7ff312c00640 1 Processor -- start 2024-09-10T07:46:17.613 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.613+0000 7ff312c00640 1 -- start start 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c0b5b20 0x7ff30c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 0x7ff30c14b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c15d300 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c15d020 con 0x7ff30c0b5b20 2024-09-10T07:46:17.614 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.614+0000 7ff312c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c072700 con 0x7ff30c0cd6b0 2024-09-10T07:46:17.615 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.615 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37798/0 (socket says 172.21.15.3:37798) 2024-09-10T07:46:17.615 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 -- 172.21.15.3:0/2108466519 learned_addr learned my addr 172.21.15.3:0/2108466519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:17.615 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff30be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c0b5b20 0x7ff30c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.615 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff303e00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 0x7ff30c14b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 -- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 msgr2=0x7ff30c14b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 0x7ff30c14b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 -- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c0b5b20 msgr2=0x7ff30c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c0b5b20 0x7ff30c1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 -- 172.21.15.3:0/2108466519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c14b370 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff303e00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 0x7ff30c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff310c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c760 tx=0x7ff30400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304003da0 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.616 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304005070 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.617 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304005460 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.617 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.615+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c157eb0 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.617 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.616+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c158470 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.617 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.617+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c0cda90 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.617 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.618+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7ff304005600 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.618 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.618+0000 7ff308c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 0x7ff2e806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:17.618 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.619+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2009+0+0 (secure 0 0 0) 0x7ff304089d40 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.618 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.619+0000 7ff30be00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 0x7ff2e806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:17.619 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.619+0000 7ff30be00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 0x7ff2e806e290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc034000 tx=0x7ff2fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:17.623 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.624+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff30c14b370 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.718 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-09-10T07:46:17.719+0000 7fe0ba600640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T07:46:17.722 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-09-10T07:46:17.723+0000 7fe0afc00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:17.838 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.838+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff30c15ac90 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.839 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.839+0000 7ff308c00640 1 -- 172.21.15.3:0/2108466519 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7555 (secure 0 0 0) 0x7ff304059640 con 0x7ff30c0cf3b0 2024-09-10T07:46:17.840 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-10T07:46:17.840 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":13,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:17.729835+0000","last_up_change":"2024-09-10T07:46:17.729835+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:17.842 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 msgr2=0x7ff2e806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.842 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 0x7ff2e806e290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc034000 tx=0x7ff2fc02e000 comp rx=0 tx=0).stop 2024-09-10T07:46:17.842 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 msgr2=0x7ff30c14ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:17.842 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c760 tx=0x7ff30400cc20 comp rx=0 tx=0).stop 2024-09-10T07:46:17.842 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 shutdown_connections 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff2e806bdd0 0x7ff2e806e290 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff30c0cf3b0 0x7ff30c14ae30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff30c0cd6b0 0x7ff30c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.842+0000 7ff312c00640 1 --2- 172.21.15.3:0/2108466519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30c0b5b20 0x7ff30c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.843+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 >> 172.21.15.3:0/2108466519 conn(0x7ff30c06c240 msgr2=0x7ff30c0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.843+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 shutdown_connections 2024-09-10T07:46:17.843 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:17.843+0000 7ff312c00640 1 -- 172.21.15.3:0/2108466519 wait complete. 2024-09-10T07:46:17.856 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-10T07:46:19.095 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-09-10T07:46:19.095+0000 7fe3e3200640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T07:46:19.105 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-09-10T07:46:19.106+0000 7fe3d9c00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:19.110 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-09-10T07:46:19.110+0000 7fb354400640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T07:46:19.124 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-09-10T07:46:19.125+0000 7fb34ae00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:19.127 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-09-10T07:46:19.127+0000 7feaf7c00640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T07:46:19.146 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-09-10T07:46:19.146+0000 7feaed200640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:20.048 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-10T07:46:20.048+0000 7f2121400640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T07:46:20.054 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-10T07:46:20.054+0000 7f9e5bc00640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T07:46:20.055 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-10T07:46:20.055+0000 7f5916600640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T07:46:20.056 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-10T07:46:20.056+0000 7f2116a00640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:20.062 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-10T07:46:20.062+0000 7f590d000640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:20.063 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-10T07:46:20.064+0000 7f9e52600640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:20.075 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-10T07:46:20.076+0000 7fbdb4600640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T07:46:20.083 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-10T07:46:20.083+0000 7fbdab000640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:46:20.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:46:20.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:46:24.663 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:24.806 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.806+0000 7f2986e00640 1 Processor -- start 2024-09-10T07:46:24.806 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.806+0000 7f2986e00640 1 -- start start 2024-09-10T07:46:24.808 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2986e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29800cd6b0 0x7f29800cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.808 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2986e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2980072320 0x7f2980072790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.808 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2986e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2980160950 con 0x7f2980160570 2024-09-10T07:46:24.808 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2986e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f29800ce040 con 0x7f29800cd6b0 2024-09-10T07:46:24.808 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2986e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2980163db0 con 0x7f2980072320 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f297fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2980160570 0x7f29800b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42474/0 (socket says 172.21.15.3:42474) 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f297fe00640 1 -- 172.21.15.3:0/1814349481 learned_addr learned my addr 172.21.15.3:0/1814349481 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f297f400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29800cd6b0 0x7f29800cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.808+0000 7f2984e00640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2980072320 0x7f2980072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.809+0000 7f297f400640 1 -- 172.21.15.3:0/1814349481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2980072320 msgr2=0x7f2980072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.809+0000 7f297f400640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2980072320 0x7f2980072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.809+0000 7f297f400640 1 -- 172.21.15.3:0/1814349481 >> v1:172.21.15.3:6789/0 conn(0x7f2980160570 legacy=0x7f29800b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.809+0000 7f297ea00640 1 -- 172.21.15.3:0/1814349481 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177848757 0 0) 0x7f2980160950 con 0x7f2980160570 2024-09-10T07:46:24.809 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.809+0000 7f297f400640 1 -- 172.21.15.3:0/1814349481 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29800d0210 con 0x7f29800cd6b0 2024-09-10T07:46:24.810 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f297f400640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29800cd6b0 0x7f29800cdb00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097d0 tx=0x7f296c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2459e80003b5b896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.810 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f297ea00640 1 -- 172.21.15.3:0/1814349481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c037070 con 0x7f29800cd6b0 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f297ea00640 1 -- 172.21.15.3:0/1814349481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c032440 con 0x7f29800cd6b0 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f297ea00640 1 -- 172.21.15.3:0/1814349481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c032810 con 0x7f29800cd6b0 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f2986e00640 1 -- 172.21.15.3:0/1814349481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29800cd6b0 msgr2=0x7f29800cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f2986e00640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29800cd6b0 0x7f29800cdb00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097d0 tx=0x7f296c02a1c0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.810+0000 7f2986e00640 1 -- 172.21.15.3:0/1814349481 shutdown_connections 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.811+0000 7f2986e00640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2980072320 0x7f2980072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.811+0000 7f2986e00640 1 --2- 172.21.15.3:0/1814349481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29800cd6b0 0x7f29800cdb00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.811+0000 7f2986e00640 1 -- 172.21.15.3:0/1814349481 >> 172.21.15.3:0/1814349481 conn(0x7f298006c240 msgr2=0x7f29800ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.811+0000 7f2986e00640 1 -- 172.21.15.3:0/1814349481 shutdown_connections 2024-09-10T07:46:24.811 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.811+0000 7f2986e00640 1 -- 172.21.15.3:0/1814349481 wait complete. 2024-09-10T07:46:24.812 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.812+0000 7f2986e00640 1 Processor -- start 2024-09-10T07:46:24.812 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.812+0000 7f2986e00640 1 -- start start 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 0x7f29801f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29800cd6b0 0x7f29801ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29800cf730 con 0x7f2980160570 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29800cf450 con 0x7f2980072320 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.813+0000 7f2986e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29800cfa10 con 0x7f29800cd6b0 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48902/0 (socket says 172.21.15.3:48902) 2024-09-10T07:46:24.814 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 -- 172.21.15.3:0/3932722207 learned_addr learned my addr 172.21.15.3:0/3932722207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f297fe00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 0x7f29801f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f297f400640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29800cd6b0 0x7f29801ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 -- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29800cd6b0 msgr2=0x7f29801ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29800cd6b0 0x7f29801ec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 -- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 msgr2=0x7f29801f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 0x7f29801f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 -- 172.21.15.3:0/3932722207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2980164f30 con 0x7f2980160570 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f297fe00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 0x7f29801f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:24.815 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2984e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f297800efc0 tx=0x7f297800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:24.816 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.814+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978018020 con 0x7f2980160570 2024-09-10T07:46:24.816 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.815+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2978014480 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.815+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2980165210 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.815+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978014870 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.815+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801657d0 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.816+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29800cfcf0 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.817+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f2978005d10 con 0x7f2980160570 2024-09-10T07:46:24.817 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.818+0000 7f2977e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 0x7f296006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:24.818 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.818+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f297808b300 con 0x7f2980160570 2024-09-10T07:46:24.823 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.823+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2978022070 con 0x7f2980160570 2024-09-10T07:46:24.827 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.828+0000 7f297fe00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 0x7f296006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:24.828 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:24.828+0000 7f297fe00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 0x7f296006e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2970000f60 tx=0x7f2970007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:25.008 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.008+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29800cc7c0 con 0x7f2980160570 2024-09-10T07:46:25.009 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.009+0000 7f2977e00640 1 -- 172.21.15.3:0/3932722207 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12824 (secure 0 0 0) 0x7f2978010040 con 0x7f2980160570 2024-09-10T07:46:25.009 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-10T07:46:25.009 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":19,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:24.080215+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:25.011 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.011+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 msgr2=0x7f296006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.012 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.011+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 0x7f296006e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2970000f60 tx=0x7f2970007d50 comp rx=0 tx=0).stop 2024-09-10T07:46:25.012 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 msgr2=0x7f29801ec850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.012 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f297800efc0 tx=0x7f297800c490 comp rx=0 tx=0).stop 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 shutdown_connections 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f296006bda0 0x7f296006e260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2980160570 0x7f29801ec850 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29800cd6b0 0x7f29801ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 --2- 172.21.15.3:0/3932722207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2980072320 0x7f29801f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 >> 172.21.15.3:0/3932722207 conn(0x7f298006c240 msgr2=0x7f29800b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 shutdown_connections 2024-09-10T07:46:25.013 INFO:teuthology.misc.health.smithi003.stderr:2024-09-10T07:46:25.012+0000 7f2986e00640 1 -- 172.21.15.3:0/3932722207 wait complete. 2024-09-10T07:46:25.024 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T07:46:25.025 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T07:46:25.025 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:46:25.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:46:25.029 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:46:25.029 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:46:25.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:46:25.033 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:46:25.033 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:46:25.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:46:25.037 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:46:25.038 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:46:25.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:46:25.042 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:46:25.042 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T07:46:25.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:25.161 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:46:25.162 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:46:25.172 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:46:25.173 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:46:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.210+0000 7f3153200640 1 Processor -- start 2024-09-10T07:46:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.211+0000 7f3153200640 1 -- start start 2024-09-10T07:46:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3153200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3153200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 0x7f314c0d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3153200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314c0cff10 con 0x7f314c0b6a70 2024-09-10T07:46:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3153200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f314c0d1b70 con 0x7f314c0d16e0 2024-09-10T07:46:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3153200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f314c0cf540 con 0x7f314c0b6690 2024-09-10T07:46:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3143e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f314c0b6a70 0x7f314c0d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50536/0 (socket says 172.21.15.3:50536) 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3143e00640 1 -- 172.21.15.3:0/1089181658 learned_addr learned my addr 172.21.15.3:0/1089181658 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3151200640 1 --2- 172.21.15.3:0/1089181658 >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 0x7f314c0d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314be00640 1 -- 172.21.15.3:0/1089181658 >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 msgr2=0x7f314c0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314be00640 1 --2- 172.21.15.3:0/1089181658 >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 0x7f314c0d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314be00640 1 -- 172.21.15.3:0/1089181658 >> v1:172.21.15.70:6789/0 conn(0x7f314c0b6a70 legacy=0x7f314c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314be00640 1 -- 172.21.15.3:0/1089181658 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c0d1d90 con 0x7f314c0b6690 2024-09-10T07:46:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f3151200640 1 --2- 172.21.15.3:0/1089181658 >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 0x7f314c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.212+0000 7f314b400640 1 -- 172.21.15.3:0/1089181658 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043848856 0 0) 0x7f314c0cff10 con 0x7f314c0b6a70 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.213+0000 7f314be00640 1 --2- 172.21.15.3:0/1089181658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c0cf9d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097a0 tx=0x7f313c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a6a8ee54a14684 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f314b400640 1 -- 172.21.15.3:0/1089181658 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c037070 con 0x7f314c0b6690 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f314b400640 1 -- 172.21.15.3:0/1089181658 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c032440 con 0x7f314c0b6690 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f314b400640 1 -- 172.21.15.3:0/1089181658 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c032810 con 0x7f314c0b6690 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 -- 172.21.15.3:0/1089181658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 msgr2=0x7f314c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 --2- 172.21.15.3:0/1089181658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c0cf9d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097a0 tx=0x7f313c02a180 comp rx=0 tx=0).stop 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 -- 172.21.15.3:0/1089181658 shutdown_connections 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 --2- 172.21.15.3:0/1089181658 >> v2:172.21.15.3:3300/0 conn(0x7f314c0d16e0 0x7f314c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 --2- 172.21.15.3:0/1089181658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c0cf9d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 -- 172.21.15.3:0/1089181658 >> 172.21.15.3:0/1089181658 conn(0x7f314c072560 msgr2=0x7f314c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 -- 172.21.15.3:0/1089181658 shutdown_connections 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.214+0000 7f3153200640 1 -- 172.21.15.3:0/1089181658 wait complete. 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.216+0000 7f3153200640 1 Processor -- start 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.216+0000 7f3153200640 1 -- start start 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f314c0b6a70 0x7f314c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314c0d16e0 0x7f314c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0b6ed0 con 0x7f314c0b6a70 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0cf1d0 con 0x7f314c0d16e0 2024-09-10T07:46:25.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.217+0000 7f3153200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f314c15c530 con 0x7f314c0b6690 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f3143e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f314c0b6a70 0x7f314c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f3151200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314c0d16e0 0x7f314c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:43236/0 (socket says 172.21.15.3:43236) 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 -- 172.21.15.3:0/93492805 learned_addr learned my addr 172.21.15.3:0/93492805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 -- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314c0d16e0 msgr2=0x7f314c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314c0d16e0 0x7f314c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 -- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f314c0b6a70 msgr2=0x7f314c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f314c0b6a70 0x7f314c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 -- 172.21.15.3:0/93492805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c1f5a20 con 0x7f314c0b6690 2024-09-10T07:46:25.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f314be00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f313c000c00 tx=0x7f313c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.218+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c037050 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.219+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c038040 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.219+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c004470 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.219+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c1f5180 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.219+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1578d0 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.220+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c1f5540 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.221+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f313c030980 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.221+0000 7f3148c00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 0x7f312806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.222+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f313c0ab550 con 0x7f314c0b6690 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.222+0000 7f3143e00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 0x7f312806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.222+0000 7f3143e00640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 0x7f312806e260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f314c1598f0 tx=0x7f3134007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.227+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f313c079d60 con 0x7f314c0b6690 2024-09-10T07:46:25.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.453+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f314c0cf9d0 con 0x7f314c0b6690 2024-09-10T07:46:25.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.454+0000 7f3148c00640 1 -- 172.21.15.3:0/93492805 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f313c07e060 con 0x7f314c0b6690 2024-09-10T07:46:25.454 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:25.454 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"btime":"2024-09-10T07:46:25:171289+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T07:46:25.454 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-09-10T07:46:25.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 msgr2=0x7f312806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 0x7f312806e260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f314c1598f0 tx=0x7f3134007d50 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 msgr2=0x7f314c1f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f313c000c00 tx=0x7f313c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 shutdown_connections 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f312806bda0 0x7f312806e260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f314c0d16e0 0x7f314c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f314c0b6a70 0x7f314c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 --2- 172.21.15.3:0/93492805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f314c0b6690 0x7f314c1f48a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 >> 172.21.15.3:0/93492805 conn(0x7f314c072560 msgr2=0x7f314c0cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 shutdown_connections 2024-09-10T07:46:25.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:25.456+0000 7f3153200640 1 -- 172.21.15.3:0/93492805 wait complete. 2024-09-10T07:46:27.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.640+0000 7f77d6a00640 1 Processor -- start 2024-09-10T07:46:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.640+0000 7f77d6a00640 1 -- start start 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.641+0000 7f77d6a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77d00b6a70 0x7f77d00d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.641+0000 7f77d6a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77d00d1510 0x7f77d00d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.641+0000 7f77d6a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d00d1d90 con 0x7f77d00b6690 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.641+0000 7f77d6a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f77d00cf540 con 0x7f77d00d1510 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.641+0000 7f77d6a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f77d00ceef0 con 0x7f77d00b6a70 2024-09-10T07:46:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77d00d1510 0x7f77d00d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77d00d1510 0x7f77d00d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48964/0 (socket says 172.21.15.3:48964) 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 -- 172.21.15.3:0/3226465458 learned_addr learned my addr 172.21.15.3:0/3226465458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77cfe00640 1 --1- 172.21.15.3:0/3226465458 >> v1:172.21.15.3:6789/0 conn(0x7f77d00b6690 0x7f77d00cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42516/0 (socket says 172.21.15.3:42516) 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77c7e00640 1 --2- 172.21.15.3:0/3226465458 >> v2:172.21.15.70:3300/0 conn(0x7f77d00b6a70 0x7f77d00d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 -- 172.21.15.3:0/3226465458 >> v2:172.21.15.70:3300/0 conn(0x7f77d00b6a70 msgr2=0x7f77d00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 --2- 172.21.15.3:0/3226465458 >> v2:172.21.15.70:3300/0 conn(0x7f77d00b6a70 0x7f77d00d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 -- 172.21.15.3:0/3226465458 >> v1:172.21.15.3:6789/0 conn(0x7f77d00b6690 legacy=0x7f77d00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.642+0000 7f77d4a00640 1 -- 172.21.15.3:0/3226465458 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d00cf1d0 con 0x7f77d00d1510 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.643+0000 7f77d4a00640 1 --2- 172.21.15.3:0/3226465458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00d1510 0x7f77d00d2b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b7690 tx=0x7f77c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d081cd24b18ce40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.643+0000 7f77cf400640 1 -- 172.21.15.3:0/3226465458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8036070 con 0x7f77d00d1510 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.643+0000 7f77cf400640 1 -- 172.21.15.3:0/3226465458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8031600 con 0x7f77d00d1510 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.643+0000 7f77cf400640 1 -- 172.21.15.3:0/3226465458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80319d0 con 0x7f77d00d1510 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 -- 172.21.15.3:0/3226465458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00d1510 msgr2=0x7f77d00d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3226465458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00d1510 0x7f77d00d2b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b7690 tx=0x7f77c802a730 comp rx=0 tx=0).stop 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 -- 172.21.15.3:0/3226465458 shutdown_connections 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3226465458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00d1510 0x7f77d00d2b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3226465458 >> v2:172.21.15.70:3300/0 conn(0x7f77d00b6a70 0x7f77d00d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 -- 172.21.15.3:0/3226465458 >> 172.21.15.3:0/3226465458 conn(0x7f77d0072560 msgr2=0x7f77d0072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 -- 172.21.15.3:0/3226465458 shutdown_connections 2024-09-10T07:46:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.644+0000 7f77d6a00640 1 -- 172.21.15.3:0/3226465458 wait complete. 2024-09-10T07:46:27.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.646+0000 7f77d6a00640 1 Processor -- start 2024-09-10T07:46:27.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.646+0000 7f77d6a00640 1 -- start start 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77d00b6a70 0x7f77d01e5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d00d1510 0x7f77d01e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77d015c6b0 con 0x7f77d00b6690 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f77d015c530 con 0x7f77d00d1510 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d6a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f77d00d1940 con 0x7f77d00b6a70 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48966/0 (socket says 172.21.15.3:48966) 2024-09-10T07:46:27.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 -- 172.21.15.3:0/3600083423 learned_addr learned my addr 172.21.15.3:0/3600083423 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77d4a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d00d1510 0x7f77d01e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77c7e00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77d00b6a70 0x7f77d01e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 -- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77d00b6a70 msgr2=0x7f77d01e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77d00b6a70 0x7f77d01e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 -- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d00d1510 msgr2=0x7f77d01e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d00d1510 0x7f77d01e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.647+0000 7f77cfe00640 1 -- 172.21.15.3:0/3600083423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d0157cc0 con 0x7f77d00b6690 2024-09-10T07:46:27.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77cfe00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c400 tx=0x7f77c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c00074e0 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c0007680 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004a00 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d0158130 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.648+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01e6780 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.649+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d00d0fd0 con 0x7f77d00b6690 2024-09-10T07:46:27.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.650+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77c001a020 con 0x7f77d00b6690 2024-09-10T07:46:27.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.651+0000 7f77ccc00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 0x7f77ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:27.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.651+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f77c008a280 con 0x7f77d00b6690 2024-09-10T07:46:27.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.651+0000 7f77c7e00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 0x7f77ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:27.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.652+0000 7f77c7e00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 0x7f77ac06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77d015fb30 tx=0x7f77b8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:27.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.656+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77c0058890 con 0x7f77d00b6690 2024-09-10T07:46:27.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.875+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77d00cea40 con 0x7f77d00b6690 2024-09-10T07:46:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.875+0000 7f77ccc00640 1 -- 172.21.15.3:0/3600083423 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3897 (secure 0 0 0) 0x7f77c005cb90 con 0x7f77d00b6690 2024-09-10T07:46:27.875 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:27.875 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"btime":"2024-09-10T07:46:26:174498+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4287,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T07:46:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-09-10T07:46:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.877+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 msgr2=0x7f77ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.877+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 0x7f77ac06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77d015fb30 tx=0x7f77b8009190 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.877+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 msgr2=0x7f77d0157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.877+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c400 tx=0x7f77c000c8c0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 shutdown_connections 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77ac06be30 0x7f77ac06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d00d1510 0x7f77d01e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77d00b6a70 0x7f77d01e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 --2- 172.21.15.3:0/3600083423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77d00b6690 0x7f77d0157780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 >> 172.21.15.3:0/3600083423 conn(0x7f77d0072560 msgr2=0x7f77d00b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 shutdown_connections 2024-09-10T07:46:27.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:27.878+0000 7f77d6a00640 1 -- 172.21.15.3:0/3600083423 wait complete. 2024-09-10T07:46:27.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:46:28.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.049+0000 7fa4f2a00640 1 Processor -- start 2024-09-10T07:46:28.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.049+0000 7fa4f2a00640 1 -- start start 2024-09-10T07:46:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.050+0000 7fa4f2a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4ec0b5880 0x7fa4ec073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4f2a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec0b5cf0 con 0x7fa4ec15bf70 2024-09-10T07:46:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4f2a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec15f220 con 0x7fa4ec073660 2024-09-10T07:46:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4f2a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec15f500 con 0x7fa4ec0b5880 2024-09-10T07:46:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4ebe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa4ec15bf70 0x7fa4ec15ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42528/0 (socket says 172.21.15.3:42528) 2024-09-10T07:46:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4ebe00640 1 -- 172.21.15.3:0/2239620831 learned_addr learned my addr 172.21.15.3:0/2239620831 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4e3400640 1 --2- 172.21.15.3:0/2239620831 >> v2:172.21.15.70:3300/0 conn(0x7fa4ec0b5880 0x7fa4ec073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493132759 0 0) 0x7fa4ec0b5cf0 con 0x7fa4ec15bf70 2024-09-10T07:46:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.051+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4d0003520 con 0x7fa4ec15bf70 2024-09-10T07:46:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.052+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640065272 0 0) 0x7fa4ec15f220 con 0x7fa4ec073660 2024-09-10T07:46:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.052+0000 7fa4e3400640 1 -- 172.21.15.3:0/2239620831 >> v1:172.21.15.70:6789/0 conn(0x7fa4ec073660 legacy=0x7fa4ec06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.052+0000 7fa4e3400640 1 -- 172.21.15.3:0/2239620831 >> v1:172.21.15.3:6789/0 conn(0x7fa4ec15bf70 legacy=0x7fa4ec15ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.052+0000 7fa4e3400640 1 -- 172.21.15.3:0/2239620831 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec15f7e0 con 0x7fa4ec0b5880 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4e3400640 1 --2- 172.21.15.3:0/2239620831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec073120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40097a0 tx=0x7fa4d402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f435e848300060 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4037070 con 0x7fa4ec0b5880 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d402ac30 con 0x7fa4ec0b5880 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4eb400640 1 -- 172.21.15.3:0/2239620831 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc001670 con 0x7fa4ec0b5880 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2239620831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec0b5880 msgr2=0x7fa4ec073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2239620831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec073120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40097a0 tx=0x7fa4d402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2239620831 shutdown_connections 2024-09-10T07:46:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2239620831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec073120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2239620831 >> 172.21.15.3:0/2239620831 conn(0x7fa4ec0b41e0 msgr2=0x7fa4ec0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2239620831 shutdown_connections 2024-09-10T07:46:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.053+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2239620831 wait complete. 2024-09-10T07:46:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.055+0000 7fa4f2a00640 1 Processor -- start 2024-09-10T07:46:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.055+0000 7fa4f2a00640 1 -- start start 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4ec073660 0x7fa4ec160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec15bf70 0x7fa4ec0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec15c4d0 con 0x7fa4ec0b5880 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec15c350 con 0x7fa4ec15bf70 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f2a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec06cfc0 con 0x7fa4ec073660 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48970/0 (socket says 172.21.15.3:48970) 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4e3400640 1 -- 172.21.15.3:0/2868280294 learned_addr learned my addr 172.21.15.3:0/2868280294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4ec073660 0x7fa4ec160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.056+0000 7fa4f0a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec15bf70 0x7fa4ec0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 -- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4ec073660 msgr2=0x7fa4ec160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4ec073660 0x7fa4ec160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 -- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec15bf70 msgr2=0x7fa4ec0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec15bf70 0x7fa4ec0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 -- 172.21.15.3:0/2868280294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0cd100 con 0x7fa4ec0b5880 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e3400640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098c0 tx=0x7fa4d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4037040 con 0x7fa4ec0b5880 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d4004030 con 0x7fa4ec0b5880 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4004420 con 0x7fa4ec0b5880 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.057+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0cd3a0 con 0x7fa4ec0b5880 2024-09-10T07:46:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.058+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0cd960 con 0x7fa4ec0b5880 2024-09-10T07:46:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.059+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec0cb860 con 0x7fa4ec0b5880 2024-09-10T07:46:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.059+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4d403a040 con 0x7fa4ec0b5880 2024-09-10T07:46:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.061+0000 7fa4e8c00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 0x7fa4c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.061+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa4d40aaff0 con 0x7fa4ec0b5880 2024-09-10T07:46:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.061+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 0x7fa4c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.062+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 0x7fa4c806e320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001600 tx=0x7fa4dc001570 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.065+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4d4042020 con 0x7fa4ec0b5880 2024-09-10T07:46:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.253+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa4ec073100 con 0x7fa4ec0b5880 2024-09-10T07:46:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.254+0000 7fa4e8c00640 1 -- 172.21.15.3:0/2868280294 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fa4d4079720 con 0x7fa4ec0b5880 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:46:28.254 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 msgr2=0x7fa4c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 0x7fa4c806e320 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc001600 tx=0x7fa4dc001570 comp rx=0 tx=0).stop 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 msgr2=0x7fa4ec1568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098c0 tx=0x7fa4d40046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 shutdown_connections 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4c806be60 0x7fa4c806e320 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4ec15bf70 0x7fa4ec0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec0b5880 0x7fa4ec1568d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 --2- 172.21.15.3:0/2868280294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4ec073660 0x7fa4ec160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 >> 172.21.15.3:0/2868280294 conn(0x7fa4ec0b41e0 msgr2=0x7fa4ec0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 shutdown_connections 2024-09-10T07:46:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.256+0000 7fa4f2a00640 1 -- 172.21.15.3:0/2868280294 wait complete. 2024-09-10T07:46:28.271 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:46:28.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:46:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.431+0000 7f44b1e00640 1 Processor -- start 2024-09-10T07:46:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.431+0000 7f44b1e00640 1 -- start start 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ac141310 0x7f44ac141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 0x7f44ac0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ac141fb0 con 0x7f44ac0d4490 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac0d3d50 con 0x7f44ac141310 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac0d4e40 con 0x7f44ac15f0d0 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ac141310 0x7f44ac141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ac141310 0x7f44ac141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48980/0 (socket says 172.21.15.3:48980) 2024-09-10T07:46:28.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 -- 172.21.15.3:0/3367371032 learned_addr learned my addr 172.21.15.3:0/3367371032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1400640 1 --2- 172.21.15.3:0/3367371032 >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 0x7f44ac0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 -- 172.21.15.3:0/3367371032 >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 msgr2=0x7f44ac0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 --2- 172.21.15.3:0/3367371032 >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 0x7f44ac0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 -- 172.21.15.3:0/3367371032 >> v1:172.21.15.3:6789/0 conn(0x7f44ac0d4490 legacy=0x7f44ac15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b0a00640 1 -- 172.21.15.3:0/3367371032 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac15f680 con 0x7f44ac141310 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44b1400640 1 --2- 172.21.15.3:0/3367371032 >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 0x7f44ac0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.433+0000 7f44ab400640 1 -- 172.21.15.3:0/3367371032 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852206276 0 0) 0x7f44ac141fb0 con 0x7f44ac0d4490 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.434+0000 7f44b0a00640 1 --2- 172.21.15.3:0/3367371032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac141a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0631a0 tx=0x7f44a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a94f52768847e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44ab400640 1 -- 172.21.15.3:0/3367371032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0037070 con 0x7f44ac141310 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44ab400640 1 -- 172.21.15.3:0/3367371032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0032440 con 0x7f44ac141310 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44ab400640 1 -- 172.21.15.3:0/3367371032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0032810 con 0x7f44ac141310 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 -- 172.21.15.3:0/3367371032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 msgr2=0x7f44ac141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 --2- 172.21.15.3:0/3367371032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac141a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0631a0 tx=0x7f44a002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 -- 172.21.15.3:0/3367371032 shutdown_connections 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 --2- 172.21.15.3:0/3367371032 >> v2:172.21.15.70:3300/0 conn(0x7f44ac15f0d0 0x7f44ac0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 --2- 172.21.15.3:0/3367371032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac141a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 -- 172.21.15.3:0/3367371032 >> 172.21.15.3:0/3367371032 conn(0x7f44ac062a50 msgr2=0x7f44ac072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 -- 172.21.15.3:0/3367371032 shutdown_connections 2024-09-10T07:46:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.435+0000 7f44b1e00640 1 -- 172.21.15.3:0/3367371032 wait complete. 2024-09-10T07:46:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.437+0000 7f44b1e00640 1 Processor -- start 2024-09-10T07:46:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.437+0000 7f44b1e00640 1 -- start start 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 0x7f44ac13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 0x7f44ac157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0b5c50 con 0x7f44ac141310 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac15f960 con 0x7f44ac0d4490 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44b1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0b5f30 con 0x7f44ac15f0d0 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48992/0 (socket says 172.21.15.3:48992) 2024-09-10T07:46:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.438+0000 7f44abe00640 1 -- 172.21.15.3:0/1432797726 learned_addr learned my addr 172.21.15.3:0/1432797726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b0a00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 0x7f44ac13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 -- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 msgr2=0x7f44ac157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b1400640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 0x7f44ac157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 0x7f44ac157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 -- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 msgr2=0x7f44ac13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 0x7f44ac13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 -- 172.21.15.3:0/1432797726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac15bdb0 con 0x7f44ac141310 2024-09-10T07:46:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b0a00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 0x7f44ac13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b1400640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 0x7f44ac157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44abe00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f449800c4e0 tx=0x7f449800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498017070 con 0x7f44ac141310 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44980035f0 con 0x7f44ac141310 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac156830 con 0x7f44ac141310 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980039e0 con 0x7f44ac141310 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.439+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac157070 con 0x7f44ac141310 2024-09-10T07:46:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.440+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac1416f0 con 0x7f44ac141310 2024-09-10T07:46:28.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.441+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f449801a070 con 0x7f44ac141310 2024-09-10T07:46:28.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.442+0000 7f44a8c00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 0x7f448c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:28.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.442+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f4498089e60 con 0x7f44ac141310 2024-09-10T07:46:28.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.442+0000 7f44b0a00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 0x7f448c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:28.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.443+0000 7f44b0a00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 0x7f448c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44a0034000 tx=0x7f44a002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.446+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44ac156830 con 0x7f44ac141310 2024-09-10T07:46:28.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:28.629+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f44ac1449d0 con 0x7f44ac141310 2024-09-10T07:46:29.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.190+0000 7f44a8c00640 1 -- 172.21.15.3:0/1432797726 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f4498058470 con 0x7f44ac141310 2024-09-10T07:46:29.190 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:46:29.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.194+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 msgr2=0x7f448c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.194+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 0x7f448c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44a0034000 tx=0x7f44a002e000 comp rx=0 tx=0).stop 2024-09-10T07:46:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.194+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 msgr2=0x7f44ac15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.194+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f449800c4e0 tx=0x7f449800c9a0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 shutdown_connections 2024-09-10T07:46:29.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f448c06be30 0x7f448c06e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ac15f0d0 0x7f44ac157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ac141310 0x7f44ac15b1f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 --2- 172.21.15.3:0/1432797726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ac0d4490 0x7f44ac13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.195+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 >> 172.21.15.3:0/1432797726 conn(0x7f44ac062a50 msgr2=0x7f44ac0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:29.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.196+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 shutdown_connections 2024-09-10T07:46:29.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.196+0000 7f44b1e00640 1 -- 172.21.15.3:0/1432797726 wait complete. 2024-09-10T07:46:29.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:46:29.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.383+0000 7f6af7600640 1 Processor -- start 2024-09-10T07:46:29.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.384+0000 7f6af7600640 1 -- start start 2024-09-10T07:46:29.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6af7600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6af01451d0 0x7f6af006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6af7600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af00cd900 con 0x7f6af0145730 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6af7600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af00cdbe0 con 0x7f6af00cf270 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6af7600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6af006d7c0 con 0x7f6af01451d0 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6aefe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6af0145730 0x7f6af00b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42538/0 (socket says 172.21.15.3:42538) 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6aefe00640 1 -- 172.21.15.3:0/3740614393 learned_addr learned my addr 172.21.15.3:0/3740614393 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.385+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3740614393 >> v2:172.21.15.70:3300/0 conn(0x7f6af01451d0 0x7f6af006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:29.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.386+0000 7f6af4c00640 1 -- 172.21.15.3:0/3740614393 >> v1:172.21.15.70:6789/0 conn(0x7f6af00cf270 legacy=0x7f6af00cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.386+0000 7f6af4c00640 1 -- 172.21.15.3:0/3740614393 >> v1:172.21.15.3:6789/0 conn(0x7f6af0145730 legacy=0x7f6af00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.386+0000 7f6af4c00640 1 -- 172.21.15.3:0/3740614393 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af006da10 con 0x7f6af01451d0 2024-09-10T07:46:29.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.387+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3740614393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af01451d0 0x7f6af006cfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6af006d940 tx=0x7f6ae402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e277bce2c01fef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:29.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.387+0000 7f6aef400640 1 -- 172.21.15.3:0/3740614393 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4037070 con 0x7f6af01451d0 2024-09-10T07:46:29.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.387+0000 7f6aef400640 1 -- 172.21.15.3:0/3740614393 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4032440 con 0x7f6af01451d0 2024-09-10T07:46:29.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.387+0000 7f6aef400640 1 -- 172.21.15.3:0/3740614393 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4032810 con 0x7f6af01451d0 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 -- 172.21.15.3:0/3740614393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af01451d0 msgr2=0x7f6af006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 --2- 172.21.15.3:0/3740614393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af01451d0 0x7f6af006cfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6af006d940 tx=0x7f6ae402a180 comp rx=0 tx=0).stop 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 -- 172.21.15.3:0/3740614393 shutdown_connections 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 --2- 172.21.15.3:0/3740614393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af01451d0 0x7f6af006cfc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 -- 172.21.15.3:0/3740614393 >> 172.21.15.3:0/3740614393 conn(0x7f6af00b5cb0 msgr2=0x7f6af006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 -- 172.21.15.3:0/3740614393 shutdown_connections 2024-09-10T07:46:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.388+0000 7f6af7600640 1 -- 172.21.15.3:0/3740614393 wait complete. 2024-09-10T07:46:29.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.390+0000 7f6af7600640 1 Processor -- start 2024-09-10T07:46:29.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.390+0000 7f6af7600640 1 -- start start 2024-09-10T07:46:29.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.390+0000 7f6af7600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 0x7f6af015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af7600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af7600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af0145730 0x7f6af00acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af7600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6af006e0b0 con 0x7f6af01451d0 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af7600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6af006df30 con 0x7f6af0145730 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af7600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6af006e230 con 0x7f6af00cf270 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49000/0 (socket says 172.21.15.3:49000) 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 -- 172.21.15.3:0/3308109426 learned_addr learned my addr 172.21.15.3:0/3308109426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 0x7f6af015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af5600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af0145730 0x7f6af00acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:29.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 -- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 msgr2=0x7f6af015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 0x7f6af015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 -- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af0145730 msgr2=0x7f6af00acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:29.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af0145730 0x7f6af00acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6aefe00640 1 -- 172.21.15.3:0/3308109426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af00abd50 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.391+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 0x7f6af015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6aefe00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c480 tx=0x7f6adc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc017070 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6adc005ce0 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0072f0 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af00ac030 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.392+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af00ac5f0 con 0x7f6af01451d0 2024-09-10T07:46:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.393+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af006cfc0 con 0x7f6af01451d0 2024-09-10T07:46:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.394+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6adc01a070 con 0x7f6af01451d0 2024-09-10T07:46:29.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.396+0000 7f6aecc00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 0x7f6ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:29.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.396+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f6adc089eb0 con 0x7f6af01451d0 2024-09-10T07:46:29.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.396+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 0x7f6ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.397+0000 7f6af4c00640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 0x7f6ad006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4034000 tx=0x7f6ae402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:29.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.401+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6adc058480 con 0x7f6af01451d0 2024-09-10T07:46:29.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:29.586+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6af0071b00 con 0x7f6af01451d0 2024-09-10T07:46:30.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.195+0000 7f6aecc00640 1 -- 172.21.15.3:0/3308109426 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f6adc05c780 con 0x7f6af01451d0 2024-09-10T07:46:30.195 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:46:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.199+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 msgr2=0x7f6ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.199+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 0x7f6ad006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4034000 tx=0x7f6ae402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.199+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 msgr2=0x7f6af00adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.199+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c480 tx=0x7f6adc00c940 comp rx=0 tx=0).stop 2024-09-10T07:46:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 shutdown_connections 2024-09-10T07:46:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ad006be30 0x7f6ad006e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af0145730 0x7f6af00acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6af01451d0 0x7f6af00adc80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 --2- 172.21.15.3:0/3308109426 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6af00cf270 0x7f6af015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.201+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 >> 172.21.15.3:0/3308109426 conn(0x7f6af00b5cb0 msgr2=0x7f6af00b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.202+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 shutdown_connections 2024-09-10T07:46:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.202+0000 7f6af7600640 1 -- 172.21.15.3:0/3308109426 wait complete. 2024-09-10T07:46:30.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:46:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.400+0000 7fe666200640 1 Processor -- start 2024-09-10T07:46:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.400+0000 7fe666200640 1 -- start start 2024-09-10T07:46:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe666200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe666200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6600cfe90 0x7fe6600d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe666200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6600d4f00 con 0x7fe660161b70 2024-09-10T07:46:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe666200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe660161f50 con 0x7fe6600cfe90 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe666200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d3de0 con 0x7fe6600d45a0 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:43284/0 (socket says 172.21.15.3:43284) 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe65fe00640 1 -- 172.21.15.3:0/453488674 learned_addr learned my addr 172.21.15.3:0/453488674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe664e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe660161b70 0x7fe6600d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50576/0 (socket says 172.21.15.3:50576) 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.401+0000 7fe665800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6600cfe90 0x7fe6600d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.402+0000 7fe665800640 1 -- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 msgr2=0x7fe6600d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.402+0000 7fe665800640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.402+0000 7fe665800640 1 -- 172.21.15.3:0/453488674 >> v1:172.21.15.70:6789/0 conn(0x7fe660161b70 legacy=0x7fe6600d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.402+0000 7fe665800640 1 -- 172.21.15.3:0/453488674 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600d40c0 con 0x7fe6600cfe90 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.402+0000 7fe65fe00640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.403+0000 7fe665800640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6600cfe90 0x7fe6600d0320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a23406d585061b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.403+0000 7fe65f400640 1 -- 172.21.15.3:0/453488674 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658007740 con 0x7fe6600cfe90 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.403+0000 7fe65f400640 1 -- 172.21.15.3:0/453488674 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580078e0 con 0x7fe6600cfe90 2024-09-10T07:46:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.403+0000 7fe65f400640 1 -- 172.21.15.3:0/453488674 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658007cb0 con 0x7fe6600cfe90 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 -- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6600cfe90 msgr2=0x7fe6600d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6600cfe90 0x7fe6600d0320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65802a660 comp rx=0 tx=0).stop 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 -- 172.21.15.3:0/453488674 shutdown_connections 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6600cfe90 0x7fe6600d0320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 --2- 172.21.15.3:0/453488674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600d45a0 0x7fe6600d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 -- 172.21.15.3:0/453488674 >> 172.21.15.3:0/453488674 conn(0x7fe660062630 msgr2=0x7fe660062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 -- 172.21.15.3:0/453488674 shutdown_connections 2024-09-10T07:46:30.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.404+0000 7fe666200640 1 -- 172.21.15.3:0/453488674 wait complete. 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.405+0000 7fe666200640 1 Processor -- start 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 -- start start 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600cfe90 0x7fe6601515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe660161b70 0x7fe66015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600c9fc0 con 0x7fe6600d45a0 2024-09-10T07:46:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600cf970 con 0x7fe660161b70 2024-09-10T07:46:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.406+0000 7fe666200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d3a30 con 0x7fe6600cfe90 2024-09-10T07:46:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49008/0 (socket says 172.21.15.3:49008) 2024-09-10T07:46:30.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 -- 172.21.15.3:0/254171320 learned_addr learned my addr 172.21.15.3:0/254171320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe665800640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe660161b70 0x7fe66015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe664e00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600cfe90 0x7fe6601515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 -- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600cfe90 msgr2=0x7fe6601515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600cfe90 0x7fe6601515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 -- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe660161b70 msgr2=0x7fe66015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe660161b70 0x7fe66015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.407+0000 7fe65fe00640 1 -- 172.21.15.3:0/254171320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe660159760 con 0x7fe6600d45a0 2024-09-10T07:46:30.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe65fe00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9d0 tx=0x7fe64c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c017070 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64c00ca60 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00ce10 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe660159a40 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.408+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600b4f50 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.409+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6600cf2d0 con 0x7fe6600d45a0 2024-09-10T07:46:30.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.410+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe64c01a070 con 0x7fe6600d45a0 2024-09-10T07:46:30.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.411+0000 7fe65cc00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 0x7fe64006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:30.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.411+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fe64c08b0a0 con 0x7fe6600d45a0 2024-09-10T07:46:30.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.411+0000 7fe664e00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 0x7fe64006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:30.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.412+0000 7fe664e00640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 0x7fe64006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe654000f60 tx=0x7fe654000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:30.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.416+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe64c059450 con 0x7fe6600d45a0 2024-09-10T07:46:30.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:30.638+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe66015d910 con 0x7fe6600d45a0 2024-09-10T07:46:31.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.225+0000 7fe65cc00640 1 -- 172.21.15.3:0/254171320 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe64c05d750 con 0x7fe6600d45a0 2024-09-10T07:46:31.225 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T07:46:31.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.232+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 msgr2=0x7fe64006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.232+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 0x7fe64006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe654000f60 tx=0x7fe654000ed0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.232+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 msgr2=0x7fe660150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.232+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00e9d0 tx=0x7fe64c00ee90 comp rx=0 tx=0).stop 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 shutdown_connections 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe64006be30 0x7fe64006e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe660161b70 0x7fe66015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6600d45a0 0x7fe660150f40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 --2- 172.21.15.3:0/254171320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6600cfe90 0x7fe6601515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 >> 172.21.15.3:0/254171320 conn(0x7fe660062630 msgr2=0x7fe6600d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 shutdown_connections 2024-09-10T07:46:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.233+0000 7fe666200640 1 -- 172.21.15.3:0/254171320 wait complete. 2024-09-10T07:46:31.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.418+0000 7fb707600640 1 Processor -- start 2024-09-10T07:46:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.418+0000 7fb707600640 1 -- start start 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb707600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb700161dd0 0x7fb70006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb707600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 0x7fb7000b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb707600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70006fe10 con 0x7fb700062630 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb707600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000b47d0 con 0x7fb700161dd0 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb707600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb70006c870 con 0x7fb7000b5ea0 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb6ffe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 0x7fb7000b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb704c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb700161dd0 0x7fb70006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb705600640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb700062630 0x7fb700062aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50586/0 (socket says 172.21.15.3:50586) 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb6ffe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 0x7fb7000b62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:43298/0 (socket says 172.21.15.3:43298) 2024-09-10T07:46:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb705600640 1 -- 172.21.15.3:0/827104775 learned_addr learned my addr 172.21.15.3:0/827104775 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb704c00640 1 -- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 msgr2=0x7fb7000b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb704c00640 1 --2- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 0x7fb7000b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb704c00640 1 -- 172.21.15.3:0/827104775 >> v1:172.21.15.70:6789/0 conn(0x7fb700062630 legacy=0x7fb700062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.420+0000 7fb704c00640 1 -- 172.21.15.3:0/827104775 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70006cb50 con 0x7fb700161dd0 2024-09-10T07:46:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb704c00640 1 --2- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700161dd0 0x7fb70006f8d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40097a0 tx=0x7fb6f402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6569f00606742401 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb6ff400640 1 -- 172.21.15.3:0/827104775 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4037070 con 0x7fb700161dd0 2024-09-10T07:46:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb6ff400640 1 -- 172.21.15.3:0/827104775 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4032440 con 0x7fb700161dd0 2024-09-10T07:46:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb6ff400640 1 -- 172.21.15.3:0/827104775 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4032810 con 0x7fb700161dd0 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb707600640 1 -- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700161dd0 msgr2=0x7fb70006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.422+0000 7fb707600640 1 --2- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700161dd0 0x7fb70006f8d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40097a0 tx=0x7fb6f402a180 comp rx=0 tx=0).stop 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 -- 172.21.15.3:0/827104775 shutdown_connections 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 --2- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7000b5ea0 0x7fb7000b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 --2- 172.21.15.3:0/827104775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700161dd0 0x7fb70006f8d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 -- 172.21.15.3:0/827104775 >> 172.21.15.3:0/827104775 conn(0x7fb7000b3540 msgr2=0x7fb7000b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 -- 172.21.15.3:0/827104775 shutdown_connections 2024-09-10T07:46:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.423+0000 7fb707600640 1 -- 172.21.15.3:0/827104775 wait complete. 2024-09-10T07:46:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.424+0000 7fb707600640 1 Processor -- start 2024-09-10T07:46:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.424+0000 7fb707600640 1 -- start start 2024-09-10T07:46:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700062630 0x7fb70015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 0x7fb70014fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7000b6820 con 0x7fb7000b5ea0 2024-09-10T07:46:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7000b06c0 con 0x7fb700062630 2024-09-10T07:46:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.425+0000 7fb707600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb70006f3a0 con 0x7fb700161dd0 2024-09-10T07:46:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49016/0 (socket says 172.21.15.3:49016) 2024-09-10T07:46:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 -- 172.21.15.3:0/1484484412 learned_addr learned my addr 172.21.15.3:0/1484484412 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb705600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 0x7fb70014fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb704c00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700062630 0x7fb70015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 -- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 msgr2=0x7fb70014fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 0x7fb70014fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 -- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700062630 msgr2=0x7fb70015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700062630 0x7fb70015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 -- 172.21.15.3:0/1484484412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb700156d60 con 0x7fb7000b5ea0 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb705600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 0x7fb70014fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.426+0000 7fb6ffe00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e9b0 tx=0x7fb6ec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.427+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec017070 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.427+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec00c9a0 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.427+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00cd90 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.427+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb700155cc0 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.427+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb700156280 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.428+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7000b43f0 con 0x7fb7000b5ea0 2024-09-10T07:46:31.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.429+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6ec01a070 con 0x7fb7000b5ea0 2024-09-10T07:46:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.430+0000 7fb6fcc00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 0x7fb6e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.430+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb6ec08af30 con 0x7fb7000b5ea0 2024-09-10T07:46:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.430+0000 7fb704c00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 0x7fb6e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.431+0000 7fb704c00640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 0x7fb6e006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4034000 tx=0x7fb6f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.435+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6ec059840 con 0x7fb7000b5ea0 2024-09-10T07:46:31.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.626+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7000af5f0 con 0x7fb7000b5ea0 2024-09-10T07:46:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.627+0000 7fb6fcc00640 1 -- 172.21.15.3:0/1484484412 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16610 (secure 0 0 0) 0x7fb6ec059140 con 0x7fb7000b5ea0 2024-09-10T07:46:31.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:31.627 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:30.640326+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 msgr2=0x7fb6e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 0x7fb6e006e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4034000 tx=0x7fb6f402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 msgr2=0x7fb700155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e9b0 tx=0x7fb6ec00ee70 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 shutdown_connections 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb6e006be30 0x7fb6e006e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb700161dd0 0x7fb70014fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7000b5ea0 0x7fb700155550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 --2- 172.21.15.3:0/1484484412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb700062630 0x7fb70015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 >> 172.21.15.3:0/1484484412 conn(0x7fb7000b3540 msgr2=0x7fb7000afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 shutdown_connections 2024-09-10T07:46:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.630+0000 7fb707600640 1 -- 172.21.15.3:0/1484484412 wait complete. 2024-09-10T07:46:31.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.803+0000 7f6d59a00640 1 Processor -- start 2024-09-10T07:46:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.803+0000 7f6d59a00640 1 -- start start 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d59a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d540b6690 0x7f6d540cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d59a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d540cff10 con 0x7f6d540d16e0 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d59a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d540d1b10 con 0x7f6d540b6a70 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d59a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6d540cf540 con 0x7f6d540b6690 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d53400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d540d16e0 0x7f6d540d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42550/0 (socket says 172.21.15.3:42550) 2024-09-10T07:46:31.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.804+0000 7f6d53400640 1 -- 172.21.15.3:0/2955474254 learned_addr learned my addr 172.21.15.3:0/2955474254 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d52a00640 1 --2- 172.21.15.3:0/2955474254 >> v2:172.21.15.70:3300/0 conn(0x7f6d540b6690 0x7f6d540cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352367052 0 0) 0x7f6d540cff10 con 0x7f6d540d16e0 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d38003520 con 0x7f6d540d16e0 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923750379 0 0) 0x7f6d40001110 con 0x7f6d540b6a70 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d540cff10 con 0x7f6d540b6a70 2024-09-10T07:46:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d52a00640 1 -- 172.21.15.3:0/2955474254 >> v1:172.21.15.70:6789/0 conn(0x7f6d540b6a70 legacy=0x7f6d540d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d52a00640 1 -- 172.21.15.3:0/2955474254 >> v1:172.21.15.3:6789/0 conn(0x7f6d540d16e0 legacy=0x7f6d540d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.805+0000 7f6d52a00640 1 -- 172.21.15.3:0/2955474254 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540d1d90 con 0x7f6d540b6690 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.806+0000 7f6d52a00640 1 --2- 172.21.15.3:0/2955474254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d540cf9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097a0 tx=0x7f6d4402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53034fdc30d40557 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.806+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44037070 con 0x7f6d540b6690 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.806+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d44032440 con 0x7f6d540b6690 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.806+0000 7f6d51600640 1 -- 172.21.15.3:0/2955474254 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44032810 con 0x7f6d540b6690 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 -- 172.21.15.3:0/2955474254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 msgr2=0x7f6d540cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2955474254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d540cf9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097a0 tx=0x7f6d4402a230 comp rx=0 tx=0).stop 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 -- 172.21.15.3:0/2955474254 shutdown_connections 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2955474254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d540cf9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 -- 172.21.15.3:0/2955474254 >> 172.21.15.3:0/2955474254 conn(0x7f6d54072560 msgr2=0x7f6d54072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 -- 172.21.15.3:0/2955474254 shutdown_connections 2024-09-10T07:46:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.807+0000 7f6d59a00640 1 -- 172.21.15.3:0/2955474254 wait complete. 2024-09-10T07:46:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.809+0000 7f6d59a00640 1 Processor -- start 2024-09-10T07:46:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.809+0000 7f6d59a00640 1 -- start start 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d54156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d540d16e0 0x7f6d540ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540b6f00 con 0x7f6d540b6a70 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540cf1d0 con 0x7f6d540b6690 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d59a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5415c530 con 0x7f6d540d16e0 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49030/0 (socket says 172.21.15.3:49030) 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 -- 172.21.15.3:0/2956531473 learned_addr learned my addr 172.21.15.3:0/2956531473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d54156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d53400640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d540d16e0 0x7f6d540ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 -- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d540d16e0 msgr2=0x7f6d540ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d540d16e0 0x7f6d540ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 -- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 msgr2=0x7f6d54156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d54156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.810+0000 7f6d52000640 1 -- 172.21.15.3:0/2956531473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540cb760 con 0x7f6d540b6a70 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d52000640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000c700 tx=0x7f6d4000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400052d0 con 0x7f6d540b6a70 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d40005470 con 0x7f6d540b6a70 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d540cba40 con 0x7f6d540b6a70 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000a860 con 0x7f6d540b6a70 2024-09-10T07:46:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.811+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d540cc2e0 con 0x7f6d540b6a70 2024-09-10T07:46:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.812+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d54069140 con 0x7f6d540b6a70 2024-09-10T07:46:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.813+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d4000aa00 con 0x7f6d540b6a70 2024-09-10T07:46:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.814+0000 7f6d4aa00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 0x7f6d3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:31.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.814+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6d4008aa60 con 0x7f6d540b6a70 2024-09-10T07:46:31.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.814+0000 7f6d52a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 0x7f6d3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:31.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.815+0000 7f6d52a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 0x7f6d3406e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d44034000 tx=0x7f6d4402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:31.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:31.818+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d540d1b10 con 0x7f6d540b6a70 2024-09-10T07:46:32.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.004+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d540b98b0 con 0x7f6d540b6a70 2024-09-10T07:46:32.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.005+0000 7f6d4aa00640 1 -- 172.21.15.3:0/2956531473 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16610 (secure 0 0 0) 0x7f6d4005f020 con 0x7f6d540b6a70 2024-09-10T07:46:32.005 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:32.005 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:30.640326+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:32.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 msgr2=0x7f6d3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 0x7f6d3406e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d44034000 tx=0x7f6d4402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:32.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 msgr2=0x7f6d54155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000c700 tx=0x7f6d4000cbc0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 shutdown_connections 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6d3406be30 0x7f6d3406e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6d540d16e0 0x7f6d540ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d540b6a70 0x7f6d54155290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 --2- 172.21.15.3:0/2956531473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d540b6690 0x7f6d54156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.007+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 >> 172.21.15.3:0/2956531473 conn(0x7f6d54072560 msgr2=0x7f6d540b9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.008+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 shutdown_connections 2024-09-10T07:46:32.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.008+0000 7f6d59a00640 1 -- 172.21.15.3:0/2956531473 wait complete. 2024-09-10T07:46:32.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:46:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.182+0000 7f765d000640 1 Processor -- start 2024-09-10T07:46:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.182+0000 7f765d000640 1 -- start start 2024-09-10T07:46:32.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f765d000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f765d000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f765d000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76581620f0 con 0x7f76580b76c0 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f765d000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7658072140 con 0x7f76580cf6a0 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f765d000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76580d4200 con 0x7f7658161d10 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f7656800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f7655e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f7655400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f76580b76c0 0x7f76580cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50596/0 (socket says 172.21.15.3:50596) 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655400640 1 -- 172.21.15.3:0/1228984066 learned_addr learned my addr 172.21.15.3:0/1228984066 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.183+0000 7f7656800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49042/0 (socket says 172.21.15.3:49042) 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:43336/0 (socket says 172.21.15.3:43336) 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655e00640 1 -- 172.21.15.3:0/1228984066 >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 msgr2=0x7f76580d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655e00640 1 --2- 172.21.15.3:0/1228984066 >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655e00640 1 -- 172.21.15.3:0/1228984066 >> v1:172.21.15.70:6789/0 conn(0x7f76580b76c0 legacy=0x7f76580cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7655e00640 1 -- 172.21.15.3:0/1228984066 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7658072390 con 0x7f7658161d10 2024-09-10T07:46:32.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7654a00640 1 -- 172.21.15.3:0/1228984066 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547647192 0 0) 0x7f76581620f0 con 0x7f76580b76c0 2024-09-10T07:46:32.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.184+0000 7f7656800640 1 --2- 172.21.15.3:0/1228984066 >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.185+0000 7f7655e00640 1 --2- 172.21.15.3:0/1228984066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76480097a0 tx=0x7f764802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb5c80fabfef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.185+0000 7f7654a00640 1 -- 172.21.15.3:0/1228984066 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648037070 con 0x7f7658161d10 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.185+0000 7f7654a00640 1 -- 172.21.15.3:0/1228984066 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764802ae40 con 0x7f7658161d10 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.185+0000 7f7654a00640 1 -- 172.21.15.3:0/1228984066 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648032660 con 0x7f7658161d10 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 -- 172.21.15.3:0/1228984066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 msgr2=0x7f76580d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 --2- 172.21.15.3:0/1228984066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76480097a0 tx=0x7f764802a0f0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 -- 172.21.15.3:0/1228984066 shutdown_connections 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 --2- 172.21.15.3:0/1228984066 >> v2:172.21.15.3:3300/0 conn(0x7f76580cf6a0 0x7f76580d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 --2- 172.21.15.3:0/1228984066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580d3b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 -- 172.21.15.3:0/1228984066 >> 172.21.15.3:0/1228984066 conn(0x7f76580d4730 msgr2=0x7f76580d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 -- 172.21.15.3:0/1228984066 shutdown_connections 2024-09-10T07:46:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.186+0000 7f765d000640 1 -- 172.21.15.3:0/1228984066 wait complete. 2024-09-10T07:46:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.188+0000 7f765d000640 1 Processor -- start 2024-09-10T07:46:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.188+0000 7f765d000640 1 -- start start 2024-09-10T07:46:32.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b76c0 0x7f7658157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76580b7b00 con 0x7f76580cf6a0 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7658072950 con 0x7f76580b76c0 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f765d000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7658072e70 con 0x7f7658161d10 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49050/0 (socket says 172.21.15.3:49050) 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 -- 172.21.15.3:0/1302317332 learned_addr learned my addr 172.21.15.3:0/1302317332 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7656800640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655e00640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b76c0 0x7f7658157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 -- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 msgr2=0x7f76580cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 -- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b76c0 msgr2=0x7f7658157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b76c0 0x7f7658157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.189+0000 7f7655400640 1 -- 172.21.15.3:0/1302317332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7658158560 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f7655400640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f764400c4a0 tx=0x7f764400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644018070 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76440074e0 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644004a20 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7658158840 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.190+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7658158e00 con 0x7f76580cf6a0 2024-09-10T07:46:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.191+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76580d3b20 con 0x7f76580cf6a0 2024-09-10T07:46:32.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.192+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f764401b070 con 0x7f76580cf6a0 2024-09-10T07:46:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.193+0000 7f764e000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 0x7f763806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.193+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f764408a1a0 con 0x7f76580cf6a0 2024-09-10T07:46:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.193+0000 7f7655e00640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 0x7f763806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.194+0000 7f7655e00640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 0x7f763806e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7648034000 tx=0x7f764802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:32.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.197+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7644058b40 con 0x7f76580cf6a0 2024-09-10T07:46:32.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:32.421+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f76580cf030 con 0x7f76580cf6a0 2024-09-10T07:46:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.406+0000 7f764e000640 1 -- 172.21.15.3:0/1302317332 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f7644058440 con 0x7f76580cf6a0 2024-09-10T07:46:33.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 msgr2=0x7f763806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 0x7f763806e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7648034000 tx=0x7f764802e000 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 msgr2=0x7f76580cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f764400c4a0 tx=0x7f764400c960 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 shutdown_connections 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f763806be30 0x7f763806e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7658161d10 0x7f76580cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76580cf6a0 0x7f76580cd4a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 --2- 172.21.15.3:0/1302317332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76580b76c0 0x7f7658157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 >> 172.21.15.3:0/1302317332 conn(0x7f76580d4730 msgr2=0x7f76580b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 shutdown_connections 2024-09-10T07:46:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.410+0000 7f765d000640 1 -- 172.21.15.3:0/1302317332 wait complete. 2024-09-10T07:46:33.429 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:46:33.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:46:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.593+0000 7f2c49c00640 1 Processor -- start 2024-09-10T07:46:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.593+0000 7f2c49c00640 1 -- start start 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c44145dc0 0x7f2c4406e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c49c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c44062d40 0x7f2c4406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c49c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4406e6c0 con 0x7f2c440cecb0 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c49c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4406e940 con 0x7f2c44062d40 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c49c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4406ec20 con 0x7f2c44145dc0 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c43400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c44062d40 0x7f2c4406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c43400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c44062d40 0x7f2c4406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49058/0 (socket says 172.21.15.3:49058) 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c43400640 1 -- 172.21.15.3:0/3687829147 learned_addr learned my addr 172.21.15.3:0/3687829147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.594+0000 7f2c42a00640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c44145dc0 0x7f2c4406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.595+0000 7f2c43400640 1 -- 172.21.15.3:0/3687829147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c44145dc0 msgr2=0x7f2c4406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:33.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.595+0000 7f2c43400640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c44145dc0 0x7f2c4406e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.595+0000 7f2c43400640 1 -- 172.21.15.3:0/3687829147 >> v1:172.21.15.70:6789/0 conn(0x7f2c440cecb0 legacy=0x7f2c44062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.595+0000 7f2c43400640 1 -- 172.21.15.3:0/3687829147 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4406ef00 con 0x7f2c44062d40 2024-09-10T07:46:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c43400640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c4406f9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009e50 tx=0x7f2c3c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6bd06ae0d3c10f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c42000640 1 -- 172.21.15.3:0/3687829147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c036070 con 0x7f2c44062d40 2024-09-10T07:46:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c42000640 1 -- 172.21.15.3:0/3687829147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3c031600 con 0x7f2c44062d40 2024-09-10T07:46:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c42000640 1 -- 172.21.15.3:0/3687829147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0319d0 con 0x7f2c44062d40 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c49c00640 1 -- 172.21.15.3:0/3687829147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 msgr2=0x7f2c4406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.596+0000 7f2c49c00640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c4406f9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009e50 tx=0x7f2c3c02a680 comp rx=0 tx=0).stop 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 -- 172.21.15.3:0/3687829147 shutdown_connections 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c4406f9e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 --2- 172.21.15.3:0/3687829147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c44145dc0 0x7f2c4406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 -- 172.21.15.3:0/3687829147 >> 172.21.15.3:0/3687829147 conn(0x7f2c44073060 msgr2=0x7f2c4406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 -- 172.21.15.3:0/3687829147 shutdown_connections 2024-09-10T07:46:33.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.597+0000 7f2c49c00640 1 -- 172.21.15.3:0/3687829147 wait complete. 2024-09-10T07:46:33.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.598+0000 7f2c49c00640 1 Processor -- start 2024-09-10T07:46:33.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.598+0000 7f2c49c00640 1 -- start start 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c440cecb0 0x7f2c441e9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c44145dc0 0x7f2c4415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4415cb10 con 0x7f2c44062d40 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4406f4c0 con 0x7f2c44145dc0 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c49c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c441461a0 con 0x7f2c440cecb0 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c42a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c42a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49060/0 (socket says 172.21.15.3:49060) 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.599+0000 7f2c42a00640 1 -- 172.21.15.3:0/208230315 learned_addr learned my addr 172.21.15.3:0/208230315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c43400640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c44145dc0 0x7f2c4415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c3be00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c440cecb0 0x7f2c441e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 -- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c440cecb0 msgr2=0x7f2c441e9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c440cecb0 0x7f2c441e9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 -- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c44145dc0 msgr2=0x7f2c4415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c44145dc0 0x7f2c4415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 -- 172.21.15.3:0/208230315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441e9b70 con 0x7f2c44062d40 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c42a00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400d5f0 tx=0x7f2c3400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340125f0 con 0x7f2c44062d40 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34019070 con 0x7f2c44062d40 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340129d0 con 0x7f2c44062d40 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441e5950 con 0x7f2c44062d40 2024-09-10T07:46:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.600+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441e8b50 con 0x7f2c44062d40 2024-09-10T07:46:33.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.601+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c440d0be0 con 0x7f2c44062d40 2024-09-10T07:46:33.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.602+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c34012b70 con 0x7f2c44062d40 2024-09-10T07:46:33.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.603+0000 7f2c3aa00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:33.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.603+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2c3408b390 con 0x7f2c44062d40 2024-09-10T07:46:33.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.603+0000 7f2c3be00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:33.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.604+0000 7f2c3be00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 0x7f2c2006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c441e6710 tx=0x7f2c2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.608+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c440d0be0 con 0x7f2c44062d40 2024-09-10T07:46:33.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:33.828+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2c4406e180 con 0x7f2c44062d40 2024-09-10T07:46:34.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.413+0000 7f2c3aa00640 1 -- 172.21.15.3:0/208230315 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f2c34059c20 con 0x7f2c44062d40 2024-09-10T07:46:34.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 msgr2=0x7f2c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 0x7f2c2006e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c441e6710 tx=0x7f2c2c009040 comp rx=0 tx=0).stop 2024-09-10T07:46:34.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 msgr2=0x7f2c44142c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400d5f0 tx=0x7f2c3400dab0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 shutdown_connections 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c44145dc0 0x7f2c4415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c440cecb0 0x7f2c441e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 --2- 172.21.15.3:0/208230315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c44062d40 0x7f2c44142c90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 >> 172.21.15.3:0/208230315 conn(0x7f2c44073060 msgr2=0x7f2c440cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.416+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 shutdown_connections 2024-09-10T07:46:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.417+0000 7f2c49c00640 1 -- 172.21.15.3:0/208230315 wait complete. 2024-09-10T07:46:34.435 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:46:34.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.597+0000 7f8758000640 1 Processor -- start 2024-09-10T07:46:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.597+0000 7f8758000640 1 -- start start 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8758000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6690 0x7f87500cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8758000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87500d16e0 0x7f87500d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8758000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87500cff10 con 0x7f87500b6a70 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8758000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f87500d1b70 con 0x7f87500d16e0 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8758000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87500cf540 con 0x7f87500b6690 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8756000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87500d16e0 0x7f87500d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8756000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87500d16e0 0x7f87500d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47800/0 (socket says 172.21.15.3:47800) 2024-09-10T07:46:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8756000640 1 -- 172.21.15.3:0/1946673032 learned_addr learned my addr 172.21.15.3:0/1946673032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.598+0000 7f8755600640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6690 0x7f87500cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.599+0000 7f8756000640 1 -- 172.21.15.3:0/1946673032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6690 msgr2=0x7f87500cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.599+0000 7f8756000640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6690 0x7f87500cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.599+0000 7f8756000640 1 -- 172.21.15.3:0/1946673032 >> v1:172.21.15.70:6789/0 conn(0x7f87500b6a70 legacy=0x7f87500d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.599+0000 7f8756000640 1 -- 172.21.15.3:0/1946673032 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500d1d90 con 0x7f87500d16e0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8756000640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87500d2b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f874802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9a16e5dc43a8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f874fe00640 1 -- 172.21.15.3:0/1946673032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748036070 con 0x7f87500d16e0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f874fe00640 1 -- 172.21.15.3:0/1946673032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8748031600 con 0x7f87500d16e0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f874fe00640 1 -- 172.21.15.3:0/1946673032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87480319d0 con 0x7f87500d16e0 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 -- 172.21.15.3:0/1946673032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 msgr2=0x7f87500d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87500d2b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f874802a710 comp rx=0 tx=0).stop 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 -- 172.21.15.3:0/1946673032 shutdown_connections 2024-09-10T07:46:34.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87500d2b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 --2- 172.21.15.3:0/1946673032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6690 0x7f87500cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.600+0000 7f8758000640 1 -- 172.21.15.3:0/1946673032 >> 172.21.15.3:0/1946673032 conn(0x7f8750072560 msgr2=0x7f8750072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.601+0000 7f8758000640 1 -- 172.21.15.3:0/1946673032 shutdown_connections 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.601+0000 7f8758000640 1 -- 172.21.15.3:0/1946673032 wait complete. 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.602+0000 7f8758000640 1 Processor -- start 2024-09-10T07:46:34.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.602+0000 7f8758000640 1 -- start start 2024-09-10T07:46:34.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 0x7f8750161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6a70 0x7f875014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87500b6ed0 con 0x7f87500d16e0 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f87500cf1d0 con 0x7f87500b6690 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8758000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f875015c530 con 0x7f87500b6a70 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8756000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:34.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8756000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47814/0 (socket says 172.21.15.3:47814) 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.603+0000 7f8756000640 1 -- 172.21.15.3:0/2617057566 learned_addr learned my addr 172.21.15.3:0/2617057566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 -- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6a70 msgr2=0x7f875014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6a70 0x7f875014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 -- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 msgr2=0x7f8750161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8755600640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 0x7f8750161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 0x7f8750161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 -- 172.21.15.3:0/2617057566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500ce110 con 0x7f87500d16e0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8755600640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 0x7f8750161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f8756000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f8748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748036050 con 0x7f87500d16e0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8748042070 con 0x7f87500d16e0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.604+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748009510 con 0x7f87500d16e0 2024-09-10T07:46:34.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.605+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87500ccb20 con 0x7f87500d16e0 2024-09-10T07:46:34.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.605+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8750160a10 con 0x7f87500d16e0 2024-09-10T07:46:34.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.606+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87500cf9d0 con 0x7f87500d16e0 2024-09-10T07:46:34.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.607+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8748007bf0 con 0x7f87500d16e0 2024-09-10T07:46:34.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.607+0000 7f874d600640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:34.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.607+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f87480abc40 con 0x7f87500d16e0 2024-09-10T07:46:34.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.608+0000 7f8755600640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:34.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.608+0000 7f8755600640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 0x7f873006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f875014e740 tx=0x7f8744008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.612+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f874807a5e0 con 0x7f87500d16e0 2024-09-10T07:46:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.834+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87500ce5c0 con 0x7f87500d16e0 2024-09-10T07:46:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.834+0000 7f874d600640 1 -- 172.21.15.3:0/2617057566 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f8748079ee0 con 0x7f87500d16e0 2024-09-10T07:46:34.834 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:34.834 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:46:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:46:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 msgr2=0x7f873006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 0x7f873006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f875014e740 tx=0x7f8744008040 comp rx=0 tx=0).stop 2024-09-10T07:46:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 msgr2=0x7f87501f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f8748004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 shutdown_connections 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87500d16e0 0x7f87501f4600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87500b6a70 0x7f875014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 --2- 172.21.15.3:0/2617057566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87500b6690 0x7f8750161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 >> 172.21.15.3:0/2617057566 conn(0x7f8750072560 msgr2=0x7f875015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 shutdown_connections 2024-09-10T07:46:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:34.836+0000 7f8758000640 1 -- 172.21.15.3:0/2617057566 wait complete. 2024-09-10T07:46:34.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:35.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.008+0000 7f6923200640 1 Processor -- start 2024-09-10T07:46:35.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.008+0000 7f6923200640 1 -- start start 2024-09-10T07:46:35.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.009+0000 7f6923200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 0x7f691c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.009+0000 7f6923200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.009+0000 7f6923200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691c0b4250 con 0x7f691c15b4f0 2024-09-10T07:46:35.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.009+0000 7f6923200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f691c0b44f0 con 0x7f691c15b110 2024-09-10T07:46:35.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.009+0000 7f6923200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f691c0ccd80 con 0x7f691c0d08f0 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51134/0 (socket says 172.21.15.3:51134) 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 -- 172.21.15.3:0/2055493689 learned_addr learned my addr 172.21.15.3:0/2055493689 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f691be00640 1 --2- 172.21.15.3:0/2055493689 >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 0x7f691c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 -- 172.21.15.3:0/2055493689 >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 msgr2=0x7f691c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f691aa00640 1 -- 172.21.15.3:0/2055493689 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226401694 0 0) 0x7f691c0b4250 con 0x7f691c15b4f0 2024-09-10T07:46:35.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 --2- 172.21.15.3:0/2055493689 >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 0x7f691c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 -- 172.21.15.3:0/2055493689 >> v1:172.21.15.70:6789/0 conn(0x7f691c15b4f0 legacy=0x7f691c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f6921200640 1 -- 172.21.15.3:0/2055493689 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c0cd060 con 0x7f691c0d08f0 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.010+0000 7f691be00640 1 --2- 172.21.15.3:0/2055493689 >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 0x7f691c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.011+0000 7f6921200640 1 --2- 172.21.15.3:0/2055493689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f691c0b4670 tx=0x7f691402c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c35f27a12acc306e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f691aa00640 1 -- 172.21.15.3:0/2055493689 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914009600 con 0x7f691c0d08f0 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f691aa00640 1 -- 172.21.15.3:0/2055493689 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914037070 con 0x7f691c0d08f0 2024-09-10T07:46:35.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f691aa00640 1 -- 172.21.15.3:0/2055493689 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69140099c0 con 0x7f691c0d08f0 2024-09-10T07:46:35.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 -- 172.21.15.3:0/2055493689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 msgr2=0x7f691c0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 --2- 172.21.15.3:0/2055493689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f691c0b4670 tx=0x7f691402c910 comp rx=0 tx=0).stop 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 -- 172.21.15.3:0/2055493689 shutdown_connections 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 --2- 172.21.15.3:0/2055493689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c0d08f0 0x7f691c0cdb40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 --2- 172.21.15.3:0/2055493689 >> v2:172.21.15.70:3300/0 conn(0x7f691c15b110 0x7f691c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 -- 172.21.15.3:0/2055493689 >> 172.21.15.3:0/2055493689 conn(0x7f691c06d000 msgr2=0x7f691c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 -- 172.21.15.3:0/2055493689 shutdown_connections 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.012+0000 7f6923200640 1 -- 172.21.15.3:0/2055493689 wait complete. 2024-09-10T07:46:35.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.014+0000 7f6923200640 1 Processor -- start 2024-09-10T07:46:35.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.014+0000 7f6923200640 1 -- start start 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f691c0d08f0 0x7f691c072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c15b110 0x7f691c15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f691c073d10 con 0x7f691c15b4f0 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f691c0cd620 con 0x7f691c0d08f0 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6923200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f691c073e90 con 0x7f691c15b110 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6921200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6921200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47824/0 (socket says 172.21.15.3:47824) 2024-09-10T07:46:35.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f6921200640 1 -- 172.21.15.3:0/1889541793 learned_addr learned my addr 172.21.15.3:0/1889541793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f691be00640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f691c0d08f0 0x7f691c072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.015+0000 7f691b400640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c15b110 0x7f691c15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 -- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c15b110 msgr2=0x7f691c15bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c15b110 0x7f691c15bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 -- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f691c0d08f0 msgr2=0x7f691c072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f691c0d08f0 0x7f691c072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 -- 172.21.15.3:0/1889541793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c149630 con 0x7f691c15b4f0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6921200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6914034040 tx=0x7f6914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691400b720 con 0x7f691c15b4f0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914003bb0 con 0x7f691c15b4f0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914009e30 con 0x7f691c15b4f0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c149910 con 0x7f691c15b4f0 2024-09-10T07:46:35.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.016+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c149ed0 con 0x7f691c15b4f0 2024-09-10T07:46:35.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.017+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c0b3d10 con 0x7f691c15b4f0 2024-09-10T07:46:35.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.018+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6914031070 con 0x7f691c15b4f0 2024-09-10T07:46:35.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.020+0000 7f6913e00640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 0x7f68fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.020+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69140abb60 con 0x7f691c15b4f0 2024-09-10T07:46:35.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.020+0000 7f691be00640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 0x7f68fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.020+0000 7f691be00640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 0x7f68fc06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f691c0ce220 tx=0x7f690c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.024+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6914079e30 con 0x7f691c15b4f0 2024-09-10T07:46:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.208+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f691c0ce250 con 0x7f691c15b4f0 2024-09-10T07:46:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.209+0000 7f6913e00640 1 -- 172.21.15.3:0/1889541793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f691407e130 con 0x7f691c15b4f0 2024-09-10T07:46:35.209 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:35.209 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.211+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 msgr2=0x7f68fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.211+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 0x7f68fc06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f691c0ce220 tx=0x7f690c009190 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.211+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 msgr2=0x7f691c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.211+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6914034040 tx=0x7f6914004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 shutdown_connections 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f68fc06be30 0x7f68fc06e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f691c15b4f0 0x7f691c0ae8a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f691c15b110 0x7f691c15bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 --2- 172.21.15.3:0/1889541793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f691c0d08f0 0x7f691c072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 >> 172.21.15.3:0/1889541793 conn(0x7f691c06d000 msgr2=0x7f691c0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 shutdown_connections 2024-09-10T07:46:35.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.212+0000 7f6923200640 1 -- 172.21.15.3:0/1889541793 wait complete. 2024-09-10T07:46:35.228 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:46:35.228 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.383+0000 7fdf36600640 1 Processor -- start 2024-09-10T07:46:35.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.383+0000 7fdf36600640 1 -- start start 2024-09-10T07:46:35.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.384+0000 7fdf36600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf30071f30 0x7fdf3015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.384+0000 7fdf36600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3015d040 0x7fdf300af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.384+0000 7fdf36600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf30156f80 con 0x7fdf300d0bb0 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.384+0000 7fdf36600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdf3015d780 con 0x7fdf3015d040 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.384+0000 7fdf36600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf30071450 con 0x7fdf30071f30 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3015d040 0x7fdf300af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf3015d040 0x7fdf300af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47832/0 (socket says 172.21.15.3:47832) 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 -- 172.21.15.3:0/2385530282 learned_addr learned my addr 172.21.15.3:0/2385530282 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf27e00640 1 --2- 172.21.15.3:0/2385530282 >> v2:172.21.15.70:3300/0 conn(0x7fdf30071f30 0x7fdf3015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 -- 172.21.15.3:0/2385530282 >> v2:172.21.15.70:3300/0 conn(0x7fdf30071f30 msgr2=0x7fdf3015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 --2- 172.21.15.3:0/2385530282 >> v2:172.21.15.70:3300/0 conn(0x7fdf30071f30 0x7fdf3015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 -- 172.21.15.3:0/2385530282 >> v1:172.21.15.70:6789/0 conn(0x7fdf300d0bb0 legacy=0x7fdf300d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.385+0000 7fdf2fe00640 1 -- 172.21.15.3:0/2385530282 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3015da60 con 0x7fdf3015d040 2024-09-10T07:46:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.386+0000 7fdf2fe00640 1 --2- 172.21.15.3:0/2385530282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3015d040 0x7fdf300af4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf30072b30 tx=0x7fdf2802a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16514a9e0caec437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.386+0000 7fdf2ea00640 1 -- 172.21.15.3:0/2385530282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28036070 con 0x7fdf3015d040 2024-09-10T07:46:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.386+0000 7fdf2ea00640 1 -- 172.21.15.3:0/2385530282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf28031600 con 0x7fdf3015d040 2024-09-10T07:46:35.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.386+0000 7fdf2ea00640 1 -- 172.21.15.3:0/2385530282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf280319d0 con 0x7fdf3015d040 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 -- 172.21.15.3:0/2385530282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3015d040 msgr2=0x7fdf300af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 --2- 172.21.15.3:0/2385530282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3015d040 0x7fdf300af4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf30072b30 tx=0x7fdf2802a740 comp rx=0 tx=0).stop 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 -- 172.21.15.3:0/2385530282 shutdown_connections 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 --2- 172.21.15.3:0/2385530282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf3015d040 0x7fdf300af4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 --2- 172.21.15.3:0/2385530282 >> v2:172.21.15.70:3300/0 conn(0x7fdf30071f30 0x7fdf3015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 -- 172.21.15.3:0/2385530282 >> 172.21.15.3:0/2385530282 conn(0x7fdf3006bf40 msgr2=0x7fdf3006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 -- 172.21.15.3:0/2385530282 shutdown_connections 2024-09-10T07:46:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.387+0000 7fdf36600640 1 -- 172.21.15.3:0/2385530282 wait complete. 2024-09-10T07:46:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.389+0000 7fdf36600640 1 Processor -- start 2024-09-10T07:46:35.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.389+0000 7fdf36600640 1 -- start start 2024-09-10T07:46:35.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf30071f30 0x7fdf300aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf3015d040 0x7fdf30148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3015d450 con 0x7fdf300d0bb0 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30071a10 con 0x7fdf30071f30 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf36600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf300d0100 con 0x7fdf3015d040 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47848/0 (socket says 172.21.15.3:47848) 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 -- 172.21.15.3:0/5451702 learned_addr learned my addr 172.21.15.3:0/5451702 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf2f400640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf30071f30 0x7fdf300aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf2fe00640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf3015d040 0x7fdf30148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 -- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf3015d040 msgr2=0x7fdf30148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf3015d040 0x7fdf30148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 -- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf30071f30 msgr2=0x7fdf300aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf30071f30 0x7fdf300aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.390+0000 7fdf27e00640 1 -- 172.21.15.3:0/5451702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf300ab760 con 0x7fdf300d0bb0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf27e00640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800d5d0 tx=0x7fdf1800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180125f0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf18019070 con 0x7fdf300d0bb0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180129f0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf300abaa0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.391+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf300ac2e0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.392+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30069190 con 0x7fdf300d0bb0 2024-09-10T07:46:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.393+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf18012b90 con 0x7fdf300d0bb0 2024-09-10T07:46:35.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.394+0000 7fdf27400640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 0x7fdf0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.394+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdf1808b1b0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.394+0000 7fdf2f400640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 0x7fdf0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.395+0000 7fdf2f400640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 0x7fdf0c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf20001780 tx=0x7fdf20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.398+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf300abaa0 con 0x7fdf300d0bb0 2024-09-10T07:46:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.578+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf300ac4e0 con 0x7fdf0c06be30 2024-09-10T07:46:35.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.586+0000 7fdf27400640 1 -- 172.21.15.3:0/5451702 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250090 (secure 0 0 0) 0x7fdf300ac4e0 con 0x7fdf0c06be30 2024-09-10T07:46:35.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:35.590 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:35.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 msgr2=0x7fdf0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 0x7fdf0c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf20001780 tx=0x7fdf20009040 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 msgr2=0x7fdf3015b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800d5d0 tx=0x7fdf1800da90 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 shutdown_connections 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdf0c06be30 0x7fdf0c06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf3015d040 0x7fdf30148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf300d0bb0 0x7fdf3015b970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 --2- 172.21.15.3:0/5451702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf30071f30 0x7fdf300aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 >> 172.21.15.3:0/5451702 conn(0x7fdf3006bf40 msgr2=0x7fdf300ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 shutdown_connections 2024-09-10T07:46:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.590+0000 7fdf36600640 1 -- 172.21.15.3:0/5451702 wait complete. 2024-09-10T07:46:35.606 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T07:46:35.490007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592584,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235668,"kb_used_data":2784,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739052,"statfs":{"total":773094113280,"available":772852789248,"internally_reserved":0,"allocated":2850816,"data_stored":1720922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247657+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:31.247657+0000","last_peered":"2024-09-10T07:46:31.247657+0000","last_clean":"2024-09-10T07:46:31.247657+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:31.247657+0000","last_undegraded":"2024-09-10T07:46:31.247657+0000","last_fullsized":"2024-09-10T07:46:31.247657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203393+0000","last_change":"2024-09-10T07:46:30.203393+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203393+0000","last_undegraded":"2024-09-10T07:46:30.203393+0000","last_fullsized":"2024-09-10T07:46:30.203393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200935+0000","last_change":"2024-09-10T07:46:30.200935+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200935+0000","last_undegraded":"2024-09-10T07:46:30.200935+0000","last_fullsized":"2024-09-10T07:46:30.200935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223177+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:31.223177+0000","last_peered":"2024-09-10T07:46:31.223177+0000","last_clean":"2024-09-10T07:46:31.223177+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:31.223177+0000","last_undegraded":"2024-09-10T07:46:31.223177+0000","last_fullsized":"2024-09-10T07:46:31.223177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.253444+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:31.253444+0000","last_peered":"2024-09-10T07:46:31.253444+0000","last_clean":"2024-09-10T07:46:31.253444+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:31.253444+0000","last_undegraded":"2024-09-10T07:46:31.253444+0000","last_fullsized":"2024-09-10T07:46:31.253444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.232052+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:31.232052+0000","last_peered":"2024-09-10T07:46:31.232052+0000","last_clean":"2024-09-10T07:46:31.232052+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:31.232052+0000","last_undegraded":"2024-09-10T07:46:31.232052+0000","last_fullsized":"2024-09-10T07:46:31.232052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241195+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:31.241195+0000","last_peered":"2024-09-10T07:46:31.241195+0000","last_clean":"2024-09-10T07:46:31.241195+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:31.241195+0000","last_undegraded":"2024-09-10T07:46:31.241195+0000","last_fullsized":"2024-09-10T07:46:31.241195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202948+0000","last_change":"2024-09-10T07:46:30.202948+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202948+0000","last_undegraded":"2024-09-10T07:46:30.202948+0000","last_fullsized":"2024-09-10T07:46:30.202948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.238669+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:31.238669+0000","last_peered":"2024-09-10T07:46:31.238669+0000","last_clean":"2024-09-10T07:46:31.238669+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:31.238669+0000","last_undegraded":"2024-09-10T07:46:31.238669+0000","last_fullsized":"2024-09-10T07:46:31.238669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244605+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:31.244605+0000","last_peered":"2024-09-10T07:46:31.244605+0000","last_clean":"2024-09-10T07:46:31.244605+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:31.244605+0000","last_undegraded":"2024-09-10T07:46:31.244605+0000","last_fullsized":"2024-09-10T07:46:31.244605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246200+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:31.246200+0000","last_peered":"2024-09-10T07:46:31.246200+0000","last_clean":"2024-09-10T07:46:31.246200+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:31.246200+0000","last_undegraded":"2024-09-10T07:46:31.246200+0000","last_fullsized":"2024-09-10T07:46:31.246200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248044+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:31.248044+0000","last_peered":"2024-09-10T07:46:31.248044+0000","last_clean":"2024-09-10T07:46:31.248044+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:31.248044+0000","last_undegraded":"2024-09-10T07:46:31.248044+0000","last_fullsized":"2024-09-10T07:46:31.248044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209709+0000","last_change":"2024-09-10T07:46:30.209709+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209709+0000","last_undegraded":"2024-09-10T07:46:30.209709+0000","last_fullsized":"2024-09-10T07:46:30.209709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.212144+0000","last_change":"2024-09-10T07:46:30.212144+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.212144+0000","last_undegraded":"2024-09-10T07:46:30.212144+0000","last_fullsized":"2024-09-10T07:46:30.212144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203055+0000","last_change":"2024-09-10T07:46:30.203055+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203055+0000","last_undegraded":"2024-09-10T07:46:30.203055+0000","last_fullsized":"2024-09-10T07:46:30.203055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234849+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:31.234849+0000","last_peered":"2024-09-10T07:46:31.234849+0000","last_clean":"2024-09-10T07:46:31.234849+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:31.234849+0000","last_undegraded":"2024-09-10T07:46:31.234849+0000","last_fullsized":"2024-09-10T07:46:31.234849+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236363+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:31.236363+0000","last_peered":"2024-09-10T07:46:31.236363+0000","last_clean":"2024-09-10T07:46:31.236363+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:31.236363+0000","last_undegraded":"2024-09-10T07:46:31.236363+0000","last_fullsized":"2024-09-10T07:46:31.236363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209207+0000","last_change":"2024-09-10T07:46:30.209207+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209207+0000","last_undegraded":"2024-09-10T07:46:30.209207+0000","last_fullsized":"2024-09-10T07:46:30.209207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.218669+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:31.218669+0000","last_peered":"2024-09-10T07:46:31.218669+0000","last_clean":"2024-09-10T07:46:31.218669+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:31.218669+0000","last_undegraded":"2024-09-10T07:46:31.218669+0000","last_fullsized":"2024-09-10T07:46:31.218669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246121+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:31.246121+0000","last_peered":"2024-09-10T07:46:31.246121+0000","last_clean":"2024-09-10T07:46:31.246121+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:31.246121+0000","last_undegraded":"2024-09-10T07:46:31.246121+0000","last_fullsized":"2024-09-10T07:46:31.246121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236449+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:31.236449+0000","last_peered":"2024-09-10T07:46:31.236449+0000","last_clean":"2024-09-10T07:46:31.236449+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:31.236449+0000","last_undegraded":"2024-09-10T07:46:31.236449+0000","last_fullsized":"2024-09-10T07:46:31.236449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201489+0000","last_change":"2024-09-10T07:46:30.201489+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201489+0000","last_undegraded":"2024-09-10T07:46:30.201489+0000","last_fullsized":"2024-09-10T07:46:30.201489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245315+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:31.245315+0000","last_peered":"2024-09-10T07:46:31.245315+0000","last_clean":"2024-09-10T07:46:31.245315+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:31.245315+0000","last_undegraded":"2024-09-10T07:46:31.245315+0000","last_fullsized":"2024-09-10T07:46:31.245315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204563+0000","last_change":"2024-09-10T07:46:30.204563+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204563+0000","last_undegraded":"2024-09-10T07:46:30.204563+0000","last_fullsized":"2024-09-10T07:46:30.204563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235466+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:31.235466+0000","last_peered":"2024-09-10T07:46:31.235466+0000","last_clean":"2024-09-10T07:46:31.235466+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:31.235466+0000","last_undegraded":"2024-09-10T07:46:31.235466+0000","last_fullsized":"2024-09-10T07:46:31.235466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208574+0000","last_change":"2024-09-10T07:46:30.208574+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208574+0000","last_undegraded":"2024-09-10T07:46:30.208574+0000","last_fullsized":"2024-09-10T07:46:30.208574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.210647+0000","last_change":"2024-09-10T07:46:30.210647+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.210647+0000","last_undegraded":"2024-09-10T07:46:30.210647+0000","last_fullsized":"2024-09-10T07:46:30.210647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201619+0000","last_change":"2024-09-10T07:46:30.201619+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201619+0000","last_undegraded":"2024-09-10T07:46:30.201619+0000","last_fullsized":"2024-09-10T07:46:30.201619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245190+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:31.245190+0000","last_peered":"2024-09-10T07:46:31.245190+0000","last_clean":"2024-09-10T07:46:31.245190+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:31.245190+0000","last_undegraded":"2024-09-10T07:46:31.245190+0000","last_fullsized":"2024-09-10T07:46:31.245190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204987+0000","last_change":"2024-09-10T07:46:30.204987+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204987+0000","last_undegraded":"2024-09-10T07:46:30.204987+0000","last_fullsized":"2024-09-10T07:46:30.204987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236511+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:31.236511+0000","last_peered":"2024-09-10T07:46:31.236511+0000","last_clean":"2024-09-10T07:46:31.236511+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:31.236511+0000","last_undegraded":"2024-09-10T07:46:31.236511+0000","last_fullsized":"2024-09-10T07:46:31.236511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248657+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:31.248657+0000","last_peered":"2024-09-10T07:46:31.248657+0000","last_clean":"2024-09-10T07:46:31.248657+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:31.248657+0000","last_undegraded":"2024-09-10T07:46:31.248657+0000","last_fullsized":"2024-09-10T07:46:31.248657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223295+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:30.223295+0000","last_peered":"2024-09-10T07:46:30.223295+0000","last_clean":"2024-09-10T07:46:30.223295+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:30.223295+0000","last_undegraded":"2024-09-10T07:46:30.223295+0000","last_fullsized":"2024-09-10T07:46:30.223295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.256925+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:31.256925+0000","last_peered":"2024-09-10T07:46:31.256925+0000","last_clean":"2024-09-10T07:46:31.256925+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:31.256925+0000","last_undegraded":"2024-09-10T07:46:31.256925+0000","last_fullsized":"2024-09-10T07:46:31.256925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232303+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:30.232303+0000","last_peered":"2024-09-10T07:46:30.232303+0000","last_clean":"2024-09-10T07:46:30.232303+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:30.232303+0000","last_undegraded":"2024-09-10T07:46:30.232303+0000","last_fullsized":"2024-09-10T07:46:30.232303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200915+0000","last_change":"2024-09-10T07:46:30.200915+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200915+0000","last_undegraded":"2024-09-10T07:46:30.200915+0000","last_fullsized":"2024-09-10T07:46:30.200915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212725+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:31.212725+0000","last_peered":"2024-09-10T07:46:31.212725+0000","last_clean":"2024-09-10T07:46:31.212725+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:31.212725+0000","last_undegraded":"2024-09-10T07:46:31.212725+0000","last_fullsized":"2024-09-10T07:46:31.212725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223415+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:31.223415+0000","last_peered":"2024-09-10T07:46:31.223415+0000","last_clean":"2024-09-10T07:46:31.223415+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:31.223415+0000","last_undegraded":"2024-09-10T07:46:31.223415+0000","last_fullsized":"2024-09-10T07:46:31.223415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235488+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:31.235488+0000","last_peered":"2024-09-10T07:46:31.235488+0000","last_clean":"2024-09-10T07:46:31.235488+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:31.235488+0000","last_undegraded":"2024-09-10T07:46:31.235488+0000","last_fullsized":"2024-09-10T07:46:31.235488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.228068+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:31.228068+0000","last_peered":"2024-09-10T07:46:31.228068+0000","last_clean":"2024-09-10T07:46:31.228068+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:31.228068+0000","last_undegraded":"2024-09-10T07:46:31.228068+0000","last_fullsized":"2024-09-10T07:46:31.228068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202517+0000","last_change":"2024-09-10T07:46:30.202517+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202517+0000","last_undegraded":"2024-09-10T07:46:30.202517+0000","last_fullsized":"2024-09-10T07:46:30.202517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.226640+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:30.226640+0000","last_peered":"2024-09-10T07:46:30.226640+0000","last_clean":"2024-09-10T07:46:30.226640+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:30.226640+0000","last_undegraded":"2024-09-10T07:46:30.226640+0000","last_fullsized":"2024-09-10T07:46:30.226640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235372+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:31.235372+0000","last_peered":"2024-09-10T07:46:31.235372+0000","last_clean":"2024-09-10T07:46:31.235372+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:31.235372+0000","last_undegraded":"2024-09-10T07:46:31.235372+0000","last_fullsized":"2024-09-10T07:46:31.235372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258003+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:31.258003+0000","last_peered":"2024-09-10T07:46:31.258003+0000","last_clean":"2024-09-10T07:46:31.258003+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:31.258003+0000","last_undegraded":"2024-09-10T07:46:31.258003+0000","last_fullsized":"2024-09-10T07:46:31.258003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.240265+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:31.240265+0000","last_peered":"2024-09-10T07:46:31.240265+0000","last_clean":"2024-09-10T07:46:31.240265+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:31.240265+0000","last_undegraded":"2024-09-10T07:46:31.240265+0000","last_fullsized":"2024-09-10T07:46:31.240265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.235636+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:30.235636+0000","last_peered":"2024-09-10T07:46:30.235636+0000","last_clean":"2024-09-10T07:46:30.235636+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:30.235636+0000","last_undegraded":"2024-09-10T07:46:30.235636+0000","last_fullsized":"2024-09-10T07:46:30.235636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208560+0000","last_change":"2024-09-10T07:46:30.208560+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208560+0000","last_undegraded":"2024-09-10T07:46:30.208560+0000","last_fullsized":"2024-09-10T07:46:30.208560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214230+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:31.214230+0000","last_peered":"2024-09-10T07:46:31.214230+0000","last_clean":"2024-09-10T07:46:31.214230+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:31.214230+0000","last_undegraded":"2024-09-10T07:46:31.214230+0000","last_fullsized":"2024-09-10T07:46:31.214230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235477+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:31.235477+0000","last_peered":"2024-09-10T07:46:31.235477+0000","last_clean":"2024-09-10T07:46:31.235477+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:31.235477+0000","last_undegraded":"2024-09-10T07:46:31.235477+0000","last_fullsized":"2024-09-10T07:46:31.235477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.222561+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:31.222561+0000","last_peered":"2024-09-10T07:46:31.222561+0000","last_clean":"2024-09-10T07:46:31.222561+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:31.222561+0000","last_undegraded":"2024-09-10T07:46:31.222561+0000","last_fullsized":"2024-09-10T07:46:31.222561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248841+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:31.248841+0000","last_peered":"2024-09-10T07:46:31.248841+0000","last_clean":"2024-09-10T07:46:31.248841+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:31.248841+0000","last_undegraded":"2024-09-10T07:46:31.248841+0000","last_fullsized":"2024-09-10T07:46:31.248841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234830+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:31.234830+0000","last_peered":"2024-09-10T07:46:31.234830+0000","last_clean":"2024-09-10T07:46:31.234830+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:31.234830+0000","last_undegraded":"2024-09-10T07:46:31.234830+0000","last_fullsized":"2024-09-10T07:46:31.234830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257933+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:31.257933+0000","last_peered":"2024-09-10T07:46:31.257933+0000","last_clean":"2024-09-10T07:46:31.257933+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:31.257933+0000","last_undegraded":"2024-09-10T07:46:31.257933+0000","last_fullsized":"2024-09-10T07:46:31.257933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237018+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:31.237018+0000","last_peered":"2024-09-10T07:46:31.237018+0000","last_clean":"2024-09-10T07:46:31.237018+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:31.237018+0000","last_undegraded":"2024-09-10T07:46:31.237018+0000","last_fullsized":"2024-09-10T07:46:31.237018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247784+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:31.247784+0000","last_peered":"2024-09-10T07:46:31.247784+0000","last_clean":"2024-09-10T07:46:31.247784+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:31.247784+0000","last_undegraded":"2024-09-10T07:46:31.247784+0000","last_fullsized":"2024-09-10T07:46:31.247784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.219503+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:30.219503+0000","last_peered":"2024-09-10T07:46:30.219503+0000","last_clean":"2024-09-10T07:46:30.219503+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:30.219503+0000","last_undegraded":"2024-09-10T07:46:30.219503+0000","last_fullsized":"2024-09-10T07:46:30.219503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241918+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:31.241918+0000","last_peered":"2024-09-10T07:46:31.241918+0000","last_clean":"2024-09-10T07:46:31.241918+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:31.241918+0000","last_undegraded":"2024-09-10T07:46:31.241918+0000","last_fullsized":"2024-09-10T07:46:31.241918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232317+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:30.232317+0000","last_peered":"2024-09-10T07:46:30.232317+0000","last_clean":"2024-09-10T07:46:30.232317+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:30.232317+0000","last_undegraded":"2024-09-10T07:46:30.232317+0000","last_fullsized":"2024-09-10T07:46:30.232317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235419+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:31.235419+0000","last_peered":"2024-09-10T07:46:31.235419+0000","last_clean":"2024-09-10T07:46:31.235419+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:31.235419+0000","last_undegraded":"2024-09-10T07:46:31.235419+0000","last_fullsized":"2024-09-10T07:46:31.235419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223016+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:31.223016+0000","last_peered":"2024-09-10T07:46:31.223016+0000","last_clean":"2024-09-10T07:46:31.223016+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:31.223016+0000","last_undegraded":"2024-09-10T07:46:31.223016+0000","last_fullsized":"2024-09-10T07:46:31.223016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.213999+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:31.213999+0000","last_peered":"2024-09-10T07:46:31.213999+0000","last_clean":"2024-09-10T07:46:31.213999+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:31.213999+0000","last_undegraded":"2024-09-10T07:46:31.213999+0000","last_fullsized":"2024-09-10T07:46:31.213999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212930+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:31.212930+0000","last_peered":"2024-09-10T07:46:31.212930+0000","last_clean":"2024-09-10T07:46:31.212930+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:31.212930+0000","last_undegraded":"2024-09-10T07:46:31.212930+0000","last_fullsized":"2024-09-10T07:46:31.212930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206969+0000","last_change":"2024-09-10T07:46:30.206969+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206969+0000","last_undegraded":"2024-09-10T07:46:30.206969+0000","last_fullsized":"2024-09-10T07:46:30.206969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222953+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:30.222953+0000","last_peered":"2024-09-10T07:46:30.222953+0000","last_clean":"2024-09-10T07:46:30.222953+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:30.222953+0000","last_undegraded":"2024-09-10T07:46:30.222953+0000","last_fullsized":"2024-09-10T07:46:30.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202494+0000","last_change":"2024-09-10T07:46:30.202494+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202494+0000","last_undegraded":"2024-09-10T07:46:30.202494+0000","last_fullsized":"2024-09-10T07:46:30.202494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214242+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:31.214242+0000","last_peered":"2024-09-10T07:46:31.214242+0000","last_clean":"2024-09-10T07:46:31.214242+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:31.214242+0000","last_undegraded":"2024-09-10T07:46:31.214242+0000","last_fullsized":"2024-09-10T07:46:31.214242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247133+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:31.247133+0000","last_peered":"2024-09-10T07:46:31.247133+0000","last_clean":"2024-09-10T07:46:31.247133+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:31.247133+0000","last_undegraded":"2024-09-10T07:46:31.247133+0000","last_fullsized":"2024-09-10T07:46:31.247133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222035+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:30.222035+0000","last_peered":"2024-09-10T07:46:30.222035+0000","last_clean":"2024-09-10T07:46:30.222035+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:30.222035+0000","last_undegraded":"2024-09-10T07:46:30.222035+0000","last_fullsized":"2024-09-10T07:46:30.222035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237055+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:31.237055+0000","last_peered":"2024-09-10T07:46:31.237055+0000","last_clean":"2024-09-10T07:46:31.237055+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:31.237055+0000","last_undegraded":"2024-09-10T07:46:31.237055+0000","last_fullsized":"2024-09-10T07:46:31.237055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:34.661912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214679+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:31.214679+0000","last_peered":"2024-09-10T07:46:31.214679+0000","last_clean":"2024-09-10T07:46:31.214679+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:31.214679+0000","last_undegraded":"2024-09-10T07:46:31.214679+0000","last_fullsized":"2024-09-10T07:46:31.214679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.207029+0000","last_change":"2024-09-10T07:46:30.207029+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.207029+0000","last_undegraded":"2024-09-10T07:46:30.207029+0000","last_fullsized":"2024-09-10T07:46:30.207029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258157+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:31.258157+0000","last_peered":"2024-09-10T07:46:31.258157+0000","last_clean":"2024-09-10T07:46:31.258157+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:31.258157+0000","last_undegraded":"2024-09-10T07:46:31.258157+0000","last_fullsized":"2024-09-10T07:46:31.258157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.217759+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:31.217759+0000","last_peered":"2024-09-10T07:46:31.217759+0000","last_clean":"2024-09-10T07:46:31.217759+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:31.217759+0000","last_undegraded":"2024-09-10T07:46:31.217759+0000","last_fullsized":"2024-09-10T07:46:31.217759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.227462+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:31.227462+0000","last_peered":"2024-09-10T07:46:31.227462+0000","last_clean":"2024-09-10T07:46:31.227462+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:31.227462+0000","last_undegraded":"2024-09-10T07:46:31.227462+0000","last_fullsized":"2024-09-10T07:46:31.227462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223401+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:30.223401+0000","last_peered":"2024-09-10T07:46:30.223401+0000","last_clean":"2024-09-10T07:46:30.223401+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:30.223401+0000","last_undegraded":"2024-09-10T07:46:30.223401+0000","last_fullsized":"2024-09-10T07:46:30.223401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:33.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206604+0000","last_change":"2024-09-10T07:46:30.206604+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206604+0000","last_undegraded":"2024-09-10T07:46:30.206604+0000","last_fullsized":"2024-09-10T07:46:30.206604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257046+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:31.257046+0000","last_peered":"2024-09-10T07:46:31.257046+0000","last_clean":"2024-09-10T07:46:31.257046+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:31.257046+0000","last_undegraded":"2024-09-10T07:46:31.257046+0000","last_fullsized":"2024-09-10T07:46:31.257046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244686+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:31.244686+0000","last_peered":"2024-09-10T07:46:31.244686+0000","last_clean":"2024-09-10T07:46:31.244686+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:31.244686+0000","last_undegraded":"2024-09-10T07:46:31.244686+0000","last_fullsized":"2024-09-10T07:46:31.244686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235456+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:31.235456+0000","last_peered":"2024-09-10T07:46:31.235456+0000","last_clean":"2024-09-10T07:46:31.235456+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:31.235456+0000","last_undegraded":"2024-09-10T07:46:31.235456+0000","last_fullsized":"2024-09-10T07:46:31.235456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257895+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:31.257895+0000","last_peered":"2024-09-10T07:46:31.257895+0000","last_clean":"2024-09-10T07:46:31.257895+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:31.257895+0000","last_undegraded":"2024-09-10T07:46:31.257895+0000","last_fullsized":"2024-09-10T07:46:31.257895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.239138+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:31.239138+0000","last_peered":"2024-09-10T07:46:31.239138+0000","last_clean":"2024-09-10T07:46:31.239138+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:31.239138+0000","last_undegraded":"2024-09-10T07:46:31.239138+0000","last_fullsized":"2024-09-10T07:46:31.239138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257456+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:31.257456+0000","last_peered":"2024-09-10T07:46:31.257456+0000","last_clean":"2024-09-10T07:46:31.257456+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:31.257456+0000","last_undegraded":"2024-09-10T07:46:31.257456+0000","last_fullsized":"2024-09-10T07:46:31.257456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204027+0000","last_change":"2024-09-10T07:46:30.204027+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204027+0000","last_undegraded":"2024-09-10T07:46:30.204027+0000","last_fullsized":"2024-09-10T07:46:30.204027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.218500+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:30.218500+0000","last_peered":"2024-09-10T07:46:30.218500+0000","last_clean":"2024-09-10T07:46:30.218500+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:30.218500+0000","last_undegraded":"2024-09-10T07:46:30.218500+0000","last_fullsized":"2024-09-10T07:46:30.218500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202785+0000","last_change":"2024-09-10T07:46:30.202785+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202785+0000","last_undegraded":"2024-09-10T07:46:30.202785+0000","last_fullsized":"2024-09-10T07:46:30.202785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476739,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":657408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":60765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":76099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:35.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.801+0000 7f189a400640 1 Processor -- start 2024-09-10T07:46:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.801+0000 7f189a400640 1 -- start start 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.802+0000 7f189a400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1894160710 0x7f18940cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f189a400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1894160af0 0x7f18940d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f189a400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18940cfdd0 con 0x7f18940cf800 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f189a400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f189415cc90 con 0x7f1894160710 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f189a400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f189415cf70 con 0x7f1894160af0 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1894160710 0x7f18940cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1894160710 0x7f18940cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47854/0 (socket says 172.21.15.3:47854) 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 -- 172.21.15.3:0/159052076 learned_addr learned my addr 172.21.15.3:0/159052076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f188be00640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1894160af0 0x7f18940d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 -- 172.21.15.3:0/159052076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1894160af0 msgr2=0x7f18940d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1894160af0 0x7f18940d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 -- 172.21.15.3:0/159052076 >> v1:172.21.15.3:6789/0 conn(0x7f18940cf800 legacy=0x7f189406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.803+0000 7f1893400640 1 -- 172.21.15.3:0/159052076 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189415d250 con 0x7f1894160710 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.804+0000 7f1893400640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160710 0x7f18940cf2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1884009870 tx=0x7f188402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e937bb72ef1bf18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.804+0000 7f1892a00640 1 -- 172.21.15.3:0/159052076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884037070 con 0x7f1894160710 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f1892a00640 1 -- 172.21.15.3:0/159052076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884032440 con 0x7f1894160710 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f1892a00640 1 -- 172.21.15.3:0/159052076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884032810 con 0x7f1894160710 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 -- 172.21.15.3:0/159052076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160710 msgr2=0x7f18940cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160710 0x7f18940cf2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1884009870 tx=0x7f188402a250 comp rx=0 tx=0).stop 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 -- 172.21.15.3:0/159052076 shutdown_connections 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1894160af0 0x7f18940d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 --2- 172.21.15.3:0/159052076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160710 0x7f18940cf2c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 -- 172.21.15.3:0/159052076 >> 172.21.15.3:0/159052076 conn(0x7f189406d000 msgr2=0x7f189406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 -- 172.21.15.3:0/159052076 shutdown_connections 2024-09-10T07:46:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.805+0000 7f189a400640 1 -- 172.21.15.3:0/159052076 wait complete. 2024-09-10T07:46:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.807+0000 7f189a400640 1 Processor -- start 2024-09-10T07:46:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.807+0000 7f189a400640 1 -- start start 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 0x7f1894152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894160710 0x7f18941e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f189415d770 con 0x7f1894160af0 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1894073080 con 0x7f1894160710 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f189a400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f18941f4690 con 0x7f18940cf800 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 0x7f1894152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47866/0 (socket says 172.21.15.3:47866) 2024-09-10T07:46:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 0x7f1894152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51174/0 (socket says 172.21.15.3:51174) 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 -- 172.21.15.3:0/1236340043 learned_addr learned my addr 172.21.15.3:0/1236340043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f188be00640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894160710 0x7f18941e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 -- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 msgr2=0x7f1894152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 0x7f1894152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 -- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894160710 msgr2=0x7f18941e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894160710 0x7f18941e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.808+0000 7f1893e00640 1 -- 172.21.15.3:0/1236340043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18941e86d0 con 0x7f1894160af0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f1893e00640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c7a0 tx=0x7f188c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c003b70 con 0x7f1894160af0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188c003d30 con 0x7f1894160af0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18941e89b0 con 0x7f1894160af0 2024-09-10T07:46:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c00aa90 con 0x7f1894160af0 2024-09-10T07:46:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.809+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1894152fc0 con 0x7f1894160af0 2024-09-10T07:46:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.810+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18940afd50 con 0x7f1894160af0 2024-09-10T07:46:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.811+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f188c00ac30 con 0x7f1894160af0 2024-09-10T07:46:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.812+0000 7f188b400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 0x7f187006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.812+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f188c08ac80 con 0x7f1894160af0 2024-09-10T07:46:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.812+0000 7f1893400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 0x7f187006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.813+0000 7f1893400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 0x7f187006e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1884034000 tx=0x7f188402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.816+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f188c058fd0 con 0x7f1894160af0 2024-09-10T07:46:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:35.997+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18941e8d40 con 0x7f187006be30 2024-09-10T07:46:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.003+0000 7f188b400640 1 -- 172.21.15.3:0/1236340043 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250090 (secure 0 0 0) 0x7f18941e8d40 con 0x7f187006be30 2024-09-10T07:46:36.005 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:36.006 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 msgr2=0x7f187006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 0x7f187006e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1884034000 tx=0x7f188402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 msgr2=0x7f189415eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c7a0 tx=0x7f188c00cc60 comp rx=0 tx=0).stop 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 shutdown_connections 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f187006be30 0x7f187006e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1894160af0 0x7f189415eca0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894160710 0x7f18941e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 --2- 172.21.15.3:0/1236340043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18940cf800 0x7f1894152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 >> 172.21.15.3:0/1236340043 conn(0x7f189406d000 msgr2=0x7f18940ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 shutdown_connections 2024-09-10T07:46:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.007+0000 7f189a400640 1 -- 172.21.15.3:0/1236340043 wait complete. 2024-09-10T07:46:36.022 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T07:46:35.490007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592584,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235668,"kb_used_data":2784,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739052,"statfs":{"total":773094113280,"available":772852789248,"internally_reserved":0,"allocated":2850816,"data_stored":1720922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247657+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:31.247657+0000","last_peered":"2024-09-10T07:46:31.247657+0000","last_clean":"2024-09-10T07:46:31.247657+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:31.247657+0000","last_undegraded":"2024-09-10T07:46:31.247657+0000","last_fullsized":"2024-09-10T07:46:31.247657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203393+0000","last_change":"2024-09-10T07:46:30.203393+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203393+0000","last_undegraded":"2024-09-10T07:46:30.203393+0000","last_fullsized":"2024-09-10T07:46:30.203393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200935+0000","last_change":"2024-09-10T07:46:30.200935+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200935+0000","last_undegraded":"2024-09-10T07:46:30.200935+0000","last_fullsized":"2024-09-10T07:46:30.200935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223177+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:31.223177+0000","last_peered":"2024-09-10T07:46:31.223177+0000","last_clean":"2024-09-10T07:46:31.223177+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:31.223177+0000","last_undegraded":"2024-09-10T07:46:31.223177+0000","last_fullsized":"2024-09-10T07:46:31.223177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.253444+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:31.253444+0000","last_peered":"2024-09-10T07:46:31.253444+0000","last_clean":"2024-09-10T07:46:31.253444+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:31.253444+0000","last_undegraded":"2024-09-10T07:46:31.253444+0000","last_fullsized":"2024-09-10T07:46:31.253444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.232052+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:31.232052+0000","last_peered":"2024-09-10T07:46:31.232052+0000","last_clean":"2024-09-10T07:46:31.232052+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:31.232052+0000","last_undegraded":"2024-09-10T07:46:31.232052+0000","last_fullsized":"2024-09-10T07:46:31.232052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241195+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:31.241195+0000","last_peered":"2024-09-10T07:46:31.241195+0000","last_clean":"2024-09-10T07:46:31.241195+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:31.241195+0000","last_undegraded":"2024-09-10T07:46:31.241195+0000","last_fullsized":"2024-09-10T07:46:31.241195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202948+0000","last_change":"2024-09-10T07:46:30.202948+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202948+0000","last_undegraded":"2024-09-10T07:46:30.202948+0000","last_fullsized":"2024-09-10T07:46:30.202948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.238669+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:31.238669+0000","last_peered":"2024-09-10T07:46:31.238669+0000","last_clean":"2024-09-10T07:46:31.238669+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:31.238669+0000","last_undegraded":"2024-09-10T07:46:31.238669+0000","last_fullsized":"2024-09-10T07:46:31.238669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244605+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:31.244605+0000","last_peered":"2024-09-10T07:46:31.244605+0000","last_clean":"2024-09-10T07:46:31.244605+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:31.244605+0000","last_undegraded":"2024-09-10T07:46:31.244605+0000","last_fullsized":"2024-09-10T07:46:31.244605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246200+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:31.246200+0000","last_peered":"2024-09-10T07:46:31.246200+0000","last_clean":"2024-09-10T07:46:31.246200+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:31.246200+0000","last_undegraded":"2024-09-10T07:46:31.246200+0000","last_fullsized":"2024-09-10T07:46:31.246200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248044+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:31.248044+0000","last_peered":"2024-09-10T07:46:31.248044+0000","last_clean":"2024-09-10T07:46:31.248044+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:31.248044+0000","last_undegraded":"2024-09-10T07:46:31.248044+0000","last_fullsized":"2024-09-10T07:46:31.248044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209709+0000","last_change":"2024-09-10T07:46:30.209709+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209709+0000","last_undegraded":"2024-09-10T07:46:30.209709+0000","last_fullsized":"2024-09-10T07:46:30.209709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.212144+0000","last_change":"2024-09-10T07:46:30.212144+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.212144+0000","last_undegraded":"2024-09-10T07:46:30.212144+0000","last_fullsized":"2024-09-10T07:46:30.212144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203055+0000","last_change":"2024-09-10T07:46:30.203055+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203055+0000","last_undegraded":"2024-09-10T07:46:30.203055+0000","last_fullsized":"2024-09-10T07:46:30.203055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234849+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:31.234849+0000","last_peered":"2024-09-10T07:46:31.234849+0000","last_clean":"2024-09-10T07:46:31.234849+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:31.234849+0000","last_undegraded":"2024-09-10T07:46:31.234849+0000","last_fullsized":"2024-09-10T07:46:31.234849+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236363+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:31.236363+0000","last_peered":"2024-09-10T07:46:31.236363+0000","last_clean":"2024-09-10T07:46:31.236363+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:31.236363+0000","last_undegraded":"2024-09-10T07:46:31.236363+0000","last_fullsized":"2024-09-10T07:46:31.236363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209207+0000","last_change":"2024-09-10T07:46:30.209207+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209207+0000","last_undegraded":"2024-09-10T07:46:30.209207+0000","last_fullsized":"2024-09-10T07:46:30.209207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.218669+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:31.218669+0000","last_peered":"2024-09-10T07:46:31.218669+0000","last_clean":"2024-09-10T07:46:31.218669+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:31.218669+0000","last_undegraded":"2024-09-10T07:46:31.218669+0000","last_fullsized":"2024-09-10T07:46:31.218669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246121+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:31.246121+0000","last_peered":"2024-09-10T07:46:31.246121+0000","last_clean":"2024-09-10T07:46:31.246121+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:31.246121+0000","last_undegraded":"2024-09-10T07:46:31.246121+0000","last_fullsized":"2024-09-10T07:46:31.246121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236449+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:31.236449+0000","last_peered":"2024-09-10T07:46:31.236449+0000","last_clean":"2024-09-10T07:46:31.236449+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:31.236449+0000","last_undegraded":"2024-09-10T07:46:31.236449+0000","last_fullsized":"2024-09-10T07:46:31.236449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201489+0000","last_change":"2024-09-10T07:46:30.201489+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201489+0000","last_undegraded":"2024-09-10T07:46:30.201489+0000","last_fullsized":"2024-09-10T07:46:30.201489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245315+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:31.245315+0000","last_peered":"2024-09-10T07:46:31.245315+0000","last_clean":"2024-09-10T07:46:31.245315+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:31.245315+0000","last_undegraded":"2024-09-10T07:46:31.245315+0000","last_fullsized":"2024-09-10T07:46:31.245315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204563+0000","last_change":"2024-09-10T07:46:30.204563+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204563+0000","last_undegraded":"2024-09-10T07:46:30.204563+0000","last_fullsized":"2024-09-10T07:46:30.204563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235466+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:31.235466+0000","last_peered":"2024-09-10T07:46:31.235466+0000","last_clean":"2024-09-10T07:46:31.235466+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:31.235466+0000","last_undegraded":"2024-09-10T07:46:31.235466+0000","last_fullsized":"2024-09-10T07:46:31.235466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208574+0000","last_change":"2024-09-10T07:46:30.208574+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208574+0000","last_undegraded":"2024-09-10T07:46:30.208574+0000","last_fullsized":"2024-09-10T07:46:30.208574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.210647+0000","last_change":"2024-09-10T07:46:30.210647+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.210647+0000","last_undegraded":"2024-09-10T07:46:30.210647+0000","last_fullsized":"2024-09-10T07:46:30.210647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201619+0000","last_change":"2024-09-10T07:46:30.201619+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201619+0000","last_undegraded":"2024-09-10T07:46:30.201619+0000","last_fullsized":"2024-09-10T07:46:30.201619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245190+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:31.245190+0000","last_peered":"2024-09-10T07:46:31.245190+0000","last_clean":"2024-09-10T07:46:31.245190+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:31.245190+0000","last_undegraded":"2024-09-10T07:46:31.245190+0000","last_fullsized":"2024-09-10T07:46:31.245190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204987+0000","last_change":"2024-09-10T07:46:30.204987+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204987+0000","last_undegraded":"2024-09-10T07:46:30.204987+0000","last_fullsized":"2024-09-10T07:46:30.204987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236511+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:31.236511+0000","last_peered":"2024-09-10T07:46:31.236511+0000","last_clean":"2024-09-10T07:46:31.236511+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:31.236511+0000","last_undegraded":"2024-09-10T07:46:31.236511+0000","last_fullsized":"2024-09-10T07:46:31.236511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248657+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:31.248657+0000","last_peered":"2024-09-10T07:46:31.248657+0000","last_clean":"2024-09-10T07:46:31.248657+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:31.248657+0000","last_undegraded":"2024-09-10T07:46:31.248657+0000","last_fullsized":"2024-09-10T07:46:31.248657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223295+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:30.223295+0000","last_peered":"2024-09-10T07:46:30.223295+0000","last_clean":"2024-09-10T07:46:30.223295+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:30.223295+0000","last_undegraded":"2024-09-10T07:46:30.223295+0000","last_fullsized":"2024-09-10T07:46:30.223295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.256925+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:31.256925+0000","last_peered":"2024-09-10T07:46:31.256925+0000","last_clean":"2024-09-10T07:46:31.256925+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:31.256925+0000","last_undegraded":"2024-09-10T07:46:31.256925+0000","last_fullsized":"2024-09-10T07:46:31.256925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232303+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:30.232303+0000","last_peered":"2024-09-10T07:46:30.232303+0000","last_clean":"2024-09-10T07:46:30.232303+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:30.232303+0000","last_undegraded":"2024-09-10T07:46:30.232303+0000","last_fullsized":"2024-09-10T07:46:30.232303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200915+0000","last_change":"2024-09-10T07:46:30.200915+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200915+0000","last_undegraded":"2024-09-10T07:46:30.200915+0000","last_fullsized":"2024-09-10T07:46:30.200915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212725+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:31.212725+0000","last_peered":"2024-09-10T07:46:31.212725+0000","last_clean":"2024-09-10T07:46:31.212725+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:31.212725+0000","last_undegraded":"2024-09-10T07:46:31.212725+0000","last_fullsized":"2024-09-10T07:46:31.212725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223415+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:31.223415+0000","last_peered":"2024-09-10T07:46:31.223415+0000","last_clean":"2024-09-10T07:46:31.223415+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:31.223415+0000","last_undegraded":"2024-09-10T07:46:31.223415+0000","last_fullsized":"2024-09-10T07:46:31.223415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235488+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:31.235488+0000","last_peered":"2024-09-10T07:46:31.235488+0000","last_clean":"2024-09-10T07:46:31.235488+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:31.235488+0000","last_undegraded":"2024-09-10T07:46:31.235488+0000","last_fullsized":"2024-09-10T07:46:31.235488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.228068+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:31.228068+0000","last_peered":"2024-09-10T07:46:31.228068+0000","last_clean":"2024-09-10T07:46:31.228068+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:31.228068+0000","last_undegraded":"2024-09-10T07:46:31.228068+0000","last_fullsized":"2024-09-10T07:46:31.228068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202517+0000","last_change":"2024-09-10T07:46:30.202517+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202517+0000","last_undegraded":"2024-09-10T07:46:30.202517+0000","last_fullsized":"2024-09-10T07:46:30.202517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.226640+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:30.226640+0000","last_peered":"2024-09-10T07:46:30.226640+0000","last_clean":"2024-09-10T07:46:30.226640+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:30.226640+0000","last_undegraded":"2024-09-10T07:46:30.226640+0000","last_fullsized":"2024-09-10T07:46:30.226640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235372+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:31.235372+0000","last_peered":"2024-09-10T07:46:31.235372+0000","last_clean":"2024-09-10T07:46:31.235372+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:31.235372+0000","last_undegraded":"2024-09-10T07:46:31.235372+0000","last_fullsized":"2024-09-10T07:46:31.235372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258003+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:31.258003+0000","last_peered":"2024-09-10T07:46:31.258003+0000","last_clean":"2024-09-10T07:46:31.258003+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:31.258003+0000","last_undegraded":"2024-09-10T07:46:31.258003+0000","last_fullsized":"2024-09-10T07:46:31.258003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.240265+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:31.240265+0000","last_peered":"2024-09-10T07:46:31.240265+0000","last_clean":"2024-09-10T07:46:31.240265+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:31.240265+0000","last_undegraded":"2024-09-10T07:46:31.240265+0000","last_fullsized":"2024-09-10T07:46:31.240265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.235636+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:30.235636+0000","last_peered":"2024-09-10T07:46:30.235636+0000","last_clean":"2024-09-10T07:46:30.235636+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:30.235636+0000","last_undegraded":"2024-09-10T07:46:30.235636+0000","last_fullsized":"2024-09-10T07:46:30.235636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208560+0000","last_change":"2024-09-10T07:46:30.208560+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208560+0000","last_undegraded":"2024-09-10T07:46:30.208560+0000","last_fullsized":"2024-09-10T07:46:30.208560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214230+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:31.214230+0000","last_peered":"2024-09-10T07:46:31.214230+0000","last_clean":"2024-09-10T07:46:31.214230+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:31.214230+0000","last_undegraded":"2024-09-10T07:46:31.214230+0000","last_fullsized":"2024-09-10T07:46:31.214230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235477+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:31.235477+0000","last_peered":"2024-09-10T07:46:31.235477+0000","last_clean":"2024-09-10T07:46:31.235477+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:31.235477+0000","last_undegraded":"2024-09-10T07:46:31.235477+0000","last_fullsized":"2024-09-10T07:46:31.235477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.222561+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:31.222561+0000","last_peered":"2024-09-10T07:46:31.222561+0000","last_clean":"2024-09-10T07:46:31.222561+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:31.222561+0000","last_undegraded":"2024-09-10T07:46:31.222561+0000","last_fullsized":"2024-09-10T07:46:31.222561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248841+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:31.248841+0000","last_peered":"2024-09-10T07:46:31.248841+0000","last_clean":"2024-09-10T07:46:31.248841+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:31.248841+0000","last_undegraded":"2024-09-10T07:46:31.248841+0000","last_fullsized":"2024-09-10T07:46:31.248841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234830+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:31.234830+0000","last_peered":"2024-09-10T07:46:31.234830+0000","last_clean":"2024-09-10T07:46:31.234830+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:31.234830+0000","last_undegraded":"2024-09-10T07:46:31.234830+0000","last_fullsized":"2024-09-10T07:46:31.234830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257933+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:31.257933+0000","last_peered":"2024-09-10T07:46:31.257933+0000","last_clean":"2024-09-10T07:46:31.257933+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:31.257933+0000","last_undegraded":"2024-09-10T07:46:31.257933+0000","last_fullsized":"2024-09-10T07:46:31.257933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237018+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:31.237018+0000","last_peered":"2024-09-10T07:46:31.237018+0000","last_clean":"2024-09-10T07:46:31.237018+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:31.237018+0000","last_undegraded":"2024-09-10T07:46:31.237018+0000","last_fullsized":"2024-09-10T07:46:31.237018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247784+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:31.247784+0000","last_peered":"2024-09-10T07:46:31.247784+0000","last_clean":"2024-09-10T07:46:31.247784+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:31.247784+0000","last_undegraded":"2024-09-10T07:46:31.247784+0000","last_fullsized":"2024-09-10T07:46:31.247784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.219503+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:30.219503+0000","last_peered":"2024-09-10T07:46:30.219503+0000","last_clean":"2024-09-10T07:46:30.219503+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:30.219503+0000","last_undegraded":"2024-09-10T07:46:30.219503+0000","last_fullsized":"2024-09-10T07:46:30.219503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241918+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:31.241918+0000","last_peered":"2024-09-10T07:46:31.241918+0000","last_clean":"2024-09-10T07:46:31.241918+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:31.241918+0000","last_undegraded":"2024-09-10T07:46:31.241918+0000","last_fullsized":"2024-09-10T07:46:31.241918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232317+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:30.232317+0000","last_peered":"2024-09-10T07:46:30.232317+0000","last_clean":"2024-09-10T07:46:30.232317+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:30.232317+0000","last_undegraded":"2024-09-10T07:46:30.232317+0000","last_fullsized":"2024-09-10T07:46:30.232317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235419+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:31.235419+0000","last_peered":"2024-09-10T07:46:31.235419+0000","last_clean":"2024-09-10T07:46:31.235419+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:31.235419+0000","last_undegraded":"2024-09-10T07:46:31.235419+0000","last_fullsized":"2024-09-10T07:46:31.235419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223016+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:31.223016+0000","last_peered":"2024-09-10T07:46:31.223016+0000","last_clean":"2024-09-10T07:46:31.223016+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:31.223016+0000","last_undegraded":"2024-09-10T07:46:31.223016+0000","last_fullsized":"2024-09-10T07:46:31.223016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.213999+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:31.213999+0000","last_peered":"2024-09-10T07:46:31.213999+0000","last_clean":"2024-09-10T07:46:31.213999+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:31.213999+0000","last_undegraded":"2024-09-10T07:46:31.213999+0000","last_fullsized":"2024-09-10T07:46:31.213999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212930+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:31.212930+0000","last_peered":"2024-09-10T07:46:31.212930+0000","last_clean":"2024-09-10T07:46:31.212930+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:31.212930+0000","last_undegraded":"2024-09-10T07:46:31.212930+0000","last_fullsized":"2024-09-10T07:46:31.212930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206969+0000","last_change":"2024-09-10T07:46:30.206969+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206969+0000","last_undegraded":"2024-09-10T07:46:30.206969+0000","last_fullsized":"2024-09-10T07:46:30.206969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222953+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:30.222953+0000","last_peered":"2024-09-10T07:46:30.222953+0000","last_clean":"2024-09-10T07:46:30.222953+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:30.222953+0000","last_undegraded":"2024-09-10T07:46:30.222953+0000","last_fullsized":"2024-09-10T07:46:30.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202494+0000","last_change":"2024-09-10T07:46:30.202494+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202494+0000","last_undegraded":"2024-09-10T07:46:30.202494+0000","last_fullsized":"2024-09-10T07:46:30.202494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214242+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:31.214242+0000","last_peered":"2024-09-10T07:46:31.214242+0000","last_clean":"2024-09-10T07:46:31.214242+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:31.214242+0000","last_undegraded":"2024-09-10T07:46:31.214242+0000","last_fullsized":"2024-09-10T07:46:31.214242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247133+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:31.247133+0000","last_peered":"2024-09-10T07:46:31.247133+0000","last_clean":"2024-09-10T07:46:31.247133+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:31.247133+0000","last_undegraded":"2024-09-10T07:46:31.247133+0000","last_fullsized":"2024-09-10T07:46:31.247133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222035+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:30.222035+0000","last_peered":"2024-09-10T07:46:30.222035+0000","last_clean":"2024-09-10T07:46:30.222035+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:30.222035+0000","last_undegraded":"2024-09-10T07:46:30.222035+0000","last_fullsized":"2024-09-10T07:46:30.222035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237055+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:31.237055+0000","last_peered":"2024-09-10T07:46:31.237055+0000","last_clean":"2024-09-10T07:46:31.237055+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:31.237055+0000","last_undegraded":"2024-09-10T07:46:31.237055+0000","last_fullsized":"2024-09-10T07:46:31.237055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:34.661912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214679+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:31.214679+0000","last_peered":"2024-09-10T07:46:31.214679+0000","last_clean":"2024-09-10T07:46:31.214679+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:31.214679+0000","last_undegraded":"2024-09-10T07:46:31.214679+0000","last_fullsized":"2024-09-10T07:46:31.214679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.207029+0000","last_change":"2024-09-10T07:46:30.207029+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.207029+0000","last_undegraded":"2024-09-10T07:46:30.207029+0000","last_fullsized":"2024-09-10T07:46:30.207029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258157+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:31.258157+0000","last_peered":"2024-09-10T07:46:31.258157+0000","last_clean":"2024-09-10T07:46:31.258157+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:31.258157+0000","last_undegraded":"2024-09-10T07:46:31.258157+0000","last_fullsized":"2024-09-10T07:46:31.258157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.217759+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:31.217759+0000","last_peered":"2024-09-10T07:46:31.217759+0000","last_clean":"2024-09-10T07:46:31.217759+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:31.217759+0000","last_undegraded":"2024-09-10T07:46:31.217759+0000","last_fullsized":"2024-09-10T07:46:31.217759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.227462+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:31.227462+0000","last_peered":"2024-09-10T07:46:31.227462+0000","last_clean":"2024-09-10T07:46:31.227462+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:31.227462+0000","last_undegraded":"2024-09-10T07:46:31.227462+0000","last_fullsized":"2024-09-10T07:46:31.227462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223401+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:30.223401+0000","last_peered":"2024-09-10T07:46:30.223401+0000","last_clean":"2024-09-10T07:46:30.223401+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:30.223401+0000","last_undegraded":"2024-09-10T07:46:30.223401+0000","last_fullsized":"2024-09-10T07:46:30.223401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:33.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206604+0000","last_change":"2024-09-10T07:46:30.206604+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206604+0000","last_undegraded":"2024-09-10T07:46:30.206604+0000","last_fullsized":"2024-09-10T07:46:30.206604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257046+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:31.257046+0000","last_peered":"2024-09-10T07:46:31.257046+0000","last_clean":"2024-09-10T07:46:31.257046+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:31.257046+0000","last_undegraded":"2024-09-10T07:46:31.257046+0000","last_fullsized":"2024-09-10T07:46:31.257046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244686+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:31.244686+0000","last_peered":"2024-09-10T07:46:31.244686+0000","last_clean":"2024-09-10T07:46:31.244686+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:31.244686+0000","last_undegraded":"2024-09-10T07:46:31.244686+0000","last_fullsized":"2024-09-10T07:46:31.244686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235456+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:31.235456+0000","last_peered":"2024-09-10T07:46:31.235456+0000","last_clean":"2024-09-10T07:46:31.235456+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:31.235456+0000","last_undegraded":"2024-09-10T07:46:31.235456+0000","last_fullsized":"2024-09-10T07:46:31.235456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257895+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:31.257895+0000","last_peered":"2024-09-10T07:46:31.257895+0000","last_clean":"2024-09-10T07:46:31.257895+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:31.257895+0000","last_undegraded":"2024-09-10T07:46:31.257895+0000","last_fullsized":"2024-09-10T07:46:31.257895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.239138+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:31.239138+0000","last_peered":"2024-09-10T07:46:31.239138+0000","last_clean":"2024-09-10T07:46:31.239138+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:31.239138+0000","last_undegraded":"2024-09-10T07:46:31.239138+0000","last_fullsized":"2024-09-10T07:46:31.239138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257456+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:31.257456+0000","last_peered":"2024-09-10T07:46:31.257456+0000","last_clean":"2024-09-10T07:46:31.257456+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:31.257456+0000","last_undegraded":"2024-09-10T07:46:31.257456+0000","last_fullsized":"2024-09-10T07:46:31.257456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204027+0000","last_change":"2024-09-10T07:46:30.204027+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204027+0000","last_undegraded":"2024-09-10T07:46:30.204027+0000","last_fullsized":"2024-09-10T07:46:30.204027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.218500+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:30.218500+0000","last_peered":"2024-09-10T07:46:30.218500+0000","last_clean":"2024-09-10T07:46:30.218500+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:30.218500+0000","last_undegraded":"2024-09-10T07:46:30.218500+0000","last_fullsized":"2024-09-10T07:46:30.218500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202785+0000","last_change":"2024-09-10T07:46:30.202785+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202785+0000","last_undegraded":"2024-09-10T07:46:30.202785+0000","last_fullsized":"2024-09-10T07:46:30.202785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476739,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":657408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":60765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":76099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:36.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:36.185 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:46:36.186+0000 7f9fe8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:46:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.218+0000 7f6ae2a00640 1 Processor -- start 2024-09-10T07:46:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.218+0000 7f6ae2a00640 1 -- start start 2024-09-10T07:46:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae2a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 0x7f6adc06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae2a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6adc06fe10 con 0x7f6adc06e390 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae2a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6adc0b47d0 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae2a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc06c870 con 0x7f6adc161dd0 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae0a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6adc06e390 0x7f6adc06e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52282/0 (socket says 172.21.15.3:52282) 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6ae0a00640 1 -- 172.21.15.3:0/3411724968 learned_addr learned my addr 172.21.15.3:0/3411724968 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:36.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6adbe00640 1 --2- 172.21.15.3:0/3411724968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 0x7f6adc06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191819006 0 0) 0x7f6adc06fe10 con 0x7f6adc06e390 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.220+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ac0003520 con 0x7f6adc06e390 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.221+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700316060 0 0) 0x7f6adc0b47d0 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.221+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6adc06fe10 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.221+0000 7f6adbe00640 1 -- 172.21.15.3:0/3411724968 >> v1:172.21.15.70:6789/0 conn(0x7f6adc0b5ea0 legacy=0x7f6adc0b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.221+0000 7f6adbe00640 1 -- 172.21.15.3:0/3411724968 >> v1:172.21.15.3:6789/0 conn(0x7f6adc06e390 legacy=0x7f6adc06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.221+0000 7f6adbe00640 1 -- 172.21.15.3:0/3411724968 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc06cb50 con 0x7f6adc161dd0 2024-09-10T07:46:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6adbe00640 1 --2- 172.21.15.3:0/3411724968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 0x7f6adc06f8d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0097d0 tx=0x7f6acc02a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282599dcd89afbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc037070 con 0x7f6adc161dd0 2024-09-10T07:46:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6acc032440 con 0x7f6adc161dd0 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6adb400640 1 -- 172.21.15.3:0/3411724968 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc032810 con 0x7f6adc161dd0 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3411724968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 msgr2=0x7f6adc06f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.222+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3411724968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 0x7f6adc06f8d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0097d0 tx=0x7f6acc02a200 comp rx=0 tx=0).stop 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.223+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3411724968 shutdown_connections 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.223+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3411724968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc161dd0 0x7f6adc06f8d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.223+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3411724968 >> 172.21.15.3:0/3411724968 conn(0x7f6adc0b3540 msgr2=0x7f6adc0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.223+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3411724968 shutdown_connections 2024-09-10T07:46:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.223+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3411724968 wait complete. 2024-09-10T07:46:36.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.224+0000 7f6ae2a00640 1 Processor -- start 2024-09-10T07:46:36.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.224+0000 7f6ae2a00640 1 -- start start 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc06e390 0x7f6adc15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc161dd0 0x7f6adc15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc0b43f0 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc0b06c0 con 0x7f6adc161dd0 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ae2a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc0b4570 con 0x7f6adc06e390 2024-09-10T07:46:36.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47878/0 (socket says 172.21.15.3:47878) 2024-09-10T07:46:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.225+0000 7f6ad3e00640 1 -- 172.21.15.3:0/3048704076 learned_addr learned my addr 172.21.15.3:0/3048704076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6adbe00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc06e390 0x7f6adc15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ae0a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc161dd0 0x7f6adc15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 -- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc06e390 msgr2=0x7f6adc15d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc06e390 0x7f6adc15d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 -- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc161dd0 msgr2=0x7f6adc15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc161dd0 0x7f6adc15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 -- 172.21.15.3:0/3048704076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc15f580 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad3e00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c430 tx=0x7f6ac400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.226+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4017070 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.227+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac40052d0 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.227+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc15f860 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.227+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4003810 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.227+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc15fe20 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.228+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc0b2270 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.229+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ac401a070 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.229+0000 7f6ad8c00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 0x7f6ab806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.230+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6ac408a240 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.230+0000 7f6adbe00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 0x7f6ab806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.230+0000 7f6adbe00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 0x7f6ab806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6acc034000 tx=0x7f6acc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.234+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ac4058590 con 0x7f6adc0b5ea0 2024-09-10T07:46:36.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.412+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6adc15f270 con 0x7f6ab806be30 2024-09-10T07:46:36.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.421+0000 7f6ad8c00640 1 -- 172.21.15.3:0/3048704076 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250090 (secure 0 0 0) 0x7f6adc15f270 con 0x7f6ab806be30 2024-09-10T07:46:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:36.425 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 msgr2=0x7f6ab806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 0x7f6ab806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6acc034000 tx=0x7f6acc02e000 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 msgr2=0x7f6adc1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c430 tx=0x7f6ac400c8f0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 shutdown_connections 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ab806be30 0x7f6ab806e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc161dd0 0x7f6adc15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6adc0b5ea0 0x7f6adc1530e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 --2- 172.21.15.3:0/3048704076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6adc06e390 0x7f6adc15d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 >> 172.21.15.3:0/3048704076 conn(0x7f6adc0b3540 msgr2=0x7f6adc0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 shutdown_connections 2024-09-10T07:46:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.425+0000 7f6ae2a00640 1 -- 172.21.15.3:0/3048704076 wait complete. 2024-09-10T07:46:36.440 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T07:46:35.490007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592584,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235668,"kb_used_data":2784,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739052,"statfs":{"total":773094113280,"available":772852789248,"internally_reserved":0,"allocated":2850816,"data_stored":1720922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247657+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:31.247657+0000","last_peered":"2024-09-10T07:46:31.247657+0000","last_clean":"2024-09-10T07:46:31.247657+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:31.247657+0000","last_undegraded":"2024-09-10T07:46:31.247657+0000","last_fullsized":"2024-09-10T07:46:31.247657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203393+0000","last_change":"2024-09-10T07:46:30.203393+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203393+0000","last_undegraded":"2024-09-10T07:46:30.203393+0000","last_fullsized":"2024-09-10T07:46:30.203393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200935+0000","last_change":"2024-09-10T07:46:30.200935+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200935+0000","last_undegraded":"2024-09-10T07:46:30.200935+0000","last_fullsized":"2024-09-10T07:46:30.200935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223177+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:31.223177+0000","last_peered":"2024-09-10T07:46:31.223177+0000","last_clean":"2024-09-10T07:46:31.223177+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:31.223177+0000","last_undegraded":"2024-09-10T07:46:31.223177+0000","last_fullsized":"2024-09-10T07:46:31.223177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.253444+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:31.253444+0000","last_peered":"2024-09-10T07:46:31.253444+0000","last_clean":"2024-09-10T07:46:31.253444+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:31.253444+0000","last_undegraded":"2024-09-10T07:46:31.253444+0000","last_fullsized":"2024-09-10T07:46:31.253444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.232052+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:31.232052+0000","last_peered":"2024-09-10T07:46:31.232052+0000","last_clean":"2024-09-10T07:46:31.232052+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:31.232052+0000","last_undegraded":"2024-09-10T07:46:31.232052+0000","last_fullsized":"2024-09-10T07:46:31.232052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241195+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:31.241195+0000","last_peered":"2024-09-10T07:46:31.241195+0000","last_clean":"2024-09-10T07:46:31.241195+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:31.241195+0000","last_undegraded":"2024-09-10T07:46:31.241195+0000","last_fullsized":"2024-09-10T07:46:31.241195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202948+0000","last_change":"2024-09-10T07:46:30.202948+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202948+0000","last_undegraded":"2024-09-10T07:46:30.202948+0000","last_fullsized":"2024-09-10T07:46:30.202948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.238669+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:31.238669+0000","last_peered":"2024-09-10T07:46:31.238669+0000","last_clean":"2024-09-10T07:46:31.238669+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:31.238669+0000","last_undegraded":"2024-09-10T07:46:31.238669+0000","last_fullsized":"2024-09-10T07:46:31.238669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244605+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:31.244605+0000","last_peered":"2024-09-10T07:46:31.244605+0000","last_clean":"2024-09-10T07:46:31.244605+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:31.244605+0000","last_undegraded":"2024-09-10T07:46:31.244605+0000","last_fullsized":"2024-09-10T07:46:31.244605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246200+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:31.246200+0000","last_peered":"2024-09-10T07:46:31.246200+0000","last_clean":"2024-09-10T07:46:31.246200+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:31.246200+0000","last_undegraded":"2024-09-10T07:46:31.246200+0000","last_fullsized":"2024-09-10T07:46:31.246200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248044+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:31.248044+0000","last_peered":"2024-09-10T07:46:31.248044+0000","last_clean":"2024-09-10T07:46:31.248044+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:31.248044+0000","last_undegraded":"2024-09-10T07:46:31.248044+0000","last_fullsized":"2024-09-10T07:46:31.248044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209709+0000","last_change":"2024-09-10T07:46:30.209709+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209709+0000","last_undegraded":"2024-09-10T07:46:30.209709+0000","last_fullsized":"2024-09-10T07:46:30.209709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.212144+0000","last_change":"2024-09-10T07:46:30.212144+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.212144+0000","last_undegraded":"2024-09-10T07:46:30.212144+0000","last_fullsized":"2024-09-10T07:46:30.212144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203055+0000","last_change":"2024-09-10T07:46:30.203055+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203055+0000","last_undegraded":"2024-09-10T07:46:30.203055+0000","last_fullsized":"2024-09-10T07:46:30.203055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234849+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:31.234849+0000","last_peered":"2024-09-10T07:46:31.234849+0000","last_clean":"2024-09-10T07:46:31.234849+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:31.234849+0000","last_undegraded":"2024-09-10T07:46:31.234849+0000","last_fullsized":"2024-09-10T07:46:31.234849+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236363+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:31.236363+0000","last_peered":"2024-09-10T07:46:31.236363+0000","last_clean":"2024-09-10T07:46:31.236363+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:31.236363+0000","last_undegraded":"2024-09-10T07:46:31.236363+0000","last_fullsized":"2024-09-10T07:46:31.236363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209207+0000","last_change":"2024-09-10T07:46:30.209207+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209207+0000","last_undegraded":"2024-09-10T07:46:30.209207+0000","last_fullsized":"2024-09-10T07:46:30.209207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.218669+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:31.218669+0000","last_peered":"2024-09-10T07:46:31.218669+0000","last_clean":"2024-09-10T07:46:31.218669+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:31.218669+0000","last_undegraded":"2024-09-10T07:46:31.218669+0000","last_fullsized":"2024-09-10T07:46:31.218669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246121+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:31.246121+0000","last_peered":"2024-09-10T07:46:31.246121+0000","last_clean":"2024-09-10T07:46:31.246121+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:31.246121+0000","last_undegraded":"2024-09-10T07:46:31.246121+0000","last_fullsized":"2024-09-10T07:46:31.246121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236449+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:31.236449+0000","last_peered":"2024-09-10T07:46:31.236449+0000","last_clean":"2024-09-10T07:46:31.236449+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:31.236449+0000","last_undegraded":"2024-09-10T07:46:31.236449+0000","last_fullsized":"2024-09-10T07:46:31.236449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201489+0000","last_change":"2024-09-10T07:46:30.201489+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201489+0000","last_undegraded":"2024-09-10T07:46:30.201489+0000","last_fullsized":"2024-09-10T07:46:30.201489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245315+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:31.245315+0000","last_peered":"2024-09-10T07:46:31.245315+0000","last_clean":"2024-09-10T07:46:31.245315+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:31.245315+0000","last_undegraded":"2024-09-10T07:46:31.245315+0000","last_fullsized":"2024-09-10T07:46:31.245315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204563+0000","last_change":"2024-09-10T07:46:30.204563+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204563+0000","last_undegraded":"2024-09-10T07:46:30.204563+0000","last_fullsized":"2024-09-10T07:46:30.204563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235466+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:31.235466+0000","last_peered":"2024-09-10T07:46:31.235466+0000","last_clean":"2024-09-10T07:46:31.235466+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:31.235466+0000","last_undegraded":"2024-09-10T07:46:31.235466+0000","last_fullsized":"2024-09-10T07:46:31.235466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208574+0000","last_change":"2024-09-10T07:46:30.208574+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208574+0000","last_undegraded":"2024-09-10T07:46:30.208574+0000","last_fullsized":"2024-09-10T07:46:30.208574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.210647+0000","last_change":"2024-09-10T07:46:30.210647+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.210647+0000","last_undegraded":"2024-09-10T07:46:30.210647+0000","last_fullsized":"2024-09-10T07:46:30.210647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201619+0000","last_change":"2024-09-10T07:46:30.201619+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201619+0000","last_undegraded":"2024-09-10T07:46:30.201619+0000","last_fullsized":"2024-09-10T07:46:30.201619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245190+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:31.245190+0000","last_peered":"2024-09-10T07:46:31.245190+0000","last_clean":"2024-09-10T07:46:31.245190+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:31.245190+0000","last_undegraded":"2024-09-10T07:46:31.245190+0000","last_fullsized":"2024-09-10T07:46:31.245190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204987+0000","last_change":"2024-09-10T07:46:30.204987+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204987+0000","last_undegraded":"2024-09-10T07:46:30.204987+0000","last_fullsized":"2024-09-10T07:46:30.204987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236511+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:31.236511+0000","last_peered":"2024-09-10T07:46:31.236511+0000","last_clean":"2024-09-10T07:46:31.236511+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:31.236511+0000","last_undegraded":"2024-09-10T07:46:31.236511+0000","last_fullsized":"2024-09-10T07:46:31.236511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248657+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:31.248657+0000","last_peered":"2024-09-10T07:46:31.248657+0000","last_clean":"2024-09-10T07:46:31.248657+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:31.248657+0000","last_undegraded":"2024-09-10T07:46:31.248657+0000","last_fullsized":"2024-09-10T07:46:31.248657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223295+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:30.223295+0000","last_peered":"2024-09-10T07:46:30.223295+0000","last_clean":"2024-09-10T07:46:30.223295+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:30.223295+0000","last_undegraded":"2024-09-10T07:46:30.223295+0000","last_fullsized":"2024-09-10T07:46:30.223295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.256925+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:31.256925+0000","last_peered":"2024-09-10T07:46:31.256925+0000","last_clean":"2024-09-10T07:46:31.256925+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:31.256925+0000","last_undegraded":"2024-09-10T07:46:31.256925+0000","last_fullsized":"2024-09-10T07:46:31.256925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232303+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:30.232303+0000","last_peered":"2024-09-10T07:46:30.232303+0000","last_clean":"2024-09-10T07:46:30.232303+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:30.232303+0000","last_undegraded":"2024-09-10T07:46:30.232303+0000","last_fullsized":"2024-09-10T07:46:30.232303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200915+0000","last_change":"2024-09-10T07:46:30.200915+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200915+0000","last_undegraded":"2024-09-10T07:46:30.200915+0000","last_fullsized":"2024-09-10T07:46:30.200915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212725+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:31.212725+0000","last_peered":"2024-09-10T07:46:31.212725+0000","last_clean":"2024-09-10T07:46:31.212725+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:31.212725+0000","last_undegraded":"2024-09-10T07:46:31.212725+0000","last_fullsized":"2024-09-10T07:46:31.212725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223415+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:31.223415+0000","last_peered":"2024-09-10T07:46:31.223415+0000","last_clean":"2024-09-10T07:46:31.223415+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:31.223415+0000","last_undegraded":"2024-09-10T07:46:31.223415+0000","last_fullsized":"2024-09-10T07:46:31.223415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235488+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:31.235488+0000","last_peered":"2024-09-10T07:46:31.235488+0000","last_clean":"2024-09-10T07:46:31.235488+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:31.235488+0000","last_undegraded":"2024-09-10T07:46:31.235488+0000","last_fullsized":"2024-09-10T07:46:31.235488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.228068+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:31.228068+0000","last_peered":"2024-09-10T07:46:31.228068+0000","last_clean":"2024-09-10T07:46:31.228068+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:31.228068+0000","last_undegraded":"2024-09-10T07:46:31.228068+0000","last_fullsized":"2024-09-10T07:46:31.228068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202517+0000","last_change":"2024-09-10T07:46:30.202517+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202517+0000","last_undegraded":"2024-09-10T07:46:30.202517+0000","last_fullsized":"2024-09-10T07:46:30.202517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.226640+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:30.226640+0000","last_peered":"2024-09-10T07:46:30.226640+0000","last_clean":"2024-09-10T07:46:30.226640+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:30.226640+0000","last_undegraded":"2024-09-10T07:46:30.226640+0000","last_fullsized":"2024-09-10T07:46:30.226640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235372+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:31.235372+0000","last_peered":"2024-09-10T07:46:31.235372+0000","last_clean":"2024-09-10T07:46:31.235372+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:31.235372+0000","last_undegraded":"2024-09-10T07:46:31.235372+0000","last_fullsized":"2024-09-10T07:46:31.235372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258003+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:31.258003+0000","last_peered":"2024-09-10T07:46:31.258003+0000","last_clean":"2024-09-10T07:46:31.258003+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:31.258003+0000","last_undegraded":"2024-09-10T07:46:31.258003+0000","last_fullsized":"2024-09-10T07:46:31.258003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.240265+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:31.240265+0000","last_peered":"2024-09-10T07:46:31.240265+0000","last_clean":"2024-09-10T07:46:31.240265+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:31.240265+0000","last_undegraded":"2024-09-10T07:46:31.240265+0000","last_fullsized":"2024-09-10T07:46:31.240265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.235636+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:30.235636+0000","last_peered":"2024-09-10T07:46:30.235636+0000","last_clean":"2024-09-10T07:46:30.235636+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:30.235636+0000","last_undegraded":"2024-09-10T07:46:30.235636+0000","last_fullsized":"2024-09-10T07:46:30.235636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208560+0000","last_change":"2024-09-10T07:46:30.208560+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208560+0000","last_undegraded":"2024-09-10T07:46:30.208560+0000","last_fullsized":"2024-09-10T07:46:30.208560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214230+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:31.214230+0000","last_peered":"2024-09-10T07:46:31.214230+0000","last_clean":"2024-09-10T07:46:31.214230+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:31.214230+0000","last_undegraded":"2024-09-10T07:46:31.214230+0000","last_fullsized":"2024-09-10T07:46:31.214230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235477+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:31.235477+0000","last_peered":"2024-09-10T07:46:31.235477+0000","last_clean":"2024-09-10T07:46:31.235477+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:31.235477+0000","last_undegraded":"2024-09-10T07:46:31.235477+0000","last_fullsized":"2024-09-10T07:46:31.235477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.222561+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:31.222561+0000","last_peered":"2024-09-10T07:46:31.222561+0000","last_clean":"2024-09-10T07:46:31.222561+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:31.222561+0000","last_undegraded":"2024-09-10T07:46:31.222561+0000","last_fullsized":"2024-09-10T07:46:31.222561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248841+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:31.248841+0000","last_peered":"2024-09-10T07:46:31.248841+0000","last_clean":"2024-09-10T07:46:31.248841+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:31.248841+0000","last_undegraded":"2024-09-10T07:46:31.248841+0000","last_fullsized":"2024-09-10T07:46:31.248841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234830+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:31.234830+0000","last_peered":"2024-09-10T07:46:31.234830+0000","last_clean":"2024-09-10T07:46:31.234830+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:31.234830+0000","last_undegraded":"2024-09-10T07:46:31.234830+0000","last_fullsized":"2024-09-10T07:46:31.234830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257933+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:31.257933+0000","last_peered":"2024-09-10T07:46:31.257933+0000","last_clean":"2024-09-10T07:46:31.257933+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:31.257933+0000","last_undegraded":"2024-09-10T07:46:31.257933+0000","last_fullsized":"2024-09-10T07:46:31.257933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237018+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:31.237018+0000","last_peered":"2024-09-10T07:46:31.237018+0000","last_clean":"2024-09-10T07:46:31.237018+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:31.237018+0000","last_undegraded":"2024-09-10T07:46:31.237018+0000","last_fullsized":"2024-09-10T07:46:31.237018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247784+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:31.247784+0000","last_peered":"2024-09-10T07:46:31.247784+0000","last_clean":"2024-09-10T07:46:31.247784+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:31.247784+0000","last_undegraded":"2024-09-10T07:46:31.247784+0000","last_fullsized":"2024-09-10T07:46:31.247784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.219503+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:30.219503+0000","last_peered":"2024-09-10T07:46:30.219503+0000","last_clean":"2024-09-10T07:46:30.219503+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:30.219503+0000","last_undegraded":"2024-09-10T07:46:30.219503+0000","last_fullsized":"2024-09-10T07:46:30.219503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241918+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:31.241918+0000","last_peered":"2024-09-10T07:46:31.241918+0000","last_clean":"2024-09-10T07:46:31.241918+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:31.241918+0000","last_undegraded":"2024-09-10T07:46:31.241918+0000","last_fullsized":"2024-09-10T07:46:31.241918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232317+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:30.232317+0000","last_peered":"2024-09-10T07:46:30.232317+0000","last_clean":"2024-09-10T07:46:30.232317+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:30.232317+0000","last_undegraded":"2024-09-10T07:46:30.232317+0000","last_fullsized":"2024-09-10T07:46:30.232317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235419+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:31.235419+0000","last_peered":"2024-09-10T07:46:31.235419+0000","last_clean":"2024-09-10T07:46:31.235419+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:31.235419+0000","last_undegraded":"2024-09-10T07:46:31.235419+0000","last_fullsized":"2024-09-10T07:46:31.235419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223016+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:31.223016+0000","last_peered":"2024-09-10T07:46:31.223016+0000","last_clean":"2024-09-10T07:46:31.223016+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:31.223016+0000","last_undegraded":"2024-09-10T07:46:31.223016+0000","last_fullsized":"2024-09-10T07:46:31.223016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.213999+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:31.213999+0000","last_peered":"2024-09-10T07:46:31.213999+0000","last_clean":"2024-09-10T07:46:31.213999+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:31.213999+0000","last_undegraded":"2024-09-10T07:46:31.213999+0000","last_fullsized":"2024-09-10T07:46:31.213999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212930+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:31.212930+0000","last_peered":"2024-09-10T07:46:31.212930+0000","last_clean":"2024-09-10T07:46:31.212930+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:31.212930+0000","last_undegraded":"2024-09-10T07:46:31.212930+0000","last_fullsized":"2024-09-10T07:46:31.212930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206969+0000","last_change":"2024-09-10T07:46:30.206969+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206969+0000","last_undegraded":"2024-09-10T07:46:30.206969+0000","last_fullsized":"2024-09-10T07:46:30.206969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222953+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:30.222953+0000","last_peered":"2024-09-10T07:46:30.222953+0000","last_clean":"2024-09-10T07:46:30.222953+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:30.222953+0000","last_undegraded":"2024-09-10T07:46:30.222953+0000","last_fullsized":"2024-09-10T07:46:30.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202494+0000","last_change":"2024-09-10T07:46:30.202494+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202494+0000","last_undegraded":"2024-09-10T07:46:30.202494+0000","last_fullsized":"2024-09-10T07:46:30.202494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214242+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:31.214242+0000","last_peered":"2024-09-10T07:46:31.214242+0000","last_clean":"2024-09-10T07:46:31.214242+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:31.214242+0000","last_undegraded":"2024-09-10T07:46:31.214242+0000","last_fullsized":"2024-09-10T07:46:31.214242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247133+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:31.247133+0000","last_peered":"2024-09-10T07:46:31.247133+0000","last_clean":"2024-09-10T07:46:31.247133+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:31.247133+0000","last_undegraded":"2024-09-10T07:46:31.247133+0000","last_fullsized":"2024-09-10T07:46:31.247133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222035+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:30.222035+0000","last_peered":"2024-09-10T07:46:30.222035+0000","last_clean":"2024-09-10T07:46:30.222035+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:30.222035+0000","last_undegraded":"2024-09-10T07:46:30.222035+0000","last_fullsized":"2024-09-10T07:46:30.222035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237055+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:31.237055+0000","last_peered":"2024-09-10T07:46:31.237055+0000","last_clean":"2024-09-10T07:46:31.237055+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:31.237055+0000","last_undegraded":"2024-09-10T07:46:31.237055+0000","last_fullsized":"2024-09-10T07:46:31.237055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:34.661912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214679+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:31.214679+0000","last_peered":"2024-09-10T07:46:31.214679+0000","last_clean":"2024-09-10T07:46:31.214679+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:31.214679+0000","last_undegraded":"2024-09-10T07:46:31.214679+0000","last_fullsized":"2024-09-10T07:46:31.214679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.207029+0000","last_change":"2024-09-10T07:46:30.207029+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.207029+0000","last_undegraded":"2024-09-10T07:46:30.207029+0000","last_fullsized":"2024-09-10T07:46:30.207029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258157+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:31.258157+0000","last_peered":"2024-09-10T07:46:31.258157+0000","last_clean":"2024-09-10T07:46:31.258157+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:31.258157+0000","last_undegraded":"2024-09-10T07:46:31.258157+0000","last_fullsized":"2024-09-10T07:46:31.258157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.217759+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:31.217759+0000","last_peered":"2024-09-10T07:46:31.217759+0000","last_clean":"2024-09-10T07:46:31.217759+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:31.217759+0000","last_undegraded":"2024-09-10T07:46:31.217759+0000","last_fullsized":"2024-09-10T07:46:31.217759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.227462+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:31.227462+0000","last_peered":"2024-09-10T07:46:31.227462+0000","last_clean":"2024-09-10T07:46:31.227462+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:31.227462+0000","last_undegraded":"2024-09-10T07:46:31.227462+0000","last_fullsized":"2024-09-10T07:46:31.227462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223401+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:30.223401+0000","last_peered":"2024-09-10T07:46:30.223401+0000","last_clean":"2024-09-10T07:46:30.223401+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:30.223401+0000","last_undegraded":"2024-09-10T07:46:30.223401+0000","last_fullsized":"2024-09-10T07:46:30.223401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:33.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206604+0000","last_change":"2024-09-10T07:46:30.206604+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206604+0000","last_undegraded":"2024-09-10T07:46:30.206604+0000","last_fullsized":"2024-09-10T07:46:30.206604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257046+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:31.257046+0000","last_peered":"2024-09-10T07:46:31.257046+0000","last_clean":"2024-09-10T07:46:31.257046+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:31.257046+0000","last_undegraded":"2024-09-10T07:46:31.257046+0000","last_fullsized":"2024-09-10T07:46:31.257046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244686+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:31.244686+0000","last_peered":"2024-09-10T07:46:31.244686+0000","last_clean":"2024-09-10T07:46:31.244686+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:31.244686+0000","last_undegraded":"2024-09-10T07:46:31.244686+0000","last_fullsized":"2024-09-10T07:46:31.244686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235456+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:31.235456+0000","last_peered":"2024-09-10T07:46:31.235456+0000","last_clean":"2024-09-10T07:46:31.235456+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:31.235456+0000","last_undegraded":"2024-09-10T07:46:31.235456+0000","last_fullsized":"2024-09-10T07:46:31.235456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257895+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:31.257895+0000","last_peered":"2024-09-10T07:46:31.257895+0000","last_clean":"2024-09-10T07:46:31.257895+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:31.257895+0000","last_undegraded":"2024-09-10T07:46:31.257895+0000","last_fullsized":"2024-09-10T07:46:31.257895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.239138+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:31.239138+0000","last_peered":"2024-09-10T07:46:31.239138+0000","last_clean":"2024-09-10T07:46:31.239138+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:31.239138+0000","last_undegraded":"2024-09-10T07:46:31.239138+0000","last_fullsized":"2024-09-10T07:46:31.239138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257456+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:31.257456+0000","last_peered":"2024-09-10T07:46:31.257456+0000","last_clean":"2024-09-10T07:46:31.257456+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:31.257456+0000","last_undegraded":"2024-09-10T07:46:31.257456+0000","last_fullsized":"2024-09-10T07:46:31.257456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204027+0000","last_change":"2024-09-10T07:46:30.204027+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204027+0000","last_undegraded":"2024-09-10T07:46:30.204027+0000","last_fullsized":"2024-09-10T07:46:30.204027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.218500+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:30.218500+0000","last_peered":"2024-09-10T07:46:30.218500+0000","last_clean":"2024-09-10T07:46:30.218500+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:30.218500+0000","last_undegraded":"2024-09-10T07:46:30.218500+0000","last_fullsized":"2024-09-10T07:46:30.218500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202785+0000","last_change":"2024-09-10T07:46:30.202785+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202785+0000","last_undegraded":"2024-09-10T07:46:30.202785+0000","last_fullsized":"2024-09-10T07:46:30.202785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476739,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":657408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":60765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":76099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.203393+0000', 'last_change': '2024-09-10T07:46:30.203393+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.203393+0000', 'last_undegraded': '2024-09-10T07:46:30.203393+0000', 'last_fullsized': '2024-09-10T07:46:30.203393+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.200935+0000', 'last_change': '2024-09-10T07:46:30.200935+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.200935+0000', 'last_undegraded': '2024-09-10T07:46:30.200935+0000', 'last_fullsized': '2024-09-10T07:46:30.200935+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.202948+0000', 'last_change': '2024-09-10T07:46:30.202948+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.202948+0000', 'last_undegraded': '2024-09-10T07:46:30.202948+0000', 'last_fullsized': '2024-09-10T07:46:30.202948+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.209709+0000', 'last_change': '2024-09-10T07:46:30.209709+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.209709+0000', 'last_undegraded': '2024-09-10T07:46:30.209709+0000', 'last_fullsized': '2024-09-10T07:46:30.209709+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T07:46:36.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.212144+0000', 'last_change': '2024-09-10T07:46:30.212144+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.212144+0000', 'last_undegraded': '2024-09-10T07:46:30.212144+0000', 'last_fullsized': '2024-09-10T07:46:30.212144+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.203055+0000', 'last_change': '2024-09-10T07:46:30.203055+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.203055+0000', 'last_undegraded': '2024-09-10T07:46:30.203055+0000', 'last_fullsized': '2024-09-10T07:46:30.203055+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.209207+0000', 'last_change': '2024-09-10T07:46:30.209207+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.209207+0000', 'last_undegraded': '2024-09-10T07:46:30.209207+0000', 'last_fullsized': '2024-09-10T07:46:30.209207+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.201489+0000', 'last_change': '2024-09-10T07:46:30.201489+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.201489+0000', 'last_undegraded': '2024-09-10T07:46:30.201489+0000', 'last_fullsized': '2024-09-10T07:46:30.201489+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.204563+0000', 'last_change': '2024-09-10T07:46:30.204563+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.204563+0000', 'last_undegraded': '2024-09-10T07:46:30.204563+0000', 'last_fullsized': '2024-09-10T07:46:30.204563+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.445 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.208574+0000', 'last_change': '2024-09-10T07:46:30.208574+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.208574+0000', 'last_undegraded': '2024-09-10T07:46:30.208574+0000', 'last_fullsized': '2024-09-10T07:46:30.208574+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.210647+0000', 'last_change': '2024-09-10T07:46:30.210647+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.210647+0000', 'last_undegraded': '2024-09-10T07:46:30.210647+0000', 'last_fullsized': '2024-09-10T07:46:30.210647+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.201619+0000', 'last_change': '2024-09-10T07:46:30.201619+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.201619+0000', 'last_undegraded': '2024-09-10T07:46:30.201619+0000', 'last_fullsized': '2024-09-10T07:46:30.201619+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.204987+0000', 'last_change': '2024-09-10T07:46:30.204987+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.204987+0000', 'last_undegraded': '2024-09-10T07:46:30.204987+0000', 'last_fullsized': '2024-09-10T07:46:30.204987+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.200915+0000', 'last_change': '2024-09-10T07:46:30.200915+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.200915+0000', 'last_undegraded': '2024-09-10T07:46:30.200915+0000', 'last_fullsized': '2024-09-10T07:46:30.200915+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.202517+0000', 'last_change': '2024-09-10T07:46:30.202517+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.202517+0000', 'last_undegraded': '2024-09-10T07:46:30.202517+0000', 'last_fullsized': '2024-09-10T07:46:30.202517+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.446 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.208560+0000', 'last_change': '2024-09-10T07:46:30.208560+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.208560+0000', 'last_undegraded': '2024-09-10T07:46:30.208560+0000', 'last_fullsized': '2024-09-10T07:46:30.208560+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.206969+0000', 'last_change': '2024-09-10T07:46:30.206969+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.206969+0000', 'last_undegraded': '2024-09-10T07:46:30.206969+0000', 'last_fullsized': '2024-09-10T07:46:30.206969+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.202494+0000', 'last_change': '2024-09-10T07:46:30.202494+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.202494+0000', 'last_undegraded': '2024-09-10T07:46:30.202494+0000', 'last_fullsized': '2024-09-10T07:46:30.202494+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.207029+0000', 'last_change': '2024-09-10T07:46:30.207029+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.207029+0000', 'last_undegraded': '2024-09-10T07:46:30.207029+0000', 'last_fullsized': '2024-09-10T07:46:30.207029+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.206604+0000', 'last_change': '2024-09-10T07:46:30.206604+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.206604+0000', 'last_undegraded': '2024-09-10T07:46:30.206604+0000', 'last_fullsized': '2024-09-10T07:46:30.206604+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T07:46:36.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.204027+0000', 'last_change': '2024-09-10T07:46:30.204027+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.204027+0000', 'last_undegraded': '2024-09-10T07:46:30.204027+0000', 'last_fullsized': '2024-09-10T07:46:30.204027+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.448 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-10T07:46:36.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:46:30.202785+0000', 'last_change': '2024-09-10T07:46:30.202785+0000', 'last_active': '2024-09-10T07:46:30.191950+0000', 'last_peered': '2024-09-10T07:46:30.191950+0000', 'last_clean': '2024-09-10T07:46:30.191950+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:46:30.202785+0000', 'last_undegraded': '2024-09-10T07:46:30.202785+0000', 'last_fullsized': '2024-09-10T07:46:30.202785+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'last_clean_scrub_stamp': '2024-09-10T07:46:30.191950+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:46:36.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:46:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.635+0000 7f6635600640 1 Processor -- start 2024-09-10T07:46:36.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.635+0000 7f6635600640 1 -- start start 2024-09-10T07:46:36.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.636+0000 7f6635600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f663015b410 0x7f663006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.636+0000 7f6635600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66300b4aa0 0x7f66300cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.636+0000 7f6635600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66300ccd40 con 0x7f66300cf8f0 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.636+0000 7f6635600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f663006f250 con 0x7f66300b4aa0 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.636+0000 7f6635600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f66300cfd20 con 0x7f663015b410 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66300b4aa0 0x7f66300cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66300b4aa0 0x7f66300cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47884/0 (socket says 172.21.15.3:47884) 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 -- 172.21.15.3:0/844712626 learned_addr learned my addr 172.21.15.3:0/844712626 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 -- 172.21.15.3:0/844712626 >> v2:172.21.15.70:3300/0 conn(0x7f663015b410 msgr2=0x7f663006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 --2- 172.21.15.3:0/844712626 >> v2:172.21.15.70:3300/0 conn(0x7f663015b410 0x7f663006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 -- 172.21.15.3:0/844712626 >> v1:172.21.15.70:6789/0 conn(0x7f66300cf8f0 legacy=0x7f663015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:36.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.637+0000 7f662e000640 1 -- 172.21.15.3:0/844712626 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66300cd020 con 0x7f66300b4aa0 2024-09-10T07:46:36.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.638+0000 7f662e000640 1 --2- 172.21.15.3:0/844712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f66300cf3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f663006f440 tx=0x7f661c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4667df9b81a70a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.638+0000 7f662d600640 1 -- 172.21.15.3:0/844712626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c037070 con 0x7f66300b4aa0 2024-09-10T07:46:36.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.638+0000 7f662d600640 1 -- 172.21.15.3:0/844712626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f661c032440 con 0x7f66300b4aa0 2024-09-10T07:46:36.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.638+0000 7f662d600640 1 -- 172.21.15.3:0/844712626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c032810 con 0x7f66300b4aa0 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 -- 172.21.15.3:0/844712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 msgr2=0x7f66300cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 --2- 172.21.15.3:0/844712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f66300cf3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f663006f440 tx=0x7f661c02a230 comp rx=0 tx=0).stop 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 -- 172.21.15.3:0/844712626 shutdown_connections 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 --2- 172.21.15.3:0/844712626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f66300cf3b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 --2- 172.21.15.3:0/844712626 >> v2:172.21.15.70:3300/0 conn(0x7f663015b410 0x7f663006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 -- 172.21.15.3:0/844712626 >> 172.21.15.3:0/844712626 conn(0x7f663006cfc0 msgr2=0x7f663006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 -- 172.21.15.3:0/844712626 shutdown_connections 2024-09-10T07:46:36.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.639+0000 7f6635600640 1 -- 172.21.15.3:0/844712626 wait complete. 2024-09-10T07:46:36.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.640+0000 7f6635600640 1 Processor -- start 2024-09-10T07:46:36.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.641+0000 7f6635600640 1 -- start start 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.641+0000 7f6635600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f6635600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66300cf8f0 0x7f663015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f6635600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f663015b410 0x7f663015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f6635600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66300cd8c0 con 0x7f66300b4aa0 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f6635600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66300cd5e0 con 0x7f663015b410 2024-09-10T07:46:36.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f6635600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66300cdba0 con 0x7f66300cf8f0 2024-09-10T07:46:36.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47886/0 (socket says 172.21.15.3:47886) 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 -- 172.21.15.3:0/232085900 learned_addr learned my addr 172.21.15.3:0/232085900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662e000640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66300cf8f0 0x7f663015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662f400640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f663015b410 0x7f663015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 -- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66300cf8f0 msgr2=0x7f663015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66300cf8f0 0x7f663015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 -- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f663015b410 msgr2=0x7f663015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f663015b410 0x7f663015d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 -- 172.21.15.3:0/232085900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663015e390 con 0x7f66300b4aa0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.642+0000 7f662ea00640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f662000b440 tx=0x7f662000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.643+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620015020 con 0x7f66300b4aa0 2024-09-10T07:46:36.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.643+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6620011890 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.643+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620011c80 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.643+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663015e670 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.643+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663015ec30 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.644+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6630072540 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.645+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6620018020 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.646+0000 7f6626a00640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 0x7f661006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.646+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f662008b060 con 0x7f66300b4aa0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.646+0000 7f662e000640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 0x7f661006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:36.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.647+0000 7f662e000640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 0x7f661006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f661c034000 tx=0x7f661c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:36.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.651+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f662001f020 con 0x7f66300b4aa0 2024-09-10T07:46:36.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.900+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f66300b0160 con 0x7f66300b4aa0 2024-09-10T07:46:36.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.901+0000 7f6626a00640 1 -- 172.21.15.3:0/232085900 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1138 (secure 0 0 0) 0x7f662000d040 con 0x7f66300b4aa0 2024-09-10T07:46:36.900 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:36.901 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":75},{"state_name":"creating+peering","count":22}],"num_pgs":97,"num_pools":3,"num_objects":19,"data_bytes":592584,"bytes_used":241324032,"bytes_avail":772852789248,"bytes_total":773094113280,"inactive_pgs_ratio":0.22680412232875824,"write_bytes_sec":853,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4287}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T07:46:06.430053+0000","services":{}},"progress_events":{}} 2024-09-10T07:46:36.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 msgr2=0x7f661006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 0x7f661006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f661c034000 tx=0x7f661c02e000 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 msgr2=0x7f663014aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f662000b440 tx=0x7f662000b900 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 shutdown_connections 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f661006be30 0x7f661006e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f663015b410 0x7f663015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66300cf8f0 0x7f663015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 --2- 172.21.15.3:0/232085900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66300b4aa0 0x7f663014aa70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 >> 172.21.15.3:0/232085900 conn(0x7f663006cfc0 msgr2=0x7f66300af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 shutdown_connections 2024-09-10T07:46:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:36.903+0000 7f6635600640 1 -- 172.21.15.3:0/232085900 wait complete. 2024-09-10T07:46:36.918 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:46:36.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:37.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.073+0000 7fd8b0e00640 1 Processor -- start 2024-09-10T07:46:37.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.073+0000 7fd8b0e00640 1 -- start start 2024-09-10T07:46:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.074+0000 7fd8b0e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8ac0d0d80 0x7fd8ac072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.074+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8ac0729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.074+0000 7fd8b0e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ac0d1160 con 0x7fd8ac160370 2024-09-10T07:46:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.074+0000 7fd8b0e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ac1610e0 con 0x7fd8ac0d0d80 2024-09-10T07:46:37.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.074+0000 7fd8b0e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac06e050 con 0x7fd8ac06ee10 2024-09-10T07:46:37.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8aa800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd8ac160370 0x7fd8ac1607e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52288/0 (socket says 172.21.15.3:52288) 2024-09-10T07:46:37.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8aa800640 1 -- 172.21.15.3:0/1450255838 learned_addr learned my addr 172.21.15.3:0/1450255838 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:37.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> v2:172.21.15.70:3300/0 conn(0x7fd8ac0d0d80 0x7fd8ac072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a1400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8ac0729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654369153 0 0) 0x7fd8ac0d1160 con 0x7fd8ac160370 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd890003520 con 0x7fd8ac160370 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2915905617 0 0) 0x7fd890003520 con 0x7fd8ac160370 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 msgr2=0x7fd8ac0729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8ac0729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 >> v2:172.21.15.70:3300/0 conn(0x7fd8ac0d0d80 msgr2=0x7fd8ac072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 --2- 172.21.15.3:0/1450255838 >> v2:172.21.15.70:3300/0 conn(0x7fd8ac0d0d80 0x7fd8ac072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.075+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac06e330 con 0x7fd8ac160370 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fd8a40019f0 con 0x7fd8ac160370 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 >> v1:172.21.15.3:6789/0 conn(0x7fd8ac160370 legacy=0x7fd8ac1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd890003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd890004800 0x7fd890004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac1610e0 con 0x7fd8900039b0 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac06e050 con 0x7fd890004800 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd89000b9c0 con 0x7fd8ac06ee10 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> v2:172.21.15.70:3300/0 conn(0x7fd8ac0d0d80 0x7fd8ac072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd890003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8aa800640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a1400640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd890004800 0x7fd890004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 -- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 msgr2=0x7fd8900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac06ee10 0x7fd8900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 -- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd890004800 msgr2=0x7fd890004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd890004800 0x7fd890004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.076+0000 7fd8a9e00640 1 -- 172.21.15.3:0/1450255838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd890003520 con 0x7fd8900039b0 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.077+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd890003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009c70 tx=0x7fd890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da71ba0257127cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c009840 con 0x7fd8900039b0 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c004020 con 0x7fd8900039b0 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8a9400640 1 -- 172.21.15.3:0/1450255838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0043d0 con 0x7fd8900039b0 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 -- 172.21.15.3:0/1450255838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 msgr2=0x7fd890003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/1450255838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd890003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009c70 tx=0x7fd890005830 comp rx=0 tx=0).stop 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8aa800640 1 -- 172.21.15.3:0/1450255838 reap_dead start 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 -- 172.21.15.3:0/1450255838 shutdown_connections 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 -- 172.21.15.3:0/1450255838 >> 172.21.15.3:0/1450255838 conn(0x7fd8ac0b3540 msgr2=0x7fd8ac0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:37.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 -- 172.21.15.3:0/1450255838 shutdown_connections 2024-09-10T07:46:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.078+0000 7fd8b0e00640 1 -- 172.21.15.3:0/1450255838 wait complete. 2024-09-10T07:46:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.080+0000 7fd8b0e00640 1 Processor -- start 2024-09-10T07:46:37.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.080+0000 7fd8b0e00640 1 -- start start 2024-09-10T07:46:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac1e86f0 0x7fd8ac13aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8ac13af70 0x7fd8ac13d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0729b0 con 0x7fd8900039b0 2024-09-10T07:46:37.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac06e8f0 con 0x7fd8ac13af70 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8b0e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac072b30 con 0x7fd8ac1e86f0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8a9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8a9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47910/0 (socket says 172.21.15.3:47910) 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8aa800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8ac13af70 0x7fd8ac13d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8a9e00640 1 -- 172.21.15.3:0/2200006921 learned_addr learned my addr 172.21.15.3:0/2200006921 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.081+0000 7fd8a1400640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac1e86f0 0x7fd8ac13aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 -- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac1e86f0 msgr2=0x7fd8ac13aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac1e86f0 0x7fd8ac13aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 -- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8ac13af70 msgr2=0x7fd8ac13d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8ac13af70 0x7fd8ac13d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 -- 172.21.15.3:0/2200006921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac1e7650 con 0x7fd8900039b0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a9e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd89c033040 tx=0x7fd89c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:37.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c040070 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.082+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c003820 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.083+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac1e7930 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.083+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c006e10 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.083+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1e7ef0 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.084+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac13b380 con 0x7fd8900039b0 2024-09-10T07:46:37.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.084+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd89c043070 con 0x7fd8900039b0 2024-09-10T07:46:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.085+0000 7fd8a2a00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 0x7fd88806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.085+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd89c0ab520 con 0x7fd8900039b0 2024-09-10T07:46:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.086+0000 7fd8a1400640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 0x7fd88806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:37.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.086+0000 7fd8a1400640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 0x7fd88806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac1e8ad0 tx=0x7fd894009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:37.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.090+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd8ac13b380 con 0x7fd8900039b0 2024-09-10T07:46:37.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.271+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8ac06f270 con 0x7fd88806be30 2024-09-10T07:46:37.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.277+0000 7fd8a2a00640 1 -- 172.21.15.3:0/2200006921 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250090 (secure 0 0 0) 0x7fd8ac06f270 con 0x7fd88806be30 2024-09-10T07:46:37.279 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:37.280 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:37.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 msgr2=0x7fd88806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 0x7fd88806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac1e8ad0 tx=0x7fd894009190 comp rx=0 tx=0).stop 2024-09-10T07:46:37.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 msgr2=0x7fd8ac1e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:37.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd89c033040 tx=0x7fd89c004730 comp rx=0 tx=0).stop 2024-09-10T07:46:37.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 shutdown_connections 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd88806be30 0x7fd88806e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8ac13af70 0x7fd8ac13d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8ac1e86f0 0x7fd8ac13aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 --2- 172.21.15.3:0/2200006921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8900039b0 0x7fd8ac1e6fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 >> 172.21.15.3:0/2200006921 conn(0x7fd8ac0b3540 msgr2=0x7fd8ac0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 shutdown_connections 2024-09-10T07:46:37.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:37.281+0000 7fd8b0e00640 1 -- 172.21.15.3:0/2200006921 wait complete. 2024-09-10T07:46:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T07:46:35.490007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592584,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235668,"kb_used_data":2784,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739052,"statfs":{"total":773094113280,"available":772852789248,"internally_reserved":0,"allocated":2850816,"data_stored":1720922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247657+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:31.247657+0000","last_peered":"2024-09-10T07:46:31.247657+0000","last_clean":"2024-09-10T07:46:31.247657+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:31.247657+0000","last_undegraded":"2024-09-10T07:46:31.247657+0000","last_fullsized":"2024-09-10T07:46:31.247657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203393+0000","last_change":"2024-09-10T07:46:30.203393+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203393+0000","last_undegraded":"2024-09-10T07:46:30.203393+0000","last_fullsized":"2024-09-10T07:46:30.203393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200935+0000","last_change":"2024-09-10T07:46:30.200935+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200935+0000","last_undegraded":"2024-09-10T07:46:30.200935+0000","last_fullsized":"2024-09-10T07:46:30.200935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223177+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:31.223177+0000","last_peered":"2024-09-10T07:46:31.223177+0000","last_clean":"2024-09-10T07:46:31.223177+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:31.223177+0000","last_undegraded":"2024-09-10T07:46:31.223177+0000","last_fullsized":"2024-09-10T07:46:31.223177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.253444+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:31.253444+0000","last_peered":"2024-09-10T07:46:31.253444+0000","last_clean":"2024-09-10T07:46:31.253444+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:31.253444+0000","last_undegraded":"2024-09-10T07:46:31.253444+0000","last_fullsized":"2024-09-10T07:46:31.253444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.232052+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:31.232052+0000","last_peered":"2024-09-10T07:46:31.232052+0000","last_clean":"2024-09-10T07:46:31.232052+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:31.232052+0000","last_undegraded":"2024-09-10T07:46:31.232052+0000","last_fullsized":"2024-09-10T07:46:31.232052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241195+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:31.241195+0000","last_peered":"2024-09-10T07:46:31.241195+0000","last_clean":"2024-09-10T07:46:31.241195+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:31.241195+0000","last_undegraded":"2024-09-10T07:46:31.241195+0000","last_fullsized":"2024-09-10T07:46:31.241195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202948+0000","last_change":"2024-09-10T07:46:30.202948+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202948+0000","last_undegraded":"2024-09-10T07:46:30.202948+0000","last_fullsized":"2024-09-10T07:46:30.202948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.238669+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:31.238669+0000","last_peered":"2024-09-10T07:46:31.238669+0000","last_clean":"2024-09-10T07:46:31.238669+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:31.238669+0000","last_undegraded":"2024-09-10T07:46:31.238669+0000","last_fullsized":"2024-09-10T07:46:31.238669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244605+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:31.244605+0000","last_peered":"2024-09-10T07:46:31.244605+0000","last_clean":"2024-09-10T07:46:31.244605+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:31.244605+0000","last_undegraded":"2024-09-10T07:46:31.244605+0000","last_fullsized":"2024-09-10T07:46:31.244605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246200+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:31.246200+0000","last_peered":"2024-09-10T07:46:31.246200+0000","last_clean":"2024-09-10T07:46:31.246200+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:31.246200+0000","last_undegraded":"2024-09-10T07:46:31.246200+0000","last_fullsized":"2024-09-10T07:46:31.246200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248044+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:31.248044+0000","last_peered":"2024-09-10T07:46:31.248044+0000","last_clean":"2024-09-10T07:46:31.248044+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:31.248044+0000","last_undegraded":"2024-09-10T07:46:31.248044+0000","last_fullsized":"2024-09-10T07:46:31.248044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209709+0000","last_change":"2024-09-10T07:46:30.209709+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209709+0000","last_undegraded":"2024-09-10T07:46:30.209709+0000","last_fullsized":"2024-09-10T07:46:30.209709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.212144+0000","last_change":"2024-09-10T07:46:30.212144+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.212144+0000","last_undegraded":"2024-09-10T07:46:30.212144+0000","last_fullsized":"2024-09-10T07:46:30.212144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.203055+0000","last_change":"2024-09-10T07:46:30.203055+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.203055+0000","last_undegraded":"2024-09-10T07:46:30.203055+0000","last_fullsized":"2024-09-10T07:46:30.203055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234849+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:31.234849+0000","last_peered":"2024-09-10T07:46:31.234849+0000","last_clean":"2024-09-10T07:46:31.234849+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:31.234849+0000","last_undegraded":"2024-09-10T07:46:31.234849+0000","last_fullsized":"2024-09-10T07:46:31.234849+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236363+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:31.236363+0000","last_peered":"2024-09-10T07:46:31.236363+0000","last_clean":"2024-09-10T07:46:31.236363+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:31.236363+0000","last_undegraded":"2024-09-10T07:46:31.236363+0000","last_fullsized":"2024-09-10T07:46:31.236363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.209207+0000","last_change":"2024-09-10T07:46:30.209207+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.209207+0000","last_undegraded":"2024-09-10T07:46:30.209207+0000","last_fullsized":"2024-09-10T07:46:30.209207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.218669+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:31.218669+0000","last_peered":"2024-09-10T07:46:31.218669+0000","last_clean":"2024-09-10T07:46:31.218669+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:31.218669+0000","last_undegraded":"2024-09-10T07:46:31.218669+0000","last_fullsized":"2024-09-10T07:46:31.218669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.246121+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:31.246121+0000","last_peered":"2024-09-10T07:46:31.246121+0000","last_clean":"2024-09-10T07:46:31.246121+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:31.246121+0000","last_undegraded":"2024-09-10T07:46:31.246121+0000","last_fullsized":"2024-09-10T07:46:31.246121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236449+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:31.236449+0000","last_peered":"2024-09-10T07:46:31.236449+0000","last_clean":"2024-09-10T07:46:31.236449+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:31.236449+0000","last_undegraded":"2024-09-10T07:46:31.236449+0000","last_fullsized":"2024-09-10T07:46:31.236449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:03.692671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201489+0000","last_change":"2024-09-10T07:46:30.201489+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201489+0000","last_undegraded":"2024-09-10T07:46:30.201489+0000","last_fullsized":"2024-09-10T07:46:30.201489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245315+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:31.245315+0000","last_peered":"2024-09-10T07:46:31.245315+0000","last_clean":"2024-09-10T07:46:31.245315+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:31.245315+0000","last_undegraded":"2024-09-10T07:46:31.245315+0000","last_fullsized":"2024-09-10T07:46:31.245315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204563+0000","last_change":"2024-09-10T07:46:30.204563+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204563+0000","last_undegraded":"2024-09-10T07:46:30.204563+0000","last_fullsized":"2024-09-10T07:46:30.204563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235466+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:31.235466+0000","last_peered":"2024-09-10T07:46:31.235466+0000","last_clean":"2024-09-10T07:46:31.235466+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:31.235466+0000","last_undegraded":"2024-09-10T07:46:31.235466+0000","last_fullsized":"2024-09-10T07:46:31.235466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208574+0000","last_change":"2024-09-10T07:46:30.208574+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208574+0000","last_undegraded":"2024-09-10T07:46:30.208574+0000","last_fullsized":"2024-09-10T07:46:30.208574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.210647+0000","last_change":"2024-09-10T07:46:30.210647+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.210647+0000","last_undegraded":"2024-09-10T07:46:30.210647+0000","last_fullsized":"2024-09-10T07:46:30.210647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.201619+0000","last_change":"2024-09-10T07:46:30.201619+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.201619+0000","last_undegraded":"2024-09-10T07:46:30.201619+0000","last_fullsized":"2024-09-10T07:46:30.201619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.245190+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:31.245190+0000","last_peered":"2024-09-10T07:46:31.245190+0000","last_clean":"2024-09-10T07:46:31.245190+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:31.245190+0000","last_undegraded":"2024-09-10T07:46:31.245190+0000","last_fullsized":"2024-09-10T07:46:31.245190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204987+0000","last_change":"2024-09-10T07:46:30.204987+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204987+0000","last_undegraded":"2024-09-10T07:46:30.204987+0000","last_fullsized":"2024-09-10T07:46:30.204987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.236511+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:31.236511+0000","last_peered":"2024-09-10T07:46:31.236511+0000","last_clean":"2024-09-10T07:46:31.236511+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:31.236511+0000","last_undegraded":"2024-09-10T07:46:31.236511+0000","last_fullsized":"2024-09-10T07:46:31.236511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248657+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:31.248657+0000","last_peered":"2024-09-10T07:46:31.248657+0000","last_clean":"2024-09-10T07:46:31.248657+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:31.248657+0000","last_undegraded":"2024-09-10T07:46:31.248657+0000","last_fullsized":"2024-09-10T07:46:31.248657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223295+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:30.223295+0000","last_peered":"2024-09-10T07:46:30.223295+0000","last_clean":"2024-09-10T07:46:30.223295+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:30.223295+0000","last_undegraded":"2024-09-10T07:46:30.223295+0000","last_fullsized":"2024-09-10T07:46:30.223295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.256925+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:31.256925+0000","last_peered":"2024-09-10T07:46:31.256925+0000","last_clean":"2024-09-10T07:46:31.256925+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:31.256925+0000","last_undegraded":"2024-09-10T07:46:31.256925+0000","last_fullsized":"2024-09-10T07:46:31.256925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232303+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:30.232303+0000","last_peered":"2024-09-10T07:46:30.232303+0000","last_clean":"2024-09-10T07:46:30.232303+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:30.232303+0000","last_undegraded":"2024-09-10T07:46:30.232303+0000","last_fullsized":"2024-09-10T07:46:30.232303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.200915+0000","last_change":"2024-09-10T07:46:30.200915+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.200915+0000","last_undegraded":"2024-09-10T07:46:30.200915+0000","last_fullsized":"2024-09-10T07:46:30.200915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212725+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:31.212725+0000","last_peered":"2024-09-10T07:46:31.212725+0000","last_clean":"2024-09-10T07:46:31.212725+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:31.212725+0000","last_undegraded":"2024-09-10T07:46:31.212725+0000","last_fullsized":"2024-09-10T07:46:31.212725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223415+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:31.223415+0000","last_peered":"2024-09-10T07:46:31.223415+0000","last_clean":"2024-09-10T07:46:31.223415+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:31.223415+0000","last_undegraded":"2024-09-10T07:46:31.223415+0000","last_fullsized":"2024-09-10T07:46:31.223415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235488+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:31.235488+0000","last_peered":"2024-09-10T07:46:31.235488+0000","last_clean":"2024-09-10T07:46:31.235488+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:31.235488+0000","last_undegraded":"2024-09-10T07:46:31.235488+0000","last_fullsized":"2024-09-10T07:46:31.235488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:54.946290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.228068+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:31.228068+0000","last_peered":"2024-09-10T07:46:31.228068+0000","last_clean":"2024-09-10T07:46:31.228068+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:31.228068+0000","last_undegraded":"2024-09-10T07:46:31.228068+0000","last_fullsized":"2024-09-10T07:46:31.228068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202517+0000","last_change":"2024-09-10T07:46:30.202517+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202517+0000","last_undegraded":"2024-09-10T07:46:30.202517+0000","last_fullsized":"2024-09-10T07:46:30.202517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.226640+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:30.226640+0000","last_peered":"2024-09-10T07:46:30.226640+0000","last_clean":"2024-09-10T07:46:30.226640+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:30.226640+0000","last_undegraded":"2024-09-10T07:46:30.226640+0000","last_fullsized":"2024-09-10T07:46:30.226640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235372+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:31.235372+0000","last_peered":"2024-09-10T07:46:31.235372+0000","last_clean":"2024-09-10T07:46:31.235372+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:31.235372+0000","last_undegraded":"2024-09-10T07:46:31.235372+0000","last_fullsized":"2024-09-10T07:46:31.235372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258003+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:31.258003+0000","last_peered":"2024-09-10T07:46:31.258003+0000","last_clean":"2024-09-10T07:46:31.258003+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:31.258003+0000","last_undegraded":"2024-09-10T07:46:31.258003+0000","last_fullsized":"2024-09-10T07:46:31.258003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.240265+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:31.240265+0000","last_peered":"2024-09-10T07:46:31.240265+0000","last_clean":"2024-09-10T07:46:31.240265+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:31.240265+0000","last_undegraded":"2024-09-10T07:46:31.240265+0000","last_fullsized":"2024-09-10T07:46:31.240265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.235636+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:30.235636+0000","last_peered":"2024-09-10T07:46:30.235636+0000","last_clean":"2024-09-10T07:46:30.235636+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:30.235636+0000","last_undegraded":"2024-09-10T07:46:30.235636+0000","last_fullsized":"2024-09-10T07:46:30.235636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.208560+0000","last_change":"2024-09-10T07:46:30.208560+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.208560+0000","last_undegraded":"2024-09-10T07:46:30.208560+0000","last_fullsized":"2024-09-10T07:46:30.208560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214230+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:31.214230+0000","last_peered":"2024-09-10T07:46:31.214230+0000","last_clean":"2024-09-10T07:46:31.214230+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:31.214230+0000","last_undegraded":"2024-09-10T07:46:31.214230+0000","last_fullsized":"2024-09-10T07:46:31.214230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235477+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:31.235477+0000","last_peered":"2024-09-10T07:46:31.235477+0000","last_clean":"2024-09-10T07:46:31.235477+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:31.235477+0000","last_undegraded":"2024-09-10T07:46:31.235477+0000","last_fullsized":"2024-09-10T07:46:31.235477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.222561+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:31.222561+0000","last_peered":"2024-09-10T07:46:31.222561+0000","last_clean":"2024-09-10T07:46:31.222561+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:31.222561+0000","last_undegraded":"2024-09-10T07:46:31.222561+0000","last_fullsized":"2024-09-10T07:46:31.222561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.248841+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:31.248841+0000","last_peered":"2024-09-10T07:46:31.248841+0000","last_clean":"2024-09-10T07:46:31.248841+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:31.248841+0000","last_undegraded":"2024-09-10T07:46:31.248841+0000","last_fullsized":"2024-09-10T07:46:31.248841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:38.570793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.234830+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:31.234830+0000","last_peered":"2024-09-10T07:46:31.234830+0000","last_clean":"2024-09-10T07:46:31.234830+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:31.234830+0000","last_undegraded":"2024-09-10T07:46:31.234830+0000","last_fullsized":"2024-09-10T07:46:31.234830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257933+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:31.257933+0000","last_peered":"2024-09-10T07:46:31.257933+0000","last_clean":"2024-09-10T07:46:31.257933+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:31.257933+0000","last_undegraded":"2024-09-10T07:46:31.257933+0000","last_fullsized":"2024-09-10T07:46:31.257933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237018+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:31.237018+0000","last_peered":"2024-09-10T07:46:31.237018+0000","last_clean":"2024-09-10T07:46:31.237018+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:31.237018+0000","last_undegraded":"2024-09-10T07:46:31.237018+0000","last_fullsized":"2024-09-10T07:46:31.237018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247784+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:31.247784+0000","last_peered":"2024-09-10T07:46:31.247784+0000","last_clean":"2024-09-10T07:46:31.247784+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:31.247784+0000","last_undegraded":"2024-09-10T07:46:31.247784+0000","last_fullsized":"2024-09-10T07:46:31.247784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:03.479317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.219503+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:30.219503+0000","last_peered":"2024-09-10T07:46:30.219503+0000","last_clean":"2024-09-10T07:46:30.219503+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:30.219503+0000","last_undegraded":"2024-09-10T07:46:30.219503+0000","last_fullsized":"2024-09-10T07:46:30.219503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.241918+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:31.241918+0000","last_peered":"2024-09-10T07:46:31.241918+0000","last_clean":"2024-09-10T07:46:31.241918+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:31.241918+0000","last_undegraded":"2024-09-10T07:46:31.241918+0000","last_fullsized":"2024-09-10T07:46:31.241918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.232317+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:30.232317+0000","last_peered":"2024-09-10T07:46:30.232317+0000","last_clean":"2024-09-10T07:46:30.232317+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:30.232317+0000","last_undegraded":"2024-09-10T07:46:30.232317+0000","last_fullsized":"2024-09-10T07:46:30.232317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:44.154957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235419+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:31.235419+0000","last_peered":"2024-09-10T07:46:31.235419+0000","last_clean":"2024-09-10T07:46:31.235419+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:31.235419+0000","last_undegraded":"2024-09-10T07:46:31.235419+0000","last_fullsized":"2024-09-10T07:46:31.235419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.223016+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:31.223016+0000","last_peered":"2024-09-10T07:46:31.223016+0000","last_clean":"2024-09-10T07:46:31.223016+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:31.223016+0000","last_undegraded":"2024-09-10T07:46:31.223016+0000","last_fullsized":"2024-09-10T07:46:31.223016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.213999+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:31.213999+0000","last_peered":"2024-09-10T07:46:31.213999+0000","last_clean":"2024-09-10T07:46:31.213999+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:31.213999+0000","last_undegraded":"2024-09-10T07:46:31.213999+0000","last_fullsized":"2024-09-10T07:46:31.213999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.212930+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:31.212930+0000","last_peered":"2024-09-10T07:46:31.212930+0000","last_clean":"2024-09-10T07:46:31.212930+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:31.212930+0000","last_undegraded":"2024-09-10T07:46:31.212930+0000","last_fullsized":"2024-09-10T07:46:31.212930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206969+0000","last_change":"2024-09-10T07:46:30.206969+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206969+0000","last_undegraded":"2024-09-10T07:46:30.206969+0000","last_fullsized":"2024-09-10T07:46:30.206969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222953+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:30.222953+0000","last_peered":"2024-09-10T07:46:30.222953+0000","last_clean":"2024-09-10T07:46:30.222953+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:30.222953+0000","last_undegraded":"2024-09-10T07:46:30.222953+0000","last_fullsized":"2024-09-10T07:46:30.222953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202494+0000","last_change":"2024-09-10T07:46:30.202494+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202494+0000","last_undegraded":"2024-09-10T07:46:30.202494+0000","last_fullsized":"2024-09-10T07:46:30.202494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214242+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:31.214242+0000","last_peered":"2024-09-10T07:46:31.214242+0000","last_clean":"2024-09-10T07:46:31.214242+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:31.214242+0000","last_undegraded":"2024-09-10T07:46:31.214242+0000","last_fullsized":"2024-09-10T07:46:31.214242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.247133+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:31.247133+0000","last_peered":"2024-09-10T07:46:31.247133+0000","last_clean":"2024-09-10T07:46:31.247133+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:31.247133+0000","last_undegraded":"2024-09-10T07:46:31.247133+0000","last_fullsized":"2024-09-10T07:46:31.247133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.222035+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:30.222035+0000","last_peered":"2024-09-10T07:46:30.222035+0000","last_clean":"2024-09-10T07:46:30.222035+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:30.222035+0000","last_undegraded":"2024-09-10T07:46:30.222035+0000","last_fullsized":"2024-09-10T07:46:30.222035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.237055+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:31.237055+0000","last_peered":"2024-09-10T07:46:31.237055+0000","last_clean":"2024-09-10T07:46:31.237055+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:31.237055+0000","last_undegraded":"2024-09-10T07:46:31.237055+0000","last_fullsized":"2024-09-10T07:46:31.237055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:34.661912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.214679+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:31.214679+0000","last_peered":"2024-09-10T07:46:31.214679+0000","last_clean":"2024-09-10T07:46:31.214679+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:31.214679+0000","last_undegraded":"2024-09-10T07:46:31.214679+0000","last_fullsized":"2024-09-10T07:46:31.214679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.207029+0000","last_change":"2024-09-10T07:46:30.207029+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.207029+0000","last_undegraded":"2024-09-10T07:46:30.207029+0000","last_fullsized":"2024-09-10T07:46:30.207029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.258157+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:31.258157+0000","last_peered":"2024-09-10T07:46:31.258157+0000","last_clean":"2024-09-10T07:46:31.258157+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:31.258157+0000","last_undegraded":"2024-09-10T07:46:31.258157+0000","last_fullsized":"2024-09-10T07:46:31.258157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.217759+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:31.217759+0000","last_peered":"2024-09-10T07:46:31.217759+0000","last_clean":"2024-09-10T07:46:31.217759+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:31.217759+0000","last_undegraded":"2024-09-10T07:46:31.217759+0000","last_fullsized":"2024-09-10T07:46:31.217759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.227462+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:31.227462+0000","last_peered":"2024-09-10T07:46:31.227462+0000","last_clean":"2024-09-10T07:46:31.227462+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:31.227462+0000","last_undegraded":"2024-09-10T07:46:31.227462+0000","last_fullsized":"2024-09-10T07:46:31.227462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:16.648167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.223401+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:30.223401+0000","last_peered":"2024-09-10T07:46:30.223401+0000","last_clean":"2024-09-10T07:46:30.223401+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:30.223401+0000","last_undegraded":"2024-09-10T07:46:30.223401+0000","last_fullsized":"2024-09-10T07:46:30.223401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:33.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.206604+0000","last_change":"2024-09-10T07:46:30.206604+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.206604+0000","last_undegraded":"2024-09-10T07:46:30.206604+0000","last_fullsized":"2024-09-10T07:46:30.206604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257046+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:31.257046+0000","last_peered":"2024-09-10T07:46:31.257046+0000","last_clean":"2024-09-10T07:46:31.257046+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:31.257046+0000","last_undegraded":"2024-09-10T07:46:31.257046+0000","last_fullsized":"2024-09-10T07:46:31.257046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.244686+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:31.244686+0000","last_peered":"2024-09-10T07:46:31.244686+0000","last_clean":"2024-09-10T07:46:31.244686+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:31.244686+0000","last_undegraded":"2024-09-10T07:46:31.244686+0000","last_fullsized":"2024-09-10T07:46:31.244686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.235456+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:31.235456+0000","last_peered":"2024-09-10T07:46:31.235456+0000","last_clean":"2024-09-10T07:46:31.235456+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:31.235456+0000","last_undegraded":"2024-09-10T07:46:31.235456+0000","last_fullsized":"2024-09-10T07:46:31.235456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257895+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:31.257895+0000","last_peered":"2024-09-10T07:46:31.257895+0000","last_clean":"2024-09-10T07:46:31.257895+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:31.257895+0000","last_undegraded":"2024-09-10T07:46:31.257895+0000","last_fullsized":"2024-09-10T07:46:31.257895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.239138+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:31.239138+0000","last_peered":"2024-09-10T07:46:31.239138+0000","last_clean":"2024-09-10T07:46:31.239138+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:31.239138+0000","last_undegraded":"2024-09-10T07:46:31.239138+0000","last_fullsized":"2024-09-10T07:46:31.239138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:18.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T07:46:31.257456+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:31.257456+0000","last_peered":"2024-09-10T07:46:31.257456+0000","last_clean":"2024-09-10T07:46:31.257456+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:31.257456+0000","last_undegraded":"2024-09-10T07:46:31.257456+0000","last_fullsized":"2024-09-10T07:46:31.257456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.204027+0000","last_change":"2024-09-10T07:46:30.204027+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.204027+0000","last_undegraded":"2024-09-10T07:46:30.204027+0000","last_fullsized":"2024-09-10T07:46:30.204027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T07:46:30.218500+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:30.218500+0000","last_peered":"2024-09-10T07:46:30.218500+0000","last_clean":"2024-09-10T07:46:30.218500+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:30.218500+0000","last_undegraded":"2024-09-10T07:46:30.218500+0000","last_fullsized":"2024-09-10T07:46:30.218500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:59.612305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T07:46:30.202785+0000","last_change":"2024-09-10T07:46:30.202785+0000","last_active":"2024-09-10T07:46:30.191950+0000","last_peered":"2024-09-10T07:46:30.191950+0000","last_clean":"2024-09-10T07:46:30.191950+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:46:30.202785+0000","last_undegraded":"2024-09-10T07:46:30.202785+0000","last_fullsized":"2024-09-10T07:46:30.202785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476739,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":657408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":204800,"data_stored":67040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":60765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":76099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:40.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:40.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.467+0000 7f84fa800640 1 Processor -- start 2024-09-10T07:46:40.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.467+0000 7f84fa800640 1 -- start start 2024-09-10T07:46:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84fa800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84f415e860 0x7f84f40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84fa800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f4073150 0x7f84f4073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84fa800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f40cee40 con 0x7f84f40cfbf0 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84fa800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f84f40d3780 con 0x7f84f415e860 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84fa800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f84f40cf120 con 0x7f84f4073150 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84f9400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84f415e860 0x7f84f40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84f8a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84f40cfbf0 0x7f84f40d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52298/0 (socket says 172.21.15.3:52298) 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84f8a00640 1 -- 172.21.15.3:0/1929461341 learned_addr learned my addr 172.21.15.3:0/1929461341 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.468+0000 7f84f9400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84f415e860 0x7f84f40cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47918/0 (socket says 172.21.15.3:47918) 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.469+0000 7f84f9e00640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f4073150 0x7f84f4073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.469+0000 7f84f9400640 1 -- 172.21.15.3:0/1929461341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f4073150 msgr2=0x7f84f4073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.469+0000 7f84f9400640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f4073150 0x7f84f4073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.469+0000 7f84f9400640 1 -- 172.21.15.3:0/1929461341 >> v1:172.21.15.3:6789/0 conn(0x7f84f40cfbf0 legacy=0x7f84f40d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.469+0000 7f84f9400640 1 -- 172.21.15.3:0/1929461341 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f40d3410 con 0x7f84f415e860 2024-09-10T07:46:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84f9400640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f415e860 0x7f84f40cf6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84e80097a0 tx=0x7f84e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c295a27ad225fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84f3e00640 1 -- 172.21.15.3:0/1929461341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8037070 con 0x7f84f415e860 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84f3e00640 1 -- 172.21.15.3:0/1929461341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8032440 con 0x7f84f415e860 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84f3e00640 1 -- 172.21.15.3:0/1929461341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8032810 con 0x7f84f415e860 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84fa800640 1 -- 172.21.15.3:0/1929461341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f415e860 msgr2=0x7f84f40cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.470+0000 7f84fa800640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f415e860 0x7f84f40cf6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84e80097a0 tx=0x7f84e802a180 comp rx=0 tx=0).stop 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 -- 172.21.15.3:0/1929461341 shutdown_connections 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f4073150 0x7f84f4073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 --2- 172.21.15.3:0/1929461341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f415e860 0x7f84f40cf6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 -- 172.21.15.3:0/1929461341 >> 172.21.15.3:0/1929461341 conn(0x7f84f40d4650 msgr2=0x7f84f40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 -- 172.21.15.3:0/1929461341 shutdown_connections 2024-09-10T07:46:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.471+0000 7f84fa800640 1 -- 172.21.15.3:0/1929461341 wait complete. 2024-09-10T07:46:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.472+0000 7f84fa800640 1 Processor -- start 2024-09-10T07:46:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.472+0000 7f84fa800640 1 -- start start 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 0x7f84f40cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84f415e860 0x7f84f40acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40d2460 con 0x7f84f4073150 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40cf400 con 0x7f84f415e860 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.473+0000 7f84fa800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f84f40b7880 con 0x7f84f40cfbf0 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47922/0 (socket says 172.21.15.3:47922) 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 -- 172.21.15.3:0/3689280774 learned_addr learned my addr 172.21.15.3:0/3689280774 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9e00640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84f415e860 0x7f84f40acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f8a00640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 0x7f84f40cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 -- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 msgr2=0x7f84f40cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 0x7f84f40cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 -- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84f415e860 msgr2=0x7f84f40acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84f415e860 0x7f84f40acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 -- 172.21.15.3:0/3689280774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f40ab070 con 0x7f84f4073150 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f8a00640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 0x7f84f40cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.474+0000 7f84f9400640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84e8035040 tx=0x7f84e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.475+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8037040 con 0x7f84f4073150 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.475+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8030770 con 0x7f84f4073150 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.475+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8030b40 con 0x7f84f4073150 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.475+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f40ab350 con 0x7f84f4073150 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.475+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f40ab910 con 0x7f84f4073150 2024-09-10T07:46:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.476+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f40cf6b0 con 0x7f84f4073150 2024-09-10T07:46:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.477+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84e8044020 con 0x7f84f4073150 2024-09-10T07:46:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.478+0000 7f84f1600640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 0x7f84d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.478+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f84e80ab3b0 con 0x7f84f4073150 2024-09-10T07:46:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.478+0000 7f84f8a00640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 0x7f84d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.479+0000 7f84f8a00640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 0x7f84d406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000f60 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.483+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f84e8032e20 con 0x7f84f4073150 2024-09-10T07:46:40.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.664+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84f4152f80 con 0x7f84d406be30 2024-09-10T07:46:40.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.670+0000 7f84f1600640 1 -- 172.21.15.3:0/3689280774 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f84f4152f80 con 0x7f84d406be30 2024-09-10T07:46:40.673 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:40.674 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 msgr2=0x7f84d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 0x7f84d406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000f60 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 msgr2=0x7f84f4152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84e8035040 tx=0x7f84e80046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 shutdown_connections 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f84d406be30 0x7f84d406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84f415e860 0x7f84f40acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f84f40cfbf0 0x7f84f40cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 --2- 172.21.15.3:0/3689280774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84f4073150 0x7f84f4152a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 >> 172.21.15.3:0/3689280774 conn(0x7f84f40d4650 msgr2=0x7f84f40d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.674+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 shutdown_connections 2024-09-10T07:46:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.675+0000 7f84fa800640 1 -- 172.21.15.3:0/3689280774 wait complete. 2024-09-10T07:46:40.690 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T07:46:39.490933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235912,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738808,"statfs":{"total":773094113280,"available":772852539392,"internally_reserved":0,"allocated":3088384,"data_stored":1826476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.282270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244567+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:32.244567+0000","last_peered":"2024-09-10T07:46:32.244567+0000","last_clean":"2024-09-10T07:46:32.244567+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:32.244567+0000","last_undegraded":"2024-09-10T07:46:32.244567+0000","last_fullsized":"2024-09-10T07:46:32.244567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470176+0000","last_change":"2024-09-10T07:46:31.233576+0000","last_active":"2024-09-10T07:46:34.470176+0000","last_peered":"2024-09-10T07:46:34.470176+0000","last_clean":"2024-09-10T07:46:34.470176+0000","last_became_active":"2024-09-10T07:46:31.230177+0000","last_became_peered":"2024-09-10T07:46:31.230177+0000","last_unstale":"2024-09-10T07:46:34.470176+0000","last_undegraded":"2024-09-10T07:46:34.470176+0000","last_fullsized":"2024-09-10T07:46:34.470176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470318+0000","last_change":"2024-09-10T07:46:31.229570+0000","last_active":"2024-09-10T07:46:34.470318+0000","last_peered":"2024-09-10T07:46:34.470318+0000","last_clean":"2024-09-10T07:46:34.470318+0000","last_became_active":"2024-09-10T07:46:31.229287+0000","last_became_peered":"2024-09-10T07:46:31.229287+0000","last_unstale":"2024-09-10T07:46:34.470318+0000","last_undegraded":"2024-09-10T07:46:34.470318+0000","last_fullsized":"2024-09-10T07:46:34.470318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732362+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:34.732362+0000","last_peered":"2024-09-10T07:46:34.732362+0000","last_clean":"2024-09-10T07:46:34.732362+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:34.732362+0000","last_undegraded":"2024-09-10T07:46:34.732362+0000","last_fullsized":"2024-09-10T07:46:34.732362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243957+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:32.243957+0000","last_peered":"2024-09-10T07:46:32.243957+0000","last_clean":"2024-09-10T07:46:32.243957+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:32.243957+0000","last_undegraded":"2024-09-10T07:46:32.243957+0000","last_fullsized":"2024-09-10T07:46:32.243957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732202+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:34.732202+0000","last_peered":"2024-09-10T07:46:34.732202+0000","last_clean":"2024-09-10T07:46:34.732202+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:34.732202+0000","last_undegraded":"2024-09-10T07:46:34.732202+0000","last_fullsized":"2024-09-10T07:46:34.732202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243828+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:32.243828+0000","last_peered":"2024-09-10T07:46:32.243828+0000","last_clean":"2024-09-10T07:46:32.243828+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:32.243828+0000","last_undegraded":"2024-09-10T07:46:32.243828+0000","last_fullsized":"2024-09-10T07:46:32.243828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470336+0000","last_change":"2024-09-10T07:46:31.242837+0000","last_active":"2024-09-10T07:46:34.470336+0000","last_peered":"2024-09-10T07:46:34.470336+0000","last_clean":"2024-09-10T07:46:34.470336+0000","last_became_active":"2024-09-10T07:46:31.242290+0000","last_became_peered":"2024-09-10T07:46:31.242290+0000","last_unstale":"2024-09-10T07:46:34.470336+0000","last_undegraded":"2024-09-10T07:46:34.470336+0000","last_fullsized":"2024-09-10T07:46:34.470336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212538+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:32.212538+0000","last_peered":"2024-09-10T07:46:32.212538+0000","last_clean":"2024-09-10T07:46:32.212538+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:32.212538+0000","last_undegraded":"2024-09-10T07:46:32.212538+0000","last_fullsized":"2024-09-10T07:46:32.212538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244315+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:32.244315+0000","last_peered":"2024-09-10T07:46:32.244315+0000","last_clean":"2024-09-10T07:46:32.244315+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:32.244315+0000","last_undegraded":"2024-09-10T07:46:32.244315+0000","last_fullsized":"2024-09-10T07:46:32.244315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244275+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:32.244275+0000","last_peered":"2024-09-10T07:46:32.244275+0000","last_clean":"2024-09-10T07:46:32.244275+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:32.244275+0000","last_undegraded":"2024-09-10T07:46:32.244275+0000","last_fullsized":"2024-09-10T07:46:32.244275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244135+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:32.244135+0000","last_peered":"2024-09-10T07:46:32.244135+0000","last_clean":"2024-09-10T07:46:32.244135+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:32.244135+0000","last_undegraded":"2024-09-10T07:46:32.244135+0000","last_fullsized":"2024-09-10T07:46:32.244135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214595+0000","last_change":"2024-09-10T07:46:31.240496+0000","last_active":"2024-09-10T07:46:32.214595+0000","last_peered":"2024-09-10T07:46:32.214595+0000","last_clean":"2024-09-10T07:46:32.214595+0000","last_became_active":"2024-09-10T07:46:31.240284+0000","last_became_peered":"2024-09-10T07:46:31.240284+0000","last_unstale":"2024-09-10T07:46:32.214595+0000","last_undegraded":"2024-09-10T07:46:32.214595+0000","last_fullsized":"2024-09-10T07:46:32.214595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212893+0000","last_change":"2024-09-10T07:46:31.234865+0000","last_active":"2024-09-10T07:46:32.212893+0000","last_peered":"2024-09-10T07:46:32.212893+0000","last_clean":"2024-09-10T07:46:32.212893+0000","last_became_active":"2024-09-10T07:46:31.234496+0000","last_became_peered":"2024-09-10T07:46:31.234496+0000","last_unstale":"2024-09-10T07:46:32.212893+0000","last_undegraded":"2024-09-10T07:46:32.212893+0000","last_fullsized":"2024-09-10T07:46:32.212893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469881+0000","last_change":"2024-09-10T07:46:31.245904+0000","last_active":"2024-09-10T07:46:34.469881+0000","last_peered":"2024-09-10T07:46:34.469881+0000","last_clean":"2024-09-10T07:46:34.469881+0000","last_became_active":"2024-09-10T07:46:31.245611+0000","last_became_peered":"2024-09-10T07:46:31.245611+0000","last_unstale":"2024-09-10T07:46:34.469881+0000","last_undegraded":"2024-09-10T07:46:34.469881+0000","last_fullsized":"2024-09-10T07:46:34.469881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959551+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:32.959551+0000","last_peered":"2024-09-10T07:46:32.959551+0000","last_clean":"2024-09-10T07:46:32.959551+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:32.959551+0000","last_undegraded":"2024-09-10T07:46:32.959551+0000","last_fullsized":"2024-09-10T07:46:32.959551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960470+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:32.960470+0000","last_peered":"2024-09-10T07:46:32.960470+0000","last_clean":"2024-09-10T07:46:32.960470+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:32.960470+0000","last_undegraded":"2024-09-10T07:46:32.960470+0000","last_fullsized":"2024-09-10T07:46:32.960470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:48.730223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487755+0000","last_change":"2024-09-10T07:46:31.237991+0000","last_active":"2024-09-10T07:46:32.487755+0000","last_peered":"2024-09-10T07:46:32.487755+0000","last_clean":"2024-09-10T07:46:32.487755+0000","last_became_active":"2024-09-10T07:46:31.237636+0000","last_became_peered":"2024-09-10T07:46:31.237636+0000","last_unstale":"2024-09-10T07:46:32.487755+0000","last_undegraded":"2024-09-10T07:46:32.487755+0000","last_fullsized":"2024-09-10T07:46:32.487755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960318+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:32.960318+0000","last_peered":"2024-09-10T07:46:32.960318+0000","last_clean":"2024-09-10T07:46:32.960318+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:32.960318+0000","last_undegraded":"2024-09-10T07:46:32.960318+0000","last_fullsized":"2024-09-10T07:46:32.960318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.816476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244209+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:32.244209+0000","last_peered":"2024-09-10T07:46:32.244209+0000","last_clean":"2024-09-10T07:46:32.244209+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:32.244209+0000","last_undegraded":"2024-09-10T07:46:32.244209+0000","last_fullsized":"2024-09-10T07:46:32.244209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211996+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:32.211996+0000","last_peered":"2024-09-10T07:46:32.211996+0000","last_clean":"2024-09-10T07:46:32.211996+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:32.211996+0000","last_undegraded":"2024-09-10T07:46:32.211996+0000","last_fullsized":"2024-09-10T07:46:32.211996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:33.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469941+0000","last_change":"2024-09-10T07:46:31.244889+0000","last_active":"2024-09-10T07:46:34.469941+0000","last_peered":"2024-09-10T07:46:34.469941+0000","last_clean":"2024-09-10T07:46:34.469941+0000","last_became_active":"2024-09-10T07:46:31.234234+0000","last_became_peered":"2024-09-10T07:46:31.234234+0000","last_unstale":"2024-09-10T07:46:34.469941+0000","last_undegraded":"2024-09-10T07:46:34.469941+0000","last_fullsized":"2024-09-10T07:46:34.469941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959527+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:32.959527+0000","last_peered":"2024-09-10T07:46:32.959527+0000","last_clean":"2024-09-10T07:46:32.959527+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:32.959527+0000","last_undegraded":"2024-09-10T07:46:32.959527+0000","last_fullsized":"2024-09-10T07:46:32.959527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470303+0000","last_change":"2024-09-10T07:46:31.238807+0000","last_active":"2024-09-10T07:46:34.470303+0000","last_peered":"2024-09-10T07:46:34.470303+0000","last_clean":"2024-09-10T07:46:34.470303+0000","last_became_active":"2024-09-10T07:46:31.238327+0000","last_became_peered":"2024-09-10T07:46:31.238327+0000","last_unstale":"2024-09-10T07:46:34.470303+0000","last_undegraded":"2024-09-10T07:46:34.470303+0000","last_fullsized":"2024-09-10T07:46:34.470303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732495+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:34.732495+0000","last_peered":"2024-09-10T07:46:34.732495+0000","last_clean":"2024-09-10T07:46:34.732495+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:34.732495+0000","last_undegraded":"2024-09-10T07:46:34.732495+0000","last_fullsized":"2024-09-10T07:46:34.732495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213178+0000","last_change":"2024-09-10T07:46:31.244052+0000","last_active":"2024-09-10T07:46:32.213178+0000","last_peered":"2024-09-10T07:46:32.213178+0000","last_clean":"2024-09-10T07:46:32.213178+0000","last_became_active":"2024-09-10T07:46:31.243687+0000","last_became_peered":"2024-09-10T07:46:31.243687+0000","last_unstale":"2024-09-10T07:46:32.213178+0000","last_undegraded":"2024-09-10T07:46:32.213178+0000","last_fullsized":"2024-09-10T07:46:32.213178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214001+0000","last_change":"2024-09-10T07:46:31.246440+0000","last_active":"2024-09-10T07:46:32.214001+0000","last_peered":"2024-09-10T07:46:32.214001+0000","last_clean":"2024-09-10T07:46:32.214001+0000","last_became_active":"2024-09-10T07:46:31.246117+0000","last_became_peered":"2024-09-10T07:46:31.246117+0000","last_unstale":"2024-09-10T07:46:32.214001+0000","last_undegraded":"2024-09-10T07:46:32.214001+0000","last_fullsized":"2024-09-10T07:46:32.214001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470149+0000","last_change":"2024-09-10T07:46:31.238490+0000","last_active":"2024-09-10T07:46:34.470149+0000","last_peered":"2024-09-10T07:46:34.470149+0000","last_clean":"2024-09-10T07:46:34.470149+0000","last_became_active":"2024-09-10T07:46:31.238151+0000","last_became_peered":"2024-09-10T07:46:31.238151+0000","last_unstale":"2024-09-10T07:46:34.470149+0000","last_undegraded":"2024-09-10T07:46:34.470149+0000","last_fullsized":"2024-09-10T07:46:34.470149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959750+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:32.959750+0000","last_peered":"2024-09-10T07:46:32.959750+0000","last_clean":"2024-09-10T07:46:32.959750+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:32.959750+0000","last_undegraded":"2024-09-10T07:46:32.959750+0000","last_fullsized":"2024-09-10T07:46:32.959750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:23.300273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487616+0000","last_change":"2024-09-10T07:46:31.235084+0000","last_active":"2024-09-10T07:46:32.487616+0000","last_peered":"2024-09-10T07:46:32.487616+0000","last_clean":"2024-09-10T07:46:32.487616+0000","last_became_active":"2024-09-10T07:46:31.234796+0000","last_became_peered":"2024-09-10T07:46:31.234796+0000","last_unstale":"2024-09-10T07:46:32.487616+0000","last_undegraded":"2024-09-10T07:46:32.487616+0000","last_fullsized":"2024-09-10T07:46:32.487616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732133+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:34.732133+0000","last_peered":"2024-09-10T07:46:34.732133+0000","last_clean":"2024-09-10T07:46:34.732133+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:34.732133+0000","last_undegraded":"2024-09-10T07:46:34.732133+0000","last_fullsized":"2024-09-10T07:46:34.732133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212250+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:32.212250+0000","last_peered":"2024-09-10T07:46:32.212250+0000","last_clean":"2024-09-10T07:46:32.212250+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:32.212250+0000","last_undegraded":"2024-09-10T07:46:32.212250+0000","last_fullsized":"2024-09-10T07:46:32.212250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470040+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:34.470040+0000","last_peered":"2024-09-10T07:46:34.470040+0000","last_clean":"2024-09-10T07:46:34.470040+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:34.470040+0000","last_undegraded":"2024-09-10T07:46:34.470040+0000","last_fullsized":"2024-09-10T07:46:34.470040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959867+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:32.959867+0000","last_peered":"2024-09-10T07:46:32.959867+0000","last_clean":"2024-09-10T07:46:32.959867+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:32.959867+0000","last_undegraded":"2024-09-10T07:46:32.959867+0000","last_fullsized":"2024-09-10T07:46:32.959867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214584+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:32.214584+0000","last_peered":"2024-09-10T07:46:32.214584+0000","last_clean":"2024-09-10T07:46:32.214584+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:32.214584+0000","last_undegraded":"2024-09-10T07:46:32.214584+0000","last_fullsized":"2024-09-10T07:46:32.214584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470527+0000","last_change":"2024-09-10T07:46:31.242643+0000","last_active":"2024-09-10T07:46:34.470527+0000","last_peered":"2024-09-10T07:46:34.470527+0000","last_clean":"2024-09-10T07:46:34.470527+0000","last_became_active":"2024-09-10T07:46:31.241930+0000","last_became_peered":"2024-09-10T07:46:31.241930+0000","last_unstale":"2024-09-10T07:46:34.470527+0000","last_undegraded":"2024-09-10T07:46:34.470527+0000","last_fullsized":"2024-09-10T07:46:34.470527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959655+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:32.959655+0000","last_peered":"2024-09-10T07:46:32.959655+0000","last_clean":"2024-09-10T07:46:32.959655+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:32.959655+0000","last_undegraded":"2024-09-10T07:46:32.959655+0000","last_fullsized":"2024-09-10T07:46:32.959655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212493+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:32.212493+0000","last_peered":"2024-09-10T07:46:32.212493+0000","last_clean":"2024-09-10T07:46:32.212493+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:32.212493+0000","last_undegraded":"2024-09-10T07:46:32.212493+0000","last_fullsized":"2024-09-10T07:46:32.212493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732520+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:34.732520+0000","last_peered":"2024-09-10T07:46:34.732520+0000","last_clean":"2024-09-10T07:46:34.732520+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:34.732520+0000","last_undegraded":"2024-09-10T07:46:34.732520+0000","last_fullsized":"2024-09-10T07:46:34.732520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244465+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:32.244465+0000","last_peered":"2024-09-10T07:46:32.244465+0000","last_clean":"2024-09-10T07:46:32.244465+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:32.244465+0000","last_undegraded":"2024-09-10T07:46:32.244465+0000","last_fullsized":"2024-09-10T07:46:32.244465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470484+0000","last_change":"2024-09-10T07:46:31.237271+0000","last_active":"2024-09-10T07:46:34.470484+0000","last_peered":"2024-09-10T07:46:34.470484+0000","last_clean":"2024-09-10T07:46:34.470484+0000","last_became_active":"2024-09-10T07:46:31.236938+0000","last_became_peered":"2024-09-10T07:46:31.236938+0000","last_unstale":"2024-09-10T07:46:34.470484+0000","last_undegraded":"2024-09-10T07:46:34.470484+0000","last_fullsized":"2024-09-10T07:46:34.470484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487630+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:32.487630+0000","last_peered":"2024-09-10T07:46:32.487630+0000","last_clean":"2024-09-10T07:46:32.487630+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:32.487630+0000","last_undegraded":"2024-09-10T07:46:32.487630+0000","last_fullsized":"2024-09-10T07:46:32.487630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959839+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:32.959839+0000","last_peered":"2024-09-10T07:46:32.959839+0000","last_clean":"2024-09-10T07:46:32.959839+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:32.959839+0000","last_undegraded":"2024-09-10T07:46:32.959839+0000","last_fullsized":"2024-09-10T07:46:32.959839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:11.790049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732340+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:34.732340+0000","last_peered":"2024-09-10T07:46:34.732340+0000","last_clean":"2024-09-10T07:46:34.732340+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:34.732340+0000","last_undegraded":"2024-09-10T07:46:34.732340+0000","last_fullsized":"2024-09-10T07:46:34.732340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244349+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:32.244349+0000","last_peered":"2024-09-10T07:46:32.244349+0000","last_clean":"2024-09-10T07:46:32.244349+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:32.244349+0000","last_undegraded":"2024-09-10T07:46:32.244349+0000","last_fullsized":"2024-09-10T07:46:32.244349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487577+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:32.487577+0000","last_peered":"2024-09-10T07:46:32.487577+0000","last_clean":"2024-09-10T07:46:32.487577+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:32.487577+0000","last_undegraded":"2024-09-10T07:46:32.487577+0000","last_fullsized":"2024-09-10T07:46:32.487577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213032+0000","last_change":"2024-09-10T07:46:31.243710+0000","last_active":"2024-09-10T07:46:32.213032+0000","last_peered":"2024-09-10T07:46:32.213032+0000","last_clean":"2024-09-10T07:46:32.213032+0000","last_became_active":"2024-09-10T07:46:31.243367+0000","last_became_peered":"2024-09-10T07:46:31.243367+0000","last_unstale":"2024-09-10T07:46:32.213032+0000","last_undegraded":"2024-09-10T07:46:32.213032+0000","last_fullsized":"2024-09-10T07:46:32.213032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960011+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:32.960011+0000","last_peered":"2024-09-10T07:46:32.960011+0000","last_clean":"2024-09-10T07:46:32.960011+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:32.960011+0000","last_undegraded":"2024-09-10T07:46:32.960011+0000","last_fullsized":"2024-09-10T07:46:32.960011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959776+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:32.959776+0000","last_peered":"2024-09-10T07:46:32.959776+0000","last_clean":"2024-09-10T07:46:32.959776+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:32.959776+0000","last_undegraded":"2024-09-10T07:46:32.959776+0000","last_fullsized":"2024-09-10T07:46:32.959776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:28:15.324769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243766+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:32.243766+0000","last_peered":"2024-09-10T07:46:32.243766+0000","last_clean":"2024-09-10T07:46:32.243766+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:32.243766+0000","last_undegraded":"2024-09-10T07:46:32.243766+0000","last_fullsized":"2024-09-10T07:46:32.243766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211818+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:32.211818+0000","last_peered":"2024-09-10T07:46:32.211818+0000","last_clean":"2024-09-10T07:46:32.211818+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:32.211818+0000","last_undegraded":"2024-09-10T07:46:32.211818+0000","last_fullsized":"2024-09-10T07:46:32.211818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959612+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:32.959612+0000","last_peered":"2024-09-10T07:46:32.959612+0000","last_clean":"2024-09-10T07:46:32.959612+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:32.959612+0000","last_undegraded":"2024-09-10T07:46:32.959612+0000","last_fullsized":"2024-09-10T07:46:32.959612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244676+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:32.244676+0000","last_peered":"2024-09-10T07:46:32.244676+0000","last_clean":"2024-09-10T07:46:32.244676+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:32.244676+0000","last_undegraded":"2024-09-10T07:46:32.244676+0000","last_fullsized":"2024-09-10T07:46:32.244676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732586+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:34.732586+0000","last_peered":"2024-09-10T07:46:34.732586+0000","last_clean":"2024-09-10T07:46:34.732586+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:34.732586+0000","last_undegraded":"2024-09-10T07:46:34.732586+0000","last_fullsized":"2024-09-10T07:46:34.732586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244571+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:32.244571+0000","last_peered":"2024-09-10T07:46:32.244571+0000","last_clean":"2024-09-10T07:46:32.244571+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:32.244571+0000","last_undegraded":"2024-09-10T07:46:32.244571+0000","last_fullsized":"2024-09-10T07:46:32.244571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470126+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:34.470126+0000","last_peered":"2024-09-10T07:46:34.470126+0000","last_clean":"2024-09-10T07:46:34.470126+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:34.470126+0000","last_undegraded":"2024-09-10T07:46:34.470126+0000","last_fullsized":"2024-09-10T07:46:34.470126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244247+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:32.244247+0000","last_peered":"2024-09-10T07:46:32.244247+0000","last_clean":"2024-09-10T07:46:32.244247+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:32.244247+0000","last_undegraded":"2024-09-10T07:46:32.244247+0000","last_fullsized":"2024-09-10T07:46:32.244247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214329+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:32.214329+0000","last_peered":"2024-09-10T07:46:32.214329+0000","last_clean":"2024-09-10T07:46:32.214329+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:32.214329+0000","last_undegraded":"2024-09-10T07:46:32.214329+0000","last_fullsized":"2024-09-10T07:46:32.214329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960136+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:32.960136+0000","last_peered":"2024-09-10T07:46:32.960136+0000","last_clean":"2024-09-10T07:46:32.960136+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:32.960136+0000","last_undegraded":"2024-09-10T07:46:32.960136+0000","last_fullsized":"2024-09-10T07:46:32.960136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.652763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211923+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:32.211923+0000","last_peered":"2024-09-10T07:46:32.211923+0000","last_clean":"2024-09-10T07:46:32.211923+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:32.211923+0000","last_undegraded":"2024-09-10T07:46:32.211923+0000","last_fullsized":"2024-09-10T07:46:32.211923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959922+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:32.959922+0000","last_peered":"2024-09-10T07:46:32.959922+0000","last_clean":"2024-09-10T07:46:32.959922+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:32.959922+0000","last_undegraded":"2024-09-10T07:46:32.959922+0000","last_fullsized":"2024-09-10T07:46:32.959922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959659+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:32.959659+0000","last_peered":"2024-09-10T07:46:32.959659+0000","last_clean":"2024-09-10T07:46:32.959659+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:32.959659+0000","last_undegraded":"2024-09-10T07:46:32.959659+0000","last_fullsized":"2024-09-10T07:46:32.959659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214503+0000","last_change":"2024-09-10T07:46:31.233650+0000","last_active":"2024-09-10T07:46:32.214503+0000","last_peered":"2024-09-10T07:46:32.214503+0000","last_clean":"2024-09-10T07:46:32.214503+0000","last_became_active":"2024-09-10T07:46:31.232959+0000","last_became_peered":"2024-09-10T07:46:31.232959+0000","last_unstale":"2024-09-10T07:46:32.214503+0000","last_undegraded":"2024-09-10T07:46:32.214503+0000","last_fullsized":"2024-09-10T07:46:32.214503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487400+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:32.487400+0000","last_peered":"2024-09-10T07:46:32.487400+0000","last_clean":"2024-09-10T07:46:32.487400+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:32.487400+0000","last_undegraded":"2024-09-10T07:46:32.487400+0000","last_fullsized":"2024-09-10T07:46:32.487400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470342+0000","last_change":"2024-09-10T07:46:31.244888+0000","last_active":"2024-09-10T07:46:34.470342+0000","last_peered":"2024-09-10T07:46:34.470342+0000","last_clean":"2024-09-10T07:46:34.470342+0000","last_became_active":"2024-09-10T07:46:31.244431+0000","last_became_peered":"2024-09-10T07:46:31.244431+0000","last_unstale":"2024-09-10T07:46:34.470342+0000","last_undegraded":"2024-09-10T07:46:34.470342+0000","last_fullsized":"2024-09-10T07:46:34.470342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960038+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:32.960038+0000","last_peered":"2024-09-10T07:46:32.960038+0000","last_clean":"2024-09-10T07:46:32.960038+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:32.960038+0000","last_undegraded":"2024-09-10T07:46:32.960038+0000","last_fullsized":"2024-09-10T07:46:32.960038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243996+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:32.243996+0000","last_peered":"2024-09-10T07:46:32.243996+0000","last_clean":"2024-09-10T07:46:32.243996+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:32.243996+0000","last_undegraded":"2024-09-10T07:46:32.243996+0000","last_fullsized":"2024-09-10T07:46:32.243996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470060+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:34.470060+0000","last_peered":"2024-09-10T07:46:34.470060+0000","last_clean":"2024-09-10T07:46:34.470060+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:34.470060+0000","last_undegraded":"2024-09-10T07:46:34.470060+0000","last_fullsized":"2024-09-10T07:46:34.470060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732286+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:34.732286+0000","last_peered":"2024-09-10T07:46:34.732286+0000","last_clean":"2024-09-10T07:46:34.732286+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:34.732286+0000","last_undegraded":"2024-09-10T07:46:34.732286+0000","last_fullsized":"2024-09-10T07:46:34.732286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960240+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:32.960240+0000","last_peered":"2024-09-10T07:46:32.960240+0000","last_clean":"2024-09-10T07:46:32.960240+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:32.960240+0000","last_undegraded":"2024-09-10T07:46:32.960240+0000","last_fullsized":"2024-09-10T07:46:32.960240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214164+0000","last_change":"2024-09-10T07:46:31.231122+0000","last_active":"2024-09-10T07:46:32.214164+0000","last_peered":"2024-09-10T07:46:32.214164+0000","last_clean":"2024-09-10T07:46:32.214164+0000","last_became_active":"2024-09-10T07:46:31.230087+0000","last_became_peered":"2024-09-10T07:46:31.230087+0000","last_unstale":"2024-09-10T07:46:32.214164+0000","last_undegraded":"2024-09-10T07:46:32.214164+0000","last_fullsized":"2024-09-10T07:46:32.214164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960429+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:32.960429+0000","last_peered":"2024-09-10T07:46:32.960429+0000","last_clean":"2024-09-10T07:46:32.960429+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:32.960429+0000","last_undegraded":"2024-09-10T07:46:32.960429+0000","last_fullsized":"2024-09-10T07:46:32.960429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243629+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:32.243629+0000","last_peered":"2024-09-10T07:46:32.243629+0000","last_clean":"2024-09-10T07:46:32.243629+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:32.243629+0000","last_undegraded":"2024-09-10T07:46:32.243629+0000","last_fullsized":"2024-09-10T07:46:32.243629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959717+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:32.959717+0000","last_peered":"2024-09-10T07:46:32.959717+0000","last_clean":"2024-09-10T07:46:32.959717+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:32.959717+0000","last_undegraded":"2024-09-10T07:46:32.959717+0000","last_fullsized":"2024-09-10T07:46:32.959717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:03.411205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213096+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:32.213096+0000","last_peered":"2024-09-10T07:46:32.213096+0000","last_clean":"2024-09-10T07:46:32.213096+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:32.213096+0000","last_undegraded":"2024-09-10T07:46:32.213096+0000","last_fullsized":"2024-09-10T07:46:32.213096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487538+0000","last_change":"2024-09-10T07:46:31.227688+0000","last_active":"2024-09-10T07:46:32.487538+0000","last_peered":"2024-09-10T07:46:32.487538+0000","last_clean":"2024-09-10T07:46:32.487538+0000","last_became_active":"2024-09-10T07:46:31.227344+0000","last_became_peered":"2024-09-10T07:46:31.227344+0000","last_unstale":"2024-09-10T07:46:32.487538+0000","last_undegraded":"2024-09-10T07:46:32.487538+0000","last_fullsized":"2024-09-10T07:46:32.487538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959762+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:32.959762+0000","last_peered":"2024-09-10T07:46:32.959762+0000","last_clean":"2024-09-10T07:46:32.959762+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:32.959762+0000","last_undegraded":"2024-09-10T07:46:32.959762+0000","last_fullsized":"2024-09-10T07:46:32.959762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959914+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:32.959914+0000","last_peered":"2024-09-10T07:46:32.959914+0000","last_clean":"2024-09-10T07:46:32.959914+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:32.959914+0000","last_undegraded":"2024-09-10T07:46:32.959914+0000","last_fullsized":"2024-09-10T07:46:32.959914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:27.770235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960085+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:32.960085+0000","last_peered":"2024-09-10T07:46:32.960085+0000","last_clean":"2024-09-10T07:46:32.960085+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:32.960085+0000","last_undegraded":"2024-09-10T07:46:32.960085+0000","last_fullsized":"2024-09-10T07:46:32.960085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:59.748478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244314+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:32.244314+0000","last_peered":"2024-09-10T07:46:32.244314+0000","last_clean":"2024-09-10T07:46:32.244314+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:32.244314+0000","last_undegraded":"2024-09-10T07:46:32.244314+0000","last_fullsized":"2024-09-10T07:46:32.244314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212216+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:32.212216+0000","last_peered":"2024-09-10T07:46:32.212216+0000","last_clean":"2024-09-10T07:46:32.212216+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:32.212216+0000","last_undegraded":"2024-09-10T07:46:32.212216+0000","last_fullsized":"2024-09-10T07:46:32.212216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244379+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:32.244379+0000","last_peered":"2024-09-10T07:46:32.244379+0000","last_clean":"2024-09-10T07:46:32.244379+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:32.244379+0000","last_undegraded":"2024-09-10T07:46:32.244379+0000","last_fullsized":"2024-09-10T07:46:32.244379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470327+0000","last_change":"2024-09-10T07:46:31.238515+0000","last_active":"2024-09-10T07:46:34.470327+0000","last_peered":"2024-09-10T07:46:34.470327+0000","last_clean":"2024-09-10T07:46:34.470327+0000","last_became_active":"2024-09-10T07:46:31.237949+0000","last_became_peered":"2024-09-10T07:46:31.237949+0000","last_unstale":"2024-09-10T07:46:34.470327+0000","last_undegraded":"2024-09-10T07:46:34.470327+0000","last_fullsized":"2024-09-10T07:46:34.470327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470350+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:34.470350+0000","last_peered":"2024-09-10T07:46:34.470350+0000","last_clean":"2024-09-10T07:46:34.470350+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:34.470350+0000","last_undegraded":"2024-09-10T07:46:34.470350+0000","last_fullsized":"2024-09-10T07:46:34.470350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470203+0000","last_change":"2024-09-10T07:46:31.238308+0000","last_active":"2024-09-10T07:46:34.470203+0000","last_peered":"2024-09-10T07:46:34.470203+0000","last_clean":"2024-09-10T07:46:34.470203+0000","last_became_active":"2024-09-10T07:46:31.238032+0000","last_became_peered":"2024-09-10T07:46:31.238032+0000","last_unstale":"2024-09-10T07:46:34.470203+0000","last_undegraded":"2024-09-10T07:46:34.470203+0000","last_fullsized":"2024-09-10T07:46:34.470203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476742,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476740,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":670553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":80171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":80239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":80115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":670575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":249856,"data_stored":82399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:40.693 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:46:40.694 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T07:46:40.694 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T07:46:40.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T07:46:40.694 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T07:46:40.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.887+0000 7f441b200640 1 Processor -- start 2024-09-10T07:46:40.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.887+0000 7f441b200640 1 -- start start 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f441b200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f441415e860 0x7f44140cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f441b200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 0x7f4414073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f441b200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140cee40 con 0x7f44140cfbf0 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f441b200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f44140d3780 con 0x7f441415e860 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f441b200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f44140cf120 con 0x7f4414073150 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f4419e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f441415e860 0x7f44140cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f4419400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44140cfbf0 0x7f44140d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52310/0 (socket says 172.21.15.3:52310) 2024-09-10T07:46:40.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.888+0000 7f4419400640 1 -- 172.21.15.3:0/906679005 learned_addr learned my addr 172.21.15.3:0/906679005 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:40.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f441a800640 1 --2- 172.21.15.3:0/906679005 >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 0x7f4414073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f4419e00640 1 -- 172.21.15.3:0/906679005 >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 msgr2=0x7f4414073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f4419e00640 1 --2- 172.21.15.3:0/906679005 >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 0x7f4414073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f4419e00640 1 -- 172.21.15.3:0/906679005 >> v1:172.21.15.3:6789/0 conn(0x7f44140cfbf0 legacy=0x7f44140d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f4419e00640 1 -- 172.21.15.3:0/906679005 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140d3410 con 0x7f441415e860 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f4418a00640 1 -- 172.21.15.3:0/906679005 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004442089 0 0) 0x7f44140cee40 con 0x7f44140cfbf0 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.889+0000 7f441a800640 1 --2- 172.21.15.3:0/906679005 >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 0x7f4414073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f4419e00640 1 --2- 172.21.15.3:0/906679005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441415e860 0x7f44140cf6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44140b4060 tx=0x7f440802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18027476327b46e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f4418a00640 1 -- 172.21.15.3:0/906679005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408037070 con 0x7f441415e860 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f4418a00640 1 -- 172.21.15.3:0/906679005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4408032440 con 0x7f441415e860 2024-09-10T07:46:40.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f4418a00640 1 -- 172.21.15.3:0/906679005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408032810 con 0x7f441415e860 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f441b200640 1 -- 172.21.15.3:0/906679005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441415e860 msgr2=0x7f44140cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.890+0000 7f441b200640 1 --2- 172.21.15.3:0/906679005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441415e860 0x7f44140cf6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44140b4060 tx=0x7f440802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 -- 172.21.15.3:0/906679005 shutdown_connections 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 --2- 172.21.15.3:0/906679005 >> v2:172.21.15.70:3300/0 conn(0x7f4414073150 0x7f4414073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 --2- 172.21.15.3:0/906679005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441415e860 0x7f44140cf6b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 -- 172.21.15.3:0/906679005 >> 172.21.15.3:0/906679005 conn(0x7f44140d4650 msgr2=0x7f44140d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 -- 172.21.15.3:0/906679005 shutdown_connections 2024-09-10T07:46:40.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.891+0000 7f441b200640 1 -- 172.21.15.3:0/906679005 wait complete. 2024-09-10T07:46:40.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.892+0000 7f441b200640 1 Processor -- start 2024-09-10T07:46:40.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.892+0000 7f441b200640 1 -- start start 2024-09-10T07:46:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44140cfbf0 0x7f44141f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f441415e860 0x7f44141f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d2460 con 0x7f4414073150 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44140cf400 con 0x7f44140cfbf0 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f441b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44140b7880 con 0x7f441415e860 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f4419e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f4419e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47930/0 (socket says 172.21.15.3:47930) 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.893+0000 7f4419e00640 1 -- 172.21.15.3:0/2457674615 learned_addr learned my addr 172.21.15.3:0/2457674615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 -- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f441415e860 msgr2=0x7f44141f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f441415e860 0x7f44141f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 -- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44140cfbf0 msgr2=0x7f44141f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44140cfbf0 0x7f44141f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 -- 172.21.15.3:0/2457674615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441415bc50 con 0x7f4414073150 2024-09-10T07:46:40.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f4419e00640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4408009890 tx=0x7f44080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408037040 con 0x7f4414073150 2024-09-10T07:46:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4408042070 con 0x7f4414073150 2024-09-10T07:46:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408030a50 con 0x7f4414073150 2024-09-10T07:46:40.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.894+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44140abba0 con 0x7f4414073150 2024-09-10T07:46:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.895+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44140ac160 con 0x7f4414073150 2024-09-10T07:46:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.895+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441406d260 con 0x7f4414073150 2024-09-10T07:46:40.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.896+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4408030bf0 con 0x7f4414073150 2024-09-10T07:46:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.897+0000 7f440e000640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 0x7f43f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.897+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f44080ab920 con 0x7f4414073150 2024-09-10T07:46:40.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.898+0000 7f4419400640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 0x7f43f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:40.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.898+0000 7f4419400640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 0x7f43f806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4404007fd0 tx=0x7f4404007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:40.902+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f441406d260 con 0x7f4414073150 2024-09-10T07:46:41.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.135+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f44140d40a0 con 0x7f4414073150 2024-09-10T07:46:41.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.139+0000 7f440e000640 1 -- 172.21.15.3:0/2457674615 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172931 (secure 0 0 0) 0x7f44140d40a0 con 0x7f4414073150 2024-09-10T07:46:41.140 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 msgr2=0x7f43f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 0x7f43f806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4404007fd0 tx=0x7f4404007920 comp rx=0 tx=0).stop 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 msgr2=0x7f441415b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4408009890 tx=0x7f44080046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 shutdown_connections 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be30 0x7f43f806e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f441415e860 0x7f44141f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44140cfbf0 0x7f44141f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 --2- 172.21.15.3:0/2457674615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4414073150 0x7f441415b710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 >> 172.21.15.3:0/2457674615 conn(0x7f44140d4650 msgr2=0x7f4414072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 shutdown_connections 2024-09-10T07:46:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.142+0000 7f441b200640 1 -- 172.21.15.3:0/2457674615 wait complete. 2024-09-10T07:46:41.157 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3286701713},{"type":"v1","addr":"172.21.15.3:6833","nonce":3286701713}]},"active_addr":"172.21.15.3:6833/3286701713","active_change":"2024-09-10T07:46:11.475114+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":749344021}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2745071978}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2868838216}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2509410213}]}]} 2024-09-10T07:46:41.161 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T07:46:41.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T07:46:41.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.317+0000 7f8ad5a00640 1 Processor -- start 2024-09-10T07:46:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.318+0000 7f8ad5a00640 1 -- start start 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ad5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 0x7f8ad00cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ad5a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ad00b40d0 0x7f8ad00b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ad5a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad00cf930 con 0x7f8ad00cd7d0 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ad5a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad00cfc10 con 0x7f8ad00b40d0 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ad5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad00b38d0 con 0x7f8ad0162ac0 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ac7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ad00cd7d0 0x7f8ad0070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52312/0 (socket says 172.21.15.3:52312) 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ac7e00640 1 -- 172.21.15.3:0/3083604112 learned_addr learned my addr 172.21.15.3:0/3083604112 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8acea00640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 0x7f8ad00cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8acf400640 1 --2- 172.21.15.3:0/3083604112 >> v2:172.21.15.70:3300/0 conn(0x7f8ad00b40d0 0x7f8ad00b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ace000640 1 -- 172.21.15.3:0/3083604112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007944571 0 0) 0x7f8ad00cf930 con 0x7f8ad00cd7d0 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.319+0000 7f8ace000640 1 -- 172.21.15.3:0/3083604112 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ab4003520 con 0x7f8ad00cd7d0 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.320+0000 7f8acf400640 1 -- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 msgr2=0x7f8ad00cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.320+0000 7f8acf400640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 0x7f8ad00cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.320+0000 7f8acf400640 1 -- 172.21.15.3:0/3083604112 >> v1:172.21.15.3:6789/0 conn(0x7f8ad00cd7d0 legacy=0x7f8ad0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.320+0000 7f8acf400640 1 -- 172.21.15.3:0/3083604112 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad00b3bb0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.320+0000 7f8acea00640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 0x7f8ad00cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8acf400640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b4560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d90 tx=0x7f8ac802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9053001fb1108730 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8ace000640 1 -- 172.21.15.3:0/3083604112 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8007740 con 0x7f8ad00b40d0 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8ace000640 1 -- 172.21.15.3:0/3083604112 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac80078e0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8ace000640 1 -- 172.21.15.3:0/3083604112 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8007c90 con 0x7f8ad00b40d0 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8ad5a00640 1 -- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad00b40d0 msgr2=0x7f8ad00b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.321+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b4560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009d90 tx=0x7f8ac802a660 comp rx=0 tx=0).stop 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 -- 172.21.15.3:0/3083604112 shutdown_connections 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b4560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/3083604112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad0162ac0 0x7f8ad00cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 -- 172.21.15.3:0/3083604112 >> 172.21.15.3:0/3083604112 conn(0x7f8ad006d000 msgr2=0x7f8ad006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 -- 172.21.15.3:0/3083604112 shutdown_connections 2024-09-10T07:46:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.322+0000 7f8ad5a00640 1 -- 172.21.15.3:0/3083604112 wait complete. 2024-09-10T07:46:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.323+0000 7f8ad5a00640 1 Processor -- start 2024-09-10T07:46:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.323+0000 7f8ad5a00640 1 -- start start 2024-09-10T07:46:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad00cd7d0 0x7f8ad0164250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad0162ac0 0x7f8ad0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad0162ea0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad0071590 con 0x7f8ad0162ac0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8ad5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad0163020 con 0x7f8ad00cd7d0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8acea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8acea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47942/0 (socket says 172.21.15.3:47942) 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.324+0000 7f8acea00640 1 -- 172.21.15.3:0/2797899239 learned_addr learned my addr 172.21.15.3:0/2797899239 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acf400640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad0162ac0 0x7f8ad0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ac7e00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad00cd7d0 0x7f8ad0164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 -- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad00cd7d0 msgr2=0x7f8ad0164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad00cd7d0 0x7f8ad0164250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 -- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad0162ac0 msgr2=0x7f8ad0157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad0162ac0 0x7f8ad0157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 -- 172.21.15.3:0/2797899239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad00b0b20 con 0x7f8ad00b40d0 2024-09-10T07:46:41.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8acea00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e960 tx=0x7f8ac000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0017070 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac000ca00 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad00b18a0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000cdd0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.325+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad00b10a0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.326+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad00cdc10 con 0x7f8ad00b40d0 2024-09-10T07:46:41.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.327+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ac001a070 con 0x7f8ad00b40d0 2024-09-10T07:46:41.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.329+0000 7f8ac6a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 0x7f8aac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.329+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8ac008ad50 con 0x7f8ad00b40d0 2024-09-10T07:46:41.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.329+0000 7f8ac7e00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 0x7f8aac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.329+0000 7f8ac7e00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 0x7f8aac06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ad01fbba0 tx=0x7f8ab8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.333+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ac00596f0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.521+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ad00b4790 con 0x7f8ad00b40d0 2024-09-10T07:46:41.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.522+0000 7f8ac6a00640 1 -- 172.21.15.3:0/2797899239 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f8ac0058ff0 con 0x7f8ad00b40d0 2024-09-10T07:46:41.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:41.522 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.524+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 msgr2=0x7f8aac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.524+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 0x7f8aac06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ad01fbba0 tx=0x7f8ab8009040 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.524+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 msgr2=0x7f8ad00b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.524+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e960 tx=0x7f8ac000ee20 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 shutdown_connections 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8aac06be30 0x7f8aac06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ad0162ac0 0x7f8ad0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ad00cd7d0 0x7f8ad0164250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 --2- 172.21.15.3:0/2797899239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad00b40d0 0x7f8ad00b27d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 >> 172.21.15.3:0/2797899239 conn(0x7f8ad006d000 msgr2=0x7f8ad0071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 shutdown_connections 2024-09-10T07:46:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.525+0000 7f8ad5a00640 1 -- 172.21.15.3:0/2797899239 wait complete. 2024-09-10T07:46:41.541 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T07:46:41.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.698+0000 7fec1fe00640 1 Processor -- start 2024-09-10T07:46:41.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.698+0000 7fec1fe00640 1 -- start start 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.699+0000 7fec1fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec20161a00 0x7fec200d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec20161de0 0x7fec200cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec200d3060 con 0x7fec200d10f0 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1fe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fec200d3340 con 0x7fec20161a00 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fec200d3620 con 0x7fec20161de0 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec20161a00 0x7fec200d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fec200d10f0 0x7fec200b6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52318/0 (socket says 172.21.15.3:52318) 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1f400640 1 -- 172.21.15.3:0/1372068368 learned_addr learned my addr 172.21.15.3:0/1372068368 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec17e00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec20161de0 0x7fec200cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1ea00640 1 -- 172.21.15.3:0/1372068368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec20161de0 msgr2=0x7fec200cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1ea00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec20161de0 0x7fec200cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1ea00640 1 -- 172.21.15.3:0/1372068368 >> v1:172.21.15.3:6789/0 conn(0x7fec200d10f0 legacy=0x7fec200b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1ea00640 1 -- 172.21.15.3:0/1372068368 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec200d3900 con 0x7fec20161a00 2024-09-10T07:46:41.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.700+0000 7fec1e000640 1 -- 172.21.15.3:0/1372068368 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299900799 0 0) 0x7fec200d3060 con 0x7fec200d10f0 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.701+0000 7fec1ea00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161a00 0x7fec200d2a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2977efb05c7d986 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1e000640 1 -- 172.21.15.3:0/1372068368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10037070 con 0x7fec20161a00 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1e000640 1 -- 172.21.15.3:0/1372068368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec10032440 con 0x7fec20161a00 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1e000640 1 -- 172.21.15.3:0/1372068368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10032810 con 0x7fec20161a00 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 -- 172.21.15.3:0/1372068368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161a00 msgr2=0x7fec200d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161a00 0x7fec200d2a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec100097a0 tx=0x7fec1002a180 comp rx=0 tx=0).stop 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 -- 172.21.15.3:0/1372068368 shutdown_connections 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec20161de0 0x7fec200cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 --2- 172.21.15.3:0/1372068368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161a00 0x7fec200d2a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 -- 172.21.15.3:0/1372068368 >> 172.21.15.3:0/1372068368 conn(0x7fec20072770 msgr2=0x7fec2006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 -- 172.21.15.3:0/1372068368 shutdown_connections 2024-09-10T07:46:41.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.702+0000 7fec1fe00640 1 -- 172.21.15.3:0/1372068368 wait complete. 2024-09-10T07:46:41.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.704+0000 7fec1fe00640 1 Processor -- start 2024-09-10T07:46:41.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.704+0000 7fec1fe00640 1 -- start start 2024-09-10T07:46:41.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec200d10f0 0x7fec20157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec20161a00 0x7fec200b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec200b8a00 con 0x7fec20161de0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fec200cfe40 con 0x7fec20161a00 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fec200b8b80 con 0x7fec200d10f0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47966/0 (socket says 172.21.15.3:47966) 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1f400640 1 -- 172.21.15.3:0/388729559 learned_addr learned my addr 172.21.15.3:0/388729559 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec1ea00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec200d10f0 0x7fec20157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.705+0000 7fec17e00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec20161a00 0x7fec200b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 -- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec200d10f0 msgr2=0x7fec20157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec200d10f0 0x7fec20157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 -- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec20161a00 msgr2=0x7fec200b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec20161a00 0x7fec200b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 -- 172.21.15.3:0/388729559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec20158160 con 0x7fec20161de0 2024-09-10T07:46:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1f400640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c7a0 tx=0x7fec1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18018070 con 0x7fec20161de0 2024-09-10T07:46:41.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1800a640 con 0x7fec20161de0 2024-09-10T07:46:41.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800aab0 con 0x7fec20161de0 2024-09-10T07:46:41.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec200b60f0 con 0x7fec20161de0 2024-09-10T07:46:41.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.706+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec200b66b0 con 0x7fec20161de0 2024-09-10T07:46:41.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.707+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec200d2a80 con 0x7fec20161de0 2024-09-10T07:46:41.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.708+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec1800ac50 con 0x7fec20161de0 2024-09-10T07:46:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.709+0000 7fec16a00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 0x7febfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.709+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fec1808a4e0 con 0x7fec20161de0 2024-09-10T07:46:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.709+0000 7fec1ea00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 0x7febfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.710+0000 7fec1ea00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 0x7febfc06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec10034000 tx=0x7fec1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:41.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.714+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec180587a0 con 0x7fec20161de0 2024-09-10T07:46:41.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.898+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec200cec40 con 0x7fec20161de0 2024-09-10T07:46:41.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.900+0000 7fec16a00640 1 -- 172.21.15.3:0/388729559 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7fec1805caa0 con 0x7fec20161de0 2024-09-10T07:46:41.900 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:41.900 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 msgr2=0x7febfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 0x7febfc06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec10034000 tx=0x7fec1002e000 comp rx=0 tx=0).stop 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 msgr2=0x7fec200b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c7a0 tx=0x7fec1800cc60 comp rx=0 tx=0).stop 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 shutdown_connections 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7febfc06be30 0x7febfc06e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec20161de0 0x7fec200b7da0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec20161a00 0x7fec200b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 --2- 172.21.15.3:0/388729559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fec200d10f0 0x7fec20157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:41.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 >> 172.21.15.3:0/388729559 conn(0x7fec20072770 msgr2=0x7fec20160620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:41.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 shutdown_connections 2024-09-10T07:46:41.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:41.902+0000 7fec1fe00640 1 -- 172.21.15.3:0/388729559 wait complete. 2024-09-10T07:46:41.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T07:46:41.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T07:46:41.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T07:46:41.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T07:46:41.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T07:46:41.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T07:46:41.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T07:46:41.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T07:46:42.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.197+0000 7f0f0f200640 1 Processor -- start 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.198+0000 7f0f0f200640 1 -- start start 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.199+0000 7f0f0f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f080e2ba0 0x7f0f08007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f08006280 con 0x7f0f08037b20 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0f08006560 con 0x7f0f080e2ba0 2024-09-10T07:46:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f08006840 con 0x7f0f08005d80 2024-09-10T07:46:42.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f64a1200640 1 Processor -- start 2024-09-10T07:46:42.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51278/0 (socket says 172.21.15.3:51278) 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.200+0000 7f0f0e800640 1 -- 172.21.15.3:0/1770852040 learned_addr learned my addr 172.21.15.3:0/1770852040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f64a1200640 1 -- start start 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f0f0de00640 1 --2- 172.21.15.3:0/1770852040 >> v2:172.21.15.3:3300/0 conn(0x7f0f080e2ba0 0x7f0f08007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f0f0e800640 1 -- 172.21.15.3:0/1770852040 >> v2:172.21.15.3:3300/0 conn(0x7f0f080e2ba0 msgr2=0x7f0f08007320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f0f0e800640 1 --2- 172.21.15.3:0/1770852040 >> v2:172.21.15.3:3300/0 conn(0x7f0f080e2ba0 0x7f0f08007320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f0f0e800640 1 -- 172.21.15.3:0/1770852040 >> v1:172.21.15.3:6789/0 conn(0x7f0f08037b20 legacy=0x7f0f0803bbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.201+0000 7f0f0e800640 1 -- 172.21.15.3:0/1770852040 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08006b20 con 0x7f0f08005d80 2024-09-10T07:46:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.202+0000 7f0f0e800640 1 --2- 172.21.15.3:0/1770852040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f1007b950 tx=0x7f0f100a60b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca0f468e39b3007 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f64a1200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c0b3590 0x7f649c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f64a1200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c15acf0 con 0x7f649c15a7e0 2024-09-10T07:46:42.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7fb9ed200640 1 Processor -- start 2024-09-10T07:46:42.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f0f0ca00640 1 -- 172.21.15.3:0/1770852040 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f100b3070 con 0x7f0f08005d80 2024-09-10T07:46:42.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7fb9ed200640 1 -- start start 2024-09-10T07:46:42.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f64a1200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c15afd0 con 0x7f649c0cd760 2024-09-10T07:46:42.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f64a1200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f649c15b900 con 0x7f649c0b3590 2024-09-10T07:46:42.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f0f0ca00640 1 -- 172.21.15.3:0/1770852040 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f100ae440 con 0x7f0f08005d80 2024-09-10T07:46:42.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.203+0000 7f0f0ca00640 1 -- 172.21.15.3:0/1770852040 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f100ae8b0 con 0x7f0f08005d80 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.204+0000 7f649be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f649c15a7e0 0x7f649c06bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52334/0 (socket says 172.21.15.3:52334) 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.204+0000 7f649be00640 1 -- 172.21.15.3:0/269894904 learned_addr learned my addr 172.21.15.3:0/269894904 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.204+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1154990906 0 0) 0x7f649c15acf0 con 0x7f649c15a7e0 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6480003520 con 0x7f649c15a7e0 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 143734274 0 0) 0x7f6480003520 con 0x7f649c15a7e0 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c0b3590 msgr2=0x7f649c0b2000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c0b3590 0x7f649c0b2000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 >> v1:172.21.15.70:6789/0 conn(0x7f649c0cd760 legacy=0x7f649c0cdb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T07:46:42.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c15b2b0 con 0x7f649c15a7e0 2024-09-10T07:46:42.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f64940019d0 con 0x7f649c15a7e0 2024-09-10T07:46:42.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 >> v1:172.21.15.3:6789/0 conn(0x7f649c15a7e0 legacy=0x7f649c06bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f0f0f200640 1 -- 172.21.15.3:0/1770852040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 msgr2=0x7f0f0803c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003b00 0x7f6480003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f0f0f200640 1 --2- 172.21.15.3:0/1770852040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f1007b950 tx=0x7f0f100a60b0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f0f0f200640 1 -- 172.21.15.3:0/1770852040 shutdown_connections 2024-09-10T07:46:42.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f0f0f200640 1 --2- 172.21.15.3:0/1770852040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08005d80 0x7f0f0803c2d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f0f0f200640 1 --2- 172.21.15.3:0/1770852040 >> v2:172.21.15.3:3300/0 conn(0x7f0f080e2ba0 0x7f0f08007320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f0f0f200640 1 -- 172.21.15.3:0/1770852040 >> 172.21.15.3:0/1770852040 conn(0x7f0f0803a510 msgr2=0x7f0f0803a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9ed200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e80cf930 con 0x7fb9e80b40d0 2024-09-10T07:46:42.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f0f0f200640 1 -- 172.21.15.3:0/1770852040 shutdown_connections 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51308/0 (socket says 172.21.15.3:51308) 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9e5e00640 1 -- 172.21.15.3:0/1565003273 learned_addr learned my addr 172.21.15.3:0/1565003273 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9ed200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9e80cfc10 con 0x7fb9e80cd7d0 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e80b38d0 con 0x7fb9e8162ac0 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9e6800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9e80b40d0 0x7fb9e80d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52338/0 (socket says 172.21.15.3:52338) 2024-09-10T07:46:42.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7fb9e5400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb9e80cd7d0 0x7fb9e8070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46970/0 (socket says 172.21.15.3:46970) 2024-09-10T07:46:42.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.207+0000 7f0f0f200640 1 -- 172.21.15.3:0/1770852040 wait complete. 2024-09-10T07:46:42.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800053c0 0x7f648000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f649c15b900 con 0x7f6480004800 2024-09-10T07:46:42.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f64800106f0 con 0x7f64800053c0 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.205+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f64800108f0 con 0x7f6480003b00 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7fb9e5e00640 1 -- 172.21.15.3:0/1565003273 >> v1:172.21.15.70:6789/0 conn(0x7fb9e80cd7d0 legacy=0x7fb9e8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7fb9e5e00640 1 -- 172.21.15.3:0/1565003273 >> v1:172.21.15.3:6789/0 conn(0x7fb9e80b40d0 legacy=0x7fb9e80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7fb9e5e00640 1 -- 172.21.15.3:0/1565003273 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e80b3bb0 con 0x7fb9e8162ac0 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f789ec00640 1 Processor -- start 2024-09-10T07:46:42.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f789ec00640 1 -- start start 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f649b400640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003b00 0x7f6480003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.206+0000 7f649aa00640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.207+0000 7f649be00640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800053c0 0x7f648000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f649b400640 1 -- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800053c0 msgr2=0x7f648000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f649b400640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800053c0 0x7f648000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f649b400640 1 -- 172.21.15.3:0/269894904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480004800 msgr2=0x7f6480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f649b400640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.208+0000 7f649b400640 1 -- 172.21.15.3:0/269894904 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c15b2b0 con 0x7f6480003b00 2024-09-10T07:46:42.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.209+0000 7f649aa00640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480004800 0x7f6480004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.209+0000 7f0f0f200640 1 Processor -- start 2024-09-10T07:46:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7f0f0f200640 1 -- start start 2024-09-10T07:46:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7fb9e5e00640 1 --2- 172.21.15.3:0/1565003273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098a0 tx=0x7fb9d802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd1bc4847fd09dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7f649b400640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003b00 0x7f6480003f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f648c009370 tx=0x7f6480005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52fbf532eb88437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7fb9e4a00640 1 -- 172.21.15.3:0/1565003273 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9e8162ac0 2024-09-10T07:46:42.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7fb9e4a00640 1 -- 172.21.15.3:0/1565003273 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d802ae50 con 0x7fb9e8162ac0 2024-09-10T07:46:42.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7f789ec00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f789806daf0 0x7f78981516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7f789ec00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78980d01b0 0x7f78980d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.210+0000 7f789ec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789815e1c0 con 0x7f78980b35b0 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f789ec00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f78980d0b80 con 0x7f789806daf0 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f789ec00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f78980b3a40 con 0x7f78980d01b0 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c048070 con 0x7f6480003b00 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c03ba20 con 0x7f6480003b00 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f7897400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f78980b35b0 0x7f78980cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46976/0 (socket says 172.21.15.3:46976) 2024-09-10T07:46:42.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f7897400640 1 -- 172.21.15.3:0/4193797068 learned_addr learned my addr 172.21.15.3:0/4193797068 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7f7897e00640 1 --2- 172.21.15.3:0/4193797068 >> v2:172.21.15.3:3300/0 conn(0x7f789806daf0 0x7f78981516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 msgr2=0x7fb9e80cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965152633 0 0) 0x7f789815e1c0 con 0x7f78980b35b0 2024-09-10T07:46:42.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f789cc00640 1 --2- 172.21.15.3:0/4193797068 >> v2:172.21.15.70:3300/0 conn(0x7f78980d01b0 0x7f78980d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7ff89ee00640 1 Processor -- start 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f649a000640 1 -- 172.21.15.3:0/269894904 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0093b0 con 0x7f6480003b00 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f64a1200640 1 -- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003b00 msgr2=0x7f6480003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f64a1200640 1 --2- 172.21.15.3:0/269894904 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6480003b00 0x7f6480003f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f648c009370 tx=0x7f6480005830 comp rx=0 tx=0).stop 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f649be00640 1 -- 172.21.15.3:0/269894904 reap_dead start 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f64a1200640 1 -- 172.21.15.3:0/269894904 shutdown_connections 2024-09-10T07:46:42.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f64a1200640 1 -- 172.21.15.3:0/269894904 >> 172.21.15.3:0/269894904 conn(0x7f649c06d000 msgr2=0x7f649c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.211+0000 7fb9ed200640 1 --2- 172.21.15.3:0/1565003273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098a0 tx=0x7fb9d802a300 comp rx=0 tx=0).stop 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f787c003520 con 0x7f78980b35b0 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7897e00640 1 -- 172.21.15.3:0/4193797068 >> v2:172.21.15.70:3300/0 conn(0x7f78980d01b0 msgr2=0x7f78980d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7897e00640 1 --2- 172.21.15.3:0/4193797068 >> v2:172.21.15.70:3300/0 conn(0x7f78980d01b0 0x7f78980d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7897e00640 1 -- 172.21.15.3:0/4193797068 >> v1:172.21.15.70:6789/0 conn(0x7f78980b35b0 legacy=0x7f78980cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7897e00640 1 -- 172.21.15.3:0/4193797068 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789815e4a0 con 0x7f789806daf0 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722917777 0 0) 0x7f787c003520 con 0x7f78980b35b0 2024-09-10T07:46:42.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.212+0000 7ff89ee00640 1 -- start start 2024-09-10T07:46:42.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7f64a1200640 1 -- 172.21.15.3:0/269894904 shutdown_connections 2024-09-10T07:46:42.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 shutdown_connections 2024-09-10T07:46:42.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7fb9ed200640 1 --2- 172.21.15.3:0/1565003273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e8162ac0 0x7fb9e80cf250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 >> 172.21.15.3:0/1565003273 conn(0x7fb9e806d000 msgr2=0x7fb9e806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f08005d80 0x7f0f0804c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.213+0000 7f64a1200640 1 -- 172.21.15.3:0/269894904 wait complete. 2024-09-10T07:46:42.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08037b20 0x7f0f08178560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f080e2ba0 0x7f0f08051e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 0x7ff8980cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 0x7ff8980d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8980d1510 0x7ff8980d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff8980cf540 con 0x7ff8980b6690 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff8980cff10 con 0x7ff8980d1510 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff8980d1d90 con 0x7ff8980b6a70 2024-09-10T07:46:42.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7f0f0f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0803c990 con 0x7f0f080e2ba0 2024-09-10T07:46:42.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 shutdown_connections 2024-09-10T07:46:42.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7f7897e00640 1 --2- 172.21.15.3:0/4193797068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f789806daf0 0x7f78981516c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f789815abd0 tx=0x7f788802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cbb4ed36788d95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7ff89ce00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8980d1510 0x7ff8980d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff897400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 0x7ff8980d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff897400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 0x7ff8980d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51312/0 (socket says 172.21.15.3:51312) 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8980d1510 0x7ff8980d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48134/0 (socket says 172.21.15.3:48134) 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff897400640 1 -- 172.21.15.3:0/1211090629 learned_addr learned my addr 172.21.15.3:0/1211090629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff897e00640 1 --2- 172.21.15.3:0/1211090629 >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 0x7ff8980cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 -- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 msgr2=0x7ff8980d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 --2- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 0x7ff8980d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 -- 172.21.15.3:0/1211090629 >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 msgr2=0x7ff8980cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 --2- 172.21.15.3:0/1211090629 >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 0x7ff8980cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7ff89ce00640 1 -- 172.21.15.3:0/1211090629 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8980ceef0 con 0x7ff8980d1510 2024-09-10T07:46:42.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888037070 con 0x7f789806daf0 2024-09-10T07:46:42.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788802ab40 con 0x7f789806daf0 2024-09-10T07:46:42.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f7896a00640 1 -- 172.21.15.3:0/4193797068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884001670 con 0x7f789806daf0 2024-09-10T07:46:42.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f64a1200640 1 Processor -- start 2024-09-10T07:46:42.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f64a1200640 1 -- start start 2024-09-10T07:46:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7f0f0e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f080e2ba0 0x7f0f08051e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7f0f0e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f080e2ba0 0x7f0f08051e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48012/0 (socket says 172.21.15.3:48012) 2024-09-10T07:46:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.215+0000 7f0f0e800640 1 -- 172.21.15.3:0/3193594936 learned_addr learned my addr 172.21.15.3:0/3193594936 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f0f0de00640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f08005d80 0x7f0f0804c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7ff897e00640 1 --2- 172.21.15.3:0/1211090629 >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 0x7ff8980cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0803c810 con 0x7f0f08005d80 2024-09-10T07:46:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.216+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080374d0 con 0x7f0f08037b20 2024-09-10T07:46:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f0f0d400640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08037b20 0x7f0f08178560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.214+0000 7fb9ed200640 1 -- 172.21.15.3:0/1565003273 wait complete. 2024-09-10T07:46:42.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7fb9ed200640 1 Processor -- start 2024-09-10T07:46:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7fcd1b400640 1 Processor -- start 2024-09-10T07:46:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7fcd1b400640 1 -- start start 2024-09-10T07:46:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480003b00 0x7f649c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 0x7f649c1efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7fb9ed200640 1 -- start start 2024-09-10T07:46:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649c1f0260 0x7f649c1f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f649c06dca0 con 0x7f6480003b00 2024-09-10T07:46:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f649c06db20 con 0x7f649c1f0260 2024-09-10T07:46:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.217+0000 7f64a1200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f649c06c3e0 con 0x7f649c1578a0 2024-09-10T07:46:42.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 -- 172.21.15.3:0/4193797068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f789806daf0 msgr2=0x7f78981516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f0f0de00640 1 -- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08037b20 msgr2=0x7f0f08178560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 --2- 172.21.15.3:0/4193797068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f789806daf0 0x7f78981516c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f789815abd0 tx=0x7f788802a0b0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f0f0de00640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f08037b20 0x7f0f08178560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f0f0de00640 1 -- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f080e2ba0 msgr2=0x7f0f08051e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f0f0de00640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f080e2ba0 0x7f0f08051e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f0f0de00640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f080d5590 con 0x7f0f08005d80 2024-09-10T07:46:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f649aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 0x7f649c1efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 -- 172.21.15.3:0/4193797068 shutdown_connections 2024-09-10T07:46:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 --2- 172.21.15.3:0/4193797068 >> v2:172.21.15.70:3300/0 conn(0x7f78980d01b0 0x7f78980d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 --2- 172.21.15.3:0/4193797068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f789806daf0 0x7f78981516c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7efc1ec00640 1 Processor -- start 2024-09-10T07:46:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f649be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649c1f0260 0x7f649c1f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f649aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 0x7f649c1efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51326/0 (socket says 172.21.15.3:51326) 2024-09-10T07:46:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f649aa00640 1 -- 172.21.15.3:0/1008734352 learned_addr learned my addr 172.21.15.3:0/1008734352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f789ec00640 1 -- 172.21.15.3:0/4193797068 >> 172.21.15.3:0/4193797068 conn(0x7f789806d000 msgr2=0x7f789806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.218+0000 7f649b400640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480003b00 0x7f649c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649be00640 1 -- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 msgr2=0x7f649c1efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649be00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 0x7f649c1efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649be00640 1 -- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480003b00 msgr2=0x7f649c157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649be00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6480003b00 0x7f649c157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649be00640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c157db0 con 0x7f649c1f0260 2024-09-10T07:46:42.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f649aa00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f649c1578a0 0x7f649c1efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f0f0de00640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f08005d80 0x7f0f0804c530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c5f0 tx=0x7f0efc00cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fcd1b400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd140d1130 0x7fcd140cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 0x7fb9e80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9e80cd7d0 0x7fb9e8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e8162ac0 0x7fb9e815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7ff89ce00640 1 --2- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980d1510 0x7ff8980d2b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff89808d960 tx=0x7ff88802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb3adc053db0f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f789ec00640 1 -- 172.21.15.3:0/4193797068 shutdown_connections 2024-09-10T07:46:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7ff896a00640 1 -- 172.21.15.3:0/1211090629 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888037070 con 0x7ff8980d1510 2024-09-10T07:46:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fcd1b400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd1415f520 0x7fcd140cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fcd1b400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd140d0090 con 0x7fcd1415e590 2024-09-10T07:46:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fcd1b400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1406fe20 con 0x7fcd1415f520 2024-09-10T07:46:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fcd1b400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcd140d0370 con 0x7fcd140d1130 2024-09-10T07:46:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7efc1ec00640 1 -- start start 2024-09-10T07:46:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8162ea0 con 0x7fb9e80cd7d0 2024-09-10T07:46:42.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8071590 con 0x7fb9e8162ac0 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7ff896a00640 1 -- 172.21.15.3:0/1211090629 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff888032440 con 0x7ff8980d1510 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7ff896a00640 1 -- 172.21.15.3:0/1211090629 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888032850 con 0x7ff8980d1510 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9ed200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e8163020 con 0x7fb9e80b40d0 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 0x7fb9e80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 0x7fb9e80b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51334/0 (socket says 172.21.15.3:51334) 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7fb9e5e00640 1 -- 172.21.15.3:0/3812628132 learned_addr learned my addr 172.21.15.3:0/3812628132 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e6800640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e8162ac0 0x7fb9e815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5400640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9e80cd7d0 0x7fb9e8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7f649be00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649c1f0260 0x7f649c1f06e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f649c15abc0 tx=0x7f6494007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5e00640 1 -- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e8162ac0 msgr2=0x7fb9e815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fcd13e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd140d1130 0x7fcd140cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fcd13e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd140d1130 0x7fcd140cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48170/0 (socket says 172.21.15.3:48170) 2024-09-10T07:46:42.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fcd13e00640 1 -- 172.21.15.3:0/3337228470 learned_addr learned my addr 172.21.15.3:0/3337228470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fcd19400640 1 --2- 172.21.15.3:0/3337228470 >> v2:172.21.15.3:3300/0 conn(0x7fcd1415f520 0x7fcd140cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5e00640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e8162ac0 0x7fb9e815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 -- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980d1510 msgr2=0x7ff8980d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 --2- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980d1510 0x7ff8980d2b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff89808d960 tx=0x7ff88802a130 comp rx=0 tx=0).stop 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5e00640 1 -- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9e80cd7d0 msgr2=0x7fb9e8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5e00640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9e80cd7d0 0x7fb9e8155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e5e00640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e8156010 con 0x7fb9e80b40d0 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.220+0000 7fb9e6800640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9e8162ac0 0x7fb9e815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 -- 172.21.15.3:0/1211090629 shutdown_connections 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 --2- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980d1510 0x7ff8980d2b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 --2- 172.21.15.3:0/1211090629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980b6a70 0x7ff8980d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 --2- 172.21.15.3:0/1211090629 >> v2:172.21.15.3:3300/0 conn(0x7ff8980b6690 0x7ff8980cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 -- 172.21.15.3:0/1211090629 >> 172.21.15.3:0/1211090629 conn(0x7ff898072560 msgr2=0x7ff898072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7fcd13e00640 1 -- 172.21.15.3:0/3337228470 >> v2:172.21.15.3:3300/0 conn(0x7fcd1415f520 msgr2=0x7fcd140cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64880093f0 con 0x7f649c1f0260 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940123b0 con 0x7f649c1f0260 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c158090 con 0x7f649c1f0260 2024-09-10T07:46:42.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c0add40 con 0x7f649c1f0260 2024-09-10T07:46:42.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7efc1ec00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efc180d16e0 0x7efc180d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc1ec00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc180d1b10 con 0x7efc180b6690 2024-09-10T07:46:42.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc1ec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc180d1d90 con 0x7efc180b6a70 2024-09-10T07:46:42.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc1ec00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7efc180cf540 con 0x7efc180d16e0 2024-09-10T07:46:42.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7fb9e5e00640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 0x7fb9e80b2040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098a0 tx=0x7fb9d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 -- 172.21.15.3:0/1211090629 shutdown_connections 2024-09-10T07:46:42.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7ff89ee00640 1 -- 172.21.15.3:0/1211090629 wait complete. 2024-09-10T07:46:42.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494006030 con 0x7f649c1f0260 2024-09-10T07:46:42.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc015070 con 0x7f0f08005d80 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.219+0000 7f789ec00640 1 -- 172.21.15.3:0/4193797068 wait complete. 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7f789ec00640 1 Processor -- start 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc17400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efc180b6a70 0x7efc180d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46980/0 (socket says 172.21.15.3:46980) 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc17400640 1 -- 172.21.15.3:0/1674487966 learned_addr learned my addr 172.21.15.3:0/1674487966 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0efc011800 con 0x7f0f08005d80 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc011bd0 con 0x7f0f08005d80 2024-09-10T07:46:42.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f080d5870 con 0x7f0f08005d80 2024-09-10T07:46:42.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7efc1cc00640 1 --2- 172.21.15.3:0/1674487966 >> v2:172.21.15.70:3300/0 conn(0x7efc180d16e0 0x7efc180d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7fcd13e00640 1 --2- 172.21.15.3:0/3337228470 >> v2:172.21.15.3:3300/0 conn(0x7fcd1415f520 0x7fcd140cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7fcd13e00640 1 -- 172.21.15.3:0/3337228470 >> v1:172.21.15.3:6789/0 conn(0x7fcd1415e590 legacy=0x7fcd1415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7fcd13e00640 1 -- 172.21.15.3:0/3337228470 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140d0650 con 0x7fcd140d1130 2024-09-10T07:46:42.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.221+0000 7fcd13400640 1 -- 172.21.15.3:0/3337228470 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521988195 0 0) 0x7fcd140d0090 con 0x7fcd1415e590 2024-09-10T07:46:42.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.222+0000 7ffa7b800640 1 Processor -- start 2024-09-10T07:46:42.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7ffa7b800640 1 -- start start 2024-09-10T07:46:42.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f649c06c870 con 0x7f649c1f0260 2024-09-10T07:46:42.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8042020 con 0x7fb9e80b40d0 2024-09-10T07:46:42.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d8030a70 con 0x7fb9e80b40d0 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e80b2700 con 0x7fb9e80b40d0 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e80b2cc0 con 0x7fb9e80b40d0 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037040 con 0x7fb9e80b40d0 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7efc1cc00640 1 -- 172.21.15.3:0/1674487966 >> v1:172.21.15.70:6789/0 conn(0x7efc180b6a70 legacy=0x7efc180d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7efc1cc00640 1 -- 172.21.15.3:0/1674487966 >> v1:172.21.15.3:6789/0 conn(0x7efc180b6690 legacy=0x7efc180cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7efc1cc00640 1 -- 172.21.15.3:0/1674487966 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc180ceef0 con 0x7efc180d16e0 2024-09-10T07:46:42.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0804cc40 con 0x7f0f08005d80 2024-09-10T07:46:42.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.223+0000 7f789ec00640 1 -- start start 2024-09-10T07:46:42.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7fcd13e00640 1 --2- 172.21.15.3:0/3337228470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140d1130 0x7fcd140cf440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd140cefb0 tx=0x7fcd0002c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b395a6e17daa6715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ee8001030 con 0x7f0f08005d80 2024-09-10T07:46:42.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa74062d80 0x7ffa740ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740cea50 0x7ffa740d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 0x7ffa740d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7406bce0 con 0x7ffa740b0890 2024-09-10T07:46:42.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7406bfc0 con 0x7ffa74062d80 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.224+0000 7ffa7b800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffa740cdd10 con 0x7ffa740cea50 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ffa78e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa74062d80 0x7ffa740ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ffa78e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa74062d80 0x7ffa740ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48174/0 (socket says 172.21.15.3:48174) 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ffa78e00640 1 -- 172.21.15.3:0/183563449 learned_addr learned my addr 172.21.15.3:0/183563449 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ffa73e00640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740cea50 0x7ffa740d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f789ec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f789806daf0 0x7f78980739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f789ec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78980b35b0 0x7f78982016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f789ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78980d01b0 0x7f789815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ffa79800640 1 --2- 172.21.15.3:0/183563449 >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 0x7ffa740d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ff89ee00640 1 Processor -- start 2024-09-10T07:46:42.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7ff89ee00640 1 -- start start 2024-09-10T07:46:42.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64940061d0 con 0x7f649c1f0260 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7f7897400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78980b35b0 0x7f78982016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7f7897400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78980b35b0 0x7f78982016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51354/0 (socket says 172.21.15.3:51354) 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7f7897400640 1 -- 172.21.15.3:0/3692516114 learned_addr learned my addr 172.21.15.3:0/3692516114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7fcd13400640 1 -- 172.21.15.3:0/3337228470 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00036020 con 0x7fcd140d1130 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7fcd13400640 1 -- 172.21.15.3:0/3337228470 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd00032830 con 0x7fcd140d1130 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7fcd13400640 1 -- 172.21.15.3:0/3337228470 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00032c00 con 0x7fcd140d1130 2024-09-10T07:46:42.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7efc1cc00640 1 --2- 172.21.15.3:0/1674487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180d16e0 0x7efc180d2b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d90 tx=0x7efc1002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19cad13ebdb41552 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa78e00640 1 -- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740cea50 msgr2=0x7ffa740d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa78e00640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740cea50 0x7ffa740d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa78e00640 1 -- 172.21.15.3:0/183563449 >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 msgr2=0x7ffa740d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa78e00640 1 --2- 172.21.15.3:0/183563449 >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 0x7ffa740d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa78e00640 1 -- 172.21.15.3:0/183563449 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa740cdff0 con 0x7ffa74062d80 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.226+0000 7ffa79800640 1 --2- 172.21.15.3:0/183563449 >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 0x7ffa740d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7fcd1b400640 1 -- 172.21.15.3:0/3337228470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140d1130 msgr2=0x7fcd140cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7fcd1b400640 1 --2- 172.21.15.3:0/3337228470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140d1130 0x7fcd140cf440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd140cefb0 tx=0x7fcd0002c930 comp rx=0 tx=0).stop 2024-09-10T07:46:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.225+0000 7f789ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f789815ed40 con 0x7f78980d01b0 2024-09-10T07:46:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f789815ea60 con 0x7f789806daf0 2024-09-10T07:46:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f789815f0b0 con 0x7f78980b35b0 2024-09-10T07:46:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f789cc00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78980d01b0 0x7f789815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7efc16a00640 1 -- 172.21.15.3:0/1674487966 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10007740 con 0x7efc180d16e0 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7efc16a00640 1 -- 172.21.15.3:0/1674487966 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc100078e0 con 0x7efc180d16e0 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7efc16a00640 1 -- 172.21.15.3:0/1674487966 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10007cb0 con 0x7efc180d16e0 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f7897e00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f789806daf0 0x7f78980739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8980b6690 0x7ff898151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980b6a70 0x7ff8981f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 0x7ff8981612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8980d19a0 con 0x7ff8980b6690 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff8980b6ed0 con 0x7ff8980b6a70 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7ff89ee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff8980d1b20 con 0x7ff8980d1510 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9d8045020 con 0x7fb9e80b40d0 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f6493400640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f647c06bf30 0x7f647c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f649408b170 con 0x7f649c1f0260 2024-09-10T07:46:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f6493400640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7f647c074150 0x7f647c076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6494010e70 con 0x7f647c074150 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f649408b560 con 0x7f649c1f0260 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f789cc00640 1 -- 172.21.15.3:0/3692516114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78980b35b0 msgr2=0x7f78982016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.227+0000 7f789cc00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78980b35b0 0x7f78982016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f789cc00640 1 -- 172.21.15.3:0/3692516114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f789806daf0 msgr2=0x7f78980739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f789cc00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f789806daf0 0x7f78980739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f789cc00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7898074090 con 0x7f78980d01b0 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 -- 172.21.15.3:0/3337228470 shutdown_connections 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 --2- 172.21.15.3:0/3337228470 >> v2:172.21.15.3:3300/0 conn(0x7fcd1415f520 0x7fcd140cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 --2- 172.21.15.3:0/3337228470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd140d1130 0x7fcd140cf440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 -- 172.21.15.3:0/3337228470 >> 172.21.15.3:0/3337228470 conn(0x7fcd14072ef0 msgr2=0x7fcd1406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7ff89ce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 0x7ff8981612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7efc1ec00640 1 -- 172.21.15.3:0/1674487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180d16e0 msgr2=0x7efc180d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7efc1ec00640 1 --2- 172.21.15.3:0/1674487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180d16e0 0x7efc180d2b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d90 tx=0x7efc1002a6a0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f789cc00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78980d01b0 0x7f789815c5b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f789000efc0 tx=0x7f789000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f649b400640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f647c06bf30 0x7f647c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 -- 172.21.15.3:0/3337228470 shutdown_connections 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7ff897e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8980b6690 0x7ff898151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7ff897400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980b6a70 0x7ff8981f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7ff89ce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 0x7ff8981612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51366/0 (socket says 172.21.15.3:51366) 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7ff89ce00640 1 -- 172.21.15.3:0/809337703 learned_addr learned my addr 172.21.15.3:0/809337703 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7efc1ec00640 1 -- 172.21.15.3:0/1674487966 shutdown_connections 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7efc1ec00640 1 --2- 172.21.15.3:0/1674487966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180d16e0 0x7efc180d2b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7efc1ec00640 1 -- 172.21.15.3:0/1674487966 >> 172.21.15.3:0/1674487966 conn(0x7efc18072560 msgr2=0x7efc18072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7f649aa00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7f647c074150 0x7f647c076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890018020 con 0x7f78980d01b0 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7890014480 con 0x7f78980d01b0 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890014870 con 0x7f78980d01b0 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7898071230 con 0x7f78980d01b0 2024-09-10T07:46:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78980717f0 con 0x7f78980d01b0 2024-09-10T07:46:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.228+0000 7fcd1b400640 1 -- 172.21.15.3:0/3337228470 wait complete. 2024-09-10T07:46:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ffa78e00640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa74062d80 0x7ffa740ce510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa740d2340 tx=0x7ffa6802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e37c84707a7804 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7fb9de000640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb9c806be30 0x7fb9c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9c0001030 con 0x7fb9e80b40d0 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ff89ce00640 1 -- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980b6a70 msgr2=0x7ff8981f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ff89ce00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980b6a70 0x7ff8981f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ff89ce00640 1 -- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8980b6690 msgr2=0x7ff898151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7efc1ec00640 1 -- 172.21.15.3:0/1674487966 shutdown_connections 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ff89ce00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8980b6690 0x7ff898151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.229+0000 7ff89ce00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8981f6390 con 0x7ff8980d1510 2024-09-10T07:46:42.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0efc018070 con 0x7f0f08005d80 2024-09-10T07:46:42.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7f649b400640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f647c06bf30 0x7f647c06e3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f648c045000 tx=0x7f648c03f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7fb9e5400640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb9c806be30 0x7fb9c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7ffa73400640 1 -- 172.21.15.3:0/183563449 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68036070 con 0x7ffa74062d80 2024-09-10T07:46:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7ffa73400640 1 -- 172.21.15.3:0/183563449 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa68031600 con 0x7ffa74062d80 2024-09-10T07:46:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7ffa73400640 1 -- 172.21.15.3:0/183563449 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa680319d0 con 0x7ffa74062d80 2024-09-10T07:46:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7ff89ce00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 0x7ff8981612b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff888009890 tx=0x7ff8880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ff897400640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8980b6a70 0x7ff8981f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7f649aa00640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7f647c074150 0x7f647c076570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ff897e00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8980b6690 0x7ff898151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888037050 con 0x7ff8980d1510 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7f0f06000640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0ef006bf00 0x7f0ef006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0efc08aa50 con 0x7f0f08005d80 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7f0f06000640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f0ef0074180 0x7f0ef00765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ef0076c50 con 0x7f0ef0074180 2024-09-10T07:46:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0efc08f2a0 con 0x7f0f08005d80 2024-09-10T07:46:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff888043070 con 0x7ff8980d1510 2024-09-10T07:46:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888030cc0 con 0x7ff8980d1510 2024-09-10T07:46:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7fcd1b400640 1 Processor -- start 2024-09-10T07:46:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7fcd1b400640 1 -- start start 2024-09-10T07:46:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ffa7b800640 1 -- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa74062d80 msgr2=0x7ffa740ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ffa7b800640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa74062d80 0x7ffa740ce510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa740d2340 tx=0x7ffa6802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7f0f0e800640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f0ef0074180 0x7f0ef00765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8981f6670 con 0x7ff8980d1510 2024-09-10T07:46:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.230+0000 7efc1ec00640 1 -- 172.21.15.3:0/1674487966 wait complete. 2024-09-10T07:46:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8981f6c30 con 0x7ff8980d1510 2024-09-10T07:46:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7efc1ec00640 1 Processor -- start 2024-09-10T07:46:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb9d80abb00 con 0x7fb9e80b40d0 2024-09-10T07:46:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fb9de000640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7fb9c8073fd0 0x7fb9c80763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9d8001d30 con 0x7fb9c8073fd0 2024-09-10T07:46:42.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb9d8075880 con 0x7fb9e80b40d0 2024-09-10T07:46:42.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6494010e70 con 0x7f647c074150 2024-09-10T07:46:42.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1415e590 0x7fcd1414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd1415f520 0x7fcd14156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7efc1ec00640 1 -- start start 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fb9e6800640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7fb9c8073fd0 0x7fb9c80763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fb9e5400640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb9c806be30 0x7fb9c806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8156720 tx=0x7fb9d4007ce0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.231+0000 7ffa7b800640 1 -- 172.21.15.3:0/183563449 shutdown_connections 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ffa7b800640 1 --2- 172.21.15.3:0/183563449 >> v2:172.21.15.3:3300/0 conn(0x7ffa740b0890 0x7ffa740d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ffa7b800640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740cea50 0x7ffa740d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ffa7b800640 1 --2- 172.21.15.3:0/183563449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa74062d80 0x7ffa740ce510 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa740d2340 tx=0x7ffa6802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ffa7b800640 1 -- 172.21.15.3:0/183563449 >> 172.21.15.3:0/183563449 conn(0x7ffa740732b0 msgr2=0x7ffa740736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.232+0000 7ffa7b800640 1 -- 172.21.15.3:0/183563449 shutdown_connections 2024-09-10T07:46:42.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7ffa7b800640 1 -- 172.21.15.3:0/183563449 wait complete. 2024-09-10T07:46:42.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff898069140 con 0x7ff8980d1510 2024-09-10T07:46:42.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 0x7fcd140b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1415e970 con 0x7fcd1415e590 2024-09-10T07:46:42.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140d0c10 con 0x7fcd141570e0 2024-09-10T07:46:42.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7fcd1b400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1415eaf0 con 0x7fcd1415f520 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7fcd19400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 0x7fcd140b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7fcd19400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 0x7fcd140b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48218/0 (socket says 172.21.15.3:48218) 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7fcd19400640 1 -- 172.21.15.3:0/527258596 learned_addr learned my addr 172.21.15.3:0/527258596 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7fcd18a00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1415e590 0x7fcd1414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 -- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd1415f520 msgr2=0x7fcd14156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd1415f520 0x7fcd14156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 -- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1415e590 msgr2=0x7fcd1414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fb9e6800640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7fb9c8073fd0 0x7fb9c80763f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.233+0000 7f0f0d400640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0ef006bf00 0x7f0ef006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1415e590 0x7fcd1414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14157620 con 0x7fcd141570e0 2024-09-10T07:46:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd18a00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1415e590 0x7fcd1414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7efc1ec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efc180b6690 0x7efc18154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.234+0000 7f0f0e800640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f0ef0074180 0x7f0ef00765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7efc1ec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 0x7efc181596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== osd.1 v2:172.21.15.3:6800/2429800591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ef0076c50 con 0x7f0ef0074180 2024-09-10T07:46:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc1ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc180d16e0 0x7efc181e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7fcd19400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 0x7fcd140b5c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd04007e40 tx=0x7fcd0400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7ffa7b800640 1 Processor -- start 2024-09-10T07:46:42.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.235+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7890005d10 con 0x7f78980d01b0 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc17400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 0x7efc181596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc17400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 0x7efc181596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48224/0 (socket says 172.21.15.3:48224) 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc17e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efc180b6690 0x7efc18154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc17e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efc180b6690 0x7efc18154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51390/0 (socket says 172.21.15.3:51390) 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc17400640 1 -- 172.21.15.3:0/1180955387 learned_addr learned my addr 172.21.15.3:0/1180955387 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc1cc00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc180d16e0 0x7efc181e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7898071020 con 0x7f78980d01b0 2024-09-10T07:46:42.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04014620 con 0x7fcd141570e0 2024-09-10T07:46:42.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc1815c530 con 0x7efc180d16e0 2024-09-10T07:46:42.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efc180b6f00 con 0x7efc180b6a70 2024-09-10T07:46:42.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.237+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efc1815c6b0 con 0x7efc180b6690 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04019070 con 0x7fcd141570e0 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04014a20 con 0x7fcd141570e0 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.237+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd14157900 con 0x7fcd141570e0 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.237+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd14157f30 con 0x7fcd141570e0 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.236+0000 7ffa7b800640 1 -- start start 2024-09-10T07:46:42.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7f0f0d400640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0ef006bf00 0x7f0ef006e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f080d4740 tx=0x7f0f000090e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.237+0000 7f788fe00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f787806bf30 0x7f787806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f789008bb50 con 0x7f78980d01b0 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7f788fe00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f7878074130 0x7f7878076550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f789000be50 con 0x7f7878074130 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7890092050 con 0x7f78980d01b0 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7f7897e00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f787806bf30 0x7f787806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7f7897400640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f7878074130 0x7f7878076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.237+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff888065480 con 0x7ff8980d1510 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fccec001030 con 0x7fcd141570e0 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7efc17400640 1 -- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efc180b6690 msgr2=0x7efc18154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7efc17400640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efc180b6690 0x7efc18154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7efc17400640 1 -- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc180d16e0 msgr2=0x7efc181e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7efc17400640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc180d16e0 0x7efc181e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7efc17400640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc18159f20 con 0x7efc180b6a70 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740b0890 0x7ffa7415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa740cea50 0x7ffa7409da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 0x7ffa7409e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa740b6df0 con 0x7ffa7409df60 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7406ff20 con 0x7ffa740cea50 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.238+0000 7ffa7b800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffa740b6f70 con 0x7ffa740b0890 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 0x7ffa7409e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 0x7ffa7409e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48132/0 (socket says 172.21.15.3:48132) 2024-09-10T07:46:42.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 -- 172.21.15.3:0/3630958315 learned_addr learned my addr 172.21.15.3:0/3630958315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ff88fe00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff87806bf00 0x7ff87806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 -- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740b0890 msgr2=0x7ffa7415a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa740b0890 0x7ffa7415a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 -- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa740cea50 msgr2=0x7ffa7409da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa740cea50 0x7ffa7409da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7ffa79800640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa741f7ca0 con 0x7ffa7409df60 2024-09-10T07:46:42.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.239+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== osd.2 v2:172.21.15.3:6824/2150697764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9d8001d30 con 0x7fb9c8073fd0 2024-09-10T07:46:42.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7f7897400640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f7878074130 0x7f7878076550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa79800640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 0x7ffa7409e860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e960 tx=0x7ffa6c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff8880abe90 con 0x7ff8980d1510 2024-09-10T07:46:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c017070 con 0x7ffa7409df60 2024-09-10T07:46:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c004020 con 0x7ffa7409df60 2024-09-10T07:46:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa7b800640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa741f7f40 con 0x7ffa7409df60 2024-09-10T07:46:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ff88fe00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7ff878074120 0x7ff878076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0043f0 con 0x7ffa7409df60 2024-09-10T07:46:42.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ff897e00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff87806bf00 0x7ff87806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.241+0000 7f7897e00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f787806bf30 0x7f787806e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7888034000 tx=0x7f788802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff888034ec0 con 0x7ff878074120 2024-09-10T07:46:42.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.240+0000 7ffa7b800640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa741f8500 con 0x7ffa7409df60 2024-09-10T07:46:42.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.241+0000 7ff897400640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7ff878074120 0x7ff878076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.241+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff8880ac280 con 0x7ff8980d1510 2024-09-10T07:46:42.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.241+0000 7efc17400640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 0x7efc181596d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c460 tx=0x7efc0800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7ffa7b800640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa50001030 con 0x7ffa7409df60 2024-09-10T07:46:42.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08005ce0 con 0x7efc180b6a70 2024-09-10T07:46:42.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f789000be50 con 0x7f7878074130 2024-09-10T07:46:42.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc08007070 con 0x7efc180b6a70 2024-09-10T07:46:42.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08007460 con 0x7efc180b6a70 2024-09-10T07:46:42.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1815a200 con 0x7efc180b6a70 2024-09-10T07:46:42.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1815a7c0 con 0x7efc180b6a70 2024-09-10T07:46:42.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7ff897400640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7ff878074120 0x7ff878076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.241+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd04014bc0 con 0x7fcd141570e0 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.242+0000 7fcd10c00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fccf406bf00 0x7fccf406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.243+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcd0408c260 con 0x7fcd141570e0 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.243+0000 7fcd10c00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7fccf4074120 0x7fccf4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.243+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd04016e70 con 0x7fccf4074120 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.243+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcd0408c650 con 0x7fcd141570e0 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.243+0000 7fcd18a00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fccf406bf00 0x7fccf406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.244+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff888034ec0 con 0x7ff878074120 2024-09-10T07:46:42.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.244+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efbf0001030 con 0x7efc180b6a70 2024-09-10T07:46:42.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.244+0000 7fcd13e00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7fccf4074120 0x7fccf4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.245+0000 7fcd18a00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fccf406bf00 0x7fccf406e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000f60 tx=0x7fcd0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.245+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa6c00ca00 con 0x7ffa7409df60 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.246+0000 7ffa70c00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa5406bf00 0x7ffa5406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.246+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffa6c08af30 con 0x7ffa7409df60 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.246+0000 7ffa70c00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7ffa540740a0 0x7ffa540764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.246+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa6c007e50 con 0x7ffa540740a0 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.246+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c055320 con 0x7ffa7409df60 2024-09-10T07:46:42.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.247+0000 7fcd13e00640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7fccf4074120 0x7fccf4076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.247+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc08007600 con 0x7efc180b6a70 2024-09-10T07:46:42.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.247+0000 7ffa73e00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7ffa540740a0 0x7ffa540764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.247+0000 7ffa73e00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7ffa540740a0 0x7ffa540764c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.248+0000 7ff897e00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff87806bf00 0x7ff87806e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e490 tx=0x7ff8900075f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.248+0000 7ffa78e00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa5406bf00 0x7ffa5406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== osd.7 v2:172.21.15.70:6808/3218842598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa6c007e50 con 0x7ffa540740a0 2024-09-10T07:46:42.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7ffa78e00640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa5406bf00 0x7ffa5406e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa68033000 tx=0x7ffa68009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== osd.5 v2:172.21.15.70:6824/2669649466 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd04016e70 con 0x7fccf4074120 2024-09-10T07:46:42.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.248+0000 7efc0fe00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7efbf406be30 0x7efbf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.248+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efc0808aba0 con 0x7efc180b6a70 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7efc0fe00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7efbf4073fd0 0x7efbf40763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efbf4076aa0 con 0x7efbf4073fd0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc0808b030 con 0x7efc180b6a70 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7efc1cc00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7efbf4073fd0 0x7efbf40763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.249+0000 7efc17e00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7efbf406be30 0x7efbf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.253+0000 7efc17e00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7efbf406be30 0x7efbf406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc00000f60 tx=0x7efc00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.253+0000 7efc1cc00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7efbf4073fd0 0x7efbf40763f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.254+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efbf4076aa0 con 0x7efbf4073fd0 2024-09-10T07:46:42.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.267+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f649c075040 con 0x7f647c074150 2024-09-10T07:46:42.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.268+0000 7f6493400640 1 -- 172.21.15.3:0/1008734352 <== osd.0 v2:172.21.15.3:6808/1494406974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f649c075040 con 0x7f647c074150 2024-09-10T07:46:42.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7f647c074150 msgr2=0x7f647c076570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7f647c074150 0x7f647c076570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ee8002dd0 con 0x7f0ef0074180 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f647c06bf30 msgr2=0x7f647c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f647c06bf30 0x7f647c06e3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f648c045000 tx=0x7f648c03f000 comp rx=0 tx=0).stop 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649c1f0260 msgr2=0x7f649c1f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 --2- 172.21.15.3:0/1008734352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649c1f0260 0x7f649c1f06e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f649c15abc0 tx=0x7f6494007770 comp rx=0 tx=0).stop 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f649be00640 1 -- 172.21.15.3:0/1008734352 reap_dead start 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 shutdown_connections 2024-09-10T07:46:42.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.271+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 >> 172.21.15.3:0/1008734352 conn(0x7f649c06d000 msgr2=0x7f649c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.272+0000 7f0f06000640 1 -- 172.21.15.3:0/3193594936 <== osd.1 v2:172.21.15.3:6800/2429800591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ee8002dd0 con 0x7f0ef0074180 2024-09-10T07:46:42.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.272+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 --> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9c0002dd0 con 0x7fb9c8073fd0 2024-09-10T07:46:42.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.272+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 shutdown_connections 2024-09-10T07:46:42.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.272+0000 7f64a1200640 1 -- 172.21.15.3:0/1008734352 wait complete. 2024-09-10T07:46:42.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.273+0000 7fb9de000640 1 -- 172.21.15.3:0/3812628132 <== osd.2 v2:172.21.15.3:6824/2150697764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb9c0002dd0 con 0x7fb9c8073fd0 2024-09-10T07:46:42.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.273+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f0ef0074180 msgr2=0x7f0ef00765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.273+0000 7f0f0f200640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f0ef0074180 0x7f0ef00765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0ef006bf00 msgr2=0x7f0ef006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f0f0f200640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0ef006bf00 0x7f0ef006e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f080d4740 tx=0x7f0f000090e0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f08005d80 msgr2=0x7f0f0804c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f0f0f200640 1 --2- 172.21.15.3:0/3193594936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f08005d80 0x7f0f0804c530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c5f0 tx=0x7f0efc00cab0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78980719f0 con 0x7f7878074130 2024-09-10T07:46:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.274+0000 7f0f0e800640 1 -- 172.21.15.3:0/3193594936 reap_dead start 2024-09-10T07:46:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7fb9c8073fd0 msgr2=0x7fb9c80763f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7fb9c8073fd0 0x7fb9c80763f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 shutdown_connections 2024-09-10T07:46:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 >> 172.21.15.3:0/3193594936 conn(0x7f0f0803a510 msgr2=0x7f0f08038480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb9c806be30 msgr2=0x7fb9c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7f788fe00640 1 -- 172.21.15.3:0/3692516114 <== osd.3 v2:172.21.15.3:6816/1816157518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78980719f0 con 0x7f7878074130 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 shutdown_connections 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb9c806be30 0x7fb9c806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8156720 tx=0x7fb9d4007ce0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 msgr2=0x7fb9e80b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.275+0000 7fb9ed200640 1 --2- 172.21.15.3:0/3812628132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9e80b40d0 0x7fb9e80b2040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098a0 tx=0x7fb9d80046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7fb9e6800640 1 -- 172.21.15.3:0/3812628132 reap_dead start 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 shutdown_connections 2024-09-10T07:46:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 >> 172.21.15.3:0/3812628132 conn(0x7fb9e806d000 msgr2=0x7fb9e806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 shutdown_connections 2024-09-10T07:46:42.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7f0f0f200640 1 -- 172.21.15.3:0/3193594936 wait complete. 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f7878074130 msgr2=0x7f7878076550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.276+0000 7f789ec00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f7878074130 0x7f7878076550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f787806bf30 msgr2=0x7f787806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f787806bf30 0x7f787806e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7888034000 tx=0x7f788802e000 comp rx=0 tx=0).stop 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78980d01b0 msgr2=0x7f789815c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 --2- 172.21.15.3:0/3692516114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78980d01b0 0x7f789815c5b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f789000efc0 tx=0x7f789000c490 comp rx=0 tx=0).stop 2024-09-10T07:46:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff898160fb0 con 0x7ff878074120 2024-09-10T07:46:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789cc00640 1 -- 172.21.15.3:0/3692516114 reap_dead start 2024-09-10T07:46:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7fb9ed200640 1 -- 172.21.15.3:0/3812628132 wait complete. 2024-09-10T07:46:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 shutdown_connections 2024-09-10T07:46:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.277+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 >> 172.21.15.3:0/3692516114 conn(0x7f789806d000 msgr2=0x7f789806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.278+0000 7ff88fe00640 1 -- 172.21.15.3:0/809337703 <== osd.4 v2:172.21.15.70:6800/2990268368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff898160fb0 con 0x7ff878074120 2024-09-10T07:46:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.278+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 shutdown_connections 2024-09-10T07:46:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.278+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7ff878074120 msgr2=0x7ff878076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7ff89ee00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7ff878074120 0x7ff878076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.278+0000 7f789ec00640 1 -- 172.21.15.3:0/3692516114 wait complete. 2024-09-10T07:46:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fccec002dd0 con 0x7fccf4074120 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff87806bf00 msgr2=0x7ff87806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7ff89ee00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff87806bf00 0x7ff87806e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e490 tx=0x7ff8900075f0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 msgr2=0x7ff8981612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.279+0000 7ff89ee00640 1 --2- 172.21.15.3:0/809337703 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8980d1510 0x7ff8981612b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff888009890 tx=0x7ff8880046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ff89ce00640 1 -- 172.21.15.3:0/809337703 reap_dead start 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 shutdown_connections 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 >> 172.21.15.3:0/809337703 conn(0x7ff898072560 msgr2=0x7ff8980b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 shutdown_connections 2024-09-10T07:46:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ff89ee00640 1 -- 172.21.15.3:0/809337703 wait complete. 2024-09-10T07:46:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7fcd10c00640 1 -- 172.21.15.3:0/527258596 <== osd.5 v2:172.21.15.70:6824/2669649466 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fccec002dd0 con 0x7fccf4074120 2024-09-10T07:46:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.280+0000 7ffa7b800640 1 -- 172.21.15.3:0/3630958315 --> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa50002dd0 con 0x7ffa540740a0 2024-09-10T07:46:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.281+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7fccf4074120 msgr2=0x7fccf4076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.281+0000 7fcd1b400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7fccf4074120 0x7fccf4076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fccf406bf00 msgr2=0x7fccf406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fccf406bf00 0x7fccf406e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000f60 tx=0x7fcd0c009040 comp rx=0 tx=0).stop 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 msgr2=0x7fcd140b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 --2- 172.21.15.3:0/527258596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd141570e0 0x7fcd140b5c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd04007e40 tx=0x7fcd0400e4b0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7ffa70c00640 1 -- 172.21.15.3:0/3630958315 <== osd.7 v2:172.21.15.70:6808/3218842598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa50002dd0 con 0x7ffa540740a0 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd19400640 1 -- 172.21.15.3:0/527258596 reap_dead start 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 shutdown_connections 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.282+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 >> 172.21.15.3:0/527258596 conn(0x7fcd14072ef0 msgr2=0x7fcd1406f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.283+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 shutdown_connections 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.283+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7ffa540740a0 msgr2=0x7ffa540764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.283+0000 7ffa66000640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7ffa540740a0 0x7ffa540764c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.284+0000 7fcd1b400640 1 -- 172.21.15.3:0/527258596 wait complete. 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.285+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa5406bf00 msgr2=0x7ffa5406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.285+0000 7ffa66000640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa5406bf00 0x7ffa5406e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa68033000 tx=0x7ffa68009000 comp rx=0 tx=0).stop 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.285+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 msgr2=0x7ffa7409e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.285+0000 7ffa66000640 1 --2- 172.21.15.3:0/3630958315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa7409df60 0x7ffa7409e860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e960 tx=0x7ffa6c00ee20 comp rx=0 tx=0).stop 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.285+0000 7ffa79800640 1 -- 172.21.15.3:0/3630958315 reap_dead start 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.288+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 shutdown_connections 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.288+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 >> 172.21.15.3:0/3630958315 conn(0x7ffa740732b0 msgr2=0x7ffa740ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.288+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 shutdown_connections 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.288+0000 7ffa66000640 1 -- 172.21.15.3:0/3630958315 wait complete. 2024-09-10T07:46:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.289+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efbf0002dd0 con 0x7efbf4073fd0 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.290+0000 7efc0fe00640 1 -- 172.21.15.3:0/1180955387 <== osd.6 v2:172.21.15.70:6816/3978225537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efbf0002dd0 con 0x7efbf4073fd0 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.291+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7efbf4073fd0 msgr2=0x7efbf40763f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.291+0000 7efc1ec00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7efbf4073fd0 0x7efbf40763f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.291+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7efbf406be30 msgr2=0x7efbf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1ec00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7efbf406be30 0x7efbf406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc00000f60 tx=0x7efc00009040 comp rx=0 tx=0).stop 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 msgr2=0x7efc181596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1ec00640 1 --2- 172.21.15.3:0/1180955387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc180b6a70 0x7efc181596d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c460 tx=0x7efc0800c920 comp rx=0 tx=0).stop 2024-09-10T07:46:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1cc00640 1 -- 172.21.15.3:0/1180955387 reap_dead start 2024-09-10T07:46:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 shutdown_connections 2024-09-10T07:46:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.292+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 >> 172.21.15.3:0/1180955387 conn(0x7efc18072560 msgr2=0x7efc180cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.293+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 shutdown_connections 2024-09-10T07:46:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.293+0000 7efc1ec00640 1 -- 172.21.15.3:0/1180955387 wait complete. 2024-09-10T07:46:42.318 INFO:teuthology.orchestra.run.smithi003.stdout:68719476743 2024-09-10T07:46:42.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T07:46:42.319 INFO:teuthology.orchestra.run.smithi003.stdout:68719476742 2024-09-10T07:46:42.320 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-09-10T07:46:42.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T07:46:42.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T07:46:42.321 INFO:teuthology.orchestra.run.smithi003.stdout:68719476744 2024-09-10T07:46:42.322 INFO:teuthology.orchestra.run.smithi003.stdout:68719476744 2024-09-10T07:46:42.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T07:46:42.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T07:46:42.324 INFO:teuthology.orchestra.run.smithi003.stdout:64424509446 2024-09-10T07:46:42.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T07:46:42.325 INFO:teuthology.orchestra.run.smithi003.stdout:64424509447 2024-09-10T07:46:42.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T07:46:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:64424509447 2024-09-10T07:46:42.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T07:46:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.640+0000 7ff91f200640 1 Processor -- start 2024-09-10T07:46:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.640+0000 7ff91f200640 1 -- start start 2024-09-10T07:46:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.640+0000 7f0d59e00640 1 Processor -- start 2024-09-10T07:46:42.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.641+0000 7f0d59e00640 1 -- start start 2024-09-10T07:46:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.642+0000 7f0d59e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.643+0000 7f0d59e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d540d16e0 0x7f0d540d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.643+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.643+0000 7ff91f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 0x7ff920162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.643+0000 7ff91f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff920162790 0x7ff92015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.643+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51410/0 (socket says 172.21.15.3:51410) 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91d400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 0x7ff920162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91de00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff920144c60 0x7ff9200cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:47006/0 (socket says 172.21.15.3:47006) 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91d400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 0x7ff920162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48238/0 (socket says 172.21.15.3:48238) 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91f200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9200ce710 con 0x7ff920144c60 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff9200ce9f0 con 0x7ff920162790 2024-09-10T07:46:42.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91de00640 1 -- 172.21.15.3:0/2349759197 learned_addr learned my addr 172.21.15.3:0/2349759197 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91e800640 1 --2- 172.21.15.3:0/2349759197 >> v2:172.21.15.3:3300/0 conn(0x7ff920162790 0x7ff92015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff9200cecd0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91ca00640 1 -- 172.21.15.3:0/2349759197 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414196623 0 0) 0x7ff9200ce710 con 0x7ff920144c60 2024-09-10T07:46:42.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91ca00640 1 -- 172.21.15.3:0/2349759197 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff904003520 con 0x7ff920144c60 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d53400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d540d16e0 0x7f0d540d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d53400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d540d16e0 0x7f0d540d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48142/0 (socket says 172.21.15.3:48142) 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d52000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d540b6a70 0x7f0d540d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46990/0 (socket says 172.21.15.3:46990) 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d52a00640 1 -- 172.21.15.3:0/293545726 learned_addr learned my addr 172.21.15.3:0/293545726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d540cff10 con 0x7f0d540b6a70 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.644+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0d540d1b70 con 0x7f0d540d16e0 2024-09-10T07:46:42.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540cf540 con 0x7f0d540b6690 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d51600640 1 -- 172.21.15.3:0/293545726 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267166496 0 0) 0x7f0d540cff10 con 0x7f0d540b6a70 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d51600640 1 -- 172.21.15.3:0/293545726 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d38003520 con 0x7f0d540b6a70 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d53400640 1 -- 172.21.15.3:0/293545726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 msgr2=0x7f0d540cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d53400640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d53400640 1 -- 172.21.15.3:0/293545726 >> v1:172.21.15.70:6789/0 conn(0x7f0d540b6a70 legacy=0x7f0d540d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7f0d53400640 1 -- 172.21.15.3:0/293545726 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540d1d90 con 0x7f0d540d16e0 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91e800640 1 -- 172.21.15.3:0/2349759197 >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 msgr2=0x7ff920162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91e800640 1 --2- 172.21.15.3:0/2349759197 >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 0x7ff920162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91e800640 1 -- 172.21.15.3:0/2349759197 >> v1:172.21.15.70:6789/0 conn(0x7ff920144c60 legacy=0x7ff9200cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.645+0000 7ff91e800640 1 -- 172.21.15.3:0/2349759197 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9200cefb0 con 0x7ff920162790 2024-09-10T07:46:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.646+0000 7f0d52a00640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.647+0000 7ff91e800640 1 --2- 172.21.15.3:0/2349759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff92015f2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9200cdcc0 tx=0x7ff91002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38519ae309d9abaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.647+0000 7ff91ca00640 1 -- 172.21.15.3:0/2349759197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910037070 con 0x7ff920162790 2024-09-10T07:46:42.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7f0d53400640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540d16e0 0x7f0d540d2b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d400097a0 tx=0x7f0d4003b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e3602461f34be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7ff91ca00640 1 -- 172.21.15.3:0/2349759197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910032440 con 0x7ff920162790 2024-09-10T07:46:42.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7f0d51600640 1 -- 172.21.15.3:0/293545726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40048070 con 0x7f0d540d16e0 2024-09-10T07:46:42.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7ff91ca00640 1 -- 172.21.15.3:0/2349759197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910032850 con 0x7ff920162790 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7f0d51600640 1 -- 172.21.15.3:0/293545726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40043440 con 0x7f0d540d16e0 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.648+0000 7f0d51600640 1 -- 172.21.15.3:0/293545726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40048070 con 0x7f0d540d16e0 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540d16e0 msgr2=0x7f0d540d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7f0d59e00640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540d16e0 0x7f0d540d2b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d400097a0 tx=0x7f0d4003b190 comp rx=0 tx=0).stop 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 msgr2=0x7ff92015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 --2- 172.21.15.3:0/2349759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff92015f2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9200cdcc0 tx=0x7ff91002a1c0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 shutdown_connections 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540d16e0 0x7f0d540d2b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 --2- 172.21.15.3:0/293545726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540b6690 0x7f0d540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 >> 172.21.15.3:0/293545726 conn(0x7f0d54072560 msgr2=0x7f0d54072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 shutdown_connections 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 --2- 172.21.15.3:0/2349759197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff92015f2b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 --2- 172.21.15.3:0/2349759197 >> v2:172.21.15.70:3300/0 conn(0x7ff9200ce1c0 0x7ff920162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 >> 172.21.15.3:0/2349759197 conn(0x7ff92006d000 msgr2=0x7ff92006d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.649+0000 7fd734e00640 1 Processor -- start 2024-09-10T07:46:42.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7fb7cac00640 1 Processor -- start 2024-09-10T07:46:42.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 shutdown_connections 2024-09-10T07:46:42.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 shutdown_connections 2024-09-10T07:46:42.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7fd734e00640 1 -- start start 2024-09-10T07:46:42.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7f0d59e00640 1 -- 172.21.15.3:0/293545726 wait complete. 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd7280e6f70 0x7fd728034d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd7280333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd728049780 0x7fd728049c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd728033c80 con 0x7fd7280e6f70 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd728033f60 con 0x7fd728049780 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fd734e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd728034240 con 0x7fd728035d90 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd7280333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7fb7cac00640 1 -- start start 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd728049780 0x7fd728049c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd728049780 0x7fd728049c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48240/0 (socket says 172.21.15.3:48240) 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 -- 172.21.15.3:0/2795918186 learned_addr learned my addr 172.21.15.3:0/2795918186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fb7cac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c40cf250 0x7fb7c4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fb7cac00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c40cf690 con 0x7fb7c4162ac0 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fb7cac00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c40cf930 con 0x7fb7c40713c0 2024-09-10T07:46:42.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.652+0000 7fb7cac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c40cfc10 con 0x7fb7c40cf250 2024-09-10T07:46:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7f0d59e00640 1 Processor -- start 2024-09-10T07:46:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7f0d59e00640 1 -- start start 2024-09-10T07:46:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd7280333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51416/0 (socket says 172.21.15.3:51416) 2024-09-10T07:46:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 -- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 msgr2=0x7fd7280333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 --2- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd7280333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 -- 172.21.15.3:0/2795918186 >> v2:172.21.15.3:3300/0 conn(0x7fd7280e6f70 msgr2=0x7fd728034d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 --2- 172.21.15.3:0/2795918186 >> v2:172.21.15.3:3300/0 conn(0x7fd7280e6f70 0x7fd728034d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7fd72fe00640 1 -- 172.21.15.3:0/2795918186 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd728034520 con 0x7fd728049780 2024-09-10T07:46:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.654+0000 7fb7c3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb7c4162ac0 0x7fb7c40cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52372/0 (socket says 172.21.15.3:52372) 2024-09-10T07:46:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.654+0000 7fb7c3e00640 1 -- 172.21.15.3:0/3844810515 learned_addr learned my addr 172.21.15.3:0/3844810515 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.654+0000 7fab0ce00640 1 Processor -- start 2024-09-10T07:46:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.654+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c40cf250 0x7fb7c4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7f0d59e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.650+0000 7ff91f200640 1 -- 172.21.15.3:0/2349759197 wait complete. 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7ff91f200640 1 Processor -- start 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.653+0000 7ff91f200640 1 -- start start 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.654+0000 7ff91f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 0x7ff9200718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 0x7ff92013cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff9200b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff92015f7f0 con 0x7ff920162790 2024-09-10T07:46:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff920145040 con 0x7ff9200ce1c0 2024-09-10T07:46:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff92015f970 con 0x7ff920144c60 2024-09-10T07:46:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7f0d59e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 0x7f0d5415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 0x7ff92013cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff9200b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 0x7ff92013cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51436/0 (socket says 172.21.15.3:51436) 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7ff91d400640 1 -- 172.21.15.3:0/4001001668 learned_addr learned my addr 172.21.15.3:0/4001001668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48172/0 (socket says 172.21.15.3:48172) 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7f0d52a00640 1 -- 172.21.15.3:0/632928792 learned_addr learned my addr 172.21.15.3:0/632928792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d52000640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 0x7f0d5415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.655+0000 7fab0ce00640 1 -- start start 2024-09-10T07:46:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7ff91de00640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 0x7ff9200718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 0x7f0d5414a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fd72fe00640 1 --2- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd728049780 0x7fd728049c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd73007b950 tx=0x7fd7300a6090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ecec02a74322a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7ff91de00640 1 -- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 msgr2=0x7ff92013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7ff91de00640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 0x7ff92013cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7ff91de00640 1 -- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 msgr2=0x7ff9200b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fd72e000640 1 -- 172.21.15.3:0/2795918186 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300b3070 con 0x7fd728049780 2024-09-10T07:46:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fd72e000640 1 -- 172.21.15.3:0/2795918186 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7300ae440 con 0x7fd728049780 2024-09-10T07:46:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436461355 0 0) 0x7fb7c40cf690 con 0x7fb7c4162ac0 2024-09-10T07:46:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fd72e000640 1 -- 172.21.15.3:0/2795918186 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300ae8b0 con 0x7fd728049780 2024-09-10T07:46:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7a8003520 con 0x7fb7c4162ac0 2024-09-10T07:46:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540b6ed0 con 0x7f0d540b6690 2024-09-10T07:46:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7ff91de00640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff9200b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7ff91de00640 1 -- 172.21.15.3:0/4001001668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92013d6e0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4093713301 0 0) 0x7fb7a8003520 con 0x7fb7c4162ac0 2024-09-10T07:46:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c40cf250 msgr2=0x7fb7c4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7ff91e800640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff9200b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fab0ce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab080b6690 0x7fab080cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fab0ce00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab080b6a70 0x7fab080d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fd734e00640 1 -- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd728049780 msgr2=0x7fd728049c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 --2- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd728049780 0x7fd728049c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd73007b950 tx=0x7fd7300a6090 comp rx=0 tx=0).stop 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.657+0000 7fb7c3400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c40cf250 0x7fb7c4070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 -- 172.21.15.3:0/2795918186 shutdown_connections 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 --2- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd728049780 0x7fd728049c10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 --2- 172.21.15.3:0/2795918186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd7280333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 --2- 172.21.15.3:0/2795918186 >> v2:172.21.15.3:3300/0 conn(0x7fd7280e6f70 0x7fd728034d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd734e00640 1 -- 172.21.15.3:0/2795918186 >> 172.21.15.3:0/2795918186 conn(0x7fd72801fb40 msgr2=0x7fd7280567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fab06800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab080d1510 0x7fab080d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52374/0 (socket says 172.21.15.3:52374) 2024-09-10T07:46:42.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fab05400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab080b6a70 0x7fab080d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fab05e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab080b6690 0x7fab080cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd329e00640 1 Processor -- start 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 >> v1:172.21.15.70:6789/0 conn(0x7fb7c40713c0 legacy=0x7fb7c40b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c40b38d0 con 0x7fb7c4162ac0 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fb7ac001670 con 0x7fb7c4162ac0 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 >> v1:172.21.15.3:6789/0 conn(0x7fb7c4162ac0 legacy=0x7fb7c40cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fab05400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab080b6a70 0x7fab080d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48174/0 (socket says 172.21.15.3:48174) 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fab06800640 1 -- 172.21.15.3:0/2183966672 learned_addr learned my addr 172.21.15.3:0/2183966672 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:42.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a80039e0 0x7fb7a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a80051c0 0x7fb7a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40cfc10 con 0x7fb7a80039e0 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a800bfb0 con 0x7fb7a8004800 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a800c1b0 con 0x7fb7a80051c0 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7fd329e00640 1 -- start start 2024-09-10T07:46:42.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7ff91de00640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 0x7ff9200718f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c400 tx=0x7ff90c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540cf1d0 con 0x7f0d540b6a70 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5415c530 con 0x7f0d540d16e0 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.656+0000 7f0d53400640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 0x7f0d5414a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52000640 1 -- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 msgr2=0x7f0d5414a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52000640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 0x7f0d5414a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52000640 1 -- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 msgr2=0x7f0d5415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52000640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080cff10 con 0x7fab080d1510 2024-09-10T07:46:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52000640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5414c310 con 0x7f0d540b6a70 2024-09-10T07:46:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.658+0000 7f0d52a00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c017070 con 0x7ff9200ce1c0 2024-09-10T07:46:42.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7f0d53400640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 0x7f0d5414a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c005ce0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0072f0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fab080cf540 con 0x7fab080b6a70 2024-09-10T07:46:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab080d1d90 con 0x7fab080b6690 2024-09-10T07:46:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7ff91f200640 1 -- 172.21.15.3:0/4001001668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92013d9c0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fb7c8c00640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a80051c0 0x7fb7a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fb7c3e00640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a80039e0 0x7fb7a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983194894 0 0) 0x7fab080cff10 con 0x7fab080d1510 2024-09-10T07:46:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaec003520 con 0x7fab080d1510 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd329e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3240b6a70 0x7fd3240d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd329e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 0x7fd3240d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd329e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3240d1d90 con 0x7fd3240b6690 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd329e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd3240cf540 con 0x7fd3240b6a70 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd329e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd3240ceef0 con 0x7fd3240d1510 2024-09-10T07:46:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7f0d52000640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 0x7f0d5415a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b7690 tx=0x7f0d4400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7f4302200640 1 Processor -- start 2024-09-10T07:46:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fb7bb400640 1 -- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a80051c0 msgr2=0x7fb7a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd323400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 0x7fd3240d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd323400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 0x7fd3240d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48278/0 (socket says 172.21.15.3:48278) 2024-09-10T07:46:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.659+0000 7ff91f200640 1 -- 172.21.15.3:0/4001001668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff920071e30 con 0x7ff9200ce1c0 2024-09-10T07:46:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd323400640 1 -- 172.21.15.3:0/3813280339 learned_addr learned my addr 172.21.15.3:0/3813280339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a80051c0 0x7fb7a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3937059660 0 0) 0x7faaec003520 con 0x7fab080d1510 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab080b6690 msgr2=0x7fab080cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab080b6690 0x7fab080cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 >> v2:172.21.15.3:3300/0 conn(0x7fab080b6a70 msgr2=0x7fab080d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 --2- 172.21.15.3:0/2183966672 >> v2:172.21.15.3:3300/0 conn(0x7fab080b6a70 0x7fab080d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080ceef0 con 0x7fab080d1510 2024-09-10T07:46:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f4302200640 1 -- start start 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fd734e00640 1 -- 172.21.15.3:0/2795918186 shutdown_connections 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7faaf8001670 con 0x7fab080d1510 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 >> v1:172.21.15.3:6789/0 conn(0x7fab080d1510 legacy=0x7fab080d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaec0039b0 0x7faaec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faaec004800 0x7faaec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaec0051c0 0x7faaec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab080cf540 con 0x7faaec0039b0 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fab080d1d90 con 0x7faaec0051c0 2024-09-10T07:46:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faaec00c000 con 0x7faaec004800 2024-09-10T07:46:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fab05e00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab080b6690 0x7fab080cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fab05400640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faaec004800 0x7faaec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fd734e00640 1 -- 172.21.15.3:0/2795918186 wait complete. 2024-09-10T07:46:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fab05e00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaec0039b0 0x7faaec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fd322000640 1 --2- 172.21.15.3:0/3813280339 >> v2:172.21.15.3:3300/0 conn(0x7fd3240b6a70 0x7fd3240d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440074e0 con 0x7f0d540b6a70 2024-09-10T07:46:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d44007680 con 0x7f0d540b6a70 2024-09-10T07:46:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44004a00 con 0x7f0d540b6a70 2024-09-10T07:46:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.660+0000 7fb7bb400640 1 -- 172.21.15.3:0/3844810515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a80039e0 msgr2=0x7fb7a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a80039e0 0x7fb7a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7fb7bb400640 1 -- 172.21.15.3:0/3844810515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c40b38d0 con 0x7fb7a8004800 2024-09-10T07:46:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541572b0 con 0x7f0d540b6a70 2024-09-10T07:46:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.661+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5414c7f0 con 0x7f0d540b6a70 2024-09-10T07:46:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fb7c3e00640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a80039e0 0x7fb7a8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fab05e00640 1 -- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faaec004800 msgr2=0x7faaec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fd321600640 1 -- 172.21.15.3:0/3813280339 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1683529531 0 0) 0x7fd3240d1d90 con 0x7fd3240b6690 2024-09-10T07:46:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fd322000640 1 -- 172.21.15.3:0/3813280339 >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 msgr2=0x7fd3240d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff90c01a070 con 0x7ff9200ce1c0 2024-09-10T07:46:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7a8004c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40cc9c0 tx=0x7fb7a8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afe630f7a39ee52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fd322000640 1 --2- 172.21.15.3:0/3813280339 >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 0x7fd3240d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fd322000640 1 -- 172.21.15.3:0/3813280339 >> v1:172.21.15.70:6789/0 conn(0x7fd3240b6690 legacy=0x7fd3240cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fd322000640 1 -- 172.21.15.3:0/3813280339 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240cf1d0 con 0x7fd3240b6a70 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f4302200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42fc0b6690 0x7f42fc0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f4302200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0d16e0 0x7f42fc0d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f4302200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42fc0cff10 con 0x7f42fc0b6a70 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f4302200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f42fc0d1b50 con 0x7f42fc0b6690 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f4302200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f42fc0cf540 con 0x7f42fc0d16e0 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f42fb400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42fc0b6690 0x7f42fc0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f42fb400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42fc0b6690 0x7f42fc0cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48194/0 (socket says 172.21.15.3:48194) 2024-09-10T07:46:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.663+0000 7f42fb400640 1 -- 172.21.15.3:0/1380370080 learned_addr learned my addr 172.21.15.3:0/1380370080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fd734e00640 1 Processor -- start 2024-09-10T07:46:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7f42fbe00640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0d16e0 0x7f42fc0d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fd734e00640 1 -- start start 2024-09-10T07:46:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4036070 con 0x7fb7a8004800 2024-09-10T07:46:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b4031400 con 0x7fb7a8004800 2024-09-10T07:46:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fb7c3400640 1 -- 172.21.15.3:0/3844810515 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4009650 con 0x7fb7a8004800 2024-09-10T07:46:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7ff916000640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 0x7ff90006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff90c08a180 con 0x7ff9200ce1c0 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd728036630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 0x7fd728051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd728033a60 con 0x7fd728049780 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7280338e0 con 0x7fd7280e6f70 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd734e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd728007700 con 0x7fd728035d90 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fab05e00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faaec004800 0x7faaec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fab05e00640 1 -- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaec0051c0 msgr2=0x7faaec00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fab05e00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaec0051c0 0x7faaec00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.662+0000 7fab05e00640 1 -- 172.21.15.3:0/2183966672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080ceef0 con 0x7faaec0039b0 2024-09-10T07:46:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7fd322000640 1 --2- 172.21.15.3:0/3813280339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3240b6a70 0x7fd3240d0fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd3241621f0 tx=0x7fd31002a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919baa9cbf500949 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fab05e00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaec0039b0 0x7faaec003dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab00007e90 tx=0x7faaec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff3ce949f327619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd728036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48292/0 (socket says 172.21.15.3:48292) 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 -- 172.21.15.3:0/3279040415 learned_addr learned my addr 172.21.15.3:0/3279040415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd321600640 1 -- 172.21.15.3:0/3813280339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310036070 con 0x7fd3240b6a70 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd321600640 1 -- 172.21.15.3:0/3813280339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310031600 con 0x7fd3240b6a70 2024-09-10T07:46:42.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7fd321600640 1 -- 172.21.15.3:0/3813280339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310031a10 con 0x7fd3240b6a70 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fb7cac00640 1 -- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 msgr2=0x7fb7a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3844810515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7a8004c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40cc9c0 tx=0x7fb7a8005830 comp rx=0 tx=0).stop 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7f42fb400640 1 -- 172.21.15.3:0/1380370080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0d16e0 msgr2=0x7f42fc0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.664+0000 7f42fb400640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0d16e0 0x7f42fc0d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7f42fb400640 1 -- 172.21.15.3:0/1380370080 >> v1:172.21.15.70:6789/0 conn(0x7f42fc0b6a70 legacy=0x7f42fc0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.665+0000 7f42fb400640 1 -- 172.21.15.3:0/1380370080 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc0d1d90 con 0x7f42fc0b6690 2024-09-10T07:46:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7ff91f200640 1 -- 172.21.15.3:0/4001001668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff920063610 con 0x7ff9200ce1c0 2024-09-10T07:46:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fb7c8c00640 1 -- 172.21.15.3:0/3844810515 reap_dead start 2024-09-10T07:46:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72ea00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 0x7fd728051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 -- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 msgr2=0x7fd728036630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fb7cac00640 1 -- 172.21.15.3:0/3844810515 shutdown_connections 2024-09-10T07:46:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00037070 con 0x7faaec0039b0 2024-09-10T07:46:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fb7cac00640 1 -- 172.21.15.3:0/3844810515 >> 172.21.15.3:0/3844810515 conn(0x7fb7c406d000 msgr2=0x7fb7c406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab00032460 con 0x7faaec0039b0 2024-09-10T07:46:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fab04a00640 1 -- 172.21.15.3:0/2183966672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00007b60 con 0x7faaec0039b0 2024-09-10T07:46:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd728036630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 -- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 msgr2=0x7fd728051ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7fd72fe00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 0x7fd728051ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fd72fe00640 1 -- 172.21.15.3:0/3279040415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72804d920 con 0x7fd7280e6f70 2024-09-10T07:46:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fd72ea00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 0x7fd728051ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fd72fe00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7300518b0 tx=0x7fd730069a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fb7cac00640 1 -- 172.21.15.3:0/3844810515 shutdown_connections 2024-09-10T07:46:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7fb7cac00640 1 -- 172.21.15.3:0/3844810515 wait complete. 2024-09-10T07:46:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.666+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d4401a020 con 0x7f0d540b6a70 2024-09-10T07:46:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c005290 con 0x7f0d540b6a70 2024-09-10T07:46:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.667+0000 7ff91d400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 0x7ff90006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7f0d4b400640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 0x7f0d3406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaec0039b0 msgr2=0x7faaec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fab0ce00640 1 --2- 172.21.15.3:0/2183966672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaec0039b0 0x7faaec003dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab00007e90 tx=0x7faaec005830 comp rx=0 tx=0).stop 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fab06800640 1 -- 172.21.15.3:0/2183966672 reap_dead start 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 -- 172.21.15.3:0/3813280339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3240b6a70 msgr2=0x7fd3240d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 --2- 172.21.15.3:0/3813280339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3240b6a70 0x7fd3240d0fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd3241621f0 tx=0x7fd31002a430 comp rx=0 tx=0).stop 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 shutdown_connections 2024-09-10T07:46:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 >> 172.21.15.3:0/2183966672 conn(0x7fab08072560 msgr2=0x7fab08072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0d4408a420 con 0x7f0d540b6a70 2024-09-10T07:46:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7f0d52a00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 0x7f0d3406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 -- 172.21.15.3:0/3813280339 shutdown_connections 2024-09-10T07:46:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 --2- 172.21.15.3:0/3813280339 >> v2:172.21.15.70:3300/0 conn(0x7fd3240d1510 0x7fd3240d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 --2- 172.21.15.3:0/3813280339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3240b6a70 0x7fd3240d0fd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 -- 172.21.15.3:0/3813280339 >> 172.21.15.3:0/3813280339 conn(0x7fd324072560 msgr2=0x7fd324072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 -- 172.21.15.3:0/3813280339 shutdown_connections 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.668+0000 7fd329e00640 1 -- 172.21.15.3:0/3813280339 wait complete. 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 shutdown_connections 2024-09-10T07:46:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7fab0ce00640 1 -- 172.21.15.3:0/2183966672 wait complete. 2024-09-10T07:46:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7f42fb400640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6690 0x7f42fc0cf9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009870 tx=0x7f42e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e97a43db276a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7f42faa00640 1 -- 172.21.15.3:0/1380370080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4037070 con 0x7f42fc0b6690 2024-09-10T07:46:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7ff91d400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 0x7ff90006e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9200aa8d0 tx=0x7ff91800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7f0d52a00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 0x7f0d3406e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d540cb090 tx=0x7f0d4c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7f42faa00640 1 -- 172.21.15.3:0/1380370080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e4032440 con 0x7f42fc0b6690 2024-09-10T07:46:42.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.669+0000 7f42faa00640 1 -- 172.21.15.3:0/1380370080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4032810 con 0x7f42fc0b6690 2024-09-10T07:46:42.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.670+0000 7fb7cac00640 1 Processor -- start 2024-09-10T07:46:42.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.670+0000 7fb7cac00640 1 -- start start 2024-09-10T07:46:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.670+0000 7f4302200640 1 -- 172.21.15.3:0/1380370080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6690 msgr2=0x7f42fc0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.670+0000 7f4302200640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6690 0x7f42fc0cf9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009870 tx=0x7f42e402a210 comp rx=0 tx=0).stop 2024-09-10T07:46:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 -- 172.21.15.3:0/1380370080 shutdown_connections 2024-09-10T07:46:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0d16e0 0x7f42fc0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 --2- 172.21.15.3:0/1380370080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6690 0x7f42fc0cf9d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 -- 172.21.15.3:0/1380370080 >> 172.21.15.3:0/1380370080 conn(0x7f42fc072560 msgr2=0x7f42fc072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 -- 172.21.15.3:0/1380370080 shutdown_connections 2024-09-10T07:46:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7fab0ce00640 1 Processor -- start 2024-09-10T07:46:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7fb7cac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7fd329e00640 1 Processor -- start 2024-09-10T07:46:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fab0ce00640 1 -- start start 2024-09-10T07:46:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fd329e00640 1 -- start start 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7cac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7c4157180 0x7fb7c4164d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7cac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c41652d0 0x7fb7c40b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7cac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40b47e0 con 0x7fb7c4157180 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7cac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40cfef0 con 0x7fb7a8004800 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7cac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c40b4960 con 0x7fb7c41652d0 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48302/0 (socket says 172.21.15.3:48302) 2024-09-10T07:46:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7c3e00640 1 -- 172.21.15.3:0/3647738635 learned_addr learned my addr 172.21.15.3:0/3647738635 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.671+0000 7f4302200640 1 -- 172.21.15.3:0/1380370080 wait complete. 2024-09-10T07:46:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb2fc200640 1 Processor -- start 2024-09-10T07:46:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300bd070 con 0x7fd7280e6f70 2024-09-10T07:46:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7280525f0 con 0x7fd7280e6f70 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd728052bb0 con 0x7fd7280e6f70 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7c4157180 0x7fb7c4164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fb7c3e00640 1 -- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c41652d0 msgr2=0x7fb7c40b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fb7c3e00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c41652d0 0x7fb7c40b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fb7c3e00640 1 -- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7c4157180 msgr2=0x7fb7c4164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fb7c3e00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7c4157180 0x7fb7c4164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fb7c3e00640 1 -- 172.21.15.3:0/3647738635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c4157610 con 0x7fb7a8004800 2024-09-10T07:46:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.672+0000 7fb2fc200640 1 -- start start 2024-09-10T07:46:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff90c0584c0 con 0x7ff9200ce1c0 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab08154b30 0x7fab08158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 0x7fab081558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab080d19a0 con 0x7fab08155400 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fab080b6ed0 con 0x7fab080cc0e0 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.673+0000 7fab0ce00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab080d1b20 con 0x7fab08154b30 2024-09-10T07:46:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fb7c3e00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00e850 tx=0x7fb7ac00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48312/0 (socket says 172.21.15.3:48312) 2024-09-10T07:46:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 -- 172.21.15.3:0/1273368657 learned_addr learned my addr 172.21.15.3:0/1273368657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05e00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab08154b30 0x7fab08158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7f4302200640 1 Processor -- start 2024-09-10T07:46:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab06800640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 0x7fab081558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7f4302200640 1 -- start start 2024-09-10T07:46:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 -- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab08154b30 msgr2=0x7fab08158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab08154b30 0x7fab08158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 -- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 msgr2=0x7fab081558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 0x7fab081558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab05400640 1 -- 172.21.15.3:0/1273368657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081561e0 con 0x7fab080cc0e0 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fab06800640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 0x7fab081558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fb2fc200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 0x7fb2f40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3240b6690 0x7fd3240cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 0x7fd3240ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd32415c6b0 con 0x7fd3241f8e00 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd32415c530 con 0x7fd3240d1510 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.674+0000 7fd329e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240d1940 con 0x7fd3240b6690 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fc200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fc200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40d1510 0x7fb2f40d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd323400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 0x7fd3240ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fc200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f40cf540 con 0x7fb2f40b6690 2024-09-10T07:46:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fc200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f40cff10 con 0x7fb2f40b6a70 2024-09-10T07:46:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fc200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f40d1d90 con 0x7fb2f40d1510 2024-09-10T07:46:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f9800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 0x7fb2f40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72804daf0 con 0x7fd7280e6f70 2024-09-10T07:46:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2fa200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40d1510 0x7fb2f40d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48320/0 (socket says 172.21.15.3:48320) 2024-09-10T07:46:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 -- 172.21.15.3:0/3143000231 learned_addr learned my addr 172.21.15.3:0/3143000231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd322000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd322000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:48328/0 (socket says 172.21.15.3:48328) 2024-09-10T07:46:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd322000640 1 -- 172.21.15.3:0/1795297830 learned_addr learned my addr 172.21.15.3:0/1795297830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fd322a00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3240b6690 0x7fd3240cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 -- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40d1510 msgr2=0x7fb2f40d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 --2- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40d1510 0x7fb2f40d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 -- 172.21.15.3:0/3143000231 >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 msgr2=0x7fb2f40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 --2- 172.21.15.3:0/3143000231 >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 0x7fb2f40cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.675+0000 7fb2f8e00640 1 -- 172.21.15.3:0/3143000231 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f40ceef0 con 0x7fb2f40b6a70 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fab05400640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaf40108f0 tx=0x7faaf4010db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd322000640 1 -- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3240b6690 msgr2=0x7fd3240cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fb2f9800640 1 --2- 172.21.15.3:0/3143000231 >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 0x7fb2f40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd322000640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3240b6690 0x7fd3240cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7f4302200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd322000640 1 -- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 msgr2=0x7fd3240ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd322000640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 0x7fd3240ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f4302200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6a70 0x7f42fc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42fb400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd322000640 1 -- 172.21.15.3:0/1795297830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240cd3e0 con 0x7fd3240d1510 2024-09-10T07:46:42.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7fb2f8e00640 1 --2- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b7690 tx=0x7fb2e002a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd000afc27e0b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42fb400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:51524/0 (socket says 172.21.15.3:51524) 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f4302200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 0x7f42fc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f4302200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc0b6f00 con 0x7f42fc0b6a70 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f4302200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc0cf1d0 con 0x7f42fc0d16e0 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f4302200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f42fc15c530 con 0x7f42fc0b6690 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42fb400640 1 -- 172.21.15.3:0/2724635300 learned_addr learned my addr 172.21.15.3:0/2724635300 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.676+0000 7fd323400640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 0x7fd3240ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:42.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42f2a00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6a70 0x7f42fc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42fbe00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 0x7f42fc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7fd322000640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd310009af0 tx=0x7fd3100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.677+0000 7f42fb400640 1 -- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 msgr2=0x7f42fc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fb400640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 0x7f42fc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fb400640 1 -- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6a70 msgr2=0x7f42fc159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fb400640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6a70 0x7f42fc159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7300ae440 con 0x7fd7280e6f70 2024-09-10T07:46:42.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fb400640 1 -- 172.21.15.3:0/2724635300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc1f9ed0 con 0x7f42fc0b6690 2024-09-10T07:46:42.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fbe00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 0x7f42fc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310041070 con 0x7fd3240d1510 2024-09-10T07:46:42.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3240ce3c0 con 0x7fd3240d1510 2024-09-10T07:46:42.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00c8d0 con 0x7fb7a8004800 2024-09-10T07:46:42.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf401a020 con 0x7fab080cc0e0 2024-09-10T07:46:42.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab08157f70 con 0x7fab080cc0e0 2024-09-10T07:46:42.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fb2f3e00640 1 -- 172.21.15.3:0/3143000231 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0037070 con 0x7fb2f40b6a70 2024-09-10T07:46:42.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7f42fb400640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009870 tx=0x7f42e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab08158530 con 0x7fab080cc0e0 2024-09-10T07:46:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3241f8600 con 0x7fd3240d1510 2024-09-10T07:46:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300b3040 con 0x7fd7280e6f70 2024-09-10T07:46:42.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fb2f3e00640 1 -- 172.21.15.3:0/3143000231 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e0032440 con 0x7fb2f40b6a70 2024-09-10T07:46:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fb2f3e00640 1 -- 172.21.15.3:0/3143000231 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0032810 con 0x7fb2f40b6a70 2024-09-10T07:46:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310031560 con 0x7fd3240d1510 2024-09-10T07:46:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf400ea80 con 0x7fab080cc0e0 2024-09-10T07:46:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf400ee30 con 0x7fab080cc0e0 2024-09-10T07:46:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310036040 con 0x7fd3240d1510 2024-09-10T07:46:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.678+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41578f0 con 0x7fb7a8004800 2024-09-10T07:46:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac018070 con 0x7fb7a8004800 2024-09-10T07:46:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d440587f0 con 0x7f0d540b6a70 2024-09-10T07:46:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7fb2fc200640 1 -- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40b6a70 msgr2=0x7fb2f40d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.679+0000 7fb2fc200640 1 --2- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b7690 tx=0x7fb2e002a1d0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00ccb0 con 0x7fb7a8004800 2024-09-10T07:46:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7300c0070 con 0x7fd7280e6f70 2024-09-10T07:46:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0005290 con 0x7fab080cc0e0 2024-09-10T07:46:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc005290 con 0x7fd3240d1510 2024-09-10T07:46:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4041020 con 0x7f42fc0b6690 2024-09-10T07:46:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e403d6e0 con 0x7f42fc0b6690 2024-09-10T07:46:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4037040 con 0x7f42fc0b6690 2024-09-10T07:46:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc0b4100 con 0x7f42fc0b6690 2024-09-10T07:46:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc0b46c0 con 0x7f42fc0b6690 2024-09-10T07:46:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.680+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c4157eb0 con 0x7fb7a8004800 2024-09-10T07:46:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fb2fc200640 1 -- 172.21.15.3:0/3143000231 shutdown_connections 2024-09-10T07:46:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fd723400640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 0x7fd71006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fb2fc200640 1 --2- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40d1510 0x7fb2f40d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fb2fc200640 1 --2- 172.21.15.3:0/3143000231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40b6a70 0x7fb2f40d0fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fb2fc200640 1 --2- 172.21.15.3:0/3143000231 >> v2:172.21.15.3:3300/0 conn(0x7fb2f40b6690 0x7fb2f40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.681+0000 7fb2fc200640 1 -- 172.21.15.3:0/3143000231 >> 172.21.15.3:0/3143000231 conn(0x7fb2f4072560 msgr2=0x7fb2f4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:42.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd730127da0 con 0x7fd7280e6f70 2024-09-10T07:46:42.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7fd72f400640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 0x7fd71006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42fc0cf9d0 con 0x7f42fc0b6690 2024-09-10T07:46:42.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7fb2fc200640 1 -- 172.21.15.3:0/3143000231 shutdown_connections 2024-09-10T07:46:42.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7fb2fc200640 1 -- 172.21.15.3:0/3143000231 wait complete. 2024-09-10T07:46:42.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.682+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c005290 con 0x7fb7a8004800 2024-09-10T07:46:42.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.683+0000 7fd72f400640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 0x7fd71006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd728024900 tx=0x7fd71c009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.684+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7300c7070 con 0x7fd7280e6f70 2024-09-10T07:46:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.685+0000 7fb2fc200640 1 Processor -- start 2024-09-10T07:46:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.685+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42e4044020 con 0x7f42fc0b6690 2024-09-10T07:46:42.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.685+0000 7fb2fc200640 1 -- start start 2024-09-10T07:46:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.686+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd310004230 con 0x7fd3240d1510 2024-09-10T07:46:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 0x7fb2f4157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 0x7fb2f4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 0x7fb2f4158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f40d1980 con 0x7fb2f40b6690 2024-09-10T07:46:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f40b6f00 con 0x7fb2f40d1510 2024-09-10T07:46:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fc200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f40d1b00 con 0x7fb2f40b6a70 2024-09-10T07:46:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7f42f3e00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 0x7f42d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42e4077c10 con 0x7f42fc0b6690 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2f9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 0x7fb2f4157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2f8e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 0x7fb2f4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2fa200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 0x7fb2f4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2f9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 0x7fb2f4157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48250/0 (socket says 172.21.15.3:48250) 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fb2f9800640 1 -- 172.21.15.3:0/848771646 learned_addr learned my addr 172.21.15.3:0/848771646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fd31b400640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 0x7fd30406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd3100abfc0 con 0x7fd3240d1510 2024-09-10T07:46:42.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fd322a00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 0x7fd30406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fb2fa200640 1 -- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 msgr2=0x7fb2f4155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fb2fa200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 0x7fb2f4155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fb2fa200640 1 -- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 msgr2=0x7fb2f4157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:42.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fb2fa200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 0x7fb2f4157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:42.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7fb2fa200640 1 -- 172.21.15.3:0/848771646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f4157610 con 0x7fb2f40d1510 2024-09-10T07:46:42.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.689+0000 7fb2f8e00640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 0x7fb2f4155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:42.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.689+0000 7fd322a00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 0x7fd30406e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3240cda40 tx=0x7fd314009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.689+0000 7fb2fa200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 0x7fb2f4158600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00efc0 tx=0x7fb2ec00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.687+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faaf401d020 con 0x7fab080cc0e0 2024-09-10T07:46:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.689+0000 7faafe000640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 0x7faae806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.689+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faaf408bab0 con 0x7fab080cc0e0 2024-09-10T07:46:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.690+0000 7fab05e00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 0x7faae806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.691+0000 7fab05e00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 0x7faae806e320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab00034000 tx=0x7fab0000a000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.691+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd31007a300 con 0x7fd3240d1510 2024-09-10T07:46:42.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.691+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7ac027430 con 0x7fb7a8004800 2024-09-10T07:46:42.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.688+0000 7f42f2a00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 0x7f42d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.691+0000 7f42f2a00640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 0x7f42d806e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42fc1f3570 tx=0x7f42ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.692+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42e404b030 con 0x7f42fc0b6690 2024-09-10T07:46:42.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.692+0000 7fb7c0c00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 0x7fb7a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.693+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb7ac08b8a0 con 0x7fb7a8004800 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.693+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 0x7fb7a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.694+0000 7fb7bb400640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 0x7fb7a006e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4033000 tx=0x7fb7b4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.695+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7ac05a230 con 0x7fb7a8004800 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.698+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec018020 con 0x7fb2f40d1510 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.699+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ec014480 con 0x7fb2f40d1510 2024-09-10T07:46:42.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.699+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec014870 con 0x7fb2f40d1510 2024-09-10T07:46:42.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.699+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41578f0 con 0x7fb2f40d1510 2024-09-10T07:46:42.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.699+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f40cbbe0 con 0x7fb2f40d1510 2024-09-10T07:46:42.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.699+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faaf4024020 con 0x7fab080cc0e0 2024-09-10T07:46:42.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.703+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2ec01b020 con 0x7fb2f40d1510 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.704+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f415c530 con 0x7fb2f40d1510 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.704+0000 7fb2f1600640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 0x7fb2d406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.705+0000 7fb2f9800640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 0x7fb2d406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.706+0000 7fb2f9800640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 0x7fb2d406e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006fd0 tx=0x7fb2e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.706+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb2ec058050 con 0x7fb2f40d1510 2024-09-10T07:46:42.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:42.710+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2ec022020 con 0x7fb2f40d1510 2024-09-10T07:46:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.081+0000 7ff91f200640 1 -- 172.21.15.3:0/4001001668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff920070f10 con 0x7ff9200ce1c0 2024-09-10T07:46:43.085 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-09-10T07:46:43.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.082+0000 7ff916000640 1 -- 172.21.15.3:0/4001001668 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff920070f10 con 0x7ff9200ce1c0 2024-09-10T07:46:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.086+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 msgr2=0x7ff90006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.086+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 0x7ff90006e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9200aa8d0 tx=0x7ff91800b040 comp rx=0 tx=0).stop 2024-09-10T07:46:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.086+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 msgr2=0x7ff9200718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.086+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 0x7ff9200718f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c400 tx=0x7ff90c00c8c0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 shutdown_connections 2024-09-10T07:46:43.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff90006be80 0x7ff90006e340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff920162790 0x7ff9200b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff920144c60 0x7ff92013cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 --2- 172.21.15.3:0/4001001668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9200ce1c0 0x7ff9200718f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.087+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 >> 172.21.15.3:0/4001001668 conn(0x7ff92006d000 msgr2=0x7ff92015fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.088+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 shutdown_connections 2024-09-10T07:46:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.088+0000 7ff90b400640 1 -- 172.21.15.3:0/4001001668 wait complete. 2024-09-10T07:46:43.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-09-10T07:46:43.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.114+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0d2c005580 con 0x7f0d540b6a70 2024-09-10T07:46:43.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.114+0000 7f0d4b400640 1 -- 172.21.15.3:0/632928792 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d4405caf0 con 0x7f0d540b6a70 2024-09-10T07:46:43.114 INFO:teuthology.orchestra.run.smithi003.stdout:68719476741 2024-09-10T07:46:43.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.117+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd7280333a0 con 0x7fd7280e6f70 2024-09-10T07:46:43.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.118+0000 7fd723400640 1 -- 172.21.15.3:0/3279040415 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7300f67c0 con 0x7fd7280e6f70 2024-09-10T07:46:43.118 INFO:teuthology.orchestra.run.smithi003.stdout:68719476740 2024-09-10T07:46:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.118+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 msgr2=0x7f0d3406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.118+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 0x7f0d3406e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d540cb090 tx=0x7f0d4c009040 comp rx=0 tx=0).stop 2024-09-10T07:46:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.118+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 msgr2=0x7f0d5415a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.118+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 0x7f0d5415a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b7690 tx=0x7f0d4400c890 comp rx=0 tx=0).stop 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 shutdown_connections 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0d3406bf00 0x7f0d3406e3c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d540d16e0 0x7f0d5414a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d540b6a70 0x7f0d5415a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 --2- 172.21.15.3:0/632928792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d540b6690 0x7f0d5415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 >> 172.21.15.3:0/632928792 conn(0x7f0d54072560 msgr2=0x7f0d5415e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 shutdown_connections 2024-09-10T07:46:43.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.119+0000 7f0d59e00640 1 -- 172.21.15.3:0/632928792 wait complete. 2024-09-10T07:46:43.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.120+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb2f415d960 con 0x7fb2f40d1510 2024-09-10T07:46:43.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.120+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd2fc005850 con 0x7fd3240d1510 2024-09-10T07:46:43.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.121+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb79c005850 con 0x7fb7a8004800 2024-09-10T07:46:43.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.121+0000 7fd31b400640 1 -- 172.21.15.3:0/1795297830 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd31007e600 con 0x7fd3240d1510 2024-09-10T07:46:43.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.121+0000 7fb2f1600640 1 -- 172.21.15.3:0/848771646 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2ec010040 con 0x7fb2f40d1510 2024-09-10T07:46:43.121 INFO:teuthology.orchestra.run.smithi003.stdout:64424509445 2024-09-10T07:46:43.121 INFO:teuthology.orchestra.run.smithi003.stdout:64424509446 2024-09-10T07:46:43.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 msgr2=0x7fd71006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fb7c0c00640 1 -- 172.21.15.3:0/3647738635 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb7ac059b30 con 0x7fb7a8004800 2024-09-10T07:46:43.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 0x7fd71006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd728024900 tx=0x7fd71c009060 comp rx=0 tx=0).stop 2024-09-10T07:46:43.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 msgr2=0x7fd72803bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7300518b0 tx=0x7fd730069a30 comp rx=0 tx=0).stop 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stdout:68719476742 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 shutdown_connections 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd71006be30 0x7fd71006e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7280e6f70 0x7fd72803bf20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd728049780 0x7fd728051ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 --2- 172.21.15.3:0/3279040415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd728035d90 0x7fd728036630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 >> 172.21.15.3:0/3279040415 conn(0x7fd72801fb40 msgr2=0x7fd728007ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.124 INFO:teuthology.orchestra.run.smithi003.stdout:64424509446 2024-09-10T07:46:43.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f42fc15d430 con 0x7f42fc0b6690 2024-09-10T07:46:43.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.122+0000 7f42f3e00640 1 -- 172.21.15.3:0/2724635300 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42e407a270 con 0x7f42fc0b6690 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.123+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 shutdown_connections 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.123+0000 7fd734e00640 1 -- 172.21.15.3:0/3279040415 wait complete. 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 msgr2=0x7fd30406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 0x7fd30406e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3240cda40 tx=0x7fd314009040 comp rx=0 tx=0).stop 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 msgr2=0x7fd3241f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 msgr2=0x7fb2d406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 0x7fb2d406e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006fd0 tx=0x7fb2e8008040 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 msgr2=0x7fb2f4158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 0x7fb2f4158600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00efc0 tx=0x7fb2ec00c490 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.125+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd310009af0 tx=0x7fd3100046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 shutdown_connections 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30406be30 0x7fd30406e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3241f8e00 0x7fd3240ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3240d1510 0x7fd3241f5610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 --2- 172.21.15.3:0/1795297830 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3240b6690 0x7fd3240cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 >> 172.21.15.3:0/1795297830 conn(0x7fd324072560 msgr2=0x7fd32415f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 shutdown_connections 2024-09-10T07:46:43.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2d406be80 0x7fb2d406e340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2f40d1510 0x7fb2f4158600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2f40b6a70 0x7fb2f4155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 --2- 172.21.15.3:0/848771646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2f40b6690 0x7fb2f4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 >> 172.21.15.3:0/848771646 conn(0x7fb2f4072560 msgr2=0x7fb2f415de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 shutdown_connections 2024-09-10T07:46:43.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 shutdown_connections 2024-09-10T07:46:43.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb2fc200640 1 -- 172.21.15.3:0/848771646 wait complete. 2024-09-10T07:46:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7faae0005850 con 0x7fab080cc0e0 2024-09-10T07:46:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 msgr2=0x7f42d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 0x7f42d806e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42fc1f3570 tx=0x7f42ec009040 comp rx=0 tx=0).stop 2024-09-10T07:46:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 msgr2=0x7f42fc0b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009870 tx=0x7f42e40046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 msgr2=0x7fb7a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 0x7fb7a006e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4033000 tx=0x7fb7b4008030 comp rx=0 tx=0).stop 2024-09-10T07:46:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 msgr2=0x7fb7c41f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00e850 tx=0x7fb7ac00ed10 comp rx=0 tx=0).stop 2024-09-10T07:46:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.126+0000 7fd329e00640 1 -- 172.21.15.3:0/1795297830 wait complete. 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 shutdown_connections 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f42d806be80 0x7f42d806e340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42fc0d16e0 0x7f42fc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42fc0b6a70 0x7f42fc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 --2- 172.21.15.3:0/2724635300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42fc0b6690 0x7f42fc0b3800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 >> 172.21.15.3:0/2724635300 conn(0x7f42fc072560 msgr2=0x7f42fc15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 shutdown_connections 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7a006be60 0x7fb7a006e320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7c41652d0 0x7fb7c40b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7c4157180 0x7fb7c4164d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 --2- 172.21.15.3:0/3647738635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a8004800 0x7fb7c41f7ca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 >> 172.21.15.3:0/3647738635 conn(0x7fb7c406d000 msgr2=0x7fb7c40b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7faafe000640 1 -- 172.21.15.3:0/1273368657 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faaf405a440 con 0x7fab080cc0e0 2024-09-10T07:46:43.132 INFO:teuthology.orchestra.run.smithi003.stdout:68719476742 2024-09-10T07:46:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 shutdown_connections 2024-09-10T07:46:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.127+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 shutdown_connections 2024-09-10T07:46:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.128+0000 7f4302200640 1 -- 172.21.15.3:0/2724635300 wait complete. 2024-09-10T07:46:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.128+0000 7fb7cac00640 1 -- 172.21.15.3:0/3647738635 wait complete. 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 msgr2=0x7faae806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 0x7faae806e320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab00034000 tx=0x7fab0000a000 comp rx=0 tx=0).stop 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 msgr2=0x7fab08157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaf40108f0 tx=0x7faaf4010db0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 shutdown_connections 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faae806be60 0x7faae806e320 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab00034000 tx=0x7fab0000a000 comp rx=0 tx=0).stop 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab08155400 0x7fab081558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab080cc0e0 0x7fab08157690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 --2- 172.21.15.3:0/1273368657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab08154b30 0x7fab08158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.130+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 >> 172.21.15.3:0/1273368657 conn(0x7fab08072560 msgr2=0x7fab080b7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.132+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 shutdown_connections 2024-09-10T07:46:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:43.133+0000 7fab0ce00640 1 -- 172.21.15.3:0/1273368657 wait complete. 2024-09-10T07:46:43.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476740 for osd.1 2024-09-10T07:46:43.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.0 2024-09-10T07:46:43.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-10T07:46:43.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-10T07:46:43.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-10T07:46:43.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476742 for osd.2 2024-09-10T07:46:43.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476742 for osd.3 2024-09-10T07:46:44.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T07:46:44.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T07:46:44.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T07:46:44.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T07:46:44.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T07:46:44.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T07:46:44.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T07:46:44.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T07:46:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.363+0000 7f450c000640 1 Processor -- start 2024-09-10T07:46:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.363+0000 7f450c000640 1 -- start start 2024-09-10T07:46:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.366+0000 7f450c000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f450a000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f45040b3a10 0x7f45040cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50102/0 (socket says 172.21.15.3:50102) 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f4509600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f4509600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34762/0 (socket says 172.21.15.3:34762) 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f450a000640 1 -- 172.21.15.3:0/3262277504 learned_addr learned my addr 172.21.15.3:0/3262277504 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45040b3ff0 con 0x7f45040d0720 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45040b42d0 con 0x7f45040b3a10 2024-09-10T07:46:44.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.367+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f45040d0b60 con 0x7f4504158ec0 2024-09-10T07:46:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.368+0000 7f4503e00640 1 -- 172.21.15.3:0/3262277504 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321410756 0 0) 0x7f45040b42d0 con 0x7f45040b3a10 2024-09-10T07:46:44.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.368+0000 7f4503e00640 1 -- 172.21.15.3:0/3262277504 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44ec003520 con 0x7f45040b3a10 2024-09-10T07:46:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.368+0000 7f4509600640 1 -- 172.21.15.3:0/3262277504 >> v1:172.21.15.70:6789/0 conn(0x7f45040b3a10 legacy=0x7f45040cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.368+0000 7f4509600640 1 -- 172.21.15.3:0/3262277504 >> v1:172.21.15.3:6789/0 conn(0x7f45040d0720 legacy=0x7f45040b3300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.368+0000 7f4509600640 1 -- 172.21.15.3:0/3262277504 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040cce80 con 0x7f4504158ec0 2024-09-10T07:46:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.370+0000 7f4509600640 1 --2- 172.21.15.3:0/3262277504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45040d0ce0 tx=0x7f44f002c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce001589970efa0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.371+0000 7f4503e00640 1 -- 172.21.15.3:0/3262277504 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0034620 con 0x7f4504158ec0 2024-09-10T07:46:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.371+0000 7f4503e00640 1 -- 172.21.15.3:0/3262277504 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f00347c0 con 0x7f4504158ec0 2024-09-10T07:46:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.372+0000 7f4503e00640 1 -- 172.21.15.3:0/3262277504 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0034620 con 0x7f4504158ec0 2024-09-10T07:46:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.372+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 msgr2=0x7f45040d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.372+0000 7f450c000640 1 --2- 172.21.15.3:0/3262277504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45040d0ce0 tx=0x7f44f002c870 comp rx=0 tx=0).stop 2024-09-10T07:46:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.373+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 shutdown_connections 2024-09-10T07:46:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.373+0000 7f450c000640 1 --2- 172.21.15.3:0/3262277504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4504158ec0 0x7f45040d01e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.373+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 >> 172.21.15.3:0/3262277504 conn(0x7f450406d000 msgr2=0x7f450406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.373+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 shutdown_connections 2024-09-10T07:46:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.374+0000 7f450c000640 1 -- 172.21.15.3:0/3262277504 wait complete. 2024-09-10T07:46:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.377+0000 7f450c000640 1 Processor -- start 2024-09-10T07:46:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.378+0000 7f450c000640 1 -- start start 2024-09-10T07:46:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45040b3a10 0x7f4504149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45040d0720 0x7f45040b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f45040cdd90 con 0x7f45040b3a10 2024-09-10T07:46:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f45041592a0 con 0x7f4504158ec0 2024-09-10T07:46:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.380+0000 7f450c000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f45040cdf10 con 0x7f45040d0720 2024-09-10T07:46:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.382+0000 7f4508c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45040d0720 0x7f45040b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.382+0000 7f450a000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.382+0000 7f450a000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54262/0 (socket says 172.21.15.3:54262) 2024-09-10T07:46:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.382+0000 7f450a000640 1 -- 172.21.15.3:0/403448312 learned_addr learned my addr 172.21.15.3:0/403448312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f4509600640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45040b3a10 0x7f4504149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f450a000640 1 -- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45040d0720 msgr2=0x7f45040b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f450a000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45040d0720 0x7f45040b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f450a000640 1 -- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45040b3a10 msgr2=0x7f4504149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f450a000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45040b3a10 0x7f4504149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.383+0000 7f450a000640 1 -- 172.21.15.3:0/403448312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4504145860 con 0x7f4504158ec0 2024-09-10T07:46:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.384+0000 7f450a000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c430 tx=0x7f44f800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.385+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f80074e0 con 0x7f4504158ec0 2024-09-10T07:46:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.385+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f80047e0 con 0x7f4504158ec0 2024-09-10T07:46:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.385+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8004c10 con 0x7f4504158ec0 2024-09-10T07:46:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.385+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45041447c0 con 0x7f4504158ec0 2024-09-10T07:46:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.386+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4504144d80 con 0x7f4504158ec0 2024-09-10T07:46:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.390+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45040d01e0 con 0x7f4504158ec0 2024-09-10T07:46:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.390+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44f8011070 con 0x7f4504158ec0 2024-09-10T07:46:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.392+0000 7f4501600640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 0x7f44e406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.392+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f44f808a9a0 con 0x7f4504158ec0 2024-09-10T07:46:44.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.393+0000 7f4509600640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 0x7f44e406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.395+0000 7f4509600640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 0x7f44e406e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44f0037000 tx=0x7f44f0032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.398+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44f80593c0 con 0x7f4504158ec0 2024-09-10T07:46:44.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.439+0000 7fd223200640 1 Processor -- start 2024-09-10T07:46:44.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.439+0000 7fd223200640 1 -- start start 2024-09-10T07:46:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.441+0000 7fd223200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd224161a20 0x7fd2240cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.441+0000 7fd223200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2240cfb10 0x7fd22406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.441+0000 7fd223200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2240d37e0 con 0x7fd2240d32f0 2024-09-10T07:46:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.441+0000 7fd223200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd2240d3ac0 con 0x7fd224161a20 2024-09-10T07:46:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.441+0000 7fd223200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd2240d3da0 con 0x7fd2240cfb10 2024-09-10T07:46:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.442+0000 7fd222800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd2240d32f0 0x7fd2240d0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60186/0 (socket says 172.21.15.3:60186) 2024-09-10T07:46:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.442+0000 7fd222800640 1 -- 172.21.15.3:0/3775593721 learned_addr learned my addr 172.21.15.3:0/3775593721 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.443+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3361501120 0 0) 0x7fd2240d37e0 con 0x7fd2240d32f0 2024-09-10T07:46:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.443+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd208003520 con 0x7fd2240d32f0 2024-09-10T07:46:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.443+0000 7fd221e00640 1 --2- 172.21.15.3:0/3775593721 >> v2:172.21.15.3:3300/0 conn(0x7fd224161a20 0x7fd2240cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.443+0000 7fd221400640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2240cfb10 0x7fd22406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 764649394 0 0) 0x7fd208003520 con 0x7fd2240d32f0 2024-09-10T07:46:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2240cfb10 msgr2=0x7fd22406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2240cfb10 0x7fd22406bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 >> v2:172.21.15.3:3300/0 conn(0x7fd224161a20 msgr2=0x7fd2240cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 --2- 172.21.15.3:0/3775593721 >> v2:172.21.15.3:3300/0 conn(0x7fd224161a20 0x7fd2240cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2240d4080 con 0x7fd2240d32f0 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fd21c0019f0 con 0x7fd2240d32f0 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 >> v1:172.21.15.3:6789/0 conn(0x7fd2240d32f0 legacy=0x7fd2240d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd208003a90 0x7fd208003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd208004800 0x7fd208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2080051c0 0x7fd20800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240d3ac0 con 0x7fd208004800 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.444+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240d3da0 con 0x7fd208003a90 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.445+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd20800c060 con 0x7fd2080051c0 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.445+0000 7fd221400640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2240cfb10 0x7fd22406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.445+0000 7fd222800640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2080051c0 0x7fd20800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd221400640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd208004800 0x7fd208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd222800640 1 -- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd208003a90 msgr2=0x7fd208003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd222800640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd208003a90 0x7fd208003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd222800640 1 -- 172.21.15.3:0/3775593721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd208004800 msgr2=0x7fd208004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd222800640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd208004800 0x7fd208004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.446+0000 7fd222800640 1 -- 172.21.15.3:0/3775593721 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd208003520 con 0x7fd2080051c0 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.447+0000 7fd221400640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd208004800 0x7fd208004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:44.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.447+0000 7fd221e00640 1 --2- 172.21.15.3:0/3775593721 >> v2:172.21.15.3:3300/0 conn(0x7fd224161a20 0x7fd2240cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:44.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.448+0000 7fd222800640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2080051c0 0x7fd20800b9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00a7d0 tx=0x7fd208005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c367ffd6b462d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.448+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c00a3f0 con 0x7fd2080051c0 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.448+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c02f070 con 0x7fd2080051c0 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.448+0000 7fd220a00640 1 -- 172.21.15.3:0/3775593721 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c02bcf0 con 0x7fd2080051c0 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.449+0000 7fd223200640 1 -- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2080051c0 msgr2=0x7fd20800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.449+0000 7fd223200640 1 --2- 172.21.15.3:0/3775593721 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2080051c0 0x7fd20800b9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00a7d0 tx=0x7fd208005830 comp rx=0 tx=0).stop 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.449+0000 7fd222800640 1 -- 172.21.15.3:0/3775593721 reap_dead start 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7fd223200640 1 -- 172.21.15.3:0/3775593721 shutdown_connections 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7fd223200640 1 -- 172.21.15.3:0/3775593721 >> 172.21.15.3:0/3775593721 conn(0x7fd2240b6090 msgr2=0x7fd2240b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7fd223200640 1 -- 172.21.15.3:0/3775593721 shutdown_connections 2024-09-10T07:46:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7f3b9f200640 1 Processor -- start 2024-09-10T07:46:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7f3b9f200640 1 -- start start 2024-09-10T07:46:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.452+0000 7f3b9f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ba0141540 0x7f3ba00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.452+0000 7f3b9f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba00b2000 0x7f3ba00b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.452+0000 7f3b9f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba015fc00 con 0x7f3ba0143de0 2024-09-10T07:46:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.452+0000 7f3b9f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba00b29a0 con 0x7f3ba0141540 2024-09-10T07:46:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.452+0000 7f3b9f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba00aea40 con 0x7f3ba00b2000 2024-09-10T07:46:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.450+0000 7fd223200640 1 -- 172.21.15.3:0/3775593721 wait complete. 2024-09-10T07:46:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.453+0000 7fd223200640 1 Processor -- start 2024-09-10T07:46:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.454+0000 7fd223200640 1 -- start start 2024-09-10T07:46:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.454+0000 7f39a9800640 1 Processor -- start 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.453+0000 7f3b9e800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3ba0143de0 0x7f3ba0144270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60190/0 (socket says 172.21.15.3:60190) 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.453+0000 7f3b9e800640 1 -- 172.21.15.3:0/3255369031 learned_addr learned my addr 172.21.15.3:0/3255369031 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.453+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3255369031 >> v2:172.21.15.3:3300/0 conn(0x7f3ba0141540 0x7f3ba00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.454+0000 7f3b9d400640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba00b2000 0x7f3ba00b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7f3b9de00640 1 -- 172.21.15.3:0/3255369031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba00b2000 msgr2=0x7f3ba00b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba00b2000 0x7f3ba00b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7f3b9de00640 1 -- 172.21.15.3:0/3255369031 >> v1:172.21.15.3:6789/0 conn(0x7f3ba0143de0 legacy=0x7f3ba0144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7f3b9de00640 1 -- 172.21.15.3:0/3255369031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba00aed20 con 0x7f3ba0141540 2024-09-10T07:46:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7f39a9800640 1 -- start start 2024-09-10T07:46:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.456+0000 7f3713200640 1 Processor -- start 2024-09-10T07:46:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 0x7fd22414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 0x7fd2241fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 0x7fd2241fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240b8d90 con 0x7fd2080051c0 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240b8c10 con 0x7fd224148ab0 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.455+0000 7fd223200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd2240b8f10 con 0x7fd224155910 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.456+0000 7fd222800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 0x7fd2241fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.456+0000 7fd222800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 0x7fd2241fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34812/0 (socket says 172.21.15.3:34812) 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.456+0000 7fd222800640 1 -- 172.21.15.3:0/3173328370 learned_addr learned my addr 172.21.15.3:0/3173328370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.456+0000 7f3713200640 1 -- start start 2024-09-10T07:46:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7fd221400640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 0x7fd2241fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 0x7f39a40b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39a40cc880 0x7f39a40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f39a40ce2f0 con 0x7f39a4162ce0 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f39a40ce5d0 con 0x7f39a40cc880 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a9800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f39a40ce8b0 con 0x7f39a4162900 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34824/0 (socket says 172.21.15.3:34824) 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.457+0000 7f39a3e00640 1 -- 172.21.15.3:0/2450457673 learned_addr learned my addr 172.21.15.3:0/2450457673 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7fd221400640 1 -- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 msgr2=0x7fd2241fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7fd221400640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 0x7fd2241fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7fd221400640 1 -- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 msgr2=0x7fd22414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3400640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 0x7f39a40b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a8e00640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.70:3300/0 conn(0x7f39a40cc880 0x7f39a40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3e00640 1 -- 172.21.15.3:0/2450457673 >> v2:172.21.15.70:3300/0 conn(0x7f39a40cc880 msgr2=0x7f39a40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.70:3300/0 conn(0x7f39a40cc880 0x7f39a40b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3e00640 1 -- 172.21.15.3:0/2450457673 >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 msgr2=0x7f39a40b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f3713200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3708005ab0 0x7f3708053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0141540 0x7f3ba00b50a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b900097a0 tx=0x7f3b900329d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3540a381785b41db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 0x7f39a40b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7f39a3e00640 1 -- 172.21.15.3:0/2450457673 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a40ceb90 con 0x7f39a4162900 2024-09-10T07:46:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.458+0000 7fd221e00640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 0x7fd22414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3711400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f37080e7120 0x7f3708005e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50112/0 (socket says 172.21.15.3:50112) 2024-09-10T07:46:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3711400640 1 -- 172.21.15.3:0/2492599747 learned_addr learned my addr 172.21.15.3:0/2492599747 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f39a3400640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 0x7f39a40b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3708052330 con 0x7f370804eb60 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3708006cd0 con 0x7f37080e7120 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3711e00640 1 --2- 172.21.15.3:0/2492599747 >> v2:172.21.15.70:3300/0 conn(0x7f3708005ab0 0x7f3708053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7fd221400640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 0x7fd22414e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7fd221400640 1 -- 172.21.15.3:0/3173328370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22414ec80 con 0x7fd224148ab0 2024-09-10T07:46:44.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f37080520e0 con 0x7f3708005ab0 2024-09-10T07:46:44.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7fd221e00640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 0x7fd22414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:44.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f8084200640 1 Processor -- start 2024-09-10T07:46:44.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7fd221400640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 0x7fd2241fcb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd2240cfef0 tx=0x7fd21400f920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3710a00640 1 -- 172.21.15.3:0/2492599747 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594891369 0 0) 0x7f3708006cd0 con 0x7f37080e7120 2024-09-10T07:46:44.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7f3710a00640 1 -- 172.21.15.3:0/2492599747 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36f4003520 con 0x7f37080e7120 2024-09-10T07:46:44.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7f8084200640 1 -- start start 2024-09-10T07:46:44.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214018020 con 0x7fd224148ab0 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214004440 con 0x7fd224148ab0 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214003830 con 0x7fd224148ab0 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3b9ca00640 1 -- 172.21.15.3:0/3255369031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9003b070 con 0x7f3ba0141540 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3b9ca00640 1 -- 172.21.15.3:0/3255369031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b90037830 con 0x7f3ba0141540 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.459+0000 7f3b9ca00640 1 -- 172.21.15.3:0/3255369031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90037c40 con 0x7f3ba0141540 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.461+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22414d6e0 con 0x7fd224148ab0 2024-09-10T07:46:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.461+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22414dc60 con 0x7fd224148ab0 2024-09-10T07:46:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7f3711e00640 1 -- 172.21.15.3:0/2492599747 >> v1:172.21.15.70:6789/0 conn(0x7f37080e7120 legacy=0x7f3708005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7f3711e00640 1 -- 172.21.15.3:0/2492599747 >> v1:172.21.15.3:6789/0 conn(0x7f370804eb60 legacy=0x7f370801d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f8084200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f807c15e3b0 0x7f807c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.460+0000 7f3711e00640 1 -- 172.21.15.3:0/2492599747 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708006fb0 con 0x7f3708005ab0 2024-09-10T07:46:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f8084200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f8084200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c0cfbb0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f8084200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f807c0d20e0 con 0x7f807c15e3b0 2024-09-10T07:46:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f8084200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f807c0d2380 con 0x7f807c0d14d0 2024-09-10T07:46:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2450457673 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39a40aff80 tx=0x7f399402a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7809cbddb79d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f3711e00640 1 --2- 172.21.15.3:0/2492599747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3708005ab0 0x7f3708053f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3700009790 tx=0x7f370002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11cc132292c88361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f3b9f200640 1 -- 172.21.15.3:0/3255369031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0141540 msgr2=0x7f3ba00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0141540 0x7f3ba00b50a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b900097a0 tx=0x7f3b900329d0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f6b1cc00640 1 Processor -- start 2024-09-10T07:46:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f3710a00640 1 -- 172.21.15.3:0/2492599747 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700037070 con 0x7f3708005ab0 2024-09-10T07:46:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f39a2a00640 1 -- 172.21.15.3:0/2450457673 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994037070 con 0x7f39a4162900 2024-09-10T07:46:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f39a2a00640 1 -- 172.21.15.3:0/2450457673 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3994032440 con 0x7f39a4162900 2024-09-10T07:46:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.462+0000 7f39a2a00640 1 -- 172.21.15.3:0/2450457673 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994032810 con 0x7f39a4162900 2024-09-10T07:46:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f6b1cc00640 1 -- start start 2024-09-10T07:46:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3710a00640 1 -- 172.21.15.3:0/2492599747 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3700032640 con 0x7f3708005ab0 2024-09-10T07:46:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3710a00640 1 -- 172.21.15.3:0/2492599747 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700032a50 con 0x7f3708005ab0 2024-09-10T07:46:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2240cf5d0 con 0x7fd224148ab0 2024-09-10T07:46:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3b9f200640 1 -- 172.21.15.3:0/3255369031 shutdown_connections 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba00b2000 0x7f3ba00b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3255369031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0141540 0x7f3ba00b50a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f3b9f200640 1 -- 172.21.15.3:0/3255369031 >> 172.21.15.3:0/3255369031 conn(0x7f3ba006bb20 msgr2=0x7f3ba006bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f39a9800640 1 -- 172.21.15.3:0/2450457673 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 msgr2=0x7f39a40b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f39a9800640 1 --2- 172.21.15.3:0/2450457673 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39a40aff80 tx=0x7f399402a140 comp rx=0 tx=0).stop 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f8081800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f807c15e3b0 0x7f807c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f8082200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f8082200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34834/0 (socket says 172.21.15.3:34834) 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f8082200640 1 -- 172.21.15.3:0/1369582945 learned_addr learned my addr 172.21.15.3:0/1369582945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.463+0000 7f8080e00640 1 --1- 172.21.15.3:0/1369582945 >> v1:172.21.15.3:6789/0 conn(0x7f807c0cf7b0 0x7f807c0d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60206/0 (socket says 172.21.15.3:60206) 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f3b9f200640 1 -- 172.21.15.3:0/3255369031 shutdown_connections 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3708005ab0 msgr2=0x7f3708053f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f3713200640 1 --2- 172.21.15.3:0/2492599747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3708005ab0 0x7f3708053f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3700009790 tx=0x7f370002a2c0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f3b9f200640 1 -- 172.21.15.3:0/3255369031 wait complete. 2024-09-10T07:46:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f8081800640 1 -- 172.21.15.3:0/1369582945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 msgr2=0x7f807c0d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 -- 172.21.15.3:0/2450457673 shutdown_connections 2024-09-10T07:46:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.70:3300/0 conn(0x7f39a40cc880 0x7f39a40b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 --2- 172.21.15.3:0/2450457673 >> v2:172.21.15.3:3300/0 conn(0x7f39a4162ce0 0x7f39a40b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 --2- 172.21.15.3:0/2450457673 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162900 0x7f39a40b3370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 -- 172.21.15.3:0/2450457673 >> 172.21.15.3:0/2450457673 conn(0x7f39a406bb20 msgr2=0x7f39a406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f39a9800640 1 -- 172.21.15.3:0/2450457673 shutdown_connections 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f8081800640 1 --2- 172.21.15.3:0/1369582945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f8081800640 1 -- 172.21.15.3:0/1369582945 >> v1:172.21.15.3:6789/0 conn(0x7f807c0cf7b0 legacy=0x7f807c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.464+0000 7f8081800640 1 -- 172.21.15.3:0/1369582945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0d2660 con 0x7f807c15e3b0 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b1cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b1806db20 0x7f6b180b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b1cc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b180b42a0 0x7f6b180ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b1cc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b180cf130 con 0x7f6b180ce900 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b1cc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6b180cf410 con 0x7f6b1806db20 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b1cc00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6b180cf6f0 con 0x7f6b180b42a0 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b0ea00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b180b42a0 0x7f6b180ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b0ea00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b180b42a0 0x7f6b180ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54300/0 (socket says 172.21.15.3:54300) 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.465+0000 7f6b0ea00640 1 -- 172.21.15.3:0/2781546686 learned_addr learned my addr 172.21.15.3:0/2781546686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd21401b020 con 0x7fd224148ab0 2024-09-10T07:46:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f8081800640 1 --2- 172.21.15.3:0/1369582945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cf270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dd54f27473f3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 shutdown_connections 2024-09-10T07:46:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f3b9f200640 1 Processor -- start 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f6b17400640 1 --2- 172.21.15.3:0/2781546686 >> v2:172.21.15.3:3300/0 conn(0x7f6b1806db20 0x7f6b180b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f6b0ea00640 1 -- 172.21.15.3:0/2781546686 >> v2:172.21.15.3:3300/0 conn(0x7f6b1806db20 msgr2=0x7f6b180b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f39a9800640 1 -- 172.21.15.3:0/2450457673 wait complete. 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f6b0ea00640 1 --2- 172.21.15.3:0/2781546686 >> v2:172.21.15.3:3300/0 conn(0x7f6b1806db20 0x7f6b180b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f6b0ea00640 1 -- 172.21.15.3:0/2781546686 >> v1:172.21.15.3:6789/0 conn(0x7f6b180ce900 legacy=0x7f6b1806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.466+0000 7f6b0ea00640 1 -- 172.21.15.3:0/2781546686 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180cf9d0 con 0x7f6b180b42a0 2024-09-10T07:46:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f3713200640 1 --2- 172.21.15.3:0/2492599747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3708005ab0 0x7f3708053f50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f807be00640 1 -- 172.21.15.3:0/1369582945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f807c15e3b0 2024-09-10T07:46:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f807be00640 1 -- 172.21.15.3:0/1369582945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070032440 con 0x7f807c15e3b0 2024-09-10T07:46:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f3b9f200640 1 -- start start 2024-09-10T07:46:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f807be00640 1 -- 172.21.15.3:0/1369582945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070032830 con 0x7f807c15e3b0 2024-09-10T07:46:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7fd21a000640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 0x7fd20406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 >> 172.21.15.3:0/2492599747 conn(0x7f3708036090 msgr2=0x7f37080364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.467+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd21408adc0 con 0x7fd224148ab0 2024-09-10T07:46:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7f39a9800640 1 Processor -- start 2024-09-10T07:46:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7fd221e00640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 0x7fd20406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7f8084200640 1 -- 172.21.15.3:0/1369582945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 msgr2=0x7f807c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 shutdown_connections 2024-09-10T07:46:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7f3713200640 1 -- 172.21.15.3:0/2492599747 wait complete. 2024-09-10T07:46:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.468+0000 7f8084200640 1 --2- 172.21.15.3:0/1369582945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cf270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a1c0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f39a9800640 1 -- start start 2024-09-10T07:46:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 0x7f3ba0139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f8084200640 1 -- 172.21.15.3:0/1369582945 shutdown_connections 2024-09-10T07:46:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9de00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f8084200640 1 --2- 172.21.15.3:0/1369582945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f8084200640 1 --2- 172.21.15.3:0/1369582945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cf270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f8084200640 1 -- 172.21.15.3:0/1369582945 >> 172.21.15.3:0/1369582945 conn(0x7f807c06bb20 msgr2=0x7f807c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9de00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54304/0 (socket says 172.21.15.3:54304) 2024-09-10T07:46:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f8084200640 1 -- 172.21.15.3:0/1369582945 shutdown_connections 2024-09-10T07:46:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.469+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34850/0 (socket says 172.21.15.3:34850) 2024-09-10T07:46:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7fd221e00640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 0x7fd20406e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd210009c50 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f3b9f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00af5c0 con 0x7f3ba0143de0 2024-09-10T07:46:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39a40cc880 0x7f39a40b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162ce0 0x7f39a40cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40cced0 con 0x7f39a4162900 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40ccd50 con 0x7f39a40cc880 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a9800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f39a40cd050 con 0x7f39a4162ce0 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f3b9f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00af2e0 con 0x7f3ba00b2000 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f3b9f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba00b55e0 con 0x7f3ba0141540 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f3b9de00640 1 -- 172.21.15.3:0/3194694468 learned_addr learned my addr 172.21.15.3:0/3194694468 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f3b9e800640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 0x7f3ba0139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f8084200640 1 -- 172.21.15.3:0/1369582945 wait complete. 2024-09-10T07:46:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45528/0 (socket says 172.21.15.3:45528) 2024-09-10T07:46:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a3400640 1 -- 172.21.15.3:0/2193108518 learned_addr learned my addr 172.21.15.3:0/2193108518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.470+0000 7f39a8e00640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162ce0 0x7f39a40cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39a40cc880 0x7f39a40b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3713200640 1 Processor -- start 2024-09-10T07:46:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3b9e800640 1 -- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 msgr2=0x7f3ba0142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3713200640 1 -- start start 2024-09-10T07:46:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f6b0ea00640 1 --2- 172.21.15.3:0/2781546686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b180b42a0 0x7f6b180ce3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b18070ea0 tx=0x7f6b0802a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4a506674a27f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd214022030 con 0x7fd224148ab0 2024-09-10T07:46:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3400640 1 -- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162ce0 msgr2=0x7f39a40cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3400640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162ce0 0x7f39a40cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3400640 1 -- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39a40cc880 msgr2=0x7f39a40b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3400640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39a40cc880 0x7f39a40b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f39a3400640 1 -- 172.21.15.3:0/2193108518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a41f87d0 con 0x7f39a4162900 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3b9e800640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3b9e800640 1 -- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 msgr2=0x7f3ba0070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3b9e800640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f3b9e800640 1 -- 172.21.15.3:0/3194694468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba013ef10 con 0x7f3ba0143de0 2024-09-10T07:46:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.471+0000 7f6b16a00640 1 -- 172.21.15.3:0/2781546686 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08037070 con 0x7f6b180b42a0 2024-09-10T07:46:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7f3b9d400640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7f39a3400640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f399000c7f0 tx=0x7f399000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7f6b16a00640 1 -- 172.21.15.3:0/2781546686 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b08032440 con 0x7f6b180b42a0 2024-09-10T07:46:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7f3b9e800640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 0x7f3ba0139480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ca70 tx=0x7f3b9800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.472+0000 7fd379800640 1 Processor -- start 2024-09-10T07:46:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990015070 con 0x7f39a4162900 2024-09-10T07:46:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3990007820 con 0x7f39a4162900 2024-09-10T07:46:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990007bf0 con 0x7f39a4162900 2024-09-10T07:46:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98017070 con 0x7f3ba0143de0 2024-09-10T07:46:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9800aa90 con 0x7f3ba0143de0 2024-09-10T07:46:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f8084200640 1 Processor -- start 2024-09-10T07:46:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f3713200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a40b1560 con 0x7f39a4162900 2024-09-10T07:46:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a40c61a0 con 0x7f39a4162900 2024-09-10T07:46:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7f8084200640 1 -- start start 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3713200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 0x7f370817d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3711e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3711e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45542/0 (socket says 172.21.15.3:45542) 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3711e00640 1 -- 172.21.15.3:0/1435329422 learned_addr learned my addr 172.21.15.3:0/1435329422 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3711400640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 0x7f370817d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37080e7120 0x7f370817d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f370801dc40 con 0x7f3708005ab0 2024-09-10T07:46:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3708053a10 con 0x7f370804eb60 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f370801ddc0 con 0x7f37080e7120 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98012400 con 0x7f3ba0143de0 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba013f1f0 con 0x7f3ba0143de0 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.474+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba013f7b0 con 0x7f3ba0143de0 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f6b16a00640 1 -- 172.21.15.3:0/2781546686 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08037070 con 0x7f6b180b42a0 2024-09-10T07:46:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f3712800640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37080e7120 0x7f370817d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f3711400640 1 -- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37080e7120 msgr2=0x7f370817d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f3711400640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37080e7120 0x7f370817d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f3711400640 1 -- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 msgr2=0x7f370817f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f8084200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.473+0000 7fd379800640 1 -- start start 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd379800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd379800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd36c0e7120 0x7fd36c005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd379800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c0063d0 con 0x7fd36c04eb60 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd379800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd36c0520e0 con 0x7fd36c0e7120 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd379800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c052330 con 0x7fd36c005ab0 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f6b1cc00640 1 -- 172.21.15.3:0/2781546686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b180b42a0 msgr2=0x7f6b180ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7fd373e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/2781546686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b180b42a0 0x7f6b180ce3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b18070ea0 tx=0x7f6b0802a280 comp rx=0 tx=0).stop 2024-09-10T07:46:44.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.475+0000 7f8084200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34870/0 (socket says 172.21.15.3:34870) 2024-09-10T07:46:44.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373e00640 1 -- 172.21.15.3:0/1562289868 learned_addr learned my addr 172.21.15.3:0/1562289868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f3711400640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f3711400640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370817d7b0 con 0x7f370804eb60 2024-09-10T07:46:44.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f3711e00640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:44.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373400640 1 --2- 172.21.15.3:0/1562289868 >> v2:172.21.15.70:3300/0 conn(0x7fd36c0e7120 0x7fd36c005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f8084200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f8081800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f8081800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54316/0 (socket says 172.21.15.3:54316) 2024-09-10T07:46:44.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373400640 1 -- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 msgr2=0x7fd36c053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7f8082200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f8081800640 1 -- 172.21.15.3:0/4119596981 learned_addr learned my addr 172.21.15.3:0/4119596981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f807c15e790 con 0x7f807c15e3b0 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0b76a0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f807c15e910 con 0x7f807c0d14d0 2024-09-10T07:46:44.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba0063610 con 0x7f3ba0143de0 2024-09-10T07:46:44.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7f3711400640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 0x7f370817d9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c430 tx=0x7f36fc00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373400640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373400640 1 -- 172.21.15.3:0/1562289868 >> v1:172.21.15.3:6789/0 conn(0x7fd36c04eb60 legacy=0x7fd36c01c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:44.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.476+0000 7fd373400640 1 -- 172.21.15.3:0/1562289868 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c006cd0 con 0x7fd36c0e7120 2024-09-10T07:46:44.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.477+0000 7fd373e00640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:44.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f6b1cc00640 1 -- 172.21.15.3:0/2781546686 shutdown_connections 2024-09-10T07:46:44.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/2781546686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b180b42a0 0x7f6b180ce3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c005290 con 0x7f39a4162900 2024-09-10T07:46:44.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7fd373400640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0e7120 0x7fd36c005e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097a0 tx=0x7fd36802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ed9a7cc0c06f95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/2781546686 >> v2:172.21.15.3:3300/0 conn(0x7f6b1806db20 0x7f6b180b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f8080e00640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f6b1cc00640 1 -- 172.21.15.3:0/2781546686 >> 172.21.15.3:0/2781546686 conn(0x7f6b1806d000 msgr2=0x7f6b1806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc017070 con 0x7f370804eb60 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8081800640 1 -- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 msgr2=0x7f807c0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8081800640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8081800640 1 -- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 msgr2=0x7f807c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8081800640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8081800640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0caec0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f8082200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370804dd90 con 0x7f370804eb60 2024-09-10T07:46:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f6b1cc00640 1 -- 172.21.15.3:0/2781546686 shutdown_connections 2024-09-10T07:46:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7fd372a00640 1 -- 172.21.15.3:0/1562289868 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368037070 con 0x7fd36c0e7120 2024-09-10T07:46:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.478+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3990018070 con 0x7f39a4162900 2024-09-10T07:46:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f399be00640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 0x7f398406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f399008ab10 con 0x7f39a4162900 2024-09-10T07:46:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f6b1cc00640 1 -- 172.21.15.3:0/2781546686 wait complete. 2024-09-10T07:46:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.480+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 0x7f398406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.480+0000 7f8081800640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f80700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc005ce0 con 0x7f370804eb60 2024-09-10T07:46:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.479+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc017070 con 0x7f370804eb60 2024-09-10T07:46:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.481+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370804bb10 con 0x7f370804eb60 2024-09-10T07:46:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.481+0000 7f39a3e00640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 0x7f398406e340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.480+0000 7fd372a00640 1 -- 172.21.15.3:0/1562289868 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd368032440 con 0x7fd36c0e7120 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.481+0000 7fd372a00640 1 -- 172.21.15.3:0/1562289868 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368032850 con 0x7fd36c0e7120 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 -- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0e7120 msgr2=0x7fd36c005e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0e7120 0x7fd36c005e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097a0 tx=0x7fd36802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 -- 172.21.15.3:0/1562289868 shutdown_connections 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0e7120 0x7fd36c005e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 --2- 172.21.15.3:0/1562289868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c005ab0 0x7fd36c053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.482+0000 7fd379800640 1 -- 172.21.15.3:0/1562289868 >> 172.21.15.3:0/1562289868 conn(0x7fd36c036090 msgr2=0x7fd36c0364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f6b1cc00640 1 Processor -- start 2024-09-10T07:46:44.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070037050 con 0x7f807c0cf7b0 2024-09-10T07:46:44.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070038040 con 0x7f807c0cf7b0 2024-09-10T07:46:44.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070003df0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7fd379800640 1 -- 172.21.15.3:0/1562289868 shutdown_connections 2024-09-10T07:46:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f6b1cc00640 1 -- start start 2024-09-10T07:46:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c0cd690 con 0x7f807c0cf7b0 2024-09-10T07:46:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c0b3fa0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7fd379800640 1 -- 172.21.15.3:0/1562289868 wait complete. 2024-09-10T07:46:44.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36fc01a070 con 0x7f370804eb60 2024-09-10T07:46:44.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3708047b50 con 0x7f370804eb60 2024-09-10T07:46:44.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.484+0000 7f6b1cc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.484+0000 7f6b1cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b180b42a0 0x7f6b1814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.484+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8058005290 con 0x7f807c0cf7b0 2024-09-10T07:46:44.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b1cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 0x7f6b1815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.483+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b9801a070 con 0x7f3ba0143de0 2024-09-10T07:46:44.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54324/0 (socket says 172.21.15.3:54324) 2024-09-10T07:46:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 -- 172.21.15.3:0/1687646772 learned_addr learned my addr 172.21.15.3:0/1687646772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180cee90 con 0x7f6b180b42a0 2024-09-10T07:46:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f3b96000640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 0x7f3b8006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180ced10 con 0x7f6b1806db20 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6b18154ef0 con 0x7f6b180ce900 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b0ea00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b180b42a0 0x7f6b1814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17e00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 0x7f6b1815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 -- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 msgr2=0x7f6b1815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 0x7f6b1815af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 -- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b180b42a0 msgr2=0x7f6b1814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3b9808ad80 con 0x7f3ba0143de0 2024-09-10T07:46:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b180b42a0 0x7f6b1814e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17400640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1815b830 con 0x7f6b1806db20 2024-09-10T07:46:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.485+0000 7f6b17e00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 0x7f6b1815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.486+0000 7f6b17400640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007ef0 tx=0x7f6b0000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.486+0000 7f3706000640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 0x7f36f006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.487+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f36fc08bcd0 con 0x7f370804eb60 2024-09-10T07:46:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.487+0000 7f3711e00640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 0x7f36f006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.487+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 0x7f3b8006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.487+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f399008f070 con 0x7f39a4162900 2024-09-10T07:46:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.488+0000 7fd379800640 1 Processor -- start 2024-09-10T07:46:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.488+0000 7f3b9de00640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 0x7f3b8006e370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b90000c00 tx=0x7f3b90042040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.488+0000 7f3711e00640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 0x7f36f006e4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3700009790 tx=0x7f370002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.488+0000 7fd379800640 1 -- start start 2024-09-10T07:46:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.489+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f370817d7b0 con 0x7f370804eb60 2024-09-10T07:46:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.489+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00014620 con 0x7f6b1806db20 2024-09-10T07:46:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.489+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b00019070 con 0x7f6b1806db20 2024-09-10T07:46:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.489+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00014a00 con 0x7f6b1806db20 2024-09-10T07:46:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.489+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1815bb10 con 0x7f6b1806db20 2024-09-10T07:46:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1815c0d0 con 0x7f6b1806db20 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 0x7fd36c0e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 0x7fd36c00b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 0x7fd36c0d2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd373e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 0x7fd36c0e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd373e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 0x7fd36c0e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54328/0 (socket says 172.21.15.3:54328) 2024-09-10T07:46:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c053a10 con 0x7fd36c04eb60 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c007290 con 0x7fd36c005ab0 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd379800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c01dc40 con 0x7fd36c0e7120 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd378e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 0x7fd36c0d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.490+0000 7fd373e00640 1 -- 172.21.15.3:0/3303998089 learned_addr learned my addr 172.21.15.3:0/3303998089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd373400640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 0x7fd36c00b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd378e00640 1 -- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 msgr2=0x7fd36c0e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd378e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 0x7fd36c0e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd378e00640 1 -- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 msgr2=0x7fd36c00b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd378e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 0x7fd36c00b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd378e00640 1 -- 172.21.15.3:0/3303998089 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c00aba0 con 0x7fd36c0e7120 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.491+0000 7fd373400640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 0x7fd36c00b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b18063610 con 0x7f6b1806db20 2024-09-10T07:46:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd378e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 0x7fd36c0d2650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd374051790 tx=0x7fd374069080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374090070 con 0x7fd36c0e7120 2024-09-10T07:46:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd374069fa0 con 0x7fd36c0e7120 2024-09-10T07:46:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740945f0 con 0x7fd36c0e7120 2024-09-10T07:46:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c0320c0 con 0x7fd36c0e7120 2024-09-10T07:46:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.492+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c00b0f0 con 0x7fd36c0e7120 2024-09-10T07:46:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.493+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8070045020 con 0x7f807c0cf7b0 2024-09-10T07:46:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.494+0000 7f8079600640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 0x7f805c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.494+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f80700ab9c0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.495+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd374093070 con 0x7fd36c0e7120 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.495+0000 7f8080e00640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 0x7f805c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.496+0000 7f8080e00640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 0x7f805c06e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f807c0c9bf0 tx=0x7f8068000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b98021020 con 0x7f3ba0143de0 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7fd367e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 0x7fd35406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd3741062b0 con 0x7fd36c0e7120 2024-09-10T07:46:44.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd350005290 con 0x7fd36c0e7120 2024-09-10T07:46:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b00014ba0 con 0x7f6b1806db20 2024-09-10T07:46:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.497+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f807c0caec0 con 0x7f807c0cf7b0 2024-09-10T07:46:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.499+0000 7fd373e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 0x7fd35406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.499+0000 7f6b0fe00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 0x7f6af406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.499+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6b0008c5f0 con 0x7f6b1806db20 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.499+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b0008ca10 con 0x7f6b1806db20 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.500+0000 7f6b0ea00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 0x7f6af406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.501+0000 7fd373e00640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 0x7fd35406e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd360006fd0 tx=0x7fd360008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.502+0000 7f6b0ea00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 0x7f6af406e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b08034000 tx=0x7f6b0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:44.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.506+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd3740d4e10 con 0x7fd36c0e7120 2024-09-10T07:46:44.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.795+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4504069f70 con 0x7f4504158ec0 2024-09-10T07:46:44.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.796+0000 7f4501600640 1 -- 172.21.15.3:0/403448312 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44f8058cc0 con 0x7f4504158ec0 2024-09-10T07:46:44.797 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-09-10T07:46:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.801+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 msgr2=0x7f44e406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.801+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 0x7f44e406e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44f0037000 tx=0x7f44f0032000 comp rx=0 tx=0).stop 2024-09-10T07:46:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.801+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 msgr2=0x7f45040b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.801+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c430 tx=0x7f44f800c8f0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.802+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 shutdown_connections 2024-09-10T07:46:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.802+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44e406bf00 0x7f44e406e3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.802+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4504158ec0 0x7f45040b1a40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.802+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f45040d0720 0x7f45040b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.802+0000 7f450c000640 1 --2- 172.21.15.3:0/403448312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45040b3a10 0x7f4504149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.803+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 >> 172.21.15.3:0/403448312 conn(0x7f450406d000 msgr2=0x7f45040af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.803+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 shutdown_connections 2024-09-10T07:46:44.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.803+0000 7f450c000640 1 -- 172.21.15.3:0/403448312 wait complete. 2024-09-10T07:46:44.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.4 2024-09-10T07:46:44.830 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.865+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f397c005850 con 0x7f39a4162900 2024-09-10T07:46:44.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.865+0000 7f399be00640 1 -- 172.21.15.3:0/2193108518 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f399008a090 con 0x7f39a4162900 2024-09-10T07:46:44.865 INFO:teuthology.orchestra.run.smithi003.stdout:64424509446 2024-09-10T07:46:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.867+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 msgr2=0x7f398406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.867+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 0x7f398406e340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 msgr2=0x7f39a40ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f399000c7f0 tx=0x7f399000ccb0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 shutdown_connections 2024-09-10T07:46:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f398406be80 0x7f398406e340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39a4162ce0 0x7f39a40cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39a4162900 0x7f39a40ca6e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 --2- 172.21.15.3:0/2193108518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39a40cc880 0x7f39a40b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 >> 172.21.15.3:0/2193108518 conn(0x7f39a406bb20 msgr2=0x7f39a40b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 shutdown_connections 2024-09-10T07:46:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.868+0000 7f39a9800640 1 -- 172.21.15.3:0/2193108518 wait complete. 2024-09-10T07:46:44.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-10T07:46:44.886 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.905+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd2240d4f60 con 0x7fd224148ab0 2024-09-10T07:46:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.905+0000 7fd21a000640 1 -- 172.21.15.3:0/3173328370 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd214059750 con 0x7fd224148ab0 2024-09-10T07:46:44.905 INFO:teuthology.orchestra.run.smithi003.stdout:68719476742 2024-09-10T07:46:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.907+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 msgr2=0x7fd20406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.907+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 0x7fd20406e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd210009c50 tx=0x7fd210008040 comp rx=0 tx=0).stop 2024-09-10T07:46:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.907+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 msgr2=0x7fd2241fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.907+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 0x7fd2241fcb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd2240cfef0 tx=0x7fd21400f920 comp rx=0 tx=0).stop 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 shutdown_connections 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd20406bf00 0x7fd20406e3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd224155910 0x7fd2241fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd224148ab0 0x7fd2241fcb50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 --2- 172.21.15.3:0/3173328370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2080051c0 0x7fd22414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 >> 172.21.15.3:0/3173328370 conn(0x7fd2240b6090 msgr2=0x7fd2240ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 shutdown_connections 2024-09-10T07:46:44.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.908+0000 7fd223200640 1 -- 172.21.15.3:0/3173328370 wait complete. 2024-09-10T07:46:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.924+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f370801d040 con 0x7f370804eb60 2024-09-10T07:46:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.924+0000 7f3706000640 1 -- 172.21.15.3:0/1435329422 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36fc057e90 con 0x7f370804eb60 2024-09-10T07:46:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.924+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8058005850 con 0x7f807c0cf7b0 2024-09-10T07:46:44.925 INFO:teuthology.orchestra.run.smithi003.stdout:64424509447 2024-09-10T07:46:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.925+0000 7f8079600640 1 -- 172.21.15.3:0/4119596981 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8070079d90 con 0x7f807c0cf7b0 2024-09-10T07:46:44.925 INFO:teuthology.orchestra.run.smithi003.stdout:68719476744 2024-09-10T07:46:44.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.927+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3ba0142aa0 con 0x7f3ba0143de0 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.927+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 msgr2=0x7f36f006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.927+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 0x7f36f006e4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3700009790 tx=0x7f370002e000 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.927+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 msgr2=0x7f370817d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.927+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 0x7f370817d9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c430 tx=0x7f36fc00c8f0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 shutdown_connections 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36f006c030 0x7f36f006e4f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37080e7120 0x7f370817d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f370804eb60 0x7f370817d9b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 --2- 172.21.15.3:0/1435329422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3708005ab0 0x7f370817f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 >> 172.21.15.3:0/1435329422 conn(0x7f3708036090 msgr2=0x7f37080342a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3b96000640 1 -- 172.21.15.3:0/3194694468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ba0142aa0 con 0x7f3ba0143de0 2024-09-10T07:46:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 msgr2=0x7f805c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 0x7f805c06e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f807c0c9bf0 tx=0x7f8068000ed0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 msgr2=0x7f807c152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f80700046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 shutdown_connections 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stdout:68719476743 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 shutdown_connections 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f805c06bf00 0x7f805c06e3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f807c15e3b0 0x7f807c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f807c0d14d0 0x7f807c0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 --2- 172.21.15.3:0/4119596981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807c0cf7b0 0x7f807c152ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 >> 172.21.15.3:0/4119596981 conn(0x7f807c06bb20 msgr2=0x7f807c0d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.928+0000 7f3713200640 1 -- 172.21.15.3:0/1435329422 wait complete. 2024-09-10T07:46:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.929+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd350005850 con 0x7fd36c0e7120 2024-09-10T07:46:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.929+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 shutdown_connections 2024-09-10T07:46:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.929+0000 7f8084200640 1 -- 172.21.15.3:0/4119596981 wait complete. 2024-09-10T07:46:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.929+0000 7fd367e00640 1 -- 172.21.15.3:0/3303998089 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd3740691b0 con 0x7fd36c0e7120 2024-09-10T07:46:44.932 INFO:teuthology.orchestra.run.smithi003.stdout:68719476744 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.930+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6b18073a90 con 0x7f6b1806db20 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.931+0000 7f6b0fe00640 1 -- 172.21.15.3:0/1687646772 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b0001b310 con 0x7f6b1806db20 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 msgr2=0x7f3b8006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 0x7f3b8006e370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b90000c00 tx=0x7f3b90042040 comp rx=0 tx=0).stop 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 msgr2=0x7f3ba0139480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 0x7f3ba0139480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ca70 tx=0x7f3b9800cf30 comp rx=0 tx=0).stop 2024-09-10T07:46:44.934 INFO:teuthology.orchestra.run.smithi003.stdout:64424509447 2024-09-10T07:46:44.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476742 for osd.1 2024-09-10T07:46:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 msgr2=0x7fd35406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 0x7fd35406e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd360006fd0 tx=0x7fd360008040 comp rx=0 tx=0).stop 2024-09-10T07:46:44.935 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 shutdown_connections 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b8006beb0 0x7f3b8006e370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ba0143de0 0x7f3ba0139480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ba0141540 0x7f3ba0142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 --2- 172.21.15.3:0/3194694468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ba00b2000 0x7f3ba0070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 >> 172.21.15.3:0/3194694468 conn(0x7f3ba006bb20 msgr2=0x7f3ba013c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 msgr2=0x7fd36c0d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.932+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 0x7fd36c0d2650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd374051790 tx=0x7fd374069080 comp rx=0 tx=0).stop 2024-09-10T07:46:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 shutdown_connections 2024-09-10T07:46:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7f3b9f200640 1 -- 172.21.15.3:0/3194694468 wait complete. 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 shutdown_connections 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd35406be80 0x7fd35406e340 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0e7120 0x7fd36c0d2650 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c04eb60 0x7fd36c00b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 --2- 172.21.15.3:0/3303998089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c005ab0 0x7fd36c0e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 >> 172.21.15.3:0/3303998089 conn(0x7fd36c036090 msgr2=0x7fd36c034570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 shutdown_connections 2024-09-10T07:46:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.933+0000 7fd379800640 1 -- 172.21.15.3:0/3303998089 wait complete. 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 msgr2=0x7f6af406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 0x7f6af406e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b08034000 tx=0x7f6b0802e000 comp rx=0 tx=0).stop 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 msgr2=0x7f6b181f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007ef0 tx=0x7f6b0000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 shutdown_connections 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6af406bf00 0x7f6af406e3c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b180ce900 0x7f6b1815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b180b42a0 0x7f6b1814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 --2- 172.21.15.3:0/1687646772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b1806db20 0x7f6b181f9030 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007ef0 tx=0x7f6b0000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:46:44.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 >> 172.21.15.3:0/1687646772 conn(0x7f6b1806d000 msgr2=0x7f6b180b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:44.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 shutdown_connections 2024-09-10T07:46:44.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:44.934+0000 7f6b1cc00640 1 -- 172.21.15.3:0/1687646772 wait complete. 2024-09-10T07:46:44.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-10T07:46:44.950 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476744 for osd.3 2024-09-10T07:46:44.953 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-10T07:46:44.953 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476744 for osd.2 2024-09-10T07:46:44.957 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.0 2024-09-10T07:46:44.961 DEBUG:teuthology.parallel:result is None 2024-09-10T07:46:44.961 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:46:44.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.156+0000 7f91d2c00640 1 Processor -- start 2024-09-10T07:46:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.156+0000 7f91d2c00640 1 -- start start 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.157+0000 7f91d2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 0x7f91cc06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.157+0000 7f91d2c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc0b36a0 con 0x7f91cc161b30 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.157+0000 7f91d2c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc0b3980 con 0x7f91cc1601f0 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.157+0000 7f91d2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0b3c60 con 0x7f91cc15fdb0 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.157+0000 7f91cbe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f91cc161b30 0x7f91cc15f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60240/0 (socket says 172.21.15.3:60240) 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91cbe00640 1 -- 172.21.15.3:0/11976726 learned_addr learned my addr 172.21.15.3:0/11976726 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91cb400640 1 --2- 172.21.15.3:0/11976726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 0x7f91cc06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832430025 0 0) 0x7f91cc0b36a0 con 0x7f91cc161b30 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b0003520 con 0x7f91cc161b30 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168619219 0 0) 0x7f91cc0b3980 con 0x7f91cc1601f0 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91cc0b36a0 con 0x7f91cc1601f0 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91cb400640 1 -- 172.21.15.3:0/11976726 >> v1:172.21.15.70:6789/0 conn(0x7f91cc1601f0 legacy=0x7f91cc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91cb400640 1 -- 172.21.15.3:0/11976726 >> v1:172.21.15.3:6789/0 conn(0x7f91cc161b30 legacy=0x7f91cc15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.158+0000 7f91cb400640 1 -- 172.21.15.3:0/11976726 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0b3f40 con 0x7f91cc15fdb0 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.159+0000 7f91cb400640 1 --2- 172.21.15.3:0/11976726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 0x7f91cc06bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91b80097d0 tx=0x7f91b802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efb022b66ab4481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8037070 con 0x7f91cc15fdb0 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b8032440 con 0x7f91cc15fdb0 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91caa00640 1 -- 172.21.15.3:0/11976726 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8032810 con 0x7f91cc15fdb0 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 -- 172.21.15.3:0/11976726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 msgr2=0x7f91cc06bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 --2- 172.21.15.3:0/11976726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 0x7f91cc06bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91b80097d0 tx=0x7f91b802a210 comp rx=0 tx=0).stop 2024-09-10T07:46:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 -- 172.21.15.3:0/11976726 shutdown_connections 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 --2- 172.21.15.3:0/11976726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc15fdb0 0x7f91cc06bb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 -- 172.21.15.3:0/11976726 >> 172.21.15.3:0/11976726 conn(0x7f91cc06d000 msgr2=0x7f91cc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 -- 172.21.15.3:0/11976726 shutdown_connections 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.160+0000 7f91d2c00640 1 -- 172.21.15.3:0/11976726 wait complete. 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.162+0000 7f91d2c00640 1 Processor -- start 2024-09-10T07:46:45.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.162+0000 7f91d2c00640 1 -- start start 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc1601f0 0x7f91cc072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 0x7f91cc1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0aefa0 con 0x7f91cc15fdb0 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc161f10 con 0x7f91cc161b30 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0b3300 con 0x7f91cc1601f0 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45582/0 (socket says 172.21.15.3:45582) 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 -- 172.21.15.3:0/2903437439 learned_addr learned my addr 172.21.15.3:0/2903437439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cb400640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc1601f0 0x7f91cc072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d0c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 0x7f91cc1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 -- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc1601f0 msgr2=0x7f91cc072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc1601f0 0x7f91cc072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 -- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 msgr2=0x7f91cc1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 0x7f91cc1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91cbe00640 1 -- 172.21.15.3:0/2903437439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc072b90 con 0x7f91cc15fdb0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.163+0000 7f91d0c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 0x7f91cc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91cbe00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c530 tx=0x7f91bc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc017070 con 0x7f91cc15fdb0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc005ce0 con 0x7f91cc15fdb0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007270 con 0x7f91cc15fdb0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc1f7940 con 0x7f91cc15fdb0 2024-09-10T07:46:45.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.164+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc0727b0 con 0x7f91cc15fdb0 2024-09-10T07:46:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.165+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc1f7f50 con 0x7f91cc15fdb0 2024-09-10T07:46:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.166+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91bc01a070 con 0x7f91cc15fdb0 2024-09-10T07:46:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.167+0000 7f91c3e00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.167+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f91bc08a180 con 0x7f91cc15fdb0 2024-09-10T07:46:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.167+0000 7f91cb400640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.168+0000 7f91cb400640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f91b8034000 tx=0x7f91b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.172+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f91bc0584d0 con 0x7f91cc15fdb0 2024-09-10T07:46:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.358+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91cc15f670 con 0x7f91ac06be30 2024-09-10T07:46:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.364+0000 7f91c3e00640 1 -- 172.21.15.3:0/2903437439 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251831 (secure 0 0 0) 0x7f91cc15f670 con 0x7f91ac06be30 2024-09-10T07:46:45.367 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.368+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 msgr2=0x7f91ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.368+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f91b8034000 tx=0x7f91b802e000 comp rx=0 tx=0).stop 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 msgr2=0x7f91cc1f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c530 tx=0x7f91bc00c9f0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 shutdown_connections 2024-09-10T07:46:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc161b30 0x7f91cc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc1601f0 0x7f91cc072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 --2- 172.21.15.3:0/2903437439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91cc15fdb0 0x7f91cc1f16a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 >> 172.21.15.3:0/2903437439 conn(0x7f91cc06d000 msgr2=0x7f91cc0af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 shutdown_connections 2024-09-10T07:46:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.369+0000 7f91d2c00640 1 -- 172.21.15.3:0/2903437439 wait complete. 2024-09-10T07:46:45.384 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T07:46:43.491917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235912,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738808,"statfs":{"total":773094113280,"available":772852539392,"internally_reserved":0,"allocated":3088384,"data_stored":1826476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.281793"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244567+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:32.244567+0000","last_peered":"2024-09-10T07:46:32.244567+0000","last_clean":"2024-09-10T07:46:32.244567+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:32.244567+0000","last_undegraded":"2024-09-10T07:46:32.244567+0000","last_fullsized":"2024-09-10T07:46:32.244567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470176+0000","last_change":"2024-09-10T07:46:31.233576+0000","last_active":"2024-09-10T07:46:34.470176+0000","last_peered":"2024-09-10T07:46:34.470176+0000","last_clean":"2024-09-10T07:46:34.470176+0000","last_became_active":"2024-09-10T07:46:31.230177+0000","last_became_peered":"2024-09-10T07:46:31.230177+0000","last_unstale":"2024-09-10T07:46:34.470176+0000","last_undegraded":"2024-09-10T07:46:34.470176+0000","last_fullsized":"2024-09-10T07:46:34.470176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470318+0000","last_change":"2024-09-10T07:46:31.229570+0000","last_active":"2024-09-10T07:46:34.470318+0000","last_peered":"2024-09-10T07:46:34.470318+0000","last_clean":"2024-09-10T07:46:34.470318+0000","last_became_active":"2024-09-10T07:46:31.229287+0000","last_became_peered":"2024-09-10T07:46:31.229287+0000","last_unstale":"2024-09-10T07:46:34.470318+0000","last_undegraded":"2024-09-10T07:46:34.470318+0000","last_fullsized":"2024-09-10T07:46:34.470318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732362+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:34.732362+0000","last_peered":"2024-09-10T07:46:34.732362+0000","last_clean":"2024-09-10T07:46:34.732362+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:34.732362+0000","last_undegraded":"2024-09-10T07:46:34.732362+0000","last_fullsized":"2024-09-10T07:46:34.732362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243957+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:32.243957+0000","last_peered":"2024-09-10T07:46:32.243957+0000","last_clean":"2024-09-10T07:46:32.243957+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:32.243957+0000","last_undegraded":"2024-09-10T07:46:32.243957+0000","last_fullsized":"2024-09-10T07:46:32.243957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732202+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:34.732202+0000","last_peered":"2024-09-10T07:46:34.732202+0000","last_clean":"2024-09-10T07:46:34.732202+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:34.732202+0000","last_undegraded":"2024-09-10T07:46:34.732202+0000","last_fullsized":"2024-09-10T07:46:34.732202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243828+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:32.243828+0000","last_peered":"2024-09-10T07:46:32.243828+0000","last_clean":"2024-09-10T07:46:32.243828+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:32.243828+0000","last_undegraded":"2024-09-10T07:46:32.243828+0000","last_fullsized":"2024-09-10T07:46:32.243828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470336+0000","last_change":"2024-09-10T07:46:31.242837+0000","last_active":"2024-09-10T07:46:34.470336+0000","last_peered":"2024-09-10T07:46:34.470336+0000","last_clean":"2024-09-10T07:46:34.470336+0000","last_became_active":"2024-09-10T07:46:31.242290+0000","last_became_peered":"2024-09-10T07:46:31.242290+0000","last_unstale":"2024-09-10T07:46:34.470336+0000","last_undegraded":"2024-09-10T07:46:34.470336+0000","last_fullsized":"2024-09-10T07:46:34.470336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212538+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:32.212538+0000","last_peered":"2024-09-10T07:46:32.212538+0000","last_clean":"2024-09-10T07:46:32.212538+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:32.212538+0000","last_undegraded":"2024-09-10T07:46:32.212538+0000","last_fullsized":"2024-09-10T07:46:32.212538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244315+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:32.244315+0000","last_peered":"2024-09-10T07:46:32.244315+0000","last_clean":"2024-09-10T07:46:32.244315+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:32.244315+0000","last_undegraded":"2024-09-10T07:46:32.244315+0000","last_fullsized":"2024-09-10T07:46:32.244315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244275+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:32.244275+0000","last_peered":"2024-09-10T07:46:32.244275+0000","last_clean":"2024-09-10T07:46:32.244275+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:32.244275+0000","last_undegraded":"2024-09-10T07:46:32.244275+0000","last_fullsized":"2024-09-10T07:46:32.244275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244135+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:32.244135+0000","last_peered":"2024-09-10T07:46:32.244135+0000","last_clean":"2024-09-10T07:46:32.244135+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:32.244135+0000","last_undegraded":"2024-09-10T07:46:32.244135+0000","last_fullsized":"2024-09-10T07:46:32.244135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214595+0000","last_change":"2024-09-10T07:46:31.240496+0000","last_active":"2024-09-10T07:46:32.214595+0000","last_peered":"2024-09-10T07:46:32.214595+0000","last_clean":"2024-09-10T07:46:32.214595+0000","last_became_active":"2024-09-10T07:46:31.240284+0000","last_became_peered":"2024-09-10T07:46:31.240284+0000","last_unstale":"2024-09-10T07:46:32.214595+0000","last_undegraded":"2024-09-10T07:46:32.214595+0000","last_fullsized":"2024-09-10T07:46:32.214595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212893+0000","last_change":"2024-09-10T07:46:31.234865+0000","last_active":"2024-09-10T07:46:32.212893+0000","last_peered":"2024-09-10T07:46:32.212893+0000","last_clean":"2024-09-10T07:46:32.212893+0000","last_became_active":"2024-09-10T07:46:31.234496+0000","last_became_peered":"2024-09-10T07:46:31.234496+0000","last_unstale":"2024-09-10T07:46:32.212893+0000","last_undegraded":"2024-09-10T07:46:32.212893+0000","last_fullsized":"2024-09-10T07:46:32.212893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469881+0000","last_change":"2024-09-10T07:46:31.245904+0000","last_active":"2024-09-10T07:46:34.469881+0000","last_peered":"2024-09-10T07:46:34.469881+0000","last_clean":"2024-09-10T07:46:34.469881+0000","last_became_active":"2024-09-10T07:46:31.245611+0000","last_became_peered":"2024-09-10T07:46:31.245611+0000","last_unstale":"2024-09-10T07:46:34.469881+0000","last_undegraded":"2024-09-10T07:46:34.469881+0000","last_fullsized":"2024-09-10T07:46:34.469881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959551+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:32.959551+0000","last_peered":"2024-09-10T07:46:32.959551+0000","last_clean":"2024-09-10T07:46:32.959551+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:32.959551+0000","last_undegraded":"2024-09-10T07:46:32.959551+0000","last_fullsized":"2024-09-10T07:46:32.959551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960470+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:32.960470+0000","last_peered":"2024-09-10T07:46:32.960470+0000","last_clean":"2024-09-10T07:46:32.960470+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:32.960470+0000","last_undegraded":"2024-09-10T07:46:32.960470+0000","last_fullsized":"2024-09-10T07:46:32.960470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:48.730223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487755+0000","last_change":"2024-09-10T07:46:31.237991+0000","last_active":"2024-09-10T07:46:32.487755+0000","last_peered":"2024-09-10T07:46:32.487755+0000","last_clean":"2024-09-10T07:46:32.487755+0000","last_became_active":"2024-09-10T07:46:31.237636+0000","last_became_peered":"2024-09-10T07:46:31.237636+0000","last_unstale":"2024-09-10T07:46:32.487755+0000","last_undegraded":"2024-09-10T07:46:32.487755+0000","last_fullsized":"2024-09-10T07:46:32.487755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960318+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:32.960318+0000","last_peered":"2024-09-10T07:46:32.960318+0000","last_clean":"2024-09-10T07:46:32.960318+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:32.960318+0000","last_undegraded":"2024-09-10T07:46:32.960318+0000","last_fullsized":"2024-09-10T07:46:32.960318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.816476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244209+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:32.244209+0000","last_peered":"2024-09-10T07:46:32.244209+0000","last_clean":"2024-09-10T07:46:32.244209+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:32.244209+0000","last_undegraded":"2024-09-10T07:46:32.244209+0000","last_fullsized":"2024-09-10T07:46:32.244209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211996+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:32.211996+0000","last_peered":"2024-09-10T07:46:32.211996+0000","last_clean":"2024-09-10T07:46:32.211996+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:32.211996+0000","last_undegraded":"2024-09-10T07:46:32.211996+0000","last_fullsized":"2024-09-10T07:46:32.211996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:33.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469941+0000","last_change":"2024-09-10T07:46:31.244889+0000","last_active":"2024-09-10T07:46:34.469941+0000","last_peered":"2024-09-10T07:46:34.469941+0000","last_clean":"2024-09-10T07:46:34.469941+0000","last_became_active":"2024-09-10T07:46:31.234234+0000","last_became_peered":"2024-09-10T07:46:31.234234+0000","last_unstale":"2024-09-10T07:46:34.469941+0000","last_undegraded":"2024-09-10T07:46:34.469941+0000","last_fullsized":"2024-09-10T07:46:34.469941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959527+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:32.959527+0000","last_peered":"2024-09-10T07:46:32.959527+0000","last_clean":"2024-09-10T07:46:32.959527+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:32.959527+0000","last_undegraded":"2024-09-10T07:46:32.959527+0000","last_fullsized":"2024-09-10T07:46:32.959527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470303+0000","last_change":"2024-09-10T07:46:31.238807+0000","last_active":"2024-09-10T07:46:34.470303+0000","last_peered":"2024-09-10T07:46:34.470303+0000","last_clean":"2024-09-10T07:46:34.470303+0000","last_became_active":"2024-09-10T07:46:31.238327+0000","last_became_peered":"2024-09-10T07:46:31.238327+0000","last_unstale":"2024-09-10T07:46:34.470303+0000","last_undegraded":"2024-09-10T07:46:34.470303+0000","last_fullsized":"2024-09-10T07:46:34.470303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732495+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:34.732495+0000","last_peered":"2024-09-10T07:46:34.732495+0000","last_clean":"2024-09-10T07:46:34.732495+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:34.732495+0000","last_undegraded":"2024-09-10T07:46:34.732495+0000","last_fullsized":"2024-09-10T07:46:34.732495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213178+0000","last_change":"2024-09-10T07:46:31.244052+0000","last_active":"2024-09-10T07:46:32.213178+0000","last_peered":"2024-09-10T07:46:32.213178+0000","last_clean":"2024-09-10T07:46:32.213178+0000","last_became_active":"2024-09-10T07:46:31.243687+0000","last_became_peered":"2024-09-10T07:46:31.243687+0000","last_unstale":"2024-09-10T07:46:32.213178+0000","last_undegraded":"2024-09-10T07:46:32.213178+0000","last_fullsized":"2024-09-10T07:46:32.213178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214001+0000","last_change":"2024-09-10T07:46:31.246440+0000","last_active":"2024-09-10T07:46:32.214001+0000","last_peered":"2024-09-10T07:46:32.214001+0000","last_clean":"2024-09-10T07:46:32.214001+0000","last_became_active":"2024-09-10T07:46:31.246117+0000","last_became_peered":"2024-09-10T07:46:31.246117+0000","last_unstale":"2024-09-10T07:46:32.214001+0000","last_undegraded":"2024-09-10T07:46:32.214001+0000","last_fullsized":"2024-09-10T07:46:32.214001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470149+0000","last_change":"2024-09-10T07:46:31.238490+0000","last_active":"2024-09-10T07:46:34.470149+0000","last_peered":"2024-09-10T07:46:34.470149+0000","last_clean":"2024-09-10T07:46:34.470149+0000","last_became_active":"2024-09-10T07:46:31.238151+0000","last_became_peered":"2024-09-10T07:46:31.238151+0000","last_unstale":"2024-09-10T07:46:34.470149+0000","last_undegraded":"2024-09-10T07:46:34.470149+0000","last_fullsized":"2024-09-10T07:46:34.470149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959750+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:32.959750+0000","last_peered":"2024-09-10T07:46:32.959750+0000","last_clean":"2024-09-10T07:46:32.959750+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:32.959750+0000","last_undegraded":"2024-09-10T07:46:32.959750+0000","last_fullsized":"2024-09-10T07:46:32.959750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:23.300273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487616+0000","last_change":"2024-09-10T07:46:31.235084+0000","last_active":"2024-09-10T07:46:32.487616+0000","last_peered":"2024-09-10T07:46:32.487616+0000","last_clean":"2024-09-10T07:46:32.487616+0000","last_became_active":"2024-09-10T07:46:31.234796+0000","last_became_peered":"2024-09-10T07:46:31.234796+0000","last_unstale":"2024-09-10T07:46:32.487616+0000","last_undegraded":"2024-09-10T07:46:32.487616+0000","last_fullsized":"2024-09-10T07:46:32.487616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732133+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:34.732133+0000","last_peered":"2024-09-10T07:46:34.732133+0000","last_clean":"2024-09-10T07:46:34.732133+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:34.732133+0000","last_undegraded":"2024-09-10T07:46:34.732133+0000","last_fullsized":"2024-09-10T07:46:34.732133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212250+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:32.212250+0000","last_peered":"2024-09-10T07:46:32.212250+0000","last_clean":"2024-09-10T07:46:32.212250+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:32.212250+0000","last_undegraded":"2024-09-10T07:46:32.212250+0000","last_fullsized":"2024-09-10T07:46:32.212250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470040+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:34.470040+0000","last_peered":"2024-09-10T07:46:34.470040+0000","last_clean":"2024-09-10T07:46:34.470040+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:34.470040+0000","last_undegraded":"2024-09-10T07:46:34.470040+0000","last_fullsized":"2024-09-10T07:46:34.470040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959867+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:32.959867+0000","last_peered":"2024-09-10T07:46:32.959867+0000","last_clean":"2024-09-10T07:46:32.959867+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:32.959867+0000","last_undegraded":"2024-09-10T07:46:32.959867+0000","last_fullsized":"2024-09-10T07:46:32.959867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214584+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:32.214584+0000","last_peered":"2024-09-10T07:46:32.214584+0000","last_clean":"2024-09-10T07:46:32.214584+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:32.214584+0000","last_undegraded":"2024-09-10T07:46:32.214584+0000","last_fullsized":"2024-09-10T07:46:32.214584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470527+0000","last_change":"2024-09-10T07:46:31.242643+0000","last_active":"2024-09-10T07:46:34.470527+0000","last_peered":"2024-09-10T07:46:34.470527+0000","last_clean":"2024-09-10T07:46:34.470527+0000","last_became_active":"2024-09-10T07:46:31.241930+0000","last_became_peered":"2024-09-10T07:46:31.241930+0000","last_unstale":"2024-09-10T07:46:34.470527+0000","last_undegraded":"2024-09-10T07:46:34.470527+0000","last_fullsized":"2024-09-10T07:46:34.470527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959655+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:32.959655+0000","last_peered":"2024-09-10T07:46:32.959655+0000","last_clean":"2024-09-10T07:46:32.959655+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:32.959655+0000","last_undegraded":"2024-09-10T07:46:32.959655+0000","last_fullsized":"2024-09-10T07:46:32.959655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212493+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:32.212493+0000","last_peered":"2024-09-10T07:46:32.212493+0000","last_clean":"2024-09-10T07:46:32.212493+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:32.212493+0000","last_undegraded":"2024-09-10T07:46:32.212493+0000","last_fullsized":"2024-09-10T07:46:32.212493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732520+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:34.732520+0000","last_peered":"2024-09-10T07:46:34.732520+0000","last_clean":"2024-09-10T07:46:34.732520+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:34.732520+0000","last_undegraded":"2024-09-10T07:46:34.732520+0000","last_fullsized":"2024-09-10T07:46:34.732520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244465+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:32.244465+0000","last_peered":"2024-09-10T07:46:32.244465+0000","last_clean":"2024-09-10T07:46:32.244465+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:32.244465+0000","last_undegraded":"2024-09-10T07:46:32.244465+0000","last_fullsized":"2024-09-10T07:46:32.244465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470484+0000","last_change":"2024-09-10T07:46:31.237271+0000","last_active":"2024-09-10T07:46:34.470484+0000","last_peered":"2024-09-10T07:46:34.470484+0000","last_clean":"2024-09-10T07:46:34.470484+0000","last_became_active":"2024-09-10T07:46:31.236938+0000","last_became_peered":"2024-09-10T07:46:31.236938+0000","last_unstale":"2024-09-10T07:46:34.470484+0000","last_undegraded":"2024-09-10T07:46:34.470484+0000","last_fullsized":"2024-09-10T07:46:34.470484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487630+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:32.487630+0000","last_peered":"2024-09-10T07:46:32.487630+0000","last_clean":"2024-09-10T07:46:32.487630+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:32.487630+0000","last_undegraded":"2024-09-10T07:46:32.487630+0000","last_fullsized":"2024-09-10T07:46:32.487630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959839+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:32.959839+0000","last_peered":"2024-09-10T07:46:32.959839+0000","last_clean":"2024-09-10T07:46:32.959839+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:32.959839+0000","last_undegraded":"2024-09-10T07:46:32.959839+0000","last_fullsized":"2024-09-10T07:46:32.959839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:11.790049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732340+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:34.732340+0000","last_peered":"2024-09-10T07:46:34.732340+0000","last_clean":"2024-09-10T07:46:34.732340+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:34.732340+0000","last_undegraded":"2024-09-10T07:46:34.732340+0000","last_fullsized":"2024-09-10T07:46:34.732340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244349+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:32.244349+0000","last_peered":"2024-09-10T07:46:32.244349+0000","last_clean":"2024-09-10T07:46:32.244349+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:32.244349+0000","last_undegraded":"2024-09-10T07:46:32.244349+0000","last_fullsized":"2024-09-10T07:46:32.244349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487577+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:32.487577+0000","last_peered":"2024-09-10T07:46:32.487577+0000","last_clean":"2024-09-10T07:46:32.487577+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:32.487577+0000","last_undegraded":"2024-09-10T07:46:32.487577+0000","last_fullsized":"2024-09-10T07:46:32.487577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213032+0000","last_change":"2024-09-10T07:46:31.243710+0000","last_active":"2024-09-10T07:46:32.213032+0000","last_peered":"2024-09-10T07:46:32.213032+0000","last_clean":"2024-09-10T07:46:32.213032+0000","last_became_active":"2024-09-10T07:46:31.243367+0000","last_became_peered":"2024-09-10T07:46:31.243367+0000","last_unstale":"2024-09-10T07:46:32.213032+0000","last_undegraded":"2024-09-10T07:46:32.213032+0000","last_fullsized":"2024-09-10T07:46:32.213032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960011+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:32.960011+0000","last_peered":"2024-09-10T07:46:32.960011+0000","last_clean":"2024-09-10T07:46:32.960011+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:32.960011+0000","last_undegraded":"2024-09-10T07:46:32.960011+0000","last_fullsized":"2024-09-10T07:46:32.960011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959776+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:32.959776+0000","last_peered":"2024-09-10T07:46:32.959776+0000","last_clean":"2024-09-10T07:46:32.959776+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:32.959776+0000","last_undegraded":"2024-09-10T07:46:32.959776+0000","last_fullsized":"2024-09-10T07:46:32.959776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:28:15.324769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243766+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:32.243766+0000","last_peered":"2024-09-10T07:46:32.243766+0000","last_clean":"2024-09-10T07:46:32.243766+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:32.243766+0000","last_undegraded":"2024-09-10T07:46:32.243766+0000","last_fullsized":"2024-09-10T07:46:32.243766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211818+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:32.211818+0000","last_peered":"2024-09-10T07:46:32.211818+0000","last_clean":"2024-09-10T07:46:32.211818+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:32.211818+0000","last_undegraded":"2024-09-10T07:46:32.211818+0000","last_fullsized":"2024-09-10T07:46:32.211818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959612+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:32.959612+0000","last_peered":"2024-09-10T07:46:32.959612+0000","last_clean":"2024-09-10T07:46:32.959612+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:32.959612+0000","last_undegraded":"2024-09-10T07:46:32.959612+0000","last_fullsized":"2024-09-10T07:46:32.959612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244676+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:32.244676+0000","last_peered":"2024-09-10T07:46:32.244676+0000","last_clean":"2024-09-10T07:46:32.244676+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:32.244676+0000","last_undegraded":"2024-09-10T07:46:32.244676+0000","last_fullsized":"2024-09-10T07:46:32.244676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732586+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:34.732586+0000","last_peered":"2024-09-10T07:46:34.732586+0000","last_clean":"2024-09-10T07:46:34.732586+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:34.732586+0000","last_undegraded":"2024-09-10T07:46:34.732586+0000","last_fullsized":"2024-09-10T07:46:34.732586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244571+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:32.244571+0000","last_peered":"2024-09-10T07:46:32.244571+0000","last_clean":"2024-09-10T07:46:32.244571+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:32.244571+0000","last_undegraded":"2024-09-10T07:46:32.244571+0000","last_fullsized":"2024-09-10T07:46:32.244571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470126+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:34.470126+0000","last_peered":"2024-09-10T07:46:34.470126+0000","last_clean":"2024-09-10T07:46:34.470126+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:34.470126+0000","last_undegraded":"2024-09-10T07:46:34.470126+0000","last_fullsized":"2024-09-10T07:46:34.470126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244247+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:32.244247+0000","last_peered":"2024-09-10T07:46:32.244247+0000","last_clean":"2024-09-10T07:46:32.244247+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:32.244247+0000","last_undegraded":"2024-09-10T07:46:32.244247+0000","last_fullsized":"2024-09-10T07:46:32.244247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214329+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:32.214329+0000","last_peered":"2024-09-10T07:46:32.214329+0000","last_clean":"2024-09-10T07:46:32.214329+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:32.214329+0000","last_undegraded":"2024-09-10T07:46:32.214329+0000","last_fullsized":"2024-09-10T07:46:32.214329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960136+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:32.960136+0000","last_peered":"2024-09-10T07:46:32.960136+0000","last_clean":"2024-09-10T07:46:32.960136+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:32.960136+0000","last_undegraded":"2024-09-10T07:46:32.960136+0000","last_fullsized":"2024-09-10T07:46:32.960136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.652763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211923+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:32.211923+0000","last_peered":"2024-09-10T07:46:32.211923+0000","last_clean":"2024-09-10T07:46:32.211923+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:32.211923+0000","last_undegraded":"2024-09-10T07:46:32.211923+0000","last_fullsized":"2024-09-10T07:46:32.211923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959922+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:32.959922+0000","last_peered":"2024-09-10T07:46:32.959922+0000","last_clean":"2024-09-10T07:46:32.959922+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:32.959922+0000","last_undegraded":"2024-09-10T07:46:32.959922+0000","last_fullsized":"2024-09-10T07:46:32.959922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959659+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:32.959659+0000","last_peered":"2024-09-10T07:46:32.959659+0000","last_clean":"2024-09-10T07:46:32.959659+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:32.959659+0000","last_undegraded":"2024-09-10T07:46:32.959659+0000","last_fullsized":"2024-09-10T07:46:32.959659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214503+0000","last_change":"2024-09-10T07:46:31.233650+0000","last_active":"2024-09-10T07:46:32.214503+0000","last_peered":"2024-09-10T07:46:32.214503+0000","last_clean":"2024-09-10T07:46:32.214503+0000","last_became_active":"2024-09-10T07:46:31.232959+0000","last_became_peered":"2024-09-10T07:46:31.232959+0000","last_unstale":"2024-09-10T07:46:32.214503+0000","last_undegraded":"2024-09-10T07:46:32.214503+0000","last_fullsized":"2024-09-10T07:46:32.214503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487400+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:32.487400+0000","last_peered":"2024-09-10T07:46:32.487400+0000","last_clean":"2024-09-10T07:46:32.487400+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:32.487400+0000","last_undegraded":"2024-09-10T07:46:32.487400+0000","last_fullsized":"2024-09-10T07:46:32.487400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470342+0000","last_change":"2024-09-10T07:46:31.244888+0000","last_active":"2024-09-10T07:46:34.470342+0000","last_peered":"2024-09-10T07:46:34.470342+0000","last_clean":"2024-09-10T07:46:34.470342+0000","last_became_active":"2024-09-10T07:46:31.244431+0000","last_became_peered":"2024-09-10T07:46:31.244431+0000","last_unstale":"2024-09-10T07:46:34.470342+0000","last_undegraded":"2024-09-10T07:46:34.470342+0000","last_fullsized":"2024-09-10T07:46:34.470342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960038+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:32.960038+0000","last_peered":"2024-09-10T07:46:32.960038+0000","last_clean":"2024-09-10T07:46:32.960038+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:32.960038+0000","last_undegraded":"2024-09-10T07:46:32.960038+0000","last_fullsized":"2024-09-10T07:46:32.960038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243996+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:32.243996+0000","last_peered":"2024-09-10T07:46:32.243996+0000","last_clean":"2024-09-10T07:46:32.243996+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:32.243996+0000","last_undegraded":"2024-09-10T07:46:32.243996+0000","last_fullsized":"2024-09-10T07:46:32.243996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470060+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:34.470060+0000","last_peered":"2024-09-10T07:46:34.470060+0000","last_clean":"2024-09-10T07:46:34.470060+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:34.470060+0000","last_undegraded":"2024-09-10T07:46:34.470060+0000","last_fullsized":"2024-09-10T07:46:34.470060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732286+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:34.732286+0000","last_peered":"2024-09-10T07:46:34.732286+0000","last_clean":"2024-09-10T07:46:34.732286+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:34.732286+0000","last_undegraded":"2024-09-10T07:46:34.732286+0000","last_fullsized":"2024-09-10T07:46:34.732286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960240+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:32.960240+0000","last_peered":"2024-09-10T07:46:32.960240+0000","last_clean":"2024-09-10T07:46:32.960240+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:32.960240+0000","last_undegraded":"2024-09-10T07:46:32.960240+0000","last_fullsized":"2024-09-10T07:46:32.960240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214164+0000","last_change":"2024-09-10T07:46:31.231122+0000","last_active":"2024-09-10T07:46:32.214164+0000","last_peered":"2024-09-10T07:46:32.214164+0000","last_clean":"2024-09-10T07:46:32.214164+0000","last_became_active":"2024-09-10T07:46:31.230087+0000","last_became_peered":"2024-09-10T07:46:31.230087+0000","last_unstale":"2024-09-10T07:46:32.214164+0000","last_undegraded":"2024-09-10T07:46:32.214164+0000","last_fullsized":"2024-09-10T07:46:32.214164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960429+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:32.960429+0000","last_peered":"2024-09-10T07:46:32.960429+0000","last_clean":"2024-09-10T07:46:32.960429+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:32.960429+0000","last_undegraded":"2024-09-10T07:46:32.960429+0000","last_fullsized":"2024-09-10T07:46:32.960429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243629+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:32.243629+0000","last_peered":"2024-09-10T07:46:32.243629+0000","last_clean":"2024-09-10T07:46:32.243629+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:32.243629+0000","last_undegraded":"2024-09-10T07:46:32.243629+0000","last_fullsized":"2024-09-10T07:46:32.243629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959717+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:32.959717+0000","last_peered":"2024-09-10T07:46:32.959717+0000","last_clean":"2024-09-10T07:46:32.959717+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:32.959717+0000","last_undegraded":"2024-09-10T07:46:32.959717+0000","last_fullsized":"2024-09-10T07:46:32.959717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:03.411205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213096+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:32.213096+0000","last_peered":"2024-09-10T07:46:32.213096+0000","last_clean":"2024-09-10T07:46:32.213096+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:32.213096+0000","last_undegraded":"2024-09-10T07:46:32.213096+0000","last_fullsized":"2024-09-10T07:46:32.213096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487538+0000","last_change":"2024-09-10T07:46:31.227688+0000","last_active":"2024-09-10T07:46:32.487538+0000","last_peered":"2024-09-10T07:46:32.487538+0000","last_clean":"2024-09-10T07:46:32.487538+0000","last_became_active":"2024-09-10T07:46:31.227344+0000","last_became_peered":"2024-09-10T07:46:31.227344+0000","last_unstale":"2024-09-10T07:46:32.487538+0000","last_undegraded":"2024-09-10T07:46:32.487538+0000","last_fullsized":"2024-09-10T07:46:32.487538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959762+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:32.959762+0000","last_peered":"2024-09-10T07:46:32.959762+0000","last_clean":"2024-09-10T07:46:32.959762+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:32.959762+0000","last_undegraded":"2024-09-10T07:46:32.959762+0000","last_fullsized":"2024-09-10T07:46:32.959762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959914+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:32.959914+0000","last_peered":"2024-09-10T07:46:32.959914+0000","last_clean":"2024-09-10T07:46:32.959914+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:32.959914+0000","last_undegraded":"2024-09-10T07:46:32.959914+0000","last_fullsized":"2024-09-10T07:46:32.959914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:27.770235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960085+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:32.960085+0000","last_peered":"2024-09-10T07:46:32.960085+0000","last_clean":"2024-09-10T07:46:32.960085+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:32.960085+0000","last_undegraded":"2024-09-10T07:46:32.960085+0000","last_fullsized":"2024-09-10T07:46:32.960085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:59.748478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244314+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:32.244314+0000","last_peered":"2024-09-10T07:46:32.244314+0000","last_clean":"2024-09-10T07:46:32.244314+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:32.244314+0000","last_undegraded":"2024-09-10T07:46:32.244314+0000","last_fullsized":"2024-09-10T07:46:32.244314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212216+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:32.212216+0000","last_peered":"2024-09-10T07:46:32.212216+0000","last_clean":"2024-09-10T07:46:32.212216+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:32.212216+0000","last_undegraded":"2024-09-10T07:46:32.212216+0000","last_fullsized":"2024-09-10T07:46:32.212216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244379+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:32.244379+0000","last_peered":"2024-09-10T07:46:32.244379+0000","last_clean":"2024-09-10T07:46:32.244379+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:32.244379+0000","last_undegraded":"2024-09-10T07:46:32.244379+0000","last_fullsized":"2024-09-10T07:46:32.244379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470327+0000","last_change":"2024-09-10T07:46:31.238515+0000","last_active":"2024-09-10T07:46:34.470327+0000","last_peered":"2024-09-10T07:46:34.470327+0000","last_clean":"2024-09-10T07:46:34.470327+0000","last_became_active":"2024-09-10T07:46:31.237949+0000","last_became_peered":"2024-09-10T07:46:31.237949+0000","last_unstale":"2024-09-10T07:46:34.470327+0000","last_undegraded":"2024-09-10T07:46:34.470327+0000","last_fullsized":"2024-09-10T07:46:34.470327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470350+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:34.470350+0000","last_peered":"2024-09-10T07:46:34.470350+0000","last_clean":"2024-09-10T07:46:34.470350+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:34.470350+0000","last_undegraded":"2024-09-10T07:46:34.470350+0000","last_fullsized":"2024-09-10T07:46:34.470350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470203+0000","last_change":"2024-09-10T07:46:31.238308+0000","last_active":"2024-09-10T07:46:34.470203+0000","last_peered":"2024-09-10T07:46:34.470203+0000","last_clean":"2024-09-10T07:46:34.470203+0000","last_became_active":"2024-09-10T07:46:31.238032+0000","last_became_peered":"2024-09-10T07:46:31.238032+0000","last_unstale":"2024-09-10T07:46:34.470203+0000","last_undegraded":"2024-09-10T07:46:34.470203+0000","last_fullsized":"2024-09-10T07:46:34.470203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":670553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":80171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":80239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":80115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":670575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":249856,"data_stored":82399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:45.388 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:46:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.581+0000 7f32e4400640 1 Processor -- start 2024-09-10T07:46:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.581+0000 7f32e4400640 1 -- start start 2024-09-10T07:46:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.582+0000 7f32e4400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f32dc0b6690 0x7f32dc0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.582+0000 7f32e4400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0b6a70 0x7f32dc0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.582+0000 7f32e4400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc0cff10 con 0x7f32dc0d1510 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.582+0000 7f32e4400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f32dc0cf540 con 0x7f32dc0b6690 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.582+0000 7f32e4400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc0d1d90 con 0x7f32dc0b6a70 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e2400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f32dc0d1510 0x7f32dc0d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60250/0 (socket says 172.21.15.3:60250) 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e2400640 1 -- 172.21.15.3:0/844064775 learned_addr learned my addr 172.21.15.3:0/844064775 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1a00640 1 --2- 172.21.15.3:0/844064775 >> v2:172.21.15.70:3300/0 conn(0x7f32dc0b6690 0x7f32dc0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1000640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0b6a70 0x7f32dc0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32dbe00640 1 -- 172.21.15.3:0/844064775 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751048326 0 0) 0x7f32dc0cff10 con 0x7f32dc0d1510 2024-09-10T07:46:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32dbe00640 1 -- 172.21.15.3:0/844064775 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c4003520 con 0x7f32dc0d1510 2024-09-10T07:46:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1a00640 1 -- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0b6a70 msgr2=0x7f32dc0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1a00640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0b6a70 0x7f32dc0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1a00640 1 -- 172.21.15.3:0/844064775 >> v1:172.21.15.3:6789/0 conn(0x7f32dc0d1510 legacy=0x7f32dc0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.583+0000 7f32e1a00640 1 -- 172.21.15.3:0/844064775 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0ceef0 con 0x7f32dc0b6690 2024-09-10T07:46:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.585+0000 7f32e1a00640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6690 0x7f32dc0cf9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32d00097a0 tx=0x7f32d002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f62d71406d4c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.588+0000 7f32dbe00640 1 -- 172.21.15.3:0/844064775 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037070 con 0x7f32dc0b6690 2024-09-10T07:46:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.588+0000 7f32dbe00640 1 -- 172.21.15.3:0/844064775 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d0032440 con 0x7f32dc0b6690 2024-09-10T07:46:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.588+0000 7f32dbe00640 1 -- 172.21.15.3:0/844064775 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0032810 con 0x7f32dc0b6690 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 -- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6690 msgr2=0x7f32dc0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6690 0x7f32dc0cf9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32d00097a0 tx=0x7f32d002a180 comp rx=0 tx=0).stop 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 -- 172.21.15.3:0/844064775 shutdown_connections 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0b6a70 0x7f32dc0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 --2- 172.21.15.3:0/844064775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6690 0x7f32dc0cf9d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 -- 172.21.15.3:0/844064775 >> 172.21.15.3:0/844064775 conn(0x7f32dc072560 msgr2=0x7f32dc072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 -- 172.21.15.3:0/844064775 shutdown_connections 2024-09-10T07:46:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.589+0000 7f32e4400640 1 -- 172.21.15.3:0/844064775 wait complete. 2024-09-10T07:46:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.591+0000 7f32e4400640 1 Processor -- start 2024-09-10T07:46:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.591+0000 7f32e4400640 1 -- start start 2024-09-10T07:46:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 0x7f32dc148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 0x7f32dc15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0d19a0 con 0x7f32dc0b6690 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0b6ed0 con 0x7f32dc0b6a70 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e4400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc0d1b20 con 0x7f32dc0d1510 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45584/0 (socket says 172.21.15.3:45584) 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e1a00640 1 -- 172.21.15.3:0/1623803833 learned_addr learned my addr 172.21.15.3:0/1623803833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e1000640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 0x7f32dc148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.592+0000 7f32e2400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 0x7f32dc15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 -- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 msgr2=0x7f32dc15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 0x7f32dc15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 -- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 msgr2=0x7f32dc148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 0x7f32dc148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 -- 172.21.15.3:0/1623803833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc1557d0 con 0x7f32dc0b6690 2024-09-10T07:46:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e2400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 0x7f32dc15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1000640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 0x7f32dc148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e1a00640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32d0035040 tx=0x7f32d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0041030 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d003d4e0 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037040 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc1492b0 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.593+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc154ea0 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.594+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc15c530 con 0x7f32dc0b6690 2024-09-10T07:46:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.595+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32d0044070 con 0x7f32dc0b6690 2024-09-10T07:46:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.596+0000 7f32d9600640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 0x7f32bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.596+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f32d00ab470 con 0x7f32dc0b6690 2024-09-10T07:46:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.596+0000 7f32e1000640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 0x7f32bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.597+0000 7f32e1000640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 0x7f32bc06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32dc1544a0 tx=0x7f32c8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.601+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f32d0079d90 con 0x7f32dc0b6690 2024-09-10T07:46:45.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.782+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32dc068d10 con 0x7f32bc06be60 2024-09-10T07:46:45.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.788+0000 7f32d9600640 1 -- 172.21.15.3:0/1623803833 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251827 (secure 0 0 0) 0x7f32dc068d10 con 0x7f32bc06be60 2024-09-10T07:46:45.790 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:45.792 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:46:45.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 msgr2=0x7f32bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 0x7f32bc06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32dc1544a0 tx=0x7f32c8009190 comp rx=0 tx=0).stop 2024-09-10T07:46:45.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 msgr2=0x7f32dc1535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32d0035040 tx=0x7f32d00046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 shutdown_connections 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32bc06be60 0x7f32bc06e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32dc0d1510 0x7f32dc15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32dc0b6a70 0x7f32dc148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 --2- 172.21.15.3:0/1623803833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32dc0b6690 0x7f32dc1535d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 >> 172.21.15.3:0/1623803833 conn(0x7f32dc072560 msgr2=0x7f32dc15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 shutdown_connections 2024-09-10T07:46:45.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:45.792+0000 7f32e4400640 1 -- 172.21.15.3:0/1623803833 wait complete. 2024-09-10T07:46:45.807 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-10T07:46:45.492483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235912,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738808,"statfs":{"total":773094113280,"available":772852539392,"internally_reserved":0,"allocated":3088384,"data_stored":1826476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":194,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002815"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244567+0000","last_change":"2024-09-10T07:46:31.240400+0000","last_active":"2024-09-10T07:46:32.244567+0000","last_peered":"2024-09-10T07:46:32.244567+0000","last_clean":"2024-09-10T07:46:32.244567+0000","last_became_active":"2024-09-10T07:46:31.239461+0000","last_became_peered":"2024-09-10T07:46:31.239461+0000","last_unstale":"2024-09-10T07:46:32.244567+0000","last_undegraded":"2024-09-10T07:46:32.244567+0000","last_fullsized":"2024-09-10T07:46:32.244567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470176+0000","last_change":"2024-09-10T07:46:31.233576+0000","last_active":"2024-09-10T07:46:34.470176+0000","last_peered":"2024-09-10T07:46:34.470176+0000","last_clean":"2024-09-10T07:46:34.470176+0000","last_became_active":"2024-09-10T07:46:31.230177+0000","last_became_peered":"2024-09-10T07:46:31.230177+0000","last_unstale":"2024-09-10T07:46:34.470176+0000","last_undegraded":"2024-09-10T07:46:34.470176+0000","last_fullsized":"2024-09-10T07:46:34.470176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470318+0000","last_change":"2024-09-10T07:46:31.229570+0000","last_active":"2024-09-10T07:46:34.470318+0000","last_peered":"2024-09-10T07:46:34.470318+0000","last_clean":"2024-09-10T07:46:34.470318+0000","last_became_active":"2024-09-10T07:46:31.229287+0000","last_became_peered":"2024-09-10T07:46:31.229287+0000","last_unstale":"2024-09-10T07:46:34.470318+0000","last_undegraded":"2024-09-10T07:46:34.470318+0000","last_fullsized":"2024-09-10T07:46:34.470318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732362+0000","last_change":"2024-09-10T07:46:31.218884+0000","last_active":"2024-09-10T07:46:34.732362+0000","last_peered":"2024-09-10T07:46:34.732362+0000","last_clean":"2024-09-10T07:46:34.732362+0000","last_became_active":"2024-09-10T07:46:31.218563+0000","last_became_peered":"2024-09-10T07:46:31.218563+0000","last_unstale":"2024-09-10T07:46:34.732362+0000","last_undegraded":"2024-09-10T07:46:34.732362+0000","last_fullsized":"2024-09-10T07:46:34.732362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243957+0000","last_change":"2024-09-10T07:46:31.246628+0000","last_active":"2024-09-10T07:46:32.243957+0000","last_peered":"2024-09-10T07:46:32.243957+0000","last_clean":"2024-09-10T07:46:32.243957+0000","last_became_active":"2024-09-10T07:46:31.246284+0000","last_became_peered":"2024-09-10T07:46:31.246284+0000","last_unstale":"2024-09-10T07:46:32.243957+0000","last_undegraded":"2024-09-10T07:46:32.243957+0000","last_fullsized":"2024-09-10T07:46:32.243957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.771486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732202+0000","last_change":"2024-09-10T07:46:31.220792+0000","last_active":"2024-09-10T07:46:34.732202+0000","last_peered":"2024-09-10T07:46:34.732202+0000","last_clean":"2024-09-10T07:46:34.732202+0000","last_became_active":"2024-09-10T07:46:31.219573+0000","last_became_peered":"2024-09-10T07:46:31.219573+0000","last_unstale":"2024-09-10T07:46:34.732202+0000","last_undegraded":"2024-09-10T07:46:34.732202+0000","last_fullsized":"2024-09-10T07:46:34.732202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243828+0000","last_change":"2024-09-10T07:46:31.231785+0000","last_active":"2024-09-10T07:46:32.243828+0000","last_peered":"2024-09-10T07:46:32.243828+0000","last_clean":"2024-09-10T07:46:32.243828+0000","last_became_active":"2024-09-10T07:46:31.231510+0000","last_became_peered":"2024-09-10T07:46:31.231510+0000","last_unstale":"2024-09-10T07:46:32.243828+0000","last_undegraded":"2024-09-10T07:46:32.243828+0000","last_fullsized":"2024-09-10T07:46:32.243828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470336+0000","last_change":"2024-09-10T07:46:31.242837+0000","last_active":"2024-09-10T07:46:34.470336+0000","last_peered":"2024-09-10T07:46:34.470336+0000","last_clean":"2024-09-10T07:46:34.470336+0000","last_became_active":"2024-09-10T07:46:31.242290+0000","last_became_peered":"2024-09-10T07:46:31.242290+0000","last_unstale":"2024-09-10T07:46:34.470336+0000","last_undegraded":"2024-09-10T07:46:34.470336+0000","last_fullsized":"2024-09-10T07:46:34.470336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212538+0000","last_change":"2024-09-10T07:46:31.233502+0000","last_active":"2024-09-10T07:46:32.212538+0000","last_peered":"2024-09-10T07:46:32.212538+0000","last_clean":"2024-09-10T07:46:32.212538+0000","last_became_active":"2024-09-10T07:46:31.233218+0000","last_became_peered":"2024-09-10T07:46:31.233218+0000","last_unstale":"2024-09-10T07:46:32.212538+0000","last_undegraded":"2024-09-10T07:46:32.212538+0000","last_fullsized":"2024-09-10T07:46:32.212538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244315+0000","last_change":"2024-09-10T07:46:31.238346+0000","last_active":"2024-09-10T07:46:32.244315+0000","last_peered":"2024-09-10T07:46:32.244315+0000","last_clean":"2024-09-10T07:46:32.244315+0000","last_became_active":"2024-09-10T07:46:31.238061+0000","last_became_peered":"2024-09-10T07:46:31.238061+0000","last_unstale":"2024-09-10T07:46:32.244315+0000","last_undegraded":"2024-09-10T07:46:32.244315+0000","last_fullsized":"2024-09-10T07:46:32.244315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244275+0000","last_change":"2024-09-10T07:46:31.239841+0000","last_active":"2024-09-10T07:46:32.244275+0000","last_peered":"2024-09-10T07:46:32.244275+0000","last_clean":"2024-09-10T07:46:32.244275+0000","last_became_active":"2024-09-10T07:46:31.239485+0000","last_became_peered":"2024-09-10T07:46:31.239485+0000","last_unstale":"2024-09-10T07:46:32.244275+0000","last_undegraded":"2024-09-10T07:46:32.244275+0000","last_fullsized":"2024-09-10T07:46:32.244275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:07.377534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244135+0000","last_change":"2024-09-10T07:46:31.242407+0000","last_active":"2024-09-10T07:46:32.244135+0000","last_peered":"2024-09-10T07:46:32.244135+0000","last_clean":"2024-09-10T07:46:32.244135+0000","last_became_active":"2024-09-10T07:46:31.241702+0000","last_became_peered":"2024-09-10T07:46:31.241702+0000","last_unstale":"2024-09-10T07:46:32.244135+0000","last_undegraded":"2024-09-10T07:46:32.244135+0000","last_fullsized":"2024-09-10T07:46:32.244135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214595+0000","last_change":"2024-09-10T07:46:31.240496+0000","last_active":"2024-09-10T07:46:32.214595+0000","last_peered":"2024-09-10T07:46:32.214595+0000","last_clean":"2024-09-10T07:46:32.214595+0000","last_became_active":"2024-09-10T07:46:31.240284+0000","last_became_peered":"2024-09-10T07:46:31.240284+0000","last_unstale":"2024-09-10T07:46:32.214595+0000","last_undegraded":"2024-09-10T07:46:32.214595+0000","last_fullsized":"2024-09-10T07:46:32.214595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:50:34.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212893+0000","last_change":"2024-09-10T07:46:31.234865+0000","last_active":"2024-09-10T07:46:32.212893+0000","last_peered":"2024-09-10T07:46:32.212893+0000","last_clean":"2024-09-10T07:46:32.212893+0000","last_became_active":"2024-09-10T07:46:31.234496+0000","last_became_peered":"2024-09-10T07:46:31.234496+0000","last_unstale":"2024-09-10T07:46:32.212893+0000","last_undegraded":"2024-09-10T07:46:32.212893+0000","last_fullsized":"2024-09-10T07:46:32.212893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469881+0000","last_change":"2024-09-10T07:46:31.245904+0000","last_active":"2024-09-10T07:46:34.469881+0000","last_peered":"2024-09-10T07:46:34.469881+0000","last_clean":"2024-09-10T07:46:34.469881+0000","last_became_active":"2024-09-10T07:46:31.245611+0000","last_became_peered":"2024-09-10T07:46:31.245611+0000","last_unstale":"2024-09-10T07:46:34.469881+0000","last_undegraded":"2024-09-10T07:46:34.469881+0000","last_fullsized":"2024-09-10T07:46:34.469881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:33.312287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959551+0000","last_change":"2024-09-10T07:46:31.226600+0000","last_active":"2024-09-10T07:46:32.959551+0000","last_peered":"2024-09-10T07:46:32.959551+0000","last_clean":"2024-09-10T07:46:32.959551+0000","last_became_active":"2024-09-10T07:46:31.226132+0000","last_became_peered":"2024-09-10T07:46:31.226132+0000","last_unstale":"2024-09-10T07:46:32.959551+0000","last_undegraded":"2024-09-10T07:46:32.959551+0000","last_fullsized":"2024-09-10T07:46:32.959551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960470+0000","last_change":"2024-09-10T07:46:31.231776+0000","last_active":"2024-09-10T07:46:32.960470+0000","last_peered":"2024-09-10T07:46:32.960470+0000","last_clean":"2024-09-10T07:46:32.960470+0000","last_became_active":"2024-09-10T07:46:31.231524+0000","last_became_peered":"2024-09-10T07:46:31.231524+0000","last_unstale":"2024-09-10T07:46:32.960470+0000","last_undegraded":"2024-09-10T07:46:32.960470+0000","last_fullsized":"2024-09-10T07:46:32.960470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:48.730223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487755+0000","last_change":"2024-09-10T07:46:31.237991+0000","last_active":"2024-09-10T07:46:32.487755+0000","last_peered":"2024-09-10T07:46:32.487755+0000","last_clean":"2024-09-10T07:46:32.487755+0000","last_became_active":"2024-09-10T07:46:31.237636+0000","last_became_peered":"2024-09-10T07:46:31.237636+0000","last_unstale":"2024-09-10T07:46:32.487755+0000","last_undegraded":"2024-09-10T07:46:32.487755+0000","last_fullsized":"2024-09-10T07:46:32.487755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960318+0000","last_change":"2024-09-10T07:46:31.216934+0000","last_active":"2024-09-10T07:46:32.960318+0000","last_peered":"2024-09-10T07:46:32.960318+0000","last_clean":"2024-09-10T07:46:32.960318+0000","last_became_active":"2024-09-10T07:46:31.216568+0000","last_became_peered":"2024-09-10T07:46:31.216568+0000","last_unstale":"2024-09-10T07:46:32.960318+0000","last_undegraded":"2024-09-10T07:46:32.960318+0000","last_fullsized":"2024-09-10T07:46:32.960318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.816476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244209+0000","last_change":"2024-09-10T07:46:31.239436+0000","last_active":"2024-09-10T07:46:32.244209+0000","last_peered":"2024-09-10T07:46:32.244209+0000","last_clean":"2024-09-10T07:46:32.244209+0000","last_became_active":"2024-09-10T07:46:31.239088+0000","last_became_peered":"2024-09-10T07:46:31.239088+0000","last_unstale":"2024-09-10T07:46:32.244209+0000","last_undegraded":"2024-09-10T07:46:32.244209+0000","last_fullsized":"2024-09-10T07:46:32.244209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213340+0000","last_change":"2024-09-10T07:46:31.232659+0000","last_active":"2024-09-10T07:46:32.213340+0000","last_peered":"2024-09-10T07:46:32.213340+0000","last_clean":"2024-09-10T07:46:32.213340+0000","last_became_active":"2024-09-10T07:46:31.232181+0000","last_became_peered":"2024-09-10T07:46:31.232181+0000","last_unstale":"2024-09-10T07:46:32.213340+0000","last_undegraded":"2024-09-10T07:46:32.213340+0000","last_fullsized":"2024-09-10T07:46:32.213340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211996+0000","last_change":"2024-09-10T07:46:31.229160+0000","last_active":"2024-09-10T07:46:32.211996+0000","last_peered":"2024-09-10T07:46:32.211996+0000","last_clean":"2024-09-10T07:46:32.211996+0000","last_became_active":"2024-09-10T07:46:31.228765+0000","last_became_peered":"2024-09-10T07:46:31.228765+0000","last_unstale":"2024-09-10T07:46:32.211996+0000","last_undegraded":"2024-09-10T07:46:32.211996+0000","last_fullsized":"2024-09-10T07:46:32.211996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:33.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.469941+0000","last_change":"2024-09-10T07:46:31.244889+0000","last_active":"2024-09-10T07:46:34.469941+0000","last_peered":"2024-09-10T07:46:34.469941+0000","last_clean":"2024-09-10T07:46:34.469941+0000","last_became_active":"2024-09-10T07:46:31.234234+0000","last_became_peered":"2024-09-10T07:46:31.234234+0000","last_unstale":"2024-09-10T07:46:34.469941+0000","last_undegraded":"2024-09-10T07:46:34.469941+0000","last_fullsized":"2024-09-10T07:46:34.469941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:36.849126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959527+0000","last_change":"2024-09-10T07:46:31.243238+0000","last_active":"2024-09-10T07:46:32.959527+0000","last_peered":"2024-09-10T07:46:32.959527+0000","last_clean":"2024-09-10T07:46:32.959527+0000","last_became_active":"2024-09-10T07:46:31.242951+0000","last_became_peered":"2024-09-10T07:46:31.242951+0000","last_unstale":"2024-09-10T07:46:32.959527+0000","last_undegraded":"2024-09-10T07:46:32.959527+0000","last_fullsized":"2024-09-10T07:46:32.959527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470303+0000","last_change":"2024-09-10T07:46:31.238807+0000","last_active":"2024-09-10T07:46:34.470303+0000","last_peered":"2024-09-10T07:46:34.470303+0000","last_clean":"2024-09-10T07:46:34.470303+0000","last_became_active":"2024-09-10T07:46:31.238327+0000","last_became_peered":"2024-09-10T07:46:31.238327+0000","last_unstale":"2024-09-10T07:46:34.470303+0000","last_undegraded":"2024-09-10T07:46:34.470303+0000","last_fullsized":"2024-09-10T07:46:34.470303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732495+0000","last_change":"2024-09-10T07:46:31.222839+0000","last_active":"2024-09-10T07:46:34.732495+0000","last_peered":"2024-09-10T07:46:34.732495+0000","last_clean":"2024-09-10T07:46:34.732495+0000","last_became_active":"2024-09-10T07:46:31.221541+0000","last_became_peered":"2024-09-10T07:46:31.221541+0000","last_unstale":"2024-09-10T07:46:34.732495+0000","last_undegraded":"2024-09-10T07:46:34.732495+0000","last_fullsized":"2024-09-10T07:46:34.732495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213178+0000","last_change":"2024-09-10T07:46:31.244052+0000","last_active":"2024-09-10T07:46:32.213178+0000","last_peered":"2024-09-10T07:46:32.213178+0000","last_clean":"2024-09-10T07:46:32.213178+0000","last_became_active":"2024-09-10T07:46:31.243687+0000","last_became_peered":"2024-09-10T07:46:31.243687+0000","last_unstale":"2024-09-10T07:46:32.213178+0000","last_undegraded":"2024-09-10T07:46:32.213178+0000","last_fullsized":"2024-09-10T07:46:32.213178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:33.120098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214001+0000","last_change":"2024-09-10T07:46:31.246440+0000","last_active":"2024-09-10T07:46:32.214001+0000","last_peered":"2024-09-10T07:46:32.214001+0000","last_clean":"2024-09-10T07:46:32.214001+0000","last_became_active":"2024-09-10T07:46:31.246117+0000","last_became_peered":"2024-09-10T07:46:31.246117+0000","last_unstale":"2024-09-10T07:46:32.214001+0000","last_undegraded":"2024-09-10T07:46:32.214001+0000","last_fullsized":"2024-09-10T07:46:32.214001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470149+0000","last_change":"2024-09-10T07:46:31.238490+0000","last_active":"2024-09-10T07:46:34.470149+0000","last_peered":"2024-09-10T07:46:34.470149+0000","last_clean":"2024-09-10T07:46:34.470149+0000","last_became_active":"2024-09-10T07:46:31.238151+0000","last_became_peered":"2024-09-10T07:46:31.238151+0000","last_unstale":"2024-09-10T07:46:34.470149+0000","last_undegraded":"2024-09-10T07:46:34.470149+0000","last_fullsized":"2024-09-10T07:46:34.470149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:27.649310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959750+0000","last_change":"2024-09-10T07:46:31.240730+0000","last_active":"2024-09-10T07:46:32.959750+0000","last_peered":"2024-09-10T07:46:32.959750+0000","last_clean":"2024-09-10T07:46:32.959750+0000","last_became_active":"2024-09-10T07:46:31.240450+0000","last_became_peered":"2024-09-10T07:46:31.240450+0000","last_unstale":"2024-09-10T07:46:32.959750+0000","last_undegraded":"2024-09-10T07:46:32.959750+0000","last_fullsized":"2024-09-10T07:46:32.959750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:23.300273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487616+0000","last_change":"2024-09-10T07:46:31.235084+0000","last_active":"2024-09-10T07:46:32.487616+0000","last_peered":"2024-09-10T07:46:32.487616+0000","last_clean":"2024-09-10T07:46:32.487616+0000","last_became_active":"2024-09-10T07:46:31.234796+0000","last_became_peered":"2024-09-10T07:46:31.234796+0000","last_unstale":"2024-09-10T07:46:32.487616+0000","last_undegraded":"2024-09-10T07:46:32.487616+0000","last_fullsized":"2024-09-10T07:46:32.487616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732133+0000","last_change":"2024-09-10T07:46:31.226987+0000","last_active":"2024-09-10T07:46:34.732133+0000","last_peered":"2024-09-10T07:46:34.732133+0000","last_clean":"2024-09-10T07:46:34.732133+0000","last_became_active":"2024-09-10T07:46:31.225913+0000","last_became_peered":"2024-09-10T07:46:31.225913+0000","last_unstale":"2024-09-10T07:46:34.732133+0000","last_undegraded":"2024-09-10T07:46:34.732133+0000","last_fullsized":"2024-09-10T07:46:34.732133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212250+0000","last_change":"2024-09-10T07:46:31.240342+0000","last_active":"2024-09-10T07:46:32.212250+0000","last_peered":"2024-09-10T07:46:32.212250+0000","last_clean":"2024-09-10T07:46:32.212250+0000","last_became_active":"2024-09-10T07:46:31.240071+0000","last_became_peered":"2024-09-10T07:46:31.240071+0000","last_unstale":"2024-09-10T07:46:32.212250+0000","last_undegraded":"2024-09-10T07:46:32.212250+0000","last_fullsized":"2024-09-10T07:46:32.212250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:19.173695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470040+0000","last_change":"2024-09-10T07:46:30.219745+0000","last_active":"2024-09-10T07:46:34.470040+0000","last_peered":"2024-09-10T07:46:34.470040+0000","last_clean":"2024-09-10T07:46:34.470040+0000","last_became_active":"2024-09-10T07:46:30.218150+0000","last_became_peered":"2024-09-10T07:46:30.218150+0000","last_unstale":"2024-09-10T07:46:34.470040+0000","last_undegraded":"2024-09-10T07:46:34.470040+0000","last_fullsized":"2024-09-10T07:46:34.470040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213462+0000","last_change":"2024-09-10T07:46:31.236587+0000","last_active":"2024-09-10T07:46:32.213462+0000","last_peered":"2024-09-10T07:46:32.213462+0000","last_clean":"2024-09-10T07:46:32.213462+0000","last_became_active":"2024-09-10T07:46:31.236300+0000","last_became_peered":"2024-09-10T07:46:31.236300+0000","last_unstale":"2024-09-10T07:46:32.213462+0000","last_undegraded":"2024-09-10T07:46:32.213462+0000","last_fullsized":"2024-09-10T07:46:32.213462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:05.918664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959867+0000","last_change":"2024-09-10T07:46:30.229760+0000","last_active":"2024-09-10T07:46:32.959867+0000","last_peered":"2024-09-10T07:46:32.959867+0000","last_clean":"2024-09-10T07:46:32.959867+0000","last_became_active":"2024-09-10T07:46:30.229493+0000","last_became_peered":"2024-09-10T07:46:30.229493+0000","last_unstale":"2024-09-10T07:46:32.959867+0000","last_undegraded":"2024-09-10T07:46:32.959867+0000","last_fullsized":"2024-09-10T07:46:32.959867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:48.047655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214584+0000","last_change":"2024-09-10T07:46:30.231405+0000","last_active":"2024-09-10T07:46:32.214584+0000","last_peered":"2024-09-10T07:46:32.214584+0000","last_clean":"2024-09-10T07:46:32.214584+0000","last_became_active":"2024-09-10T07:46:30.230959+0000","last_became_peered":"2024-09-10T07:46:30.230959+0000","last_unstale":"2024-09-10T07:46:32.214584+0000","last_undegraded":"2024-09-10T07:46:32.214584+0000","last_fullsized":"2024-09-10T07:46:32.214584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470527+0000","last_change":"2024-09-10T07:46:31.242643+0000","last_active":"2024-09-10T07:46:34.470527+0000","last_peered":"2024-09-10T07:46:34.470527+0000","last_clean":"2024-09-10T07:46:34.470527+0000","last_became_active":"2024-09-10T07:46:31.241930+0000","last_became_peered":"2024-09-10T07:46:31.241930+0000","last_unstale":"2024-09-10T07:46:34.470527+0000","last_undegraded":"2024-09-10T07:46:34.470527+0000","last_fullsized":"2024-09-10T07:46:34.470527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:30.334118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959655+0000","last_change":"2024-09-10T07:46:30.220162+0000","last_active":"2024-09-10T07:46:32.959655+0000","last_peered":"2024-09-10T07:46:32.959655+0000","last_clean":"2024-09-10T07:46:32.959655+0000","last_became_active":"2024-09-10T07:46:30.212858+0000","last_became_peered":"2024-09-10T07:46:30.212858+0000","last_unstale":"2024-09-10T07:46:32.959655+0000","last_undegraded":"2024-09-10T07:46:32.959655+0000","last_fullsized":"2024-09-10T07:46:32.959655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212946+0000","last_change":"2024-09-10T07:46:31.241738+0000","last_active":"2024-09-10T07:46:32.212946+0000","last_peered":"2024-09-10T07:46:32.212946+0000","last_clean":"2024-09-10T07:46:32.212946+0000","last_became_active":"2024-09-10T07:46:31.239002+0000","last_became_peered":"2024-09-10T07:46:31.239002+0000","last_unstale":"2024-09-10T07:46:32.212946+0000","last_undegraded":"2024-09-10T07:46:32.212946+0000","last_fullsized":"2024-09-10T07:46:32.212946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:13.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212493+0000","last_change":"2024-09-10T07:46:30.226123+0000","last_active":"2024-09-10T07:46:32.212493+0000","last_peered":"2024-09-10T07:46:32.212493+0000","last_clean":"2024-09-10T07:46:32.212493+0000","last_became_active":"2024-09-10T07:46:30.224153+0000","last_became_peered":"2024-09-10T07:46:30.224153+0000","last_unstale":"2024-09-10T07:46:32.212493+0000","last_undegraded":"2024-09-10T07:46:32.212493+0000","last_fullsized":"2024-09-10T07:46:32.212493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732520+0000","last_change":"2024-09-10T07:46:31.223078+0000","last_active":"2024-09-10T07:46:34.732520+0000","last_peered":"2024-09-10T07:46:34.732520+0000","last_clean":"2024-09-10T07:46:34.732520+0000","last_became_active":"2024-09-10T07:46:31.222543+0000","last_became_peered":"2024-09-10T07:46:31.222543+0000","last_unstale":"2024-09-10T07:46:34.732520+0000","last_undegraded":"2024-09-10T07:46:34.732520+0000","last_fullsized":"2024-09-10T07:46:34.732520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:53.518860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244465+0000","last_change":"2024-09-10T07:46:30.232907+0000","last_active":"2024-09-10T07:46:32.244465+0000","last_peered":"2024-09-10T07:46:32.244465+0000","last_clean":"2024-09-10T07:46:32.244465+0000","last_became_active":"2024-09-10T07:46:30.222981+0000","last_became_peered":"2024-09-10T07:46:30.222981+0000","last_unstale":"2024-09-10T07:46:32.244465+0000","last_undegraded":"2024-09-10T07:46:32.244465+0000","last_fullsized":"2024-09-10T07:46:32.244465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:32.051995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470484+0000","last_change":"2024-09-10T07:46:31.237271+0000","last_active":"2024-09-10T07:46:34.470484+0000","last_peered":"2024-09-10T07:46:34.470484+0000","last_clean":"2024-09-10T07:46:34.470484+0000","last_became_active":"2024-09-10T07:46:31.236938+0000","last_became_peered":"2024-09-10T07:46:31.236938+0000","last_unstale":"2024-09-10T07:46:34.470484+0000","last_undegraded":"2024-09-10T07:46:34.470484+0000","last_fullsized":"2024-09-10T07:46:34.470484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487630+0000","last_change":"2024-09-10T07:46:30.224107+0000","last_active":"2024-09-10T07:46:32.487630+0000","last_peered":"2024-09-10T07:46:32.487630+0000","last_clean":"2024-09-10T07:46:32.487630+0000","last_became_active":"2024-09-10T07:46:30.223862+0000","last_became_peered":"2024-09-10T07:46:30.223862+0000","last_unstale":"2024-09-10T07:46:32.487630+0000","last_undegraded":"2024-09-10T07:46:32.487630+0000","last_fullsized":"2024-09-10T07:46:32.487630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959839+0000","last_change":"2024-09-10T07:46:31.225703+0000","last_active":"2024-09-10T07:46:32.959839+0000","last_peered":"2024-09-10T07:46:32.959839+0000","last_clean":"2024-09-10T07:46:32.959839+0000","last_became_active":"2024-09-10T07:46:31.225414+0000","last_became_peered":"2024-09-10T07:46:31.225414+0000","last_unstale":"2024-09-10T07:46:32.959839+0000","last_undegraded":"2024-09-10T07:46:32.959839+0000","last_fullsized":"2024-09-10T07:46:32.959839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:11.790049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732340+0000","last_change":"2024-09-10T07:46:30.222955+0000","last_active":"2024-09-10T07:46:34.732340+0000","last_peered":"2024-09-10T07:46:34.732340+0000","last_clean":"2024-09-10T07:46:34.732340+0000","last_became_active":"2024-09-10T07:46:30.222638+0000","last_became_peered":"2024-09-10T07:46:30.222638+0000","last_unstale":"2024-09-10T07:46:34.732340+0000","last_undegraded":"2024-09-10T07:46:34.732340+0000","last_fullsized":"2024-09-10T07:46:34.732340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:52.145845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244349+0000","last_change":"2024-09-10T07:46:31.231061+0000","last_active":"2024-09-10T07:46:32.244349+0000","last_peered":"2024-09-10T07:46:32.244349+0000","last_clean":"2024-09-10T07:46:32.244349+0000","last_became_active":"2024-09-10T07:46:31.230814+0000","last_became_peered":"2024-09-10T07:46:31.230814+0000","last_unstale":"2024-09-10T07:46:32.244349+0000","last_undegraded":"2024-09-10T07:46:32.244349+0000","last_fullsized":"2024-09-10T07:46:32.244349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:14.720939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487577+0000","last_change":"2024-09-10T07:46:30.234858+0000","last_active":"2024-09-10T07:46:32.487577+0000","last_peered":"2024-09-10T07:46:32.487577+0000","last_clean":"2024-09-10T07:46:32.487577+0000","last_became_active":"2024-09-10T07:46:30.234622+0000","last_became_peered":"2024-09-10T07:46:30.234622+0000","last_unstale":"2024-09-10T07:46:32.487577+0000","last_undegraded":"2024-09-10T07:46:32.487577+0000","last_fullsized":"2024-09-10T07:46:32.487577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213032+0000","last_change":"2024-09-10T07:46:31.243710+0000","last_active":"2024-09-10T07:46:32.213032+0000","last_peered":"2024-09-10T07:46:32.213032+0000","last_clean":"2024-09-10T07:46:32.213032+0000","last_became_active":"2024-09-10T07:46:31.243367+0000","last_became_peered":"2024-09-10T07:46:31.243367+0000","last_unstale":"2024-09-10T07:46:32.213032+0000","last_undegraded":"2024-09-10T07:46:32.213032+0000","last_fullsized":"2024-09-10T07:46:32.213032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:49.050324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960011+0000","last_change":"2024-09-10T07:46:30.210494+0000","last_active":"2024-09-10T07:46:32.960011+0000","last_peered":"2024-09-10T07:46:32.960011+0000","last_clean":"2024-09-10T07:46:32.960011+0000","last_became_active":"2024-09-10T07:46:30.210213+0000","last_became_peered":"2024-09-10T07:46:30.210213+0000","last_unstale":"2024-09-10T07:46:32.960011+0000","last_undegraded":"2024-09-10T07:46:32.960011+0000","last_fullsized":"2024-09-10T07:46:32.960011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:32.117430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959776+0000","last_change":"2024-09-10T07:46:31.229313+0000","last_active":"2024-09-10T07:46:32.959776+0000","last_peered":"2024-09-10T07:46:32.959776+0000","last_clean":"2024-09-10T07:46:32.959776+0000","last_became_active":"2024-09-10T07:46:31.229085+0000","last_became_peered":"2024-09-10T07:46:31.229085+0000","last_unstale":"2024-09-10T07:46:32.959776+0000","last_undegraded":"2024-09-10T07:46:32.959776+0000","last_fullsized":"2024-09-10T07:46:32.959776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:28:15.324769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243766+0000","last_change":"2024-09-10T07:46:30.233176+0000","last_active":"2024-09-10T07:46:32.243766+0000","last_peered":"2024-09-10T07:46:32.243766+0000","last_clean":"2024-09-10T07:46:32.243766+0000","last_became_active":"2024-09-10T07:46:30.232789+0000","last_became_peered":"2024-09-10T07:46:30.232789+0000","last_unstale":"2024-09-10T07:46:32.243766+0000","last_undegraded":"2024-09-10T07:46:32.243766+0000","last_fullsized":"2024-09-10T07:46:32.243766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:33.659243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211818+0000","last_change":"2024-09-10T07:46:31.243266+0000","last_active":"2024-09-10T07:46:32.211818+0000","last_peered":"2024-09-10T07:46:32.211818+0000","last_clean":"2024-09-10T07:46:32.211818+0000","last_became_active":"2024-09-10T07:46:31.242933+0000","last_became_peered":"2024-09-10T07:46:31.242933+0000","last_unstale":"2024-09-10T07:46:32.211818+0000","last_undegraded":"2024-09-10T07:46:32.211818+0000","last_fullsized":"2024-09-10T07:46:32.211818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:14.615659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212801+0000","last_change":"2024-09-10T07:46:30.224029+0000","last_active":"2024-09-10T07:46:32.212801+0000","last_peered":"2024-09-10T07:46:32.212801+0000","last_clean":"2024-09-10T07:46:32.212801+0000","last_became_active":"2024-09-10T07:46:30.223744+0000","last_became_peered":"2024-09-10T07:46:30.223744+0000","last_unstale":"2024-09-10T07:46:32.212801+0000","last_undegraded":"2024-09-10T07:46:32.212801+0000","last_fullsized":"2024-09-10T07:46:32.212801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:08.193479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959612+0000","last_change":"2024-09-10T07:46:31.225058+0000","last_active":"2024-09-10T07:46:32.959612+0000","last_peered":"2024-09-10T07:46:32.959612+0000","last_clean":"2024-09-10T07:46:32.959612+0000","last_became_active":"2024-09-10T07:46:31.224874+0000","last_became_peered":"2024-09-10T07:46:31.224874+0000","last_unstale":"2024-09-10T07:46:32.959612+0000","last_undegraded":"2024-09-10T07:46:32.959612+0000","last_fullsized":"2024-09-10T07:46:32.959612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244676+0000","last_change":"2024-09-10T07:46:30.228740+0000","last_active":"2024-09-10T07:46:32.244676+0000","last_peered":"2024-09-10T07:46:32.244676+0000","last_clean":"2024-09-10T07:46:32.244676+0000","last_became_active":"2024-09-10T07:46:30.228398+0000","last_became_peered":"2024-09-10T07:46:30.228398+0000","last_unstale":"2024-09-10T07:46:32.244676+0000","last_undegraded":"2024-09-10T07:46:32.244676+0000","last_fullsized":"2024-09-10T07:46:32.244676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732586+0000","last_change":"2024-09-10T07:46:31.232774+0000","last_active":"2024-09-10T07:46:34.732586+0000","last_peered":"2024-09-10T07:46:34.732586+0000","last_clean":"2024-09-10T07:46:34.732586+0000","last_became_active":"2024-09-10T07:46:31.232525+0000","last_became_peered":"2024-09-10T07:46:31.232525+0000","last_unstale":"2024-09-10T07:46:34.732586+0000","last_undegraded":"2024-09-10T07:46:34.732586+0000","last_fullsized":"2024-09-10T07:46:34.732586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213861+0000","last_change":"2024-09-10T07:46:30.219121+0000","last_active":"2024-09-10T07:46:32.213861+0000","last_peered":"2024-09-10T07:46:32.213861+0000","last_clean":"2024-09-10T07:46:32.213861+0000","last_became_active":"2024-09-10T07:46:30.218854+0000","last_became_peered":"2024-09-10T07:46:30.218854+0000","last_unstale":"2024-09-10T07:46:32.213861+0000","last_undegraded":"2024-09-10T07:46:32.213861+0000","last_fullsized":"2024-09-10T07:46:32.213861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.970336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244571+0000","last_change":"2024-09-10T07:46:31.242521+0000","last_active":"2024-09-10T07:46:32.244571+0000","last_peered":"2024-09-10T07:46:32.244571+0000","last_clean":"2024-09-10T07:46:32.244571+0000","last_became_active":"2024-09-10T07:46:31.242234+0000","last_became_peered":"2024-09-10T07:46:31.242234+0000","last_unstale":"2024-09-10T07:46:32.244571+0000","last_undegraded":"2024-09-10T07:46:32.244571+0000","last_fullsized":"2024-09-10T07:46:32.244571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:50.570448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470126+0000","last_change":"2024-09-10T07:46:30.215842+0000","last_active":"2024-09-10T07:46:34.470126+0000","last_peered":"2024-09-10T07:46:34.470126+0000","last_clean":"2024-09-10T07:46:34.470126+0000","last_became_active":"2024-09-10T07:46:30.213088+0000","last_became_peered":"2024-09-10T07:46:30.213088+0000","last_unstale":"2024-09-10T07:46:34.470126+0000","last_undegraded":"2024-09-10T07:46:34.470126+0000","last_fullsized":"2024-09-10T07:46:34.470126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244247+0000","last_change":"2024-09-10T07:46:31.233443+0000","last_active":"2024-09-10T07:46:32.244247+0000","last_peered":"2024-09-10T07:46:32.244247+0000","last_clean":"2024-09-10T07:46:32.244247+0000","last_became_active":"2024-09-10T07:46:31.232811+0000","last_became_peered":"2024-09-10T07:46:31.232811+0000","last_unstale":"2024-09-10T07:46:32.244247+0000","last_undegraded":"2024-09-10T07:46:32.244247+0000","last_fullsized":"2024-09-10T07:46:32.244247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.755283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214329+0000","last_change":"2024-09-10T07:46:30.231504+0000","last_active":"2024-09-10T07:46:32.214329+0000","last_peered":"2024-09-10T07:46:32.214329+0000","last_clean":"2024-09-10T07:46:32.214329+0000","last_became_active":"2024-09-10T07:46:30.231198+0000","last_became_peered":"2024-09-10T07:46:30.231198+0000","last_unstale":"2024-09-10T07:46:32.214329+0000","last_undegraded":"2024-09-10T07:46:32.214329+0000","last_fullsized":"2024-09-10T07:46:32.214329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:22.155456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960136+0000","last_change":"2024-09-10T07:46:31.227337+0000","last_active":"2024-09-10T07:46:32.960136+0000","last_peered":"2024-09-10T07:46:32.960136+0000","last_clean":"2024-09-10T07:46:32.960136+0000","last_became_active":"2024-09-10T07:46:31.227077+0000","last_became_peered":"2024-09-10T07:46:31.227077+0000","last_unstale":"2024-09-10T07:46:32.960136+0000","last_undegraded":"2024-09-10T07:46:32.960136+0000","last_fullsized":"2024-09-10T07:46:32.960136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.652763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.211923+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:46:32.211923+0000","last_peered":"2024-09-10T07:46:32.211923+0000","last_clean":"2024-09-10T07:46:32.211923+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:46:32.211923+0000","last_undegraded":"2024-09-10T07:46:32.211923+0000","last_fullsized":"2024-09-10T07:46:32.211923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959922+0000","last_change":"2024-09-10T07:46:30.227899+0000","last_active":"2024-09-10T07:46:32.959922+0000","last_peered":"2024-09-10T07:46:32.959922+0000","last_clean":"2024-09-10T07:46:32.959922+0000","last_became_active":"2024-09-10T07:46:30.224373+0000","last_became_peered":"2024-09-10T07:46:30.224373+0000","last_unstale":"2024-09-10T07:46:32.959922+0000","last_undegraded":"2024-09-10T07:46:32.959922+0000","last_fullsized":"2024-09-10T07:46:32.959922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:18.171026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213294+0000","last_change":"2024-09-10T07:46:31.224123+0000","last_active":"2024-09-10T07:46:32.213294+0000","last_peered":"2024-09-10T07:46:32.213294+0000","last_clean":"2024-09-10T07:46:32.213294+0000","last_became_active":"2024-09-10T07:46:31.223573+0000","last_became_peered":"2024-09-10T07:46:31.223573+0000","last_unstale":"2024-09-10T07:46:32.213294+0000","last_undegraded":"2024-09-10T07:46:32.213294+0000","last_fullsized":"2024-09-10T07:46:32.213294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:15.850634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959659+0000","last_change":"2024-09-10T07:46:30.232161+0000","last_active":"2024-09-10T07:46:32.959659+0000","last_peered":"2024-09-10T07:46:32.959659+0000","last_clean":"2024-09-10T07:46:32.959659+0000","last_became_active":"2024-09-10T07:46:30.231438+0000","last_became_peered":"2024-09-10T07:46:30.231438+0000","last_unstale":"2024-09-10T07:46:32.959659+0000","last_undegraded":"2024-09-10T07:46:32.959659+0000","last_fullsized":"2024-09-10T07:46:32.959659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:19.235522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214503+0000","last_change":"2024-09-10T07:46:31.233650+0000","last_active":"2024-09-10T07:46:32.214503+0000","last_peered":"2024-09-10T07:46:32.214503+0000","last_clean":"2024-09-10T07:46:32.214503+0000","last_became_active":"2024-09-10T07:46:31.232959+0000","last_became_peered":"2024-09-10T07:46:31.232959+0000","last_unstale":"2024-09-10T07:46:32.214503+0000","last_undegraded":"2024-09-10T07:46:32.214503+0000","last_fullsized":"2024-09-10T07:46:32.214503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:15.761789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487400+0000","last_change":"2024-09-10T07:46:30.220043+0000","last_active":"2024-09-10T07:46:32.487400+0000","last_peered":"2024-09-10T07:46:32.487400+0000","last_clean":"2024-09-10T07:46:32.487400+0000","last_became_active":"2024-09-10T07:46:30.219731+0000","last_became_peered":"2024-09-10T07:46:30.219731+0000","last_unstale":"2024-09-10T07:46:32.487400+0000","last_undegraded":"2024-09-10T07:46:32.487400+0000","last_fullsized":"2024-09-10T07:46:32.487400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470342+0000","last_change":"2024-09-10T07:46:31.244888+0000","last_active":"2024-09-10T07:46:34.470342+0000","last_peered":"2024-09-10T07:46:34.470342+0000","last_clean":"2024-09-10T07:46:34.470342+0000","last_became_active":"2024-09-10T07:46:31.244431+0000","last_became_peered":"2024-09-10T07:46:31.244431+0000","last_unstale":"2024-09-10T07:46:34.470342+0000","last_undegraded":"2024-09-10T07:46:34.470342+0000","last_fullsized":"2024-09-10T07:46:34.470342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.069707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960038+0000","last_change":"2024-09-10T07:46:30.229082+0000","last_active":"2024-09-10T07:46:32.960038+0000","last_peered":"2024-09-10T07:46:32.960038+0000","last_clean":"2024-09-10T07:46:32.960038+0000","last_became_active":"2024-09-10T07:46:30.223658+0000","last_became_peered":"2024-09-10T07:46:30.223658+0000","last_unstale":"2024-09-10T07:46:32.960038+0000","last_undegraded":"2024-09-10T07:46:32.960038+0000","last_fullsized":"2024-09-10T07:46:32.960038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:52.516192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243996+0000","last_change":"2024-09-10T07:46:31.239891+0000","last_active":"2024-09-10T07:46:32.243996+0000","last_peered":"2024-09-10T07:46:32.243996+0000","last_clean":"2024-09-10T07:46:32.243996+0000","last_became_active":"2024-09-10T07:46:31.237865+0000","last_became_peered":"2024-09-10T07:46:31.237865+0000","last_unstale":"2024-09-10T07:46:32.243996+0000","last_undegraded":"2024-09-10T07:46:32.243996+0000","last_fullsized":"2024-09-10T07:46:32.243996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470060+0000","last_change":"2024-09-10T07:46:30.218635+0000","last_active":"2024-09-10T07:46:34.470060+0000","last_peered":"2024-09-10T07:46:34.470060+0000","last_clean":"2024-09-10T07:46:34.470060+0000","last_became_active":"2024-09-10T07:46:30.217982+0000","last_became_peered":"2024-09-10T07:46:30.217982+0000","last_unstale":"2024-09-10T07:46:34.470060+0000","last_undegraded":"2024-09-10T07:46:34.470060+0000","last_fullsized":"2024-09-10T07:46:34.470060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:17.879546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.732286+0000","last_change":"2024-09-10T07:46:31.232401+0000","last_active":"2024-09-10T07:46:34.732286+0000","last_peered":"2024-09-10T07:46:34.732286+0000","last_clean":"2024-09-10T07:46:34.732286+0000","last_became_active":"2024-09-10T07:46:31.232132+0000","last_became_peered":"2024-09-10T07:46:31.232132+0000","last_unstale":"2024-09-10T07:46:34.732286+0000","last_undegraded":"2024-09-10T07:46:34.732286+0000","last_fullsized":"2024-09-10T07:46:34.732286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:40.645573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960240+0000","last_change":"2024-09-10T07:46:30.228512+0000","last_active":"2024-09-10T07:46:32.960240+0000","last_peered":"2024-09-10T07:46:32.960240+0000","last_clean":"2024-09-10T07:46:32.960240+0000","last_became_active":"2024-09-10T07:46:30.227194+0000","last_became_peered":"2024-09-10T07:46:30.227194+0000","last_unstale":"2024-09-10T07:46:32.960240+0000","last_undegraded":"2024-09-10T07:46:32.960240+0000","last_fullsized":"2024-09-10T07:46:32.960240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:14.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214164+0000","last_change":"2024-09-10T07:46:31.231122+0000","last_active":"2024-09-10T07:46:32.214164+0000","last_peered":"2024-09-10T07:46:32.214164+0000","last_clean":"2024-09-10T07:46:32.214164+0000","last_became_active":"2024-09-10T07:46:31.230087+0000","last_became_peered":"2024-09-10T07:46:31.230087+0000","last_unstale":"2024-09-10T07:46:32.214164+0000","last_undegraded":"2024-09-10T07:46:32.214164+0000","last_fullsized":"2024-09-10T07:46:32.214164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:42.598952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243215+0000","last_change":"2024-09-10T07:46:30.238150+0000","last_active":"2024-09-10T07:46:32.243215+0000","last_peered":"2024-09-10T07:46:32.243215+0000","last_clean":"2024-09-10T07:46:32.243215+0000","last_became_active":"2024-09-10T07:46:30.237794+0000","last_became_peered":"2024-09-10T07:46:30.237794+0000","last_unstale":"2024-09-10T07:46:32.243215+0000","last_undegraded":"2024-09-10T07:46:32.243215+0000","last_fullsized":"2024-09-10T07:46:32.243215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:04.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213812+0000","last_change":"2024-09-10T07:46:31.222408+0000","last_active":"2024-09-10T07:46:32.213812+0000","last_peered":"2024-09-10T07:46:32.213812+0000","last_clean":"2024-09-10T07:46:32.213812+0000","last_became_active":"2024-09-10T07:46:31.221928+0000","last_became_peered":"2024-09-10T07:46:31.221928+0000","last_unstale":"2024-09-10T07:46:32.213812+0000","last_undegraded":"2024-09-10T07:46:32.213812+0000","last_fullsized":"2024-09-10T07:46:32.213812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:20.108016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960429+0000","last_change":"2024-09-10T07:46:30.230121+0000","last_active":"2024-09-10T07:46:32.960429+0000","last_peered":"2024-09-10T07:46:32.960429+0000","last_clean":"2024-09-10T07:46:32.960429+0000","last_became_active":"2024-09-10T07:46:30.229832+0000","last_became_peered":"2024-09-10T07:46:30.229832+0000","last_unstale":"2024-09-10T07:46:32.960429+0000","last_undegraded":"2024-09-10T07:46:32.960429+0000","last_fullsized":"2024-09-10T07:46:32.960429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:41.596283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213786+0000","last_change":"2024-09-10T07:46:31.234758+0000","last_active":"2024-09-10T07:46:32.213786+0000","last_peered":"2024-09-10T07:46:32.213786+0000","last_clean":"2024-09-10T07:46:32.213786+0000","last_became_active":"2024-09-10T07:46:31.232761+0000","last_became_peered":"2024-09-10T07:46:31.232761+0000","last_unstale":"2024-09-10T07:46:32.213786+0000","last_undegraded":"2024-09-10T07:46:32.213786+0000","last_fullsized":"2024-09-10T07:46:32.213786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:57.605251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.243629+0000","last_change":"2024-09-10T07:46:30.236809+0000","last_active":"2024-09-10T07:46:32.243629+0000","last_peered":"2024-09-10T07:46:32.243629+0000","last_clean":"2024-09-10T07:46:32.243629+0000","last_became_active":"2024-09-10T07:46:30.236229+0000","last_became_peered":"2024-09-10T07:46:30.236229+0000","last_unstale":"2024-09-10T07:46:32.243629+0000","last_undegraded":"2024-09-10T07:46:32.243629+0000","last_fullsized":"2024-09-10T07:46:32.243629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959717+0000","last_change":"2024-09-10T07:46:31.218320+0000","last_active":"2024-09-10T07:46:32.959717+0000","last_peered":"2024-09-10T07:46:32.959717+0000","last_clean":"2024-09-10T07:46:32.959717+0000","last_became_active":"2024-09-10T07:46:31.216720+0000","last_became_peered":"2024-09-10T07:46:31.216720+0000","last_unstale":"2024-09-10T07:46:32.959717+0000","last_undegraded":"2024-09-10T07:46:32.959717+0000","last_fullsized":"2024-09-10T07:46:32.959717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:03.411205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213047+0000","last_change":"2024-09-10T07:46:30.238259+0000","last_active":"2024-09-10T07:46:32.213047+0000","last_peered":"2024-09-10T07:46:32.213047+0000","last_clean":"2024-09-10T07:46:32.213047+0000","last_became_active":"2024-09-10T07:46:30.237917+0000","last_became_peered":"2024-09-10T07:46:30.237917+0000","last_unstale":"2024-09-10T07:46:32.213047+0000","last_undegraded":"2024-09-10T07:46:32.213047+0000","last_fullsized":"2024-09-10T07:46:32.213047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.353507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212764+0000","last_change":"2024-09-10T07:46:31.234756+0000","last_active":"2024-09-10T07:46:32.212764+0000","last_peered":"2024-09-10T07:46:32.212764+0000","last_clean":"2024-09-10T07:46:32.212764+0000","last_became_active":"2024-09-10T07:46:31.232465+0000","last_became_peered":"2024-09-10T07:46:31.232465+0000","last_unstale":"2024-09-10T07:46:32.212764+0000","last_undegraded":"2024-09-10T07:46:32.212764+0000","last_fullsized":"2024-09-10T07:46:32.212764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:18.368983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.213096+0000","last_change":"2024-09-10T07:46:30.222413+0000","last_active":"2024-09-10T07:46:32.213096+0000","last_peered":"2024-09-10T07:46:32.213096+0000","last_clean":"2024-09-10T07:46:32.213096+0000","last_became_active":"2024-09-10T07:46:30.222152+0000","last_became_peered":"2024-09-10T07:46:30.222152+0000","last_unstale":"2024-09-10T07:46:32.213096+0000","last_undegraded":"2024-09-10T07:46:32.213096+0000","last_fullsized":"2024-09-10T07:46:32.213096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:02.690002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.487538+0000","last_change":"2024-09-10T07:46:31.227688+0000","last_active":"2024-09-10T07:46:32.487538+0000","last_peered":"2024-09-10T07:46:32.487538+0000","last_clean":"2024-09-10T07:46:32.487538+0000","last_became_active":"2024-09-10T07:46:31.227344+0000","last_became_peered":"2024-09-10T07:46:31.227344+0000","last_unstale":"2024-09-10T07:46:32.487538+0000","last_undegraded":"2024-09-10T07:46:32.487538+0000","last_fullsized":"2024-09-10T07:46:32.487538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959762+0000","last_change":"2024-09-10T07:46:30.231440+0000","last_active":"2024-09-10T07:46:32.959762+0000","last_peered":"2024-09-10T07:46:32.959762+0000","last_clean":"2024-09-10T07:46:32.959762+0000","last_became_active":"2024-09-10T07:46:30.229074+0000","last_became_peered":"2024-09-10T07:46:30.229074+0000","last_unstale":"2024-09-10T07:46:32.959762+0000","last_undegraded":"2024-09-10T07:46:32.959762+0000","last_fullsized":"2024-09-10T07:46:32.959762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:39.642905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.959914+0000","last_change":"2024-09-10T07:46:31.236813+0000","last_active":"2024-09-10T07:46:32.959914+0000","last_peered":"2024-09-10T07:46:32.959914+0000","last_clean":"2024-09-10T07:46:32.959914+0000","last_became_active":"2024-09-10T07:46:31.236555+0000","last_became_peered":"2024-09-10T07:46:31.236555+0000","last_unstale":"2024-09-10T07:46:32.959914+0000","last_undegraded":"2024-09-10T07:46:32.959914+0000","last_fullsized":"2024-09-10T07:46:32.959914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:27.770235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.960085+0000","last_change":"2024-09-10T07:46:31.230041+0000","last_active":"2024-09-10T07:46:32.960085+0000","last_peered":"2024-09-10T07:46:32.960085+0000","last_clean":"2024-09-10T07:46:32.960085+0000","last_became_active":"2024-09-10T07:46:31.229798+0000","last_became_peered":"2024-09-10T07:46:31.229798+0000","last_unstale":"2024-09-10T07:46:32.960085+0000","last_undegraded":"2024-09-10T07:46:32.960085+0000","last_fullsized":"2024-09-10T07:46:32.960085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:59.748478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244314+0000","last_change":"2024-09-10T07:46:30.230328+0000","last_active":"2024-09-10T07:46:32.244314+0000","last_peered":"2024-09-10T07:46:32.244314+0000","last_clean":"2024-09-10T07:46:32.244314+0000","last_became_active":"2024-09-10T07:46:30.229982+0000","last_became_peered":"2024-09-10T07:46:30.229982+0000","last_unstale":"2024-09-10T07:46:32.244314+0000","last_undegraded":"2024-09-10T07:46:32.244314+0000","last_fullsized":"2024-09-10T07:46:32.244314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:13.612990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.212216+0000","last_change":"2024-09-10T07:46:31.233808+0000","last_active":"2024-09-10T07:46:32.212216+0000","last_peered":"2024-09-10T07:46:32.212216+0000","last_clean":"2024-09-10T07:46:32.212216+0000","last_became_active":"2024-09-10T07:46:31.233478+0000","last_became_peered":"2024-09-10T07:46:31.233478+0000","last_unstale":"2024-09-10T07:46:32.212216+0000","last_undegraded":"2024-09-10T07:46:32.212216+0000","last_fullsized":"2024-09-10T07:46:32.212216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:20.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.244379+0000","last_change":"2024-09-10T07:46:30.228945+0000","last_active":"2024-09-10T07:46:32.244379+0000","last_peered":"2024-09-10T07:46:32.244379+0000","last_clean":"2024-09-10T07:46:32.244379+0000","last_became_active":"2024-09-10T07:46:30.225738+0000","last_became_peered":"2024-09-10T07:46:30.225738+0000","last_unstale":"2024-09-10T07:46:32.244379+0000","last_undegraded":"2024-09-10T07:46:32.244379+0000","last_fullsized":"2024-09-10T07:46:32.244379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470327+0000","last_change":"2024-09-10T07:46:31.238515+0000","last_active":"2024-09-10T07:46:34.470327+0000","last_peered":"2024-09-10T07:46:34.470327+0000","last_clean":"2024-09-10T07:46:34.470327+0000","last_became_active":"2024-09-10T07:46:31.237949+0000","last_became_peered":"2024-09-10T07:46:31.237949+0000","last_unstale":"2024-09-10T07:46:34.470327+0000","last_undegraded":"2024-09-10T07:46:34.470327+0000","last_fullsized":"2024-09-10T07:46:34.470327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:46.646703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470350+0000","last_change":"2024-09-10T07:46:30.214632+0000","last_active":"2024-09-10T07:46:34.470350+0000","last_peered":"2024-09-10T07:46:34.470350+0000","last_clean":"2024-09-10T07:46:34.470350+0000","last_became_active":"2024-09-10T07:46:30.213972+0000","last_became_peered":"2024-09-10T07:46:30.213972+0000","last_unstale":"2024-09-10T07:46:34.470350+0000","last_undegraded":"2024-09-10T07:46:34.470350+0000","last_fullsized":"2024-09-10T07:46:34.470350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:53.943621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:34.470203+0000","last_change":"2024-09-10T07:46:31.238308+0000","last_active":"2024-09-10T07:46:34.470203+0000","last_peered":"2024-09-10T07:46:34.470203+0000","last_clean":"2024-09-10T07:46:34.470203+0000","last_became_active":"2024-09-10T07:46:31.238032+0000","last_became_peered":"2024-09-10T07:46:31.238032+0000","last_unstale":"2024-09-10T07:46:34.470203+0000","last_undegraded":"2024-09-10T07:46:34.470203+0000","last_fullsized":"2024-09-10T07:46:34.470203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:30.191950+0000","last_clean_scrub_stamp":"2024-09-10T07:46:30.191950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:17.093470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T07:46:32.214000+0000","last_change":"2024-09-10T07:46:30.225263+0000","last_active":"2024-09-10T07:46:32.214000+0000","last_peered":"2024-09-10T07:46:32.214000+0000","last_clean":"2024-09-10T07:46:32.214000+0000","last_became_active":"2024-09-10T07:46:30.224934+0000","last_became_peered":"2024-09-10T07:46:30.224934+0000","last_unstale":"2024-09-10T07:46:32.214000+0000","last_undegraded":"2024-09-10T07:46:32.214000+0000","last_fullsized":"2024-09-10T07:46:32.214000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:29.189281+0000","last_clean_scrub_stamp":"2024-09-10T07:46:29.189281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:52.146362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":81727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":670553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":80171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":80239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":80115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":670575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":249856,"data_stored":82399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:46:45.811 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:46:45.811 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T07:46:45.811 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T07:46:45.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T07:46:46.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.002+0000 7f256fc00640 1 Processor -- start 2024-09-10T07:46:46.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.002+0000 7f256fc00640 1 -- start start 2024-09-10T07:46:46.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.003+0000 7f256fc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f2568071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.003+0000 7f256fc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 0x7f256815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.003+0000 7f256fc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25680cd7f0 con 0x7f256815c720 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.003+0000 7f256fc00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2568072210 con 0x7f25680cfdc0 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.003+0000 7f256fc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25680cf880 con 0x7f2568070f30 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f2568071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f256dc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 0x7f256815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f256d200640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f256815c720 0x7f2568073610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50116/0 (socket says 172.21.15.3:50116) 2024-09-10T07:46:46.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f256d200640 1 -- 172.21.15.3:0/2419345878 learned_addr learned my addr 172.21.15.3:0/2419345878 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:46.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567e00640 1 -- 172.21.15.3:0/2419345878 >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 msgr2=0x7f256815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:46.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567e00640 1 --2- 172.21.15.3:0/2419345878 >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 0x7f256815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567e00640 1 -- 172.21.15.3:0/2419345878 >> v1:172.21.15.70:6789/0 conn(0x7f256815c720 legacy=0x7f2568073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567400640 1 -- 172.21.15.3:0/2419345878 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123356968 0 0) 0x7f25680cd7f0 con 0x7f256815c720 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f2567e00640 1 -- 172.21.15.3:0/2419345878 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25680cda90 con 0x7f2568070f30 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.004+0000 7f256dc00640 1 --2- 172.21.15.3:0/2419345878 >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 0x7f256815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.005+0000 7f2567e00640 1 --2- 172.21.15.3:0/2419345878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f2568071cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25540097a0 tx=0x7f255402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a75b69942a1afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f2567400640 1 -- 172.21.15.3:0/2419345878 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554037070 con 0x7f2568070f30 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f2567400640 1 -- 172.21.15.3:0/2419345878 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2554032440 con 0x7f2568070f30 2024-09-10T07:46:46.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f2567400640 1 -- 172.21.15.3:0/2419345878 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554032810 con 0x7f2568070f30 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 -- 172.21.15.3:0/2419345878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 msgr2=0x7f2568071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 --2- 172.21.15.3:0/2419345878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f2568071cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25540097a0 tx=0x7f255402a180 comp rx=0 tx=0).stop 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 -- 172.21.15.3:0/2419345878 shutdown_connections 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 --2- 172.21.15.3:0/2419345878 >> v2:172.21.15.70:3300/0 conn(0x7f25680cfdc0 0x7f256815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 --2- 172.21.15.3:0/2419345878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f2568071cd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 -- 172.21.15.3:0/2419345878 >> 172.21.15.3:0/2419345878 conn(0x7f256806cfc0 msgr2=0x7f256806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:46.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 -- 172.21.15.3:0/2419345878 shutdown_connections 2024-09-10T07:46:46.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.006+0000 7f256fc00640 1 -- 172.21.15.3:0/2419345878 wait complete. 2024-09-10T07:46:46.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.008+0000 7f256fc00640 1 Processor -- start 2024-09-10T07:46:46.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.008+0000 7f256fc00640 1 -- start start 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f25680b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f256815c720 0x7f256814eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2568073d20 con 0x7f25680cfdc0 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f256806ed10 con 0x7f256815c720 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256fc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2568073ea0 con 0x7f2568070f30 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f2567e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f2567e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45598/0 (socket says 172.21.15.3:45598) 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f2567e00640 1 -- 172.21.15.3:0/2820366980 learned_addr learned my addr 172.21.15.3:0/2820366980 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256d200640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f25680b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.009+0000 7f256dc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f256815c720 0x7f256814eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 -- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 msgr2=0x7f25680b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f25680b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 -- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f256815c720 msgr2=0x7f256814eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f256815c720 0x7f256814eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 -- 172.21.15.3:0/2820366980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256814f0d0 con 0x7f25680cfdc0 2024-09-10T07:46:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f256d200640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f25680b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:46.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2567e00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2554035040 tx=0x7f25540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554037040 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2554030440 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2568158f20 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554030850 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.010+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681591c0 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.012+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25680cf2b0 con 0x7f25680cfdc0 2024-09-10T07:46:46.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.012+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2554044020 con 0x7f25680cfdc0 2024-09-10T07:46:46.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.013+0000 7f2564c00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 0x7f254806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.013+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f25540aaff0 con 0x7f25680cfdc0 2024-09-10T07:46:46.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.013+0000 7f256d200640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 0x7f254806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.014+0000 7f256d200640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 0x7f254806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.017+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25540799a0 con 0x7f25680cfdc0 2024-09-10T07:46:46.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.240+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2568071cb0 con 0x7f25680cfdc0 2024-09-10T07:46:46.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.240+0000 7f2564c00640 1 -- 172.21.15.3:0/2820366980 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f25540792a0 con 0x7f25680cfdc0 2024-09-10T07:46:46.240 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:46.240 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T07:46:46.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.242+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 msgr2=0x7f254806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.242+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 0x7f254806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c007d50 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.242+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 msgr2=0x7f25680b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.242+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2554035040 tx=0x7f25540046c0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 shutdown_connections 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f254806be30 0x7f254806e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f256815c720 0x7f256814eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25680cfdc0 0x7f25680b50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 --2- 172.21.15.3:0/2820366980 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2568070f30 0x7f25680b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 >> 172.21.15.3:0/2820366980 conn(0x7f256806cfc0 msgr2=0x7f256815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 shutdown_connections 2024-09-10T07:46:46.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.243+0000 7f256fc00640 1 -- 172.21.15.3:0/2820366980 wait complete. 2024-09-10T07:46:46.257 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T07:46:46.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.414+0000 7ff471400640 1 Processor -- start 2024-09-10T07:46:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.414+0000 7ff471400640 1 -- start start 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff46c161dd0 0x7ff46c06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c0b5ea0 0x7ff46c0b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff46c062630 0x7ff46c062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff46c06fe10 con 0x7ff46c161dd0 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff46c0b47d0 con 0x7ff46c062630 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.415+0000 7ff471400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c06c870 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff46c161dd0 0x7ff46c06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff46c161dd0 0x7ff46c06f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45602/0 (socket says 172.21.15.3:45602) 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 -- 172.21.15.3:0/3784171833 learned_addr learned my addr 172.21.15.3:0/3784171833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff469e00640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c0b5ea0 0x7ff46c0b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46b200640 1 --2- 172.21.15.3:0/3784171833 >> v2:172.21.15.70:3300/0 conn(0x7ff46c062630 0x7ff46c062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 -- 172.21.15.3:0/3784171833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c0b5ea0 msgr2=0x7ff46c0b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c0b5ea0 0x7ff46c0b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 -- 172.21.15.3:0/3784171833 >> v2:172.21.15.70:3300/0 conn(0x7ff46c062630 msgr2=0x7ff46c062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 --2- 172.21.15.3:0/3784171833 >> v2:172.21.15.70:3300/0 conn(0x7ff46c062630 0x7ff46c062aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.416+0000 7ff46a800640 1 -- 172.21.15.3:0/3784171833 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c06cb50 con 0x7ff46c161dd0 2024-09-10T07:46:46.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.417+0000 7ff46a800640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c161dd0 0x7ff46c06f8d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff46c071de0 tx=0x7ff45c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fd415b4baa1ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.417+0000 7ff469400640 1 -- 172.21.15.3:0/3784171833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c037070 con 0x7ff46c161dd0 2024-09-10T07:46:46.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.417+0000 7ff469400640 1 -- 172.21.15.3:0/3784171833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45c032440 con 0x7ff46c161dd0 2024-09-10T07:46:46.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.417+0000 7ff469400640 1 -- 172.21.15.3:0/3784171833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c032810 con 0x7ff46c161dd0 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 -- 172.21.15.3:0/3784171833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c161dd0 msgr2=0x7ff46c06f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c161dd0 0x7ff46c06f8d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff46c071de0 tx=0x7ff45c02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 -- 172.21.15.3:0/3784171833 shutdown_connections 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 --2- 172.21.15.3:0/3784171833 >> v2:172.21.15.70:3300/0 conn(0x7ff46c062630 0x7ff46c062aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c0b5ea0 0x7ff46c0b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 --2- 172.21.15.3:0/3784171833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c161dd0 0x7ff46c06f8d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 -- 172.21.15.3:0/3784171833 >> 172.21.15.3:0/3784171833 conn(0x7ff46c0b3540 msgr2=0x7ff46c0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 -- 172.21.15.3:0/3784171833 shutdown_connections 2024-09-10T07:46:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.418+0000 7ff471400640 1 -- 172.21.15.3:0/3784171833 wait complete. 2024-09-10T07:46:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.420+0000 7ff471400640 1 Processor -- start 2024-09-10T07:46:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.420+0000 7ff471400640 1 -- start start 2024-09-10T07:46:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff46c062630 0x7ff46c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c161dd0 0x7ff46c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0b6820 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0b06c0 con 0x7ff46c062630 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff471400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c0b43f0 con 0x7ff46c161dd0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45604/0 (socket says 172.21.15.3:45604) 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 -- 172.21.15.3:0/1677731573 learned_addr learned my addr 172.21.15.3:0/1677731573 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff46a800640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff46c062630 0x7ff46c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff46b200640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c161dd0 0x7ff46c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 -- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c161dd0 msgr2=0x7ff46c152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c161dd0 0x7ff46c152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 -- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff46c062630 msgr2=0x7ff46c155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff46c062630 0x7ff46c155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.421+0000 7ff469e00640 1 -- 172.21.15.3:0/1677731573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c15add0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff469e00640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9e0 tx=0x7ff45800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458017070 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45800c9d0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c15b0b0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45800cdc0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.422+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c15b670 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.423+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c0b62e0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.424+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff45801a070 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.425+0000 7ff462a00640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 0x7ff44c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.425+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff45808b190 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.425+0000 7ff46a800640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 0x7ff44c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.426+0000 7ff46a800640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 0x7ff44c06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff45c034000 tx=0x7ff45c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.430+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4580594e0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.650+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff46c15bd30 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.651+0000 7ff462a00640 1 -- 172.21.15.3:0/1677731573 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7ff45805d7e0 con 0x7ff46c0b5ea0 2024-09-10T07:46:46.651 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:46.651 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:46:46.651 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:46:46.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 msgr2=0x7ff44c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 0x7ff44c06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff45c034000 tx=0x7ff45c02e000 comp rx=0 tx=0).stop 2024-09-10T07:46:46.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 msgr2=0x7ff46c1e8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9e0 tx=0x7ff45800eea0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 shutdown_connections 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff44c06be30 0x7ff44c06e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff46c161dd0 0x7ff46c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff46c0b5ea0 0x7ff46c1e8fa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 --2- 172.21.15.3:0/1677731573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff46c062630 0x7ff46c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 >> 172.21.15.3:0/1677731573 conn(0x7ff46c0b3540 msgr2=0x7ff46c0b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.653+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 shutdown_connections 2024-09-10T07:46:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.654+0000 7ff471400640 1 -- 172.21.15.3:0/1677731573 wait complete. 2024-09-10T07:46:46.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.825+0000 7ffb08200640 1 Processor -- start 2024-09-10T07:46:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.825+0000 7ffb08200640 1 -- start start 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.826+0000 7ffb08200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb00160710 0x7ffb000cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb08200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 0x7ffb000d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb08200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb000cf800 0x7ffb000d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb08200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffb000cfdd0 con 0x7ffb00160710 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb08200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ffb0015cc90 con 0x7ffb000cf800 2024-09-10T07:46:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb08200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0015cf70 con 0x7ffb00160af0 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb00160710 0x7ffb000cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb00160710 0x7ffb000cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45620/0 (socket says 172.21.15.3:45620) 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 -- 172.21.15.3:0/2557994781 learned_addr learned my addr 172.21.15.3:0/2557994781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb04e00640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 0x7ffb000d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb06200640 1 --2- 172.21.15.3:0/2557994781 >> v2:172.21.15.70:3300/0 conn(0x7ffb000cf800 0x7ffb000d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 -- 172.21.15.3:0/2557994781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 msgr2=0x7ffb000d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 0x7ffb000d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 -- 172.21.15.3:0/2557994781 >> v2:172.21.15.70:3300/0 conn(0x7ffb000cf800 msgr2=0x7ffb000d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 --2- 172.21.15.3:0/2557994781 >> v2:172.21.15.70:3300/0 conn(0x7ffb000cf800 0x7ffb000d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb05800640 1 -- 172.21.15.3:0/2557994781 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0015d250 con 0x7ffb00160710 2024-09-10T07:46:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.827+0000 7ffb04e00640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 0x7ffb000d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.828+0000 7ffb05800640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb000cf2c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb0006df40 tx=0x7ffaf402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3fe6e5fda1212c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.828+0000 7ffaffe00640 1 -- 172.21.15.3:0/2557994781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4037070 con 0x7ffb00160710 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffaffe00640 1 -- 172.21.15.3:0/2557994781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf4032440 con 0x7ffb00160710 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffaffe00640 1 -- 172.21.15.3:0/2557994781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4032810 con 0x7ffb00160710 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 -- 172.21.15.3:0/2557994781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 msgr2=0x7ffb000cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb000cf2c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb0006df40 tx=0x7ffaf402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 -- 172.21.15.3:0/2557994781 shutdown_connections 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 --2- 172.21.15.3:0/2557994781 >> v2:172.21.15.70:3300/0 conn(0x7ffb000cf800 0x7ffb000d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb00160af0 0x7ffb000d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 --2- 172.21.15.3:0/2557994781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb000cf2c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 -- 172.21.15.3:0/2557994781 >> 172.21.15.3:0/2557994781 conn(0x7ffb0006d000 msgr2=0x7ffb0006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 -- 172.21.15.3:0/2557994781 shutdown_connections 2024-09-10T07:46:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.829+0000 7ffb08200640 1 -- 172.21.15.3:0/2557994781 wait complete. 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.831+0000 7ffb08200640 1 Processor -- start 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.831+0000 7ffb08200640 1 -- start start 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb000cf800 0x7ffb000af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb00160af0 0x7ffb0014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0015d770 con 0x7ffb00160710 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0006bb20 con 0x7ffb00160af0 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb08200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0014e9f0 con 0x7ffb000cf800 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45622/0 (socket says 172.21.15.3:45622) 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 -- 172.21.15.3:0/867374012 learned_addr learned my addr 172.21.15.3:0/867374012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb05800640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb000cf800 0x7ffb000af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb06200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb00160af0 0x7ffb0014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 -- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb000cf800 msgr2=0x7ffb000af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb000cf800 0x7ffb000af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 -- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb00160af0 msgr2=0x7ffb0014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb00160af0 0x7ffb0014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.832+0000 7ffb04e00640 1 -- 172.21.15.3:0/867374012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb000cbd20 con 0x7ffb00160710 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffb04e00640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d7d0 tx=0x7ffaec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec004030 con 0x7ffb00160710 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaec0041d0 con 0x7ffb00160710 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00b6c0 con 0x7ffb00160710 2024-09-10T07:46:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb000cc000 con 0x7ffb00160710 2024-09-10T07:46:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.833+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0014ef70 con 0x7ffb00160710 2024-09-10T07:46:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.834+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb000cc1d0 con 0x7ffb00160710 2024-09-10T07:46:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.835+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaec00b860 con 0x7ffb00160710 2024-09-10T07:46:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.836+0000 7ffafd600640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 0x7ffae006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.836+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffaec08aff0 con 0x7ffb00160710 2024-09-10T07:46:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.836+0000 7ffb05800640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 0x7ffae006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.837+0000 7ffb05800640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 0x7ffae006e320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4034000 tx=0x7ffaf402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:46.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:46.840+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffaec0592f0 con 0x7ffb00160710 2024-09-10T07:46:47.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.063+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb000cc8b0 con 0x7ffb00160710 2024-09-10T07:46:47.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.064+0000 7ffafd600640 1 -- 172.21.15.3:0/867374012 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7ffaec05d5f0 con 0x7ffb00160710 2024-09-10T07:46:47.063 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:47.063 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:46:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 msgr2=0x7ffae006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 0x7ffae006e320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4034000 tx=0x7ffaf402e000 comp rx=0 tx=0).stop 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 msgr2=0x7ffb0014bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d7d0 tx=0x7ffaec00dc90 comp rx=0 tx=0).stop 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 shutdown_connections 2024-09-10T07:46:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffae006be60 0x7ffae006e320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb00160af0 0x7ffb0014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb00160710 0x7ffb0014bb10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 --2- 172.21.15.3:0/867374012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb000cf800 0x7ffb000af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 >> 172.21.15.3:0/867374012 conn(0x7ffb0006d000 msgr2=0x7ffb00072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 shutdown_connections 2024-09-10T07:46:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.066+0000 7ffb08200640 1 -- 172.21.15.3:0/867374012 wait complete. 2024-09-10T07:46:47.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:46:47.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.245+0000 7f538ea00640 1 Processor -- start 2024-09-10T07:46:47.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.245+0000 7f538ea00640 1 -- start start 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 0x7f53880b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538ea00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53880cfdc0 con 0x7f53880d3c10 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538ea00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53880cee40 con 0x7f538815e860 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538ea00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f53880d3780 con 0x7f5388073860 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538cc00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f53880d3c10 0x7f5388073150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60262/0 (socket says 172.21.15.3:60262) 2024-09-10T07:46:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538cc00640 1 -- 172.21.15.3:0/1081303682 learned_addr learned my addr 172.21.15.3:0/1081303682 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:47.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538d600640 1 --1- 172.21.15.3:0/1081303682 >> v1:172.21.15.70:6789/0 conn(0x7f538815e860 0x7f53880cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:50122/0 (socket says 172.21.15.3:50122) 2024-09-10T07:46:47.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.246+0000 7f538e000640 1 --2- 172.21.15.3:0/1081303682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 0x7f53880b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562397171 0 0) 0x7f53880cee40 con 0x7f538815e860 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5370003520 con 0x7f538815e860 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099391389 0 0) 0x7f53880cfdc0 con 0x7f53880d3c10 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53880cee40 con 0x7f53880d3c10 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f538e000640 1 -- 172.21.15.3:0/1081303682 >> v1:172.21.15.70:6789/0 conn(0x7f538815e860 legacy=0x7f53880cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f538e000640 1 -- 172.21.15.3:0/1081303682 >> v1:172.21.15.3:6789/0 conn(0x7f53880d3c10 legacy=0x7f5388073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.247+0000 7f538e000640 1 -- 172.21.15.3:0/1081303682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53880cf120 con 0x7f5388073860 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.248+0000 7f538e000640 1 --2- 172.21.15.3:0/1081303682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 0x7f53880b7880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5380009dc0 tx=0x7f538002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ccc325ff1099dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.248+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380007740 con 0x7f5388073860 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.248+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53800078e0 con 0x7f5388073860 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.248+0000 7f5387e00640 1 -- 172.21.15.3:0/1081303682 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380007c90 con 0x7f5388073860 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 -- 172.21.15.3:0/1081303682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 msgr2=0x7f53880b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 --2- 172.21.15.3:0/1081303682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 0x7f53880b7880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5380009dc0 tx=0x7f538002a6d0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 -- 172.21.15.3:0/1081303682 shutdown_connections 2024-09-10T07:46:47.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 --2- 172.21.15.3:0/1081303682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5388073860 0x7f53880b7880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 -- 172.21.15.3:0/1081303682 >> 172.21.15.3:0/1081303682 conn(0x7f53880d4650 msgr2=0x7f53880d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:47.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 -- 172.21.15.3:0/1081303682 shutdown_connections 2024-09-10T07:46:47.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.249+0000 7f538ea00640 1 -- 172.21.15.3:0/1081303682 wait complete. 2024-09-10T07:46:47.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.251+0000 7f538ea00640 1 Processor -- start 2024-09-10T07:46:47.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.251+0000 7f538ea00640 1 -- start start 2024-09-10T07:46:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5388073860 0x7f538815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f538815e860 0x7f53881f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f53880d2460 con 0x7f53880d3c10 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f53880cf400 con 0x7f5388073860 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538ea00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f53880d4050 con 0x7f538815e860 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45628/0 (socket says 172.21.15.3:45628) 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538cc00640 1 -- 172.21.15.3:0/1110410350 learned_addr learned my addr 172.21.15.3:0/1110410350 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538d600640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5388073860 0x7f538815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538e000640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f538815e860 0x7f53881f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538cc00640 1 -- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f538815e860 msgr2=0x7f53881f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.252+0000 7f538cc00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f538815e860 0x7f53881f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538cc00640 1 -- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5388073860 msgr2=0x7f538815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538cc00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5388073860 0x7f538815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538cc00640 1 -- 172.21.15.3:0/1110410350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53880abd90 con 0x7f53880d3c10 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538cc00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f537400c430 tx=0x7f537400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374017070 con 0x7f53880d3c10 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5374005ce0 con 0x7f53880d3c10 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538815f970 con 0x7f53880d3c10 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740072f0 con 0x7f53880d3c10 2024-09-10T07:46:47.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.253+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538815ff30 con 0x7f53880d3c10 2024-09-10T07:46:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.254+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53880cf6b0 con 0x7f53880d3c10 2024-09-10T07:46:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.255+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f537401a070 con 0x7f53880d3c10 2024-09-10T07:46:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.257+0000 7f5385600640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 0x7f536806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.257+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f537408a310 con 0x7f53880d3c10 2024-09-10T07:46:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.257+0000 7f538d600640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 0x7f536806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.257+0000 7f538d600640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 0x7f536806e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f537c0011e0 tx=0x7f537c007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.261+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53880abd90 con 0x7f53880d3c10 2024-09-10T07:46:47.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.451+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53880cf960 con 0x7f53880d3c10 2024-09-10T07:46:47.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.453+0000 7f5385600640 1 -- 172.21.15.3:0/1110410350 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f53740585e0 con 0x7f53880d3c10 2024-09-10T07:46:47.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:47.453 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 msgr2=0x7f536806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 0x7f536806e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f537c0011e0 tx=0x7f537c007d30 comp rx=0 tx=0).stop 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 msgr2=0x7f5388160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f537400c430 tx=0x7f537400c8f0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 shutdown_connections 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f536806be30 0x7f536806e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f538815e860 0x7f53881f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53880d3c10 0x7f5388160a10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 --2- 172.21.15.3:0/1110410350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5388073860 0x7f538815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 >> 172.21.15.3:0/1110410350 conn(0x7f53880d4650 msgr2=0x7f538815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 shutdown_connections 2024-09-10T07:46:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.455+0000 7f538ea00640 1 -- 172.21.15.3:0/1110410350 wait complete. 2024-09-10T07:46:47.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:46:47.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.627+0000 7f56f5c00640 1 Processor -- start 2024-09-10T07:46:47.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.627+0000 7f56f5c00640 1 -- start start 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.628+0000 7f56f5c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56f00d08f0 0x7f56f00ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.628+0000 7f56f5c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f00ccd80 con 0x7f56f015b4f0 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.628+0000 7f56f5c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f00cd060 con 0x7f56f015b110 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.628+0000 7f56f5c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f56f00b44f0 con 0x7f56f00d08f0 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.628+0000 7f56ee000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56f015b4f0 0x7f56f00d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60274/0 (socket says 172.21.15.3:60274) 2024-09-10T07:46:47.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ee000640 1 -- 172.21.15.3:0/3136497379 learned_addr learned my addr 172.21.15.3:0/3136497379 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ef400640 1 --2- 172.21.15.3:0/3136497379 >> v2:172.21.15.70:3300/0 conn(0x7f56f00d08f0 0x7f56f00ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847025214 0 0) 0x7f56f00ccd80 con 0x7f56f015b4f0 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56d4003520 con 0x7f56f015b4f0 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 572110745 0 0) 0x7f56d4003520 con 0x7f56f015b4f0 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 >> v2:172.21.15.70:3300/0 conn(0x7f56f00d08f0 msgr2=0x7f56f00ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 --2- 172.21.15.3:0/3136497379 >> v2:172.21.15.70:3300/0 conn(0x7f56f00d08f0 0x7f56f00ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 >> v1:172.21.15.70:6789/0 conn(0x7f56f015b110 legacy=0x7f56f00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f00cd340 con 0x7f56f015b4f0 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f56dc001670 con 0x7f56f015b4f0 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 >> v1:172.21.15.3:6789/0 conn(0x7f56f015b4f0 legacy=0x7f56f00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 0x7f56d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d4004800 0x7f56d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56d40051c0 0x7f56d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56f00b44f0 con 0x7f56d40039b0 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400c010 con 0x7f56d40051c0 2024-09-10T07:46:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.629+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f56d400c210 con 0x7f56d4004800 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 0x7f56d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56ee000640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d4004800 0x7f56d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56ef400640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56d40051c0 0x7f56d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 -- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d4004800 msgr2=0x7f56d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d4004800 0x7f56d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 -- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56d40051c0 msgr2=0x7f56d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56d40051c0 0x7f56d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56eea00640 1 -- 172.21.15.3:0/3136497379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d4003520 con 0x7f56d40039b0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.630+0000 7f56ef400640 1 -- 172.21.15.3:0/3136497379 reap_dead start 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.631+0000 7f56eea00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 0x7f56d4003d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001400 tx=0x7f56d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426230256f0bc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.631+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0001440 con 0x7f56d40039b0 2024-09-10T07:46:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.631+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e00050d0 con 0x7f56d40039b0 2024-09-10T07:46:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.631+0000 7f56ed600640 1 -- 172.21.15.3:0/3136497379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e00035f0 con 0x7f56d40039b0 2024-09-10T07:46:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 -- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 msgr2=0x7f56d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 0x7f56d4003d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001400 tx=0x7f56d4005830 comp rx=0 tx=0).stop 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 -- 172.21.15.3:0/3136497379 shutdown_connections 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 --2- 172.21.15.3:0/3136497379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40039b0 0x7f56d4003d90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 -- 172.21.15.3:0/3136497379 >> 172.21.15.3:0/3136497379 conn(0x7f56f006d000 msgr2=0x7f56f006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 -- 172.21.15.3:0/3136497379 shutdown_connections 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.632+0000 7f56f5c00640 1 -- 172.21.15.3:0/3136497379 wait complete. 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.634+0000 7f56f5c00640 1 Processor -- start 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.634+0000 7f56f5c00640 1 -- start start 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d40039b0 0x7f56f014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f01efbb0 0x7f56f01f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0073e90 con 0x7f56f00ab660 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0073d10 con 0x7f56f01efbb0 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56f5c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f56f0074010 con 0x7f56d40039b0 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45650/0 (socket says 172.21.15.3:45650) 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 -- 172.21.15.3:0/2768304884 learned_addr learned my addr 172.21.15.3:0/2768304884 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56eea00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d40039b0 0x7f56f014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ef400640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f01efbb0 0x7f56f01f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 -- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d40039b0 msgr2=0x7f56f014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d40039b0 0x7f56f014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 -- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f01efbb0 msgr2=0x7f56f01f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f01efbb0 0x7f56f01f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.635+0000 7f56ee000640 1 -- 172.21.15.3:0/2768304884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f00aa5c0 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56ee000640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56dc004f70 tx=0x7f56dc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc01a070 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56dc00f750 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00fb40 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f00aa8a0 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.636+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f00aae60 con 0x7f56f00ab660 2024-09-10T07:46:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.637+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f00b3d10 con 0x7f56f00ab660 2024-09-10T07:46:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.638+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56dc01d070 con 0x7f56f00ab660 2024-09-10T07:46:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.639+0000 7f56e6a00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 0x7f56d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.639+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f56dc08a610 con 0x7f56f00ab660 2024-09-10T07:46:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.639+0000 7f56eea00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 0x7f56d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:47.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.640+0000 7f56eea00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 0x7f56d006e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:47.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.643+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56dc058960 con 0x7f56f00ab660 2024-09-10T07:46:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.865+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56f0157970 con 0x7f56f00ab660 2024-09-10T07:46:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.865+0000 7f56e6a00640 1 -- 172.21.15.3:0/2768304884 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f56dc05cc60 con 0x7f56f00ab660 2024-09-10T07:46:47.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:46:47.866 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:46:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 msgr2=0x7f56d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 0x7f56d006e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e0030000 comp rx=0 tx=0).stop 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 msgr2=0x7f56f00abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56dc004f70 tx=0x7f56dc007430 comp rx=0 tx=0).stop 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 shutdown_connections 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56d006be30 0x7f56d006e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f01efbb0 0x7f56f01f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56f00ab660 0x7f56f00abae0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 --2- 172.21.15.3:0/2768304884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56d40039b0 0x7f56f014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 >> 172.21.15.3:0/2768304884 conn(0x7f56f006d000 msgr2=0x7f56f00b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 shutdown_connections 2024-09-10T07:46:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:47.868+0000 7f56f5c00640 1 -- 172.21.15.3:0/2768304884 wait complete. 2024-09-10T07:46:48.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:46:30.640638+0000', 'modified': '2024-09-10T07:46:33.830367+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6835/1500431536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 1500431536}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 1500431536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-09-10T07:46:48.288 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:46:48.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T07:46:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.450+0000 7f9e27a00640 1 Processor -- start 2024-09-10T07:46:48.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.450+0000 7f9e27a00640 1 -- start start 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2015e4f0 0x7f9e2015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e2006bb20 0x7f9e2015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e2006d160 0x7f9e2006d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2006ccd0 con 0x7f9e2006d160 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2006db50 con 0x7f9e2006bb20 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.451+0000 7f9e27a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2006c0c0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e2006d160 0x7f9e2006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e2006d160 0x7f9e2006d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45652/0 (socket says 172.21.15.3:45652) 2024-09-10T07:46:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 -- 172.21.15.3:0/3283188008 learned_addr learned my addr 172.21.15.3:0/3283188008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/3283188008 >> v2:172.21.15.70:3300/0 conn(0x7f9e2006bb20 0x7f9e2015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25000640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2015e4f0 0x7f9e2015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 -- 172.21.15.3:0/3283188008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2015e4f0 msgr2=0x7f9e2015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2015e4f0 0x7f9e2015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 -- 172.21.15.3:0/3283188008 >> v2:172.21.15.70:3300/0 conn(0x7f9e2006bb20 msgr2=0x7f9e2015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 --2- 172.21.15.3:0/3283188008 >> v2:172.21.15.70:3300/0 conn(0x7f9e2006bb20 0x7f9e2015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.452+0000 7f9e25a00640 1 -- 172.21.15.3:0/3283188008 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2006c3a0 con 0x7f9e2006d160 2024-09-10T07:46:48.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.453+0000 7f9e25a00640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2006d160 0x7f9e2006d610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e200d31f0 tx=0x7f9e1802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1943800358052714 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.453+0000 7f9e1f400640 1 -- 172.21.15.3:0/3283188008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18036070 con 0x7f9e2006d160 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.453+0000 7f9e1f400640 1 -- 172.21.15.3:0/3283188008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e18031600 con 0x7f9e2006d160 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.453+0000 7f9e1f400640 1 -- 172.21.15.3:0/3283188008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180319d0 con 0x7f9e2006d160 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 -- 172.21.15.3:0/3283188008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2006d160 msgr2=0x7f9e2006d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2006d160 0x7f9e2006d610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e200d31f0 tx=0x7f9e1802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 -- 172.21.15.3:0/3283188008 shutdown_connections 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2006d160 0x7f9e2006d610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 --2- 172.21.15.3:0/3283188008 >> v2:172.21.15.70:3300/0 conn(0x7f9e2006bb20 0x7f9e2015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 --2- 172.21.15.3:0/3283188008 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2015e4f0 0x7f9e2015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 -- 172.21.15.3:0/3283188008 >> 172.21.15.3:0/3283188008 conn(0x7f9e2006e5b0 msgr2=0x7f9e2006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 -- 172.21.15.3:0/3283188008 shutdown_connections 2024-09-10T07:46:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.454+0000 7f9e27a00640 1 -- 172.21.15.3:0/3283188008 wait complete. 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.456+0000 7f9e27a00640 1 Processor -- start 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.456+0000 7f9e27a00640 1 -- start start 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2006bb20 0x7f9e2014be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 0x7f9e200d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2015e8d0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2006c960 con 0x7f9e2006d160 2024-09-10T07:46:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e27a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e201601f0 con 0x7f9e2006bb20 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45666/0 (socket says 172.21.15.3:45666) 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 -- 172.21.15.3:0/2743688452 learned_addr learned my addr 172.21.15.3:0/2743688452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 0x7f9e200d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25000640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2006bb20 0x7f9e2014be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 -- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2006bb20 msgr2=0x7f9e2014be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2006bb20 0x7f9e2014be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 -- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 msgr2=0x7f9e200d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 0x7f9e200d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e25a00640 1 -- 172.21.15.3:0/2743688452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2015cdd0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.457+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 0x7f9e200d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e25a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009f40 tx=0x7f9e18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e1cc00640 1 -- 172.21.15.3:0/2743688452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18036050 con 0x7f9e2015e4f0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e1cc00640 1 -- 172.21.15.3:0/2743688452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e18042070 con 0x7f9e2015e4f0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e200cf8a0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e1cc00640 1 -- 172.21.15.3:0/2743688452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18009510 con 0x7f9e2015e4f0 2024-09-10T07:46:48.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.458+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e200cfe60 con 0x7f9e2015e4f0 2024-09-10T07:46:48.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.461+0000 7f9e1cc00640 1 -- 172.21.15.3:0/2743688452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e18007bd0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.462+0000 7f9e1cc00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 0x7f9e0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.462+0000 7f9e1cc00640 1 -- 172.21.15.3:0/2743688452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9e180abdc0 con 0x7f9e2015e4f0 2024-09-10T07:46:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.462+0000 7f9e25000640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 0x7f9e0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.462+0000 7f9e2a8a9740 1 Processor -- start 2024-09-10T07:46:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.462+0000 7f9e2a8a9740 1 -- start start 2024-09-10T07:46:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e25000640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 0x7f9e0006e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56d4b0 0x55ff5c5700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c4ce3b0 con 0x55ff5c56d4b0 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c4d3410 con 0x55ff5c56fcf0 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.463+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ff5c47b620 con 0x55ff5c571480 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:35064/0 (socket says 172.21.15.3:35064) 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25a00640 1 -- 172.21.15.3:0/4061128504 learned_addr learned my addr 172.21.15.3:0/4061128504 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:54486/0 (socket says 172.21.15.3:54486) 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56d4b0 0x55ff5c5700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 -- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 msgr2=0x55ff5c56a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 -- 172.21.15.3:0/4061128504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56d4b0 msgr2=0x55ff5c5700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56d4b0 0x55ff5c5700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 -- 172.21.15.3:0/4061128504 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5c1e62d0 con 0x55ff5c56fcf0 2024-09-10T07:46:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25a00640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.464+0000 7f9e25000640 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400f8e0 tx=0x7f9e14012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3676c4108bcc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e11600640 1 -- 172.21.15.3:0/4061128504 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14023070 con 0x55ff5c56fcf0 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e11600640 1 -- 172.21.15.3:0/4061128504 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14019070 con 0x55ff5c56fcf0 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e11600640 1 -- 172.21.15.3:0/4061128504 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14014de0 con 0x55ff5c56fcf0 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 msgr2=0x55ff5c5b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400f8e0 tx=0x7f9e14012000 comp rx=0 tx=0).stop 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/4061128504 shutdown_connections 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c56a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56d4b0 0x55ff5c5700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/4061128504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c5b5490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.465+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/4061128504 >> 172.21.15.3:0/4061128504 conn(0x55ff5c56c740 msgr2=0x55ff5c5b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.466+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/4061128504 shutdown_connections 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.466+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/4061128504 wait complete. 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.466+0000 7f9e2a8a9740 1 -- start start 2024-09-10T07:46:48.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.466+0000 7f9e2a8a9740 1 Processor -- start 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c59a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c597d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c48e780 con 0x55ff5c580690 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c4cf3d0 con 0x55ff5c56fcf0 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ff5c56bbe0 con 0x55ff5c571480 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.467+0000 7f9e2a8a9740 10 client.0 _ll_register_callbacks cb 0x55ff5c4cc5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45692/0 (socket says 172.21.15.3:45692) 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 -- 172.21.15.3:0/968764300 learned_addr learned my addr 172.21.15.3:0/968764300 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25000640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c59a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c597d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 -- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 msgr2=0x55ff5c597d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c597d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 -- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 msgr2=0x55ff5c59a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c59a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 -- 172.21.15.3:0/968764300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5c2b06b0 con 0x55ff5c580690 2024-09-10T07:46:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c597d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e25a00640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e18078310 tx=0x7f9e180799c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c79c49dead20b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e06000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180a8060 con 0x55ff5c580690 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9e180794b0 con 0x55ff5c580690 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.468+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e14019070 con 0x55ff5c580690 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f9e18092070 con 0x55ff5c580690 2024-09-10T07:46:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 10 client.4602 fetch_fsmap learned FSMap version 7 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ff5c581650 con 0x55ff5c580690 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14019070 con 0x55ff5c580690 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f9e18075db0 con 0x55ff5c580690 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 10 client.4602 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 10 client.4602 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 20 client.4602 populate_metadata read hostname 'smithi003' 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c58f260 0x55ff5c58f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 4 client.4602 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.469+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 --> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff5c4e0b40 con 0x55ff5c58f260 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.470+0000 7f9e25000640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c58f260 0x55ff5c58f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.470+0000 7f9e25000640 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c58f260 0x55ff5c58f640 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff40b6b5e637ca92 server_cookie=89c93a1f4ceb02d6 in_seq=0 out_seq=0 2024-09-10T07:46:48.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.470+0000 7f9e06000640 10 client.4602 ms_handle_connect on v2:172.21.15.70:6833/1500431536 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.471+0000 7f9e06000640 1 -- 172.21.15.3:0/968764300 <== mds.0 v2:172.21.15.70:6833/1500431536 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff5c4e0b40 con 0x55ff5c58f260 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.471+0000 7f9e06000640 10 client.4602 handle_command_reply: tid=0 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 client.4602 shutdown 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 msgr2=0x55ff5c580b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e18078310 tx=0x7f9e180799c0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 shutdown_connections 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c580690 0x55ff5c580b10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c597d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c58f260 0x55ff5c58f640 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/968764300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c56fcf0 0x55ff5c59a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 >> 172.21.15.3:0/968764300 conn(0x55ff5c56d4b0 msgr2=0x55ff5c5b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e06000640 0 client.4602 ms_handle_reset on v2:172.21.15.70:6833/1500431536 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 shutdown_connections 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/968764300 wait complete. 2024-09-10T07:46:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.472+0000 7f9e2a8a9740 20 client.4602 trim_cache size 0 max 16384 2024-09-10T07:46:48.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.491+0000 7f9e2a8a9740 1 Processor -- start 2024-09-10T07:46:48.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.491+0000 7f9e2a8a9740 1 -- start start 2024-09-10T07:46:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c5d9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c5d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e201ffa90 con 0x55ff5c56fcf0 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2014d750 con 0x55ff5c580690 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e201e8b60 con 0x55ff5c571480 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45702/0 (socket says 172.21.15.3:45702) 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 -- 172.21.15.3:0/2742208856 learned_addr learned my addr 172.21.15.3:0/2742208856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 -- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 msgr2=0x55ff5c5d9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c5d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 -- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 msgr2=0x55ff5c5d97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25a00640 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c5d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c5d97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 -- 172.21.15.3:0/2742208856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5c57ec00 con 0x55ff5c56fcf0 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25a00640 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c5d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.492+0000 7f9e25000640 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e14022000 tx=0x7f9e1401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87340c06b76973f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e10c00640 1 -- 172.21.15.3:0/2742208856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14023040 con 0x55ff5c56fcf0 2024-09-10T07:46:48.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e10c00640 1 -- 172.21.15.3:0/2742208856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14033340 con 0x55ff5c56fcf0 2024-09-10T07:46:48.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e10c00640 1 -- 172.21.15.3:0/2742208856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14033710 con 0x55ff5c56fcf0 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/2742208856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 msgr2=0x55ff5c58da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e14022000 tx=0x7f9e1401e000 comp rx=0 tx=0).stop 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/2742208856 shutdown_connections 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c5d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c571480 0x55ff5c5d9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/2742208856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c56fcf0 0x55ff5c58da70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/2742208856 >> 172.21.15.3:0/2742208856 conn(0x55ff5c56d4b0 msgr2=0x55ff5c597b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/2742208856 shutdown_connections 2024-09-10T07:46:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.493+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/2742208856 wait complete. 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.494+0000 7f9e2a8a9740 1 -- start start 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.494+0000 7f9e2a8a9740 1 Processor -- start 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c56fcf0 0x55ff5c5950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c633060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c599c90 con 0x55ff5c571480 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ff5c48e780 con 0x55ff5c580690 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ff5c598360 con 0x55ff5c56fcf0 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e2a8a9740 10 client.0 _ll_register_callbacks cb 0x7f9e180aa220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45710/0 (socket says 172.21.15.3:45710) 2024-09-10T07:46:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e1fe00640 1 -- 172.21.15.3:0/1401754561 learned_addr learned my addr 172.21.15.3:0/1401754561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.495+0000 7f9e25000640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c56fcf0 0x55ff5c5950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e25a00640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c633060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 -- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c56fcf0 msgr2=0x55ff5c5950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c56fcf0 0x55ff5c5950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 -- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 msgr2=0x55ff5c633060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c633060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 -- 172.21.15.3:0/1401754561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5c5955f0 con 0x55ff5c571480 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e1fe00640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c006fd0 tx=0x7f9e0c0147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bf242eb6c853e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e06000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c004240 con 0x55ff5c571480 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9e0c0043e0 con 0x55ff5c571480 2024-09-10T07:46:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.496+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e14033340 con 0x55ff5c571480 2024-09-10T07:46:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f9e0c003610 con 0x55ff5c571480 2024-09-10T07:46:48.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 10 client.4608 fetch_fsmap learned FSMap version 7 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ff5c594b90 con 0x55ff5c571480 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14033340 con 0x55ff5c571480 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f9e0c0037b0 con 0x55ff5c571480 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 10 client.4608 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 10 client.4608 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 20 client.4608 populate_metadata read hostname 'smithi003' 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c5c4a30 0x55ff5c5d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 4 client.4608 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.497+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 --> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] -- command(tid 0: {"prefix": "status"}) -- 0x55ff5c4dae90 con 0x55ff5c5c4a30 2024-09-10T07:46:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e25000640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c5c4a30 0x55ff5c5d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e25000640 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c5c4a30 0x55ff5c5d7fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843ac717cce0e678 server_cookie=a5679895530910ec in_seq=0 out_seq=0 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e06000640 10 client.4608 ms_handle_connect on v2:172.21.15.70:6833/1500431536 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e06000640 1 -- 172.21.15.3:0/1401754561 <== mds.0 v2:172.21.15.70:6833/1500431536 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ff5c4dae90 con 0x55ff5c5c4a30 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e06000640 10 client.4608 handle_command_reply: tid=0 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e2a8a9740 1 client.4608 shutdown 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 msgr2=0x55ff5c58dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.498+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c006fd0 tx=0x7f9e0c0147e0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 shutdown_connections 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:6833/1500431536,v1:172.21.15.70:6835/1500431536] conn(0x55ff5c5c4a30 0x55ff5c5d7fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff5c580690 0x55ff5c633060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ff5c571480 0x55ff5c58dcb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 --2- 172.21.15.3:0/1401754561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ff5c56fcf0 0x55ff5c5950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 >> 172.21.15.3:0/1401754561 conn(0x55ff5c56d4b0 msgr2=0x55ff5c5961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e06000640 0 client.4608 ms_handle_reset on v2:172.21.15.70:6833/1500431536 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 shutdown_connections 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 1 -- 172.21.15.3:0/1401754561 wait complete. 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e2a8a9740 20 client.4608 trim_cache size 0 max 16384 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:46:48.503 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4287, 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 17.273326669999999, 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 7, 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 23, 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 23, 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 23.320906236999999 2024-09-10T07:46:48.504 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:46:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 msgr2=0x7f9e0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 0x7f9e0006e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14008040 comp rx=0 tx=0).stop 2024-09-10T07:46:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 msgr2=0x7f9e200d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009f40 tx=0x7f9e18004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.499+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 shutdown_connections 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e0006be30 0x7f9e0006e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e2015e4f0 0x7f9e200d1510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e2006d160 0x7f9e200d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 --2- 172.21.15.3:0/2743688452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e2006bb20 0x7f9e2014be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 >> 172.21.15.3:0/2743688452 conn(0x7f9e2006e5b0 msgr2=0x7f9e2006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 shutdown_connections 2024-09-10T07:46:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:48.500+0000 7f9e27a00640 1 -- 172.21.15.3:0/2743688452 wait complete. 2024-09-10T07:46:48.514 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T07:46:48.525 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T07:46:48.525 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T07:46:48.525 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T07:46:48.525 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T07:46:48.552 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T07:46:48.552 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T07:46:48.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.552 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:46:48.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-10T07:46:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-10T07:46:48.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:46:48.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.691 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:46:48.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-10T07:46:48.825 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-10T07:46:48.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:46:48.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.828 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:46:48.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-09-10T07:46:48.972 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-09-10T07:46:48.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:46:48.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:48.976 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:46:49.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:49.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-09-10T07:46:49.110 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-09-10T07:46:49.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:46:49.113 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-10T07:46:49.113 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:46:49.114 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:46:49.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:46:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.273+0000 7f2636200640 1 Processor -- start 2024-09-10T07:46:49.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.273+0000 7f2636200640 1 -- start start 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2636200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26300b5ea0 0x7f26300b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2636200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f263006e390 0x7f263006e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2636200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26300b47d0 con 0x7f2630161dd0 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2636200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f263006c870 con 0x7f263006e390 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2636200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f263006cb50 con 0x7f26300b5ea0 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f263006e390 0x7f263006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f263006e390 0x7f263006e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45720/0 (socket says 172.21.15.3:45720) 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 -- 172.21.15.3:0/445803465 learned_addr learned my addr 172.21.15.3:0/445803465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:49.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f2626a00640 1 --2- 172.21.15.3:0/445803465 >> v2:172.21.15.70:3300/0 conn(0x7f26300b5ea0 0x7f26300b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 -- 172.21.15.3:0/445803465 >> v2:172.21.15.70:3300/0 conn(0x7f26300b5ea0 msgr2=0x7f26300b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 --2- 172.21.15.3:0/445803465 >> v2:172.21.15.70:3300/0 conn(0x7f26300b5ea0 0x7f26300b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 -- 172.21.15.3:0/445803465 >> v1:172.21.15.70:6789/0 conn(0x7f2630161dd0 legacy=0x7f263006f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.275+0000 7f262fe00640 1 -- 172.21.15.3:0/445803465 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26301621b0 con 0x7f263006e390 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.276+0000 7f262fe00640 1 --2- 172.21.15.3:0/445803465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263006e390 0x7f263006e790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26300b0f10 tx=0x7f262802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f354e01cc53f43e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f262ea00640 1 -- 172.21.15.3:0/445803465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628036070 con 0x7f263006e390 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f262ea00640 1 -- 172.21.15.3:0/445803465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2628031600 con 0x7f263006e390 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f262ea00640 1 -- 172.21.15.3:0/445803465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280319d0 con 0x7f263006e390 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 -- 172.21.15.3:0/445803465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263006e390 msgr2=0x7f263006e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 --2- 172.21.15.3:0/445803465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263006e390 0x7f263006e790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26300b0f10 tx=0x7f262802a730 comp rx=0 tx=0).stop 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 -- 172.21.15.3:0/445803465 shutdown_connections 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 --2- 172.21.15.3:0/445803465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f263006e390 0x7f263006e790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 --2- 172.21.15.3:0/445803465 >> v2:172.21.15.70:3300/0 conn(0x7f26300b5ea0 0x7f26300b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 -- 172.21.15.3:0/445803465 >> 172.21.15.3:0/445803465 conn(0x7f26300b3540 msgr2=0x7f26300b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 -- 172.21.15.3:0/445803465 shutdown_connections 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.277+0000 7f2636200640 1 -- 172.21.15.3:0/445803465 wait complete. 2024-09-10T07:46:49.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.279+0000 7f2636200640 1 Processor -- start 2024-09-10T07:46:49.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.279+0000 7f2636200640 1 -- start start 2024-09-10T07:46:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 0x7f2630153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26300b5ea0 0x7f2630140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26300b43f0 con 0x7f2630161dd0 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26300b6820 con 0x7f263006e390 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2636200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26300b4570 con 0x7f26300b5ea0 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45732/0 (socket says 172.21.15.3:45732) 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262fe00640 1 -- 172.21.15.3:0/3330421991 learned_addr learned my addr 172.21.15.3:0/3330421991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262f400640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 0x7f2630153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262fe00640 1 -- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26300b5ea0 msgr2=0x7f2630140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f2626a00640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26300b5ea0 0x7f2630140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.280+0000 7f262fe00640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26300b5ea0 0x7f2630140900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f262fe00640 1 -- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 msgr2=0x7f2630153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f262fe00640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 0x7f2630153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f262fe00640 1 -- 172.21.15.3:0/3330421991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26300cd6a0 con 0x7f2630161dd0 2024-09-10T07:46:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f262f400640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 0x7f2630153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:46:49.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f262fe00640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f262802ad20 tx=0x7f2628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262803f020 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2628037040 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26301438a0 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628036070 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.281+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26300cdbf0 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.282+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26300b62e0 con 0x7f2630161dd0 2024-09-10T07:46:49.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.283+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2628042020 con 0x7f2630161dd0 2024-09-10T07:46:49.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.284+0000 7f2627e00640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 0x7f260c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:49.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.284+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f26280ab670 con 0x7f2630161dd0 2024-09-10T07:46:49.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.284+0000 7f262f400640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 0x7f260c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:49.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.285+0000 7f262f400640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 0x7f260c06e320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26200015e0 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:49.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.288+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2628049020 con 0x7f2630161dd0 2024-09-10T07:46:49.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.509+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2630075640 con 0x7f2630161dd0 2024-09-10T07:46:49.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.510+0000 7f2627e00640 1 -- 172.21.15.3:0/3330421991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2628079f80 con 0x7f2630161dd0 2024-09-10T07:46:49.510 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 msgr2=0x7f260c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 0x7f260c06e320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26200015e0 tx=0x7f2620009040 comp rx=0 tx=0).stop 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 msgr2=0x7f2630143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f262802ad20 tx=0x7f2628004a40 comp rx=0 tx=0).stop 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 shutdown_connections 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f260c06be60 0x7f260c06e320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2630161dd0 0x7f2630143360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26300b5ea0 0x7f2630140900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 --2- 172.21.15.3:0/3330421991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263006e390 0x7f2630153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:49.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 >> 172.21.15.3:0/3330421991 conn(0x7f26300b3540 msgr2=0x7f263006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:49.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 shutdown_connections 2024-09-10T07:46:49.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49.512+0000 7f2636200640 1 -- 172.21.15.3:0/3330421991 wait complete. 2024-09-10T07:46:49.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T07:46:49.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:46:49.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:46:49.528 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:46:49.529 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:46:49.529 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T07:46:49.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:46:49.529 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:46:49.529 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:46:49.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:49.529 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:49.565 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42686sec preferred_lft 42686sec 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:49.566 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:49.567 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:49.567 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:46:49.567 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T07:46:49.567 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:49.567 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:49.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-10T07:46:49.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-10T07:46:49.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-10T07:46:49.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T07:46:49.568 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:49.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:49.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:49.980 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:46:50.035 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:46:50.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:50.039 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:46:50.091 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:46:50.091 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:46:50.091 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:46:50.091 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:46:50.091 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:50.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:50.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:50.667 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:46:50.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:50.703 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-10T07:46:50.778 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:50.778 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:50.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:46:50.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T07:46:50.778 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:50.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:46:50.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:46:50.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:50.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:50.955 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T07:46:50.955 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T07:46:50.956 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:51.242 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:51.242 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:51.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:46:51.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T07:46:51.242 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:51.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:51.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:51.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:51.410 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:46:51.445 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:46:51.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:51.446 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:46:51.521 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:46:51.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:51.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:46:51.656 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:46:51.656 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:46:51.656 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:52.327 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:46:52.327 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:46:52.327 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:52.327 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:52.327 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:46:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:46:52.356 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:46:52.357 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:46:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:46:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:46:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:46:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:46:52.360 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:46:52.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:52.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:46:52.419 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:52.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:52.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:client4496 2024-09-10T07:46:52.520 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:52.520 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:46:52.556 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:52.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:52.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:52.695 INFO:teuthology.orchestra.run.smithi003.stdout:client4496 2024-09-10T07:46:52.695 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:52.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:52.793 INFO:teuthology.orchestra.run.smithi003.stdout:client4496 2024-09-10T07:46:52.794 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46339922-75e0-4945-8bb6-1acaef196937.client4496/status') 2024-09-10T07:46:52.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:52.900 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4496 (3)172.21.15.3:0/2208909422 2024-09-10T07:46:52.900 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-10T07:46:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-10T07:46:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-10T07:46:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 5.6541e-05 s, 1.2 MB/s 2024-09-10T07:46:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:52.910 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-10T07:46:52.911 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:46:52.911 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T07:46:52.911 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:46:53.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.067+0000 7f363a000640 1 Processor -- start 2024-09-10T07:46:53.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.067+0000 7f363a000640 1 -- start start 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f363a000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36340d0fd0 0x7f3634162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f363a000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36340b5d80 con 0x7f36340b52c0 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f363a000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36340b6060 con 0x7f36340737f0 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f363a000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f36341621a0 con 0x7f36340d0fd0 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f362be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f36340b52c0 0x7f36340730e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60280/0 (socket says 172.21.15.3:60280) 2024-09-10T07:46:53.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.068+0000 7f362be00640 1 -- 172.21.15.3:0/836466752 learned_addr learned my addr 172.21.15.3:0/836466752 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:53.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3633400640 1 --2- 172.21.15.3:0/836466752 >> v2:172.21.15.70:3300/0 conn(0x7f36340d0fd0 0x7f3634162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646665534 0 0) 0x7f36340b5d80 con 0x7f36340b52c0 2024-09-10T07:46:53.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3618003520 con 0x7f36340b52c0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398330285 0 0) 0x7f36340b6060 con 0x7f36340737f0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36340b5d80 con 0x7f36340737f0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3421231367 0 0) 0x7f3618003520 con 0x7f36340b52c0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 >> v2:172.21.15.70:3300/0 conn(0x7f36340d0fd0 msgr2=0x7f3634162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 --2- 172.21.15.3:0/836466752 >> v2:172.21.15.70:3300/0 conn(0x7f36340d0fd0 0x7f3634162c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 >> v1:172.21.15.70:6789/0 conn(0x7f36340737f0 legacy=0x7f3634154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3634162480 con 0x7f36340b52c0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f361c001670 con 0x7f36340b52c0 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 >> v1:172.21.15.3:6789/0 conn(0x7f36340b52c0 legacy=0x7f36340730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618003b40 0x7f3618003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36180051c0 0x7f361800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36341621a0 con 0x7f3618004800 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f361800c040 con 0x7f36180051c0 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.069+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f361800c240 con 0x7f3618003b40 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f3633400640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618003b40 0x7f3618003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f3633e00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36180051c0 0x7f361800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 -- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618003b40 msgr2=0x7f3618003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618003b40 0x7f3618003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 -- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36180051c0 msgr2=0x7f361800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36180051c0 0x7f361800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f362be00640 1 -- 172.21.15.3:0/836466752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3618003520 con 0x7f3618004800 2024-09-10T07:46:53.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.070+0000 7f3633e00640 1 -- 172.21.15.3:0/836466752 reap_dead start 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.071+0000 7f362be00640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 0x7f3618004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c1c0 tx=0x7f3618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd36d9f4439b446 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.071+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c007cc0 con 0x7f3618004800 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.071+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c005ad0 con 0x7f3618004800 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.071+0000 7f3632a00640 1 -- 172.21.15.3:0/836466752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c007070 con 0x7f3618004800 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 -- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 msgr2=0x7f3618004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 0x7f3618004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c1c0 tx=0x7f3618005830 comp rx=0 tx=0).stop 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 -- 172.21.15.3:0/836466752 shutdown_connections 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 --2- 172.21.15.3:0/836466752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618004800 0x7f3618004c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 -- 172.21.15.3:0/836466752 >> 172.21.15.3:0/836466752 conn(0x7f363406bb20 msgr2=0x7f363406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 -- 172.21.15.3:0/836466752 shutdown_connections 2024-09-10T07:46:53.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.072+0000 7f363a000640 1 -- 172.21.15.3:0/836466752 wait complete. 2024-09-10T07:46:53.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.074+0000 7f363a000640 1 Processor -- start 2024-09-10T07:46:53.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.074+0000 7f363a000640 1 -- start start 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618004800 0x7f36341512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f363409cc30 0x7f36340aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36340d13b0 con 0x7f36340af2f0 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f363406f250 con 0x7f363409cc30 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f363a000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f36341f4080 con 0x7f3618004800 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f3633e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f3633e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45758/0 (socket says 172.21.15.3:45758) 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f3633e00640 1 -- 172.21.15.3:0/1732003479 learned_addr learned my addr 172.21.15.3:0/1732003479 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:53.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f3633400640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618004800 0x7f36341512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.075+0000 7f362be00640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f363409cc30 0x7f36340aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 -- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618004800 msgr2=0x7f36341512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618004800 0x7f36341512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 -- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f363409cc30 msgr2=0x7f36340aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f363409cc30 0x7f36340aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 -- 172.21.15.3:0/1732003479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36341f4310 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f3633e00640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f363415c780 tx=0x7f362c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c019070 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c006e20 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c003b90 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36341f45f0 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.076+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341f4bb0 con 0x7f36340af2f0 2024-09-10T07:46:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.077+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36340b5720 con 0x7f36340af2f0 2024-09-10T07:46:53.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.078+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f362c01c070 con 0x7f36340af2f0 2024-09-10T07:46:53.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.079+0000 7f362b400640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 0x7f361006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.079+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f362c08a190 con 0x7f36340af2f0 2024-09-10T07:46:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.079+0000 7f3633400640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 0x7f361006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.080+0000 7f3633400640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 0x7f361006e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36340518b0 tx=0x7f3624008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:53.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.084+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f362c058b30 con 0x7f36340af2f0 2024-09-10T07:46:53.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.303+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36340728c0 con 0x7f36340af2f0 2024-09-10T07:46:53.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.303+0000 7f362b400640 1 -- 172.21.15.3:0/1732003479 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f362c058430 con 0x7f36340af2f0 2024-09-10T07:46:53.303 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:46:53.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.305+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 msgr2=0x7f361006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.305+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 0x7f361006e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36340518b0 tx=0x7f3624008040 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 msgr2=0x7f363415d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f363415c780 tx=0x7f362c00ff70 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 shutdown_connections 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f361006be60 0x7f361006e320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36340af2f0 0x7f363415d210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f363409cc30 0x7f36340aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 --2- 172.21.15.3:0/1732003479 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3618004800 0x7f36341512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 >> 172.21.15.3:0/1732003479 conn(0x7f363406bb20 msgr2=0x7f363406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:53.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 shutdown_connections 2024-09-10T07:46:53.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53.306+0000 7f363a000640 1 -- 172.21.15.3:0/1732003479 wait complete. 2024-09-10T07:46:53.321 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:46:53.322 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T07:46:53.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.322 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:46:53.358 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:46:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:46:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:53.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.362 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:46:53.452 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:46:53.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.456 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:46:53.492 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:46:53.492 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:46:53.493 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:46:53.493 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:46:53.493 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:46:53.493 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:53.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:53.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.691 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:46:53.725 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:46:53.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.726 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-10T07:46:53.800 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T07:46:53.801 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:53.801 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:53.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:46:53.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T07:46:53.801 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:53.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:53.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:53.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.033 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:46:54.034 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 22:db:9c:64:68:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T07:46:54.034 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T07:46:54.034 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.034 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::20db:9cff:fe64:68ad/64 scope link 2024-09-10T07:46:54.034 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:54.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T07:46:54.091 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 22:db:9c:64:68:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::20db:9cff:fe64:68ad/64 scope link 2024-09-10T07:46:54.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:54.094 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T07:46:54.095 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:54.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:54.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:54.315 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:54.315 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:46:54.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2024-09-10T07:46:54.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T07:46:54.315 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:46:54.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:54.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:54.481 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T07:46:54.517 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T07:46:54.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:54.517 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T07:46:54.592 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:46:54.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:54.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T07:46:54.721 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T07:46:54.721 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T07:46:54.721 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T07:46:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T07:46:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:54.760 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T07:46:54.760 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:46:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:46:54.763 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-10T07:46:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:46:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:46:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:46:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:46:54.766 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:46:54.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:54.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T07:46:54.822 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:54.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:54.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:54.911 INFO:teuthology.orchestra.run.smithi003.stdout:client4623 2024-09-10T07:46:54.911 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:46:54.912 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:46:54.948 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:55.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:55.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:55.076 INFO:teuthology.orchestra.run.smithi003.stdout:client4623 2024-09-10T07:46:55.077 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:55.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:55.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:55.166 INFO:teuthology.orchestra.run.smithi003.stdout:client4623 2024-09-10T07:46:55.166 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46339922-75e0-4945-8bb6-1acaef196937.client4623/status') 2024-09-10T07:46:55.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:55.268 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4623 (3)192.168.0.2:0/134539059 2024-09-10T07:46:55.268 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-10T07:46:55.269 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-10T07:46:55.269 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-10T07:46:55.269 INFO:teuthology.orchestra.run.smithi003.stderr:68 bytes copied, 5.6206e-05 s, 1.2 MB/s 2024-09-10T07:46:55.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:55.279 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-10T07:46:55.279 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:46:55.279 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T07:46:55.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:46:55.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.435+0000 7f699f200640 1 Processor -- start 2024-09-10T07:46:55.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.435+0000 7f699f200640 1 -- start start 2024-09-10T07:46:55.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69a00d3c10 0x7f69a0073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69a00d4050 0x7f69a0073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a00cfdc0 con 0x7f69a015e860 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f69a00d3780 con 0x7f69a00d4050 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f69a00cee40 con 0x7f69a00d3c10 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699de00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f69a015e860 0x7f69a00cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56714/0 (socket says 172.21.15.3:56714) 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699de00640 1 -- 172.21.15.3:0/4042175738 learned_addr learned my addr 172.21.15.3:0/4042175738 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.436+0000 7f699e800640 1 --2- 172.21.15.3:0/4042175738 >> v2:172.21.15.3:3300/0 conn(0x7f69a00d4050 0x7f69a0073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699d400640 1 --2- 172.21.15.3:0/4042175738 >> v2:172.21.15.70:3300/0 conn(0x7f69a00d3c10 0x7f69a0073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699ca00640 1 -- 172.21.15.3:0/4042175738 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501140210 0 0) 0x7f69a00cfdc0 con 0x7f69a015e860 2024-09-10T07:46:55.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699ca00640 1 -- 172.21.15.3:0/4042175738 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6984003520 con 0x7f69a015e860 2024-09-10T07:46:55.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699e800640 1 -- 172.21.15.3:0/4042175738 >> v2:172.21.15.70:3300/0 conn(0x7f69a00d3c10 msgr2=0x7f69a0073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:55.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699e800640 1 --2- 172.21.15.3:0/4042175738 >> v2:172.21.15.70:3300/0 conn(0x7f69a00d3c10 0x7f69a0073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699e800640 1 -- 172.21.15.3:0/4042175738 >> v1:172.21.15.3:6789/0 conn(0x7f69a015e860 legacy=0x7f69a00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.437+0000 7f699e800640 1 -- 172.21.15.3:0/4042175738 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a00cf120 con 0x7f69a00d4050 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.438+0000 7f699e800640 1 --2- 172.21.15.3:0/4042175738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a0073690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f69a00d0ad0 tx=0x7f699802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4459f3c5cb522f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.438+0000 7f699ca00640 1 -- 172.21.15.3:0/4042175738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998007740 con 0x7f69a00d4050 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.438+0000 7f699ca00640 1 -- 172.21.15.3:0/4042175738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69980078e0 con 0x7f69a00d4050 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.438+0000 7f699ca00640 1 -- 172.21.15.3:0/4042175738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998007cb0 con 0x7f69a00d4050 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 -- 172.21.15.3:0/4042175738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 msgr2=0x7f69a0073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 --2- 172.21.15.3:0/4042175738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a0073690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f69a00d0ad0 tx=0x7f699802a680 comp rx=0 tx=0).stop 2024-09-10T07:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 -- 172.21.15.3:0/4042175738 shutdown_connections 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 --2- 172.21.15.3:0/4042175738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a0073690 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 --2- 172.21.15.3:0/4042175738 >> v2:172.21.15.70:3300/0 conn(0x7f69a00d3c10 0x7f69a0073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 -- 172.21.15.3:0/4042175738 >> 172.21.15.3:0/4042175738 conn(0x7f69a00d4650 msgr2=0x7f69a00d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 -- 172.21.15.3:0/4042175738 shutdown_connections 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.439+0000 7f699f200640 1 -- 172.21.15.3:0/4042175738 wait complete. 2024-09-10T07:46:55.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.440+0000 7f699f200640 1 Processor -- start 2024-09-10T07:46:55.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.441+0000 7f699f200640 1 -- start start 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.441+0000 7f699f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69a00d3c10 0x7f69a00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69a015e860 0x7f69a01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00d2460 con 0x7f69a00d4050 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00cf400 con 0x7f69a00d3c10 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f69a00b7880 con 0x7f69a015e860 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44468/0 (socket says 172.21.15.3:44468) 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 -- 172.21.15.3:0/3320775046 learned_addr learned my addr 172.21.15.3:0/3320775046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699de00640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69a00d3c10 0x7f69a00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699e800640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69a015e860 0x7f69a01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 -- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69a015e860 msgr2=0x7f69a01f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69a015e860 0x7f69a01f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 -- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69a00d3c10 msgr2=0x7f69a00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69a00d3c10 0x7f69a00b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.442+0000 7f699d400640 1 -- 172.21.15.3:0/3320775046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a00abc30 con 0x7f69a00d4050 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f699d400640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f698c00e9e0 tx=0x7f698c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c017070 con 0x7f69a00d4050 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c00c9d0 con 0x7f69a00d4050 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c00cdc0 con 0x7f69a00d4050 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a00abf10 con 0x7f69a00d4050 2024-09-10T07:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.443+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a00ac4d0 con 0x7f69a00d4050 2024-09-10T07:46:55.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.444+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a00b7c40 con 0x7f69a00d4050 2024-09-10T07:46:55.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.445+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f698c01a070 con 0x7f69a00d4050 2024-09-10T07:46:55.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.446+0000 7f6996000640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 0x7f698006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:55.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.446+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f698c08b2b0 con 0x7f69a00d4050 2024-09-10T07:46:55.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.447+0000 7f699de00640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 0x7f698006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:55.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.447+0000 7f699de00640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 0x7f698006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6990005880 tx=0x7f6990009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:55.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.451+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f698c0594e0 con 0x7f69a00d4050 2024-09-10T07:46:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.674+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f69a0069e20 con 0x7f69a00d4050 2024-09-10T07:46:55.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.675+0000 7f6996000640 1 -- 172.21.15.3:0/3320775046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f698c05d7e0 con 0x7f69a00d4050 2024-09-10T07:46:55.675 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 msgr2=0x7f698006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 0x7f698006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6990005880 tx=0x7f6990009040 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 msgr2=0x7f69a01f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f698c00e9e0 tx=0x7f698c00eea0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 shutdown_connections 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f698006be30 0x7f698006e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69a015e860 0x7f69a01f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69a00d4050 0x7f69a01f4290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 --2- 172.21.15.3:0/3320775046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69a00d3c10 0x7f69a00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 >> 172.21.15.3:0/3320775046 conn(0x7f69a00d4650 msgr2=0x7f69a00d4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 shutdown_connections 2024-09-10T07:46:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:55.677+0000 7f699f200640 1 -- 172.21.15.3:0/3320775046 wait complete. 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:46:55.692 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T07:46:55.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:55.693 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:55.733 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2024-09-10T07:46:55.734 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:55.735 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T07:46:55.735 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:55.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:56.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:56.246 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:46:56.301 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-09-10T07:46:56.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:56.304 DEBUG:teuthology.orchestra.run.smithi070:> route 2024-09-10T07:46:56.369 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2024-09-10T07:46:56.369 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:46:56.369 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T07:46:56.369 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T07:46:56.369 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T07:46:56.370 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:56.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:56.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:56.950 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:46:56.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:56.985 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2024-09-10T07:46:57.061 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:57.061 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:57.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T07:46:57.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T07:46:57.061 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:57.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:57.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:57.237 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T07:46:57.237 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:57.237 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T07:46:57.238 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:57.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:57.486 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:57.486 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:57.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2024-09-10T07:46:57.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T07:46:57.487 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:57.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:57.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:57.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:57.653 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T07:46:57.688 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T07:46:57.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:46:57.689 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T07:46:57.763 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:46:57.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:57.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T07:46:57.899 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T07:46:57.899 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T07:46:57.899 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:46:58.463 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T07:46:58.464 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T07:46:58.464 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:46:58.464 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:58.464 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:46:58.490 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:46:58.491 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-10T07:46:58.492 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:46:58.492 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:46:58.492 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:46:58.492 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:46:58.495 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:46:58.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:58.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T07:46:58.552 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:58.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:58.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:58.641 INFO:teuthology.orchestra.run.smithi070.stdout:client4635 2024-09-10T07:46:58.642 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:46:58.642 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:46:58.677 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:58.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:58.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:58.806 INFO:teuthology.orchestra.run.smithi070.stdout:client4635 2024-09-10T07:46:58.806 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:46:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:46:58.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:58.894 INFO:teuthology.orchestra.run.smithi070.stdout:client4635 2024-09-10T07:46:58.894 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46339922-75e0-4945-8bb6-1acaef196937.client4635/status') 2024-09-10T07:46:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:58.992 INFO:teuthology.orchestra.run.smithi070.stdout:instance: client.4635 (3)172.21.15.70:0/4168738592 2024-09-10T07:46:58.993 INFO:teuthology.orchestra.run.smithi070.stdout:blocklisted: false 2024-09-10T07:46:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records in 2024-09-10T07:46:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records out 2024-09-10T07:46:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:70 bytes copied, 9.4801e-05 s, 738 kB/s 2024-09-10T07:46:58.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:59.003 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-10T07:46:59.003 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T07:46:59.003 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T07:46:59.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:46:59.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.172+0000 7f2494600640 1 Processor -- start 2024-09-10T07:46:59.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.172+0000 7f2494600640 1 -- start start 2024-09-10T07:46:59.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.173+0000 7f2494600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 0x7f248c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.173+0000 7f2494600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.173+0000 7f2494600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248c0b4250 con 0x7f248c073d10 2024-09-10T07:46:59.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.173+0000 7f2494600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f248c0b44f0 con 0x7f248c15b110 2024-09-10T07:46:59.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.173+0000 7f2494600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f248c0ccd80 con 0x7f248c15b4f0 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 0x7f248c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2492600640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f248c073d10 0x7f248c15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:53594/0 (socket says 172.21.15.3:53594) 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2492600640 1 -- 172.21.15.3:0/1599317168 learned_addr learned my addr 172.21.15.3:0/1599317168 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491200640 1 -- 172.21.15.3:0/1599317168 >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 msgr2=0x7f248c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491200640 1 --2- 172.21.15.3:0/1599317168 >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 0x7f248c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491200640 1 -- 172.21.15.3:0/1599317168 >> v1:172.21.15.70:6789/0 conn(0x7f248c073d10 legacy=0x7f248c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491200640 1 -- 172.21.15.3:0/1599317168 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c0cd060 con 0x7f248c15b4f0 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.174+0000 7f2491c00640 1 --2- 172.21.15.3:0/1599317168 >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 0x7f248c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.175+0000 7f2491200640 1 --2- 172.21.15.3:0/1599317168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c0d01e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137c8043579ff3a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f248be00640 1 -- 172.21.15.3:0/1599317168 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478037070 con 0x7f248c15b4f0 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f248be00640 1 -- 172.21.15.3:0/1599317168 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478032440 con 0x7f248c15b4f0 2024-09-10T07:46:59.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f248be00640 1 -- 172.21.15.3:0/1599317168 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478032810 con 0x7f248c15b4f0 2024-09-10T07:46:59.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 -- 172.21.15.3:0/1599317168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 msgr2=0x7f248c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 --2- 172.21.15.3:0/1599317168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c0d01e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802a180 comp rx=0 tx=0).stop 2024-09-10T07:46:59.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 -- 172.21.15.3:0/1599317168 shutdown_connections 2024-09-10T07:46:59.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 --2- 172.21.15.3:0/1599317168 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c0d01e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 --2- 172.21.15.3:0/1599317168 >> v2:172.21.15.70:3300/0 conn(0x7f248c15b110 0x7f248c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 -- 172.21.15.3:0/1599317168 >> 172.21.15.3:0/1599317168 conn(0x7f248c06d000 msgr2=0x7f248c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:59.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.176+0000 7f2494600640 1 -- 172.21.15.3:0/1599317168 shutdown_connections 2024-09-10T07:46:59.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.177+0000 7f2494600640 1 -- 172.21.15.3:0/1599317168 wait complete. 2024-09-10T07:46:59.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.178+0000 7f2494600640 1 Processor -- start 2024-09-10T07:46:59.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.178+0000 7f2494600640 1 -- start start 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 0x7f248c1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0ceb50 con 0x7f248c15b110 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0cd620 con 0x7f248c073d10 2024-09-10T07:46:59.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2494600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f248c0cecd0 con 0x7f248c15b4f0 2024-09-10T07:46:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.179+0000 7f2491200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 0x7f248c1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44478/0 (socket says 172.21.15.3:44478) 2024-09-10T07:46:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491200640 1 -- 172.21.15.3:0/993500816 learned_addr learned my addr 172.21.15.3:0/993500816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:46:59.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2492600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 -- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 msgr2=0x7f248c072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c072bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 -- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 msgr2=0x7f248c072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 -- 172.21.15.3:0/993500816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c1e4ea0 con 0x7f248c073d10 2024-09-10T07:46:59.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491200640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2491c00640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 0x7f248c1f3a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f248000e900 tx=0x7f248000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.180+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480017070 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.181+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248000ca00 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.181+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c1e5180 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.181+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248000cdf0 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.181+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1e5c60 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.182+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0b3d10 con 0x7f248c073d10 2024-09-10T07:46:59.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.183+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f248001a070 con 0x7f248c073d10 2024-09-10T07:46:59.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.183+0000 7f2489600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:46:59.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.184+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f248008b0a0 con 0x7f248c073d10 2024-09-10T07:46:59.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.184+0000 7f2491200640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:46:59.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.184+0000 7f2491200640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 0x7f246c06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f247802a1c0 tx=0x7f247802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:46:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.188+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2480059360 con 0x7f248c073d10 2024-09-10T07:46:59.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.408+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f248c06d820 con 0x7f248c073d10 2024-09-10T07:46:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.409+0000 7f2489600640 1 -- 172.21.15.3:0/993500816 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f248005d660 con 0x7f248c073d10 2024-09-10T07:46:59.408 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 msgr2=0x7f246c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 0x7f246c06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f247802a1c0 tx=0x7f247802e000 comp rx=0 tx=0).stop 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 msgr2=0x7f248c1f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 0x7f248c1f3a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f248000e900 tx=0x7f248000edc0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 shutdown_connections 2024-09-10T07:46:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f246c06be30 0x7f246c06e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f248c15b4f0 0x7f248c072bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f248c15b110 0x7f248c072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 --2- 172.21.15.3:0/993500816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f248c073d10 0x7f248c1f3a80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 >> 172.21.15.3:0/993500816 conn(0x7f248c06d000 msgr2=0x7f248c0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 shutdown_connections 2024-09-10T07:46:59.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:46:59.411+0000 7f2494600640 1 -- 172.21.15.3:0/993500816 wait complete. 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:46:59.428 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T07:46:59.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:59.429 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2024-09-10T07:46:59.467 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2024-09-10T07:46:59.468 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether ce:58:cd:9b:99:c0 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38aa:e3ff:fe04:f174/64 scope link 2024-09-10T07:46:59.469 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.470 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:46:59.470 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether ce:58:cd:9b:99:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T07:46:59.470 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::cc58:cdff:fe9b:99c0/64 scope link tentative 2024-09-10T07:46:59.470 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:46:59.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:59.470 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:46:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-09-10T07:46:59.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:59.568 DEBUG:teuthology.orchestra.run.smithi070:> route 2024-09-10T07:46:59.605 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2024-09-10T07:46:59.605 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:46:59.605 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T07:46:59.605 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T07:46:59.605 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T07:46:59.606 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:46:59.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:46:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:46:59.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:59.801 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:46:59.836 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T07:46:59.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:46:59.837 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2024-09-10T07:46:59.911 INFO:teuthology.orchestra.run.smithi070.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T07:46:59.912 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:46:59.912 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:46:59.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T07:46:59.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T07:46:59.912 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:47:00.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:47:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:00.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:00.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T07:47:00.142 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 2e:0f:e4:20:b9:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.143 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::2c0f:e4ff:fe20:b914/64 scope link 2024-09-10T07:47:00.144 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:00.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:47:00.202 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 2e:0f:e4:20:b9:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T07:47:00.203 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T07:47:00.203 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.203 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::2c0f:e4ff:fe20:b914/64 scope link 2024-09-10T07:47:00.203 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:00.205 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T07:47:00.205 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:47:00.205 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:47:00.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T07:47:00.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T07:47:00.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T07:47:00.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T07:47:00.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T07:47:00.206 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:47:00.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:47:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:00.421 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:47:00.421 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T07:47:00.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.1 up 2024-09-10T07:47:00.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T07:47:00.421 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T07:47:00.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:47:00.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:00.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:47:00.587 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T07:47:00.622 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T07:47:00.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:47:00.623 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T07:47:00.698 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:47:00.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:00.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T07:47:00.826 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T07:47:00.826 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T07:47:00.826 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:47:00.842 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T07:47:00.842 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T07:47:00.842 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:47:00.842 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:47:00.842 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:invalid new device string format 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:47:00.865 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:47:00.866 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:47:00.867 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-10T07:47:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:47:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:47:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:47:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:47:00.871 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:47:00.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:00.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T07:47:00.927 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:47:01.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:47:01.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:01.014 INFO:teuthology.orchestra.run.smithi070.stdout:client4621 2024-09-10T07:47:01.015 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:01.015 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:01.050 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:47:01.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:47:01.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:01.179 INFO:teuthology.orchestra.run.smithi070.stdout:client4621 2024-09-10T07:47:01.180 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:47:01.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:47:01.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:01.269 INFO:teuthology.orchestra.run.smithi070.stdout:client4621 2024-09-10T07:47:01.269 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46339922-75e0-4945-8bb6-1acaef196937.client4621/status') 2024-09-10T07:47:01.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:47:01.368 INFO:teuthology.orchestra.run.smithi070.stdout:instance: client.4621 (3)192.168.0.2:0/2735182133 2024-09-10T07:47:01.368 INFO:teuthology.orchestra.run.smithi070.stdout:blocklisted: false 2024-09-10T07:47:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records in 2024-09-10T07:47:01.369 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records out 2024-09-10T07:47:01.369 INFO:teuthology.orchestra.run.smithi070.stderr:69 bytes copied, 5.1535e-05 s, 1.3 MB/s 2024-09-10T07:47:01.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T07:47:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:01.379 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T07:47:02.489 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:47:02.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.658+0000 7f69ce200640 1 Processor -- start 2024-09-10T07:47:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.658+0000 7f69ce200640 1 -- start start 2024-09-10T07:47:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69ce200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69c8160af0 0x7f69c80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69ce200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c80cfdd0 con 0x7f69c8160710 2024-09-10T07:47:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69ce200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c815cc90 con 0x7f69c80d05f0 2024-09-10T07:47:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69ce200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f69c815cf70 con 0x7f69c8160af0 2024-09-10T07:47:02.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69c8160af0 0x7f69c80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:02.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f69c8160af0 0x7f69c80d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44490/0 (socket says 172.21.15.3:44490) 2024-09-10T07:47:02.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 -- 172.21.15.3:0/1957631168 learned_addr learned my addr 172.21.15.3:0/1957631168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:02.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 -- 172.21.15.3:0/1957631168 >> v1:172.21.15.70:6789/0 conn(0x7f69c80d05f0 legacy=0x7f69c8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 -- 172.21.15.3:0/1957631168 >> v1:172.21.15.3:6789/0 conn(0x7f69c8160710 legacy=0x7f69c80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.660+0000 7f69bfe00640 1 -- 172.21.15.3:0/1957631168 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c815d250 con 0x7f69c8160af0 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.661+0000 7f69bfe00640 1 --2- 172.21.15.3:0/1957631168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c8160af0 0x7f69c80d00b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009870 tx=0x7f69b002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1138310cbb582813 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69c6a00640 1 -- 172.21.15.3:0/1957631168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0037070 con 0x7f69c8160af0 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69c6a00640 1 -- 172.21.15.3:0/1957631168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b0032440 con 0x7f69c8160af0 2024-09-10T07:47:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69c6a00640 1 -- 172.21.15.3:0/1957631168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0032810 con 0x7f69c8160af0 2024-09-10T07:47:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 -- 172.21.15.3:0/1957631168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c8160af0 msgr2=0x7f69c80d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 --2- 172.21.15.3:0/1957631168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c8160af0 0x7f69c80d00b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009870 tx=0x7f69b002a2a0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 -- 172.21.15.3:0/1957631168 shutdown_connections 2024-09-10T07:47:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 --2- 172.21.15.3:0/1957631168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c8160af0 0x7f69c80d00b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 -- 172.21.15.3:0/1957631168 >> 172.21.15.3:0/1957631168 conn(0x7f69c806d000 msgr2=0x7f69c806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 -- 172.21.15.3:0/1957631168 shutdown_connections 2024-09-10T07:47:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.662+0000 7f69ce200640 1 -- 172.21.15.3:0/1957631168 wait complete. 2024-09-10T07:47:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.664+0000 7f69ce200640 1 Processor -- start 2024-09-10T07:47:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.664+0000 7f69ce200640 1 -- start start 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69c8160710 0x7f69c81f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69c8160af0 0x7f69c815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f69c815d770 con 0x7f69c80d05f0 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f69c815c960 con 0x7f69c8160af0 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69ce200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f69c80b4ca0 con 0x7f69c8160710 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44506/0 (socket says 172.21.15.3:44506) 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69c7400640 1 -- 172.21.15.3:0/3988316751 learned_addr learned my addr 172.21.15.3:0/3988316751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69bfe00640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69c8160710 0x7f69c81f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.665+0000 7f69c7e00640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69c8160af0 0x7f69c815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 -- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69c8160710 msgr2=0x7f69c81f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69c8160710 0x7f69c81f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 -- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69c8160af0 msgr2=0x7f69c815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69c8160af0 0x7f69c815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 -- 172.21.15.3:0/3988316751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c80b3910 con 0x7f69c80d05f0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69c7400640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c400 tx=0x7f69b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80052d0 con 0x7f69c80d05f0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b8005490 con 0x7f69c80d05f0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8007260 con 0x7f69c80d05f0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c80b3bf0 con 0x7f69c80d05f0 2024-09-10T07:47:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.666+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c80b41b0 con 0x7f69c80d05f0 2024-09-10T07:47:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.668+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c80cfaf0 con 0x7f69c80d05f0 2024-09-10T07:47:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.668+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69b8007400 con 0x7f69c80d05f0 2024-09-10T07:47:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.669+0000 7f69bf400640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 0x7f69a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.669+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69b808a390 con 0x7f69c80d05f0 2024-09-10T07:47:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.669+0000 7f69bfe00640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 0x7f69a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.670+0000 7f69bfe00640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 0x7f69a406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69b0034000 tx=0x7f69b002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.674+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69b8058650 con 0x7f69c80d05f0 2024-09-10T07:47:02.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.870+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69c815d8f0 con 0x7f69c80d05f0 2024-09-10T07:47:02.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.871+0000 7f69bf400640 1 -- 172.21.15.3:0/3988316751 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f69b805c950 con 0x7f69c80d05f0 2024-09-10T07:47:02.871 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:02.871 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 msgr2=0x7f69a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 0x7f69a406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69b0034000 tx=0x7f69b002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 msgr2=0x7f69c80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c400 tx=0x7f69b800c8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 shutdown_connections 2024-09-10T07:47:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f69a406be30 0x7f69a406e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69c8160af0 0x7f69c815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f69c8160710 0x7f69c81f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 --2- 172.21.15.3:0/3988316751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f69c80d05f0 0x7f69c80b4ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 >> 172.21.15.3:0/3988316751 conn(0x7f69c806d000 msgr2=0x7f69c80b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 shutdown_connections 2024-09-10T07:47:02.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:02.873+0000 7f69ce200640 1 -- 172.21.15.3:0/3988316751 wait complete. 2024-09-10T07:47:02.890 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T07:46:28.671217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T07:46:29.630404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:47:02.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:47:03.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.051+0000 7fb7aa000640 1 Processor -- start 2024-09-10T07:47:03.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.051+0000 7fb7aa000640 1 -- start start 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.052+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 0x7fb7a415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.052+0000 7fb7aa000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7a40cfcf0 0x7fb7a40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7aa000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7a40ce850 con 0x7fb7a415f940 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7aa000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a40d06a0 con 0x7fb7a40cfcf0 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7aa000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a40ceb30 con 0x7fb7a415feb0 2024-09-10T07:47:03.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a8c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb7a415f940 0x7fb7a40ce040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56734/0 (socket says 172.21.15.3:56734) 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a8c00640 1 -- 172.21.15.3:0/670311633 learned_addr learned my addr 172.21.15.3:0/670311633 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3e00640 1 --2- 172.21.15.3:0/670311633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 0x7fb7a415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a9600640 1 --2- 172.21.15.3:0/670311633 >> v2:172.21.15.70:3300/0 conn(0x7fb7a40cfcf0 0x7fb7a40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3400640 1 -- 172.21.15.3:0/670311633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800227648 0 0) 0x7fb7a40ce850 con 0x7fb7a415f940 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3400640 1 -- 172.21.15.3:0/670311633 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb78c003520 con 0x7fb7a415f940 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3e00640 1 -- 172.21.15.3:0/670311633 >> v2:172.21.15.70:3300/0 conn(0x7fb7a40cfcf0 msgr2=0x7fb7a40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3e00640 1 --2- 172.21.15.3:0/670311633 >> v2:172.21.15.70:3300/0 conn(0x7fb7a40cfcf0 0x7fb7a40d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3e00640 1 -- 172.21.15.3:0/670311633 >> v1:172.21.15.3:6789/0 conn(0x7fb7a415f940 legacy=0x7fb7a40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.053+0000 7fb7a3e00640 1 -- 172.21.15.3:0/670311633 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a40cee10 con 0x7fb7a415feb0 2024-09-10T07:47:03.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7a3e00640 1 --2- 172.21.15.3:0/670311633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 0x7fb7a415d9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097a0 tx=0x7fb79002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a56501f47f5b176 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7a3400640 1 -- 172.21.15.3:0/670311633 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790037070 con 0x7fb7a415feb0 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7a3400640 1 -- 172.21.15.3:0/670311633 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79002adb0 con 0x7fb7a415feb0 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7a3400640 1 -- 172.21.15.3:0/670311633 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790032660 con 0x7fb7a415feb0 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 -- 172.21.15.3:0/670311633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 msgr2=0x7fb7a415d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 --2- 172.21.15.3:0/670311633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 0x7fb7a415d9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097a0 tx=0x7fb79002a060 comp rx=0 tx=0).stop 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 -- 172.21.15.3:0/670311633 shutdown_connections 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 --2- 172.21.15.3:0/670311633 >> v2:172.21.15.70:3300/0 conn(0x7fb7a40cfcf0 0x7fb7a40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 --2- 172.21.15.3:0/670311633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a415feb0 0x7fb7a415d9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.055+0000 7fb7aa000640 1 -- 172.21.15.3:0/670311633 >> 172.21.15.3:0/670311633 conn(0x7fb7a40b1e90 msgr2=0x7fb7a40b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.056+0000 7fb7aa000640 1 -- 172.21.15.3:0/670311633 shutdown_connections 2024-09-10T07:47:03.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.056+0000 7fb7aa000640 1 -- 172.21.15.3:0/670311633 wait complete. 2024-09-10T07:47:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.057+0000 7fb7aa000640 1 Processor -- start 2024-09-10T07:47:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.057+0000 7fb7aa000640 1 -- start start 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a40cfcf0 0x7fb7a40cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a415f940 0x7fb7a40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a4071ea0 con 0x7fb7a415feb0 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a415df10 con 0x7fb7a415f940 2024-09-10T07:47:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7aa000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a40726f0 con 0x7fb7a40cfcf0 2024-09-10T07:47:03.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7a9600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7a9600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44520/0 (socket says 172.21.15.3:44520) 2024-09-10T07:47:03.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.058+0000 7fb7a9600640 1 -- 172.21.15.3:0/2413223387 learned_addr learned my addr 172.21.15.3:0/2413223387 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:03.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a8c00640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a40cfcf0 0x7fb7a40cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a3e00640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a415f940 0x7fb7a40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 -- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a40cfcf0 msgr2=0x7fb7a40cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a40cfcf0 0x7fb7a40cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 -- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a415f940 msgr2=0x7fb7a40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a415f940 0x7fb7a40b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 -- 172.21.15.3:0/2413223387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a414d990 con 0x7fb7a415feb0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a9600640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00efc0 tx=0x7fb79c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.059+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c018020 con 0x7fb7a415feb0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.060+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c014480 con 0x7fb7a415feb0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.060+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c014870 con 0x7fb7a415feb0 2024-09-10T07:47:03.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.060+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a40cd650 con 0x7fb7a415feb0 2024-09-10T07:47:03.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.060+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a414d6b0 con 0x7fb7a415feb0 2024-09-10T07:47:03.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.061+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a40d0160 con 0x7fb7a415feb0 2024-09-10T07:47:03.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.062+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb79c005d10 con 0x7fb7a415feb0 2024-09-10T07:47:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.063+0000 7fb7a0c00640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 0x7fb78406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.063+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb79c08bb30 con 0x7fb7a415feb0 2024-09-10T07:47:03.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.063+0000 7fb7a8c00640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 0x7fb78406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.063+0000 7fb7a8c00640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 0x7fb78406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb798001200 tx=0x7fb798000bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.067+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb79c022030 con 0x7fb7a415feb0 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.257+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7a415aaf0 con 0x7fb7a415feb0 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.257+0000 7fb7a0c00640 1 -- 172.21.15.3:0/2413223387 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fb79c010040 con 0x7fb7a415feb0 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 msgr2=0x7fb78406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 0x7fb78406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb798001200 tx=0x7fb798000bc0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 msgr2=0x7fb7a40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00efc0 tx=0x7fb79c00c490 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 shutdown_connections 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb78406be30 0x7fb78406e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7a415feb0 0x7fb7a40b4720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a415f940 0x7fb7a40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 --2- 172.21.15.3:0/2413223387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7a40cfcf0 0x7fb7a40cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 >> 172.21.15.3:0/2413223387 conn(0x7fb7a40b1e90 msgr2=0x7fb7a40b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 shutdown_connections 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.260+0000 7fb7aa000640 1 -- 172.21.15.3:0/2413223387 wait complete. 2024-09-10T07:47:03.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T07:47:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.436+0000 7fc7a7600640 1 Processor -- start 2024-09-10T07:47:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.436+0000 7fc7a7600640 1 -- start start 2024-09-10T07:47:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a7600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7a015b230 0x7fc7a015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a7600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7a00b4c90 0x7fc7a0071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a7600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a015eff0 con 0x7fc7a0072c50 2024-09-10T07:47:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a7600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a015fbe0 con 0x7fc7a00b4c90 2024-09-10T07:47:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a7600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a015c8d0 con 0x7fc7a015b230 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a5600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7a00b4c90 0x7fc7a0071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a5600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7a00b4c90 0x7fc7a0071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44534/0 (socket says 172.21.15.3:44534) 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc7a5600640 1 -- 172.21.15.3:0/1997969773 learned_addr learned my addr 172.21.15.3:0/1997969773 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.437+0000 7fc79fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc7a0072c50 0x7fc7a00b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56740/0 (socket says 172.21.15.3:56740) 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.438+0000 7fc7a4c00640 1 --2- 172.21.15.3:0/1997969773 >> v2:172.21.15.70:3300/0 conn(0x7fc7a015b230 0x7fc7a015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.438+0000 7fc7a5600640 1 -- 172.21.15.3:0/1997969773 >> v2:172.21.15.70:3300/0 conn(0x7fc7a015b230 msgr2=0x7fc7a015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.438+0000 7fc7a5600640 1 --2- 172.21.15.3:0/1997969773 >> v2:172.21.15.70:3300/0 conn(0x7fc7a015b230 0x7fc7a015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.438+0000 7fc7a5600640 1 -- 172.21.15.3:0/1997969773 >> v1:172.21.15.3:6789/0 conn(0x7fc7a0072c50 legacy=0x7fc7a00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.438+0000 7fc7a5600640 1 -- 172.21.15.3:0/1997969773 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a015f2b0 con 0x7fc7a00b4c90 2024-09-10T07:47:03.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc7a5600640 1 --2- 172.21.15.3:0/1997969773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a00b4c90 0x7fc7a0071010 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00b65e0 tx=0x7fc79802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13a5c17f8025e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc79f400640 1 -- 172.21.15.3:0/1997969773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798036070 con 0x7fc7a00b4c90 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc79f400640 1 -- 172.21.15.3:0/1997969773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798031600 con 0x7fc7a00b4c90 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc79f400640 1 -- 172.21.15.3:0/1997969773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7980319d0 con 0x7fc7a00b4c90 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc7a7600640 1 -- 172.21.15.3:0/1997969773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a00b4c90 msgr2=0x7fc7a0071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.439+0000 7fc7a7600640 1 --2- 172.21.15.3:0/1997969773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a00b4c90 0x7fc7a0071010 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00b65e0 tx=0x7fc79802a730 comp rx=0 tx=0).stop 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 -- 172.21.15.3:0/1997969773 shutdown_connections 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 --2- 172.21.15.3:0/1997969773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a00b4c90 0x7fc7a0071010 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 --2- 172.21.15.3:0/1997969773 >> v2:172.21.15.70:3300/0 conn(0x7fc7a015b230 0x7fc7a015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 -- 172.21.15.3:0/1997969773 >> 172.21.15.3:0/1997969773 conn(0x7fc7a00b5cb0 msgr2=0x7fc7a006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 -- 172.21.15.3:0/1997969773 shutdown_connections 2024-09-10T07:47:03.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.440+0000 7fc7a7600640 1 -- 172.21.15.3:0/1997969773 wait complete. 2024-09-10T07:47:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.441+0000 7fc7a7600640 1 Processor -- start 2024-09-10T07:47:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.441+0000 7fc7a7600640 1 -- start start 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 0x7fc7a00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7a015b230 0x7fc7a015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a015b610 con 0x7fc7a0072c50 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a015f870 con 0x7fc7a00b4c90 2024-09-10T07:47:03.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.442+0000 7fc7a7600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a015b790 con 0x7fc7a015b230 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44538/0 (socket says 172.21.15.3:44538) 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 -- 172.21.15.3:0/3333735922 learned_addr learned my addr 172.21.15.3:0/3333735922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc79fe00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 0x7fc7a00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a5600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7a015b230 0x7fc7a015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 -- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7a015b230 msgr2=0x7fc7a015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7a015b230 0x7fc7a015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 -- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 msgr2=0x7fc7a00b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 0x7fc7a00b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 -- 172.21.15.3:0/3333735922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a01511e0 con 0x7fc7a0072c50 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc79fe00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 0x7fc7a00b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc7a4c00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d830 tx=0x7fc79400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.443+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79400b6c0 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.444+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc794019070 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.444+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79400ba80 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.444+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a014b4e0 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.444+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a014f330 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.445+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a006e1e0 con 0x7fc7a0072c50 2024-09-10T07:47:03.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.446+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc79401e460 con 0x7fc7a0072c50 2024-09-10T07:47:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.446+0000 7fc79cc00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 0x7fc78006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.446+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc79408b5d0 con 0x7fc7a0072c50 2024-09-10T07:47:03.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.447+0000 7fc79fe00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 0x7fc78006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.447+0000 7fc79fe00640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 0x7fc78006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc78c007fd0 tx=0x7fc78c007d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.451+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc794059ee0 con 0x7fc7a0072c50 2024-09-10T07:47:03.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.638+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7a015ced0 con 0x7fc7a0072c50 2024-09-10T07:47:03.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.639+0000 7fc79cc00640 1 -- 172.21.15.3:0/3333735922 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fc7940597e0 con 0x7fc7a0072c50 2024-09-10T07:47:03.639 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-10T07:47:03.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 msgr2=0x7fc78006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 0x7fc78006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc78c007fd0 tx=0x7fc78c007d60 comp rx=0 tx=0).stop 2024-09-10T07:47:03.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 msgr2=0x7fc7a014afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d830 tx=0x7fc79400dcf0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 shutdown_connections 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc78006be30 0x7fc78006e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7a015b230 0x7fc7a015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7a00b4c90 0x7fc7a00b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 --2- 172.21.15.3:0/3333735922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0072c50 0x7fc7a014afa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.641+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 >> 172.21.15.3:0/3333735922 conn(0x7fc7a00b5cb0 msgr2=0x7fc7a0151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.642+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 shutdown_connections 2024-09-10T07:47:03.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.642+0000 7fc7a7600640 1 -- 172.21.15.3:0/3333735922 wait complete. 2024-09-10T07:47:03.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T07:47:03.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.817+0000 7f033a400640 1 Processor -- start 2024-09-10T07:47:03.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.817+0000 7f033a400640 1 -- start start 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.818+0000 7f033a400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f03340d3b20 0x7f03340b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.818+0000 7f033a400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03341620f0 con 0x7f03340cf6a0 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.818+0000 7f033a400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03340d3f40 con 0x7f0334161d10 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.818+0000 7f033a400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0334072140 con 0x7f03340d3b20 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.818+0000 7f0333e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f03340d3b20 0x7f03340b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.819+0000 7f0332a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f03340cf6a0 0x7f0334072e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59638/0 (socket says 172.21.15.3:59638) 2024-09-10T07:47:03.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.819+0000 7f0332a00640 1 -- 172.21.15.3:0/3789592901 learned_addr learned my addr 172.21.15.3:0/3789592901 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.819+0000 7f0333e00640 1 -- 172.21.15.3:0/3789592901 >> v1:172.21.15.70:6789/0 conn(0x7f0334161d10 legacy=0x7f03340cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.819+0000 7f0333e00640 1 -- 172.21.15.3:0/3789592901 >> v1:172.21.15.3:6789/0 conn(0x7f03340cf6a0 legacy=0x7f0334072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.819+0000 7f0333e00640 1 -- 172.21.15.3:0/3789592901 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03340d4200 con 0x7f03340d3b20 2024-09-10T07:47:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.820+0000 7f0333e00640 1 --2- 172.21.15.3:0/3789592901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340d3b20 0x7f03340b76c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f032c009d90 tx=0x7f032c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fd54bbbf992513 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.820+0000 7f0332000640 1 -- 172.21.15.3:0/3789592901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c007740 con 0x7f03340d3b20 2024-09-10T07:47:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.820+0000 7f0332000640 1 -- 172.21.15.3:0/3789592901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c0078e0 con 0x7f03340d3b20 2024-09-10T07:47:03.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.820+0000 7f0332000640 1 -- 172.21.15.3:0/3789592901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c007cb0 con 0x7f03340d3b20 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 -- 172.21.15.3:0/3789592901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340d3b20 msgr2=0x7f03340b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 --2- 172.21.15.3:0/3789592901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340d3b20 0x7f03340b76c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f032c009d90 tx=0x7f032c02a660 comp rx=0 tx=0).stop 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 -- 172.21.15.3:0/3789592901 shutdown_connections 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 --2- 172.21.15.3:0/3789592901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340d3b20 0x7f03340b76c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 -- 172.21.15.3:0/3789592901 >> 172.21.15.3:0/3789592901 conn(0x7f03340d4730 msgr2=0x7f03340d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 -- 172.21.15.3:0/3789592901 shutdown_connections 2024-09-10T07:47:03.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.821+0000 7f033a400640 1 -- 172.21.15.3:0/3789592901 wait complete. 2024-09-10T07:47:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.822+0000 7f033a400640 1 Processor -- start 2024-09-10T07:47:03.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.823+0000 7f033a400640 1 -- start start 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03340d3b20 0x7f03340b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 0x7f03340b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0334072950 con 0x7f03340cf6a0 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0334072670 con 0x7f03340d3b20 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f033a400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f03340d3280 con 0x7f0334161d10 2024-09-10T07:47:03.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50374/0 (socket says 172.21.15.3:50374) 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 -- 172.21.15.3:0/3514264889 learned_addr learned my addr 172.21.15.3:0/3514264889 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0332a00640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03340d3b20 0x7f03340b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333e00640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 0x7f03340b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 -- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 msgr2=0x7f03340b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 0x7f03340b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 -- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03340d3b20 msgr2=0x7f03340b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03340d3b20 0x7f03340b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333400640 1 -- 172.21.15.3:0/3514264889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033415a6e0 con 0x7f03340cf6a0 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.824+0000 7f0333e00640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 0x7f03340b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:03.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f0333400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f032400c400 tx=0x7f032400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324017070 con 0x7f03340cf6a0 2024-09-10T07:47:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240052d0 con 0x7f03340cf6a0 2024-09-10T07:47:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03341f5910 con 0x7f03340cf6a0 2024-09-10T07:47:03.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324003830 con 0x7f03340cf6a0 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.825+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03340c82e0 con 0x7f03340cf6a0 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.826+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03340d3610 con 0x7f03340cf6a0 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.827+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f032401a070 con 0x7f03340cf6a0 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.828+0000 7f032b400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.828+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f032408a1b0 con 0x7f03340cf6a0 2024-09-10T07:47:03.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.828+0000 7f0332a00640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:03.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.829+0000 7f0332a00640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0334073e40 tx=0x7f0320009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:03.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:03.832+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0324058470 con 0x7f03340cf6a0 2024-09-10T07:47:04.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.015+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03340b8330 con 0x7f03340cf6a0 2024-09-10T07:47:04.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.016+0000 7f032b400640 1 -- 172.21.15.3:0/3514264889 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f032405c770 con 0x7f03340cf6a0 2024-09-10T07:47:04.015 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-10T07:47:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 msgr2=0x7f031406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 0x7f031406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0334073e40 tx=0x7f0320009040 comp rx=0 tx=0).stop 2024-09-10T07:47:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 msgr2=0x7f03340c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f032400c400 tx=0x7f032400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 shutdown_connections 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031406be30 0x7f031406e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0334161d10 0x7f03340b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03340d3b20 0x7f03340b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 --2- 172.21.15.3:0/3514264889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03340cf6a0 0x7f03340c8dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 >> 172.21.15.3:0/3514264889 conn(0x7f03340d4730 msgr2=0x7f03340cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 shutdown_connections 2024-09-10T07:47:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.018+0000 7f033a400640 1 -- 172.21.15.3:0/3514264889 wait complete. 2024-09-10T07:47:04.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-10T07:47:04.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.190+0000 7f7211200640 1 Processor -- start 2024-09-10T07:47:04.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.191+0000 7f7211200640 1 -- start start 2024-09-10T07:47:04.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 0x7f720c162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f720c0b52c0 0x7f720c0b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c073570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0b5d80 con 0x7f720c0b52c0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0b6060 con 0x7f720c0d0fd0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7211200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f720c1621a0 con 0x7f720c0730e0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7203e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f720c0b52c0 0x7f720c0b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7203e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f720c0b52c0 0x7f720c0b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50382/0 (socket says 172.21.15.3:50382) 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7203e00640 1 -- 172.21.15.3:0/1850093383 learned_addr learned my addr 172.21.15.3:0/1850093383 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f720a800640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.192+0000 7f7209e00640 1 --2- 172.21.15.3:0/1850093383 >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 0x7f720c162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7203e00640 1 -- 172.21.15.3:0/1850093383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 msgr2=0x7f720c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7203e00640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7203e00640 1 -- 172.21.15.3:0/1850093383 >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 msgr2=0x7f720c162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7203e00640 1 --2- 172.21.15.3:0/1850093383 >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 0x7f720c162c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7203e00640 1 -- 172.21.15.3:0/1850093383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c162480 con 0x7f720c0b52c0 2024-09-10T07:47:04.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.193+0000 7f7209e00640 1 --2- 172.21.15.3:0/1850093383 >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 0x7f720c162c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7203e00640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0b52c0 0x7f720c0b5720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f720c0d17d0 tx=0x7f71f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24369591e6515900 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7209400640 1 -- 172.21.15.3:0/1850093383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4037070 con 0x7f720c0b52c0 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7209400640 1 -- 172.21.15.3:0/1850093383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f4032440 con 0x7f720c0b52c0 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7209400640 1 -- 172.21.15.3:0/1850093383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4032810 con 0x7f720c0b52c0 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7211200640 1 -- 172.21.15.3:0/1850093383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0b52c0 msgr2=0x7f720c0b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.194+0000 7f7211200640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0b52c0 0x7f720c0b5720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f720c0d17d0 tx=0x7f71f402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 -- 172.21.15.3:0/1850093383 shutdown_connections 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 --2- 172.21.15.3:0/1850093383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0b52c0 0x7f720c0b5720 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 --2- 172.21.15.3:0/1850093383 >> v2:172.21.15.70:3300/0 conn(0x7f720c0d0fd0 0x7f720c162c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 -- 172.21.15.3:0/1850093383 >> 172.21.15.3:0/1850093383 conn(0x7f720c06bb20 msgr2=0x7f720c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 -- 172.21.15.3:0/1850093383 shutdown_connections 2024-09-10T07:47:04.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.195+0000 7f7211200640 1 -- 172.21.15.3:0/1850093383 wait complete. 2024-09-10T07:47:04.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.196+0000 7f7211200640 1 Processor -- start 2024-09-10T07:47:04.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.196+0000 7f7211200640 1 -- start start 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c0b52c0 0x7f720c1f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0d13b0 con 0x7f720c0d0fd0 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f720c06f250 con 0x7f720c0b52c0 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f7211200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f720c0b1420 con 0x7f720c0730e0 2024-09-10T07:47:04.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.197+0000 7f720a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50392/0 (socket says 172.21.15.3:50392) 2024-09-10T07:47:04.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 -- 172.21.15.3:0/898795312 learned_addr learned my addr 172.21.15.3:0/898795312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:04.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f7203e00640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c0b52c0 0x7f720c1f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f7209e00640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 -- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 msgr2=0x7f720c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 -- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c0b52c0 msgr2=0x7f720c1f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c0b52c0 0x7f720c1f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 -- 172.21.15.3:0/898795312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0b2520 con 0x7f720c0d0fd0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.198+0000 7f720a800640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7204007e40 tx=0x7f720400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.199+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204014620 con 0x7f720c0d0fd0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.199+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7204019070 con 0x7f720c0d0fd0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.199+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c0b2800 con 0x7f720c0d0fd0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.199+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204014a20 con 0x7f720c0d0fd0 2024-09-10T07:47:04.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.199+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c1f4c40 con 0x7f720c0d0fd0 2024-09-10T07:47:04.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.200+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c0b5720 con 0x7f720c0d0fd0 2024-09-10T07:47:04.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.201+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7204014bc0 con 0x7f720c0d0fd0 2024-09-10T07:47:04.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.202+0000 7f7202000640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.202+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f720408b5c0 con 0x7f720c0d0fd0 2024-09-10T07:47:04.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.202+0000 7f7209e00640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.202+0000 7f7209e00640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 0x7f71e806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00aa40 tx=0x7f71fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.206+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72040599b0 con 0x7f720c0d0fd0 2024-09-10T07:47:04.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.456+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7f720c06c340 con 0x7f720c0d0fd0 2024-09-10T07:47:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.557+0000 7f7202000640 1 -- 172.21.15.3:0/898795312 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v55) ==== 139+0+0 (secure 0 0 0) 0x7f720405dcb0 con 0x7f720c0d0fd0 2024-09-10T07:47:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.559+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 msgr2=0x7f71e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.559+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 0x7f71e806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00aa40 tx=0x7f71fc008040 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.559+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 msgr2=0x7f720c1f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.559+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7204007e40 tx=0x7f720400e4b0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 shutdown_connections 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f71e806be30 0x7f71e806e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f720c0d0fd0 0x7f720c1f5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c0b52c0 0x7f720c1f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 --2- 172.21.15.3:0/898795312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f720c0730e0 0x7f720c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 >> 172.21.15.3:0/898795312 conn(0x7f720c06bb20 msgr2=0x7f720c160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 shutdown_connections 2024-09-10T07:47:04.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.560+0000 7f7211200640 1 -- 172.21.15.3:0/898795312 wait complete. 2024-09-10T07:47:04.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:47:04.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.734+0000 7f71ae000640 1 Processor -- start 2024-09-10T07:47:04.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.734+0000 7f71ae000640 1 -- start start 2024-09-10T07:47:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a8161b70 0x7f71a80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 0x7f71a80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71a80d4ae0 0x7f71a80d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f71a8161f50 con 0x7f71a8161b70 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f71a80d3de0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71ae000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f71a80d40c0 con 0x7f71a80d3860 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71acc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a8161b70 0x7f71a80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71acc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a8161b70 0x7f71a80d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50400/0 (socket says 172.21.15.3:50400) 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.735+0000 7f71acc00640 1 -- 172.21.15.3:0/2427504191 learned_addr learned my addr 172.21.15.3:0/2427504191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71ad600640 1 --2- 172.21.15.3:0/2427504191 >> v2:172.21.15.70:3300/0 conn(0x7f71a80d4ae0 0x7f71a80d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71a7e00640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 0x7f71a80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71acc00640 1 -- 172.21.15.3:0/2427504191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 msgr2=0x7f71a80d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71acc00640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 0x7f71a80d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71acc00640 1 -- 172.21.15.3:0/2427504191 >> v2:172.21.15.70:3300/0 conn(0x7f71a80d4ae0 msgr2=0x7f71a80d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71acc00640 1 --2- 172.21.15.3:0/2427504191 >> v2:172.21.15.70:3300/0 conn(0x7f71a80d4ae0 0x7f71a80d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71acc00640 1 -- 172.21.15.3:0/2427504191 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a80cf690 con 0x7f71a8161b70 2024-09-10T07:47:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.736+0000 7f71a7e00640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 0x7f71a80d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.737+0000 7f71acc00640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a8161b70 0x7f71a80d3320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f71a80d28b0 tx=0x7f719c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f74f75e529149e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.737+0000 7f71a7400640 1 -- 172.21.15.3:0/2427504191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c037070 con 0x7f71a8161b70 2024-09-10T07:47:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.737+0000 7f71a7400640 1 -- 172.21.15.3:0/2427504191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719c032440 con 0x7f71a8161b70 2024-09-10T07:47:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.737+0000 7f71a7400640 1 -- 172.21.15.3:0/2427504191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c032810 con 0x7f71a8161b70 2024-09-10T07:47:04.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 -- 172.21.15.3:0/2427504191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a8161b70 msgr2=0x7f71a80d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a8161b70 0x7f71a80d3320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f71a80d28b0 tx=0x7f719c02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 -- 172.21.15.3:0/2427504191 shutdown_connections 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 --2- 172.21.15.3:0/2427504191 >> v2:172.21.15.70:3300/0 conn(0x7f71a80d4ae0 0x7f71a80d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a80d3860 0x7f71a80d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 --2- 172.21.15.3:0/2427504191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a8161b70 0x7f71a80d3320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 -- 172.21.15.3:0/2427504191 >> 172.21.15.3:0/2427504191 conn(0x7f71a8062630 msgr2=0x7f71a8062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 -- 172.21.15.3:0/2427504191 shutdown_connections 2024-09-10T07:47:04.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.738+0000 7f71ae000640 1 -- 172.21.15.3:0/2427504191 wait complete. 2024-09-10T07:47:04.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.739+0000 7f71ae000640 1 Processor -- start 2024-09-10T07:47:04.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 -- start start 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71a80d3860 0x7f71a814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a8161b70 0x7f71a80b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f71a80cf2d0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f71a80c9fc0 con 0x7f71a80d3860 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.740+0000 7f71ae000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f71a80cf450 con 0x7f71a8161b70 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50414/0 (socket says 172.21.15.3:50414) 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 -- 172.21.15.3:0/77886332 learned_addr learned my addr 172.21.15.3:0/77886332 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71acc00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71a80d3860 0x7f71a814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71ad600640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a8161b70 0x7f71a80b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 -- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a8161b70 msgr2=0x7f71a80b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a8161b70 0x7f71a80b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 -- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71a80d3860 msgr2=0x7f71a814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71a80d3860 0x7f71a814d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 -- 172.21.15.3:0/77886332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a80cb6e0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.741+0000 7f71a7e00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f719400c7f0 tx=0x7f719400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.742+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194015070 con 0x7f71a80d4ae0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.742+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7194007820 con 0x7f71a80d4ae0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.742+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194007c10 con 0x7f71a80d4ae0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.742+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a80cb9c0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.742+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a80cbf80 con 0x7f71a80d4ae0 2024-09-10T07:47:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.743+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a80b7540 con 0x7f71a80d4ae0 2024-09-10T07:47:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.744+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7194018070 con 0x7f71a80d4ae0 2024-09-10T07:47:04.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.745+0000 7f71a4c00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 0x7f718806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.745+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f719408aea0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.745+0000 7f71acc00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 0x7f718806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:04.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.746+0000 7f71acc00640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 0x7f718806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f719c034000 tx=0x7f719c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:04.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.749+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7194059210 con 0x7f71a80d4ae0 2024-09-10T07:47:04.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.932+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f71a815fdf0 con 0x7f71a80d4ae0 2024-09-10T07:47:04.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.936+0000 7f71a4c00640 1 -- 172.21.15.3:0/77886332 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f719405d510 con 0x7f71a80d4ae0 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 msgr2=0x7f718806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 0x7f718806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f719c034000 tx=0x7f719c02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 msgr2=0x7f71a80b9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f719400c7f0 tx=0x7f719400ccb0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 shutdown_connections 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f718806be30 0x7f718806e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71a8161b70 0x7f71a80b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a80d4ae0 0x7f71a80b9400 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 --2- 172.21.15.3:0/77886332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71a80d3860 0x7f71a814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:04.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 >> 172.21.15.3:0/77886332 conn(0x7f71a8062630 msgr2=0x7f71a815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:04.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 shutdown_connections 2024-09-10T07:47:04.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:04.938+0000 7f71ae000640 1 -- 172.21.15.3:0/77886332 wait complete. 2024-09-10T07:47:04.954 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:47:04.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:47:05.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:47:05.133 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:47:05.133 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:05.133 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:05.169 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:05.169 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:05.245 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:05.245 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:05.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:47:05.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.518+0000 7fea07a00640 1 Processor -- start 2024-09-10T07:47:05.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.518+0000 7fea07a00640 1 -- start start 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.519+0000 7fea07a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea000d08f0 0x7fea000ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.519+0000 7fea07a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000ccd80 con 0x7fea0015b110 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.519+0000 7fea07a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000cd060 con 0x7fea0015b4f0 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.519+0000 7fea07a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fea000b44f0 con 0x7fea000d08f0 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.519+0000 7fea05000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea0015b110 0x7fea000b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59644/0 (socket says 172.21.15.3:59644) 2024-09-10T07:47:05.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fea05000640 1 -- 172.21.15.3:0/2345791360 learned_addr learned my addr 172.21.15.3:0/2345791360 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fea05a00640 1 --2- 172.21.15.3:0/2345791360 >> v2:172.21.15.70:3300/0 conn(0x7fea000d08f0 0x7fea000ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423144103 0 0) 0x7fea000ccd80 con 0x7fea0015b110 2024-09-10T07:47:05.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9e8005730 con 0x7fea0015b110 2024-09-10T07:47:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fea05a00640 1 -- 172.21.15.3:0/2345791360 >> v1:172.21.15.70:6789/0 conn(0x7fea0015b4f0 legacy=0x7fea000d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431382112 0 0) 0x7fe9e8005730 con 0x7fea0015b110 2024-09-10T07:47:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fea05a00640 1 -- 172.21.15.3:0/2345791360 >> v1:172.21.15.3:6789/0 conn(0x7fea0015b110 legacy=0x7fea000b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.520+0000 7fea05a00640 1 -- 172.21.15.3:0/2345791360 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000cd340 con 0x7fea000d08f0 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.521+0000 7fea05a00640 1 --2- 172.21.15.3:0/2345791360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea000d08f0 0x7fea000ceb50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009d90 tx=0x7fe9f802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40866bf1704da31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4001670 con 0x7fea000d08f0 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8007540 con 0x7fea000d08f0 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fe9ff400640 1 -- 172.21.15.3:0/2345791360 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8007910 con 0x7fea000d08f0 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 -- 172.21.15.3:0/2345791360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea000d08f0 msgr2=0x7fea000ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 --2- 172.21.15.3:0/2345791360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea000d08f0 0x7fea000ceb50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009d90 tx=0x7fe9f802a680 comp rx=0 tx=0).stop 2024-09-10T07:47:05.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 -- 172.21.15.3:0/2345791360 shutdown_connections 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 --2- 172.21.15.3:0/2345791360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea000d08f0 0x7fea000ceb50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 -- 172.21.15.3:0/2345791360 >> 172.21.15.3:0/2345791360 conn(0x7fea0006d000 msgr2=0x7fea0006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 -- 172.21.15.3:0/2345791360 shutdown_connections 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.522+0000 7fea07a00640 1 -- 172.21.15.3:0/2345791360 wait complete. 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.524+0000 7fea07a00640 1 Processor -- start 2024-09-10T07:47:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.524+0000 7fea07a00640 1 -- start start 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0015b110 0x7fea00151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0015b4f0 0x7fea001f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea00073e90 con 0x7fea000d08f0 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fea00073d10 con 0x7fea0015b4f0 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea07a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea00074010 con 0x7fea0015b110 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea05000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea05000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50426/0 (socket says 172.21.15.3:50426) 2024-09-10T07:47:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.525+0000 7fea05000640 1 -- 172.21.15.3:0/3954576643 learned_addr learned my addr 172.21.15.3:0/3954576643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 -- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0015b110 msgr2=0x7fea00151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0015b110 0x7fea00151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 -- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0015b4f0 msgr2=0x7fea001f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0015b4f0 0x7fea001f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 -- 172.21.15.3:0/3954576643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea001f7ce0 con 0x7fea000d08f0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea05000640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400c750 tx=0x7fe9f400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4017070 con 0x7fea000d08f0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f4005ad0 con 0x7fea000d08f0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4007070 con 0x7fea000d08f0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.526+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea001f7fc0 con 0x7fea000d08f0 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.527+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea000729d0 con 0x7fea000d08f0 2024-09-10T07:47:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.528+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea000724e0 con 0x7fea000d08f0 2024-09-10T07:47:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.529+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9f401a070 con 0x7fea000d08f0 2024-09-10T07:47:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.530+0000 7fe9fcc00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.530+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe9f408a480 con 0x7fea000d08f0 2024-09-10T07:47:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.530+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.531+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 0x7fe9e006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000f60 tx=0x7fe9ec0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.534+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9f4058d90 con 0x7fea000d08f0 2024-09-10T07:47:05.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.729+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea000b3d10 con 0x7fea000d08f0 2024-09-10T07:47:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.730+0000 7fe9fcc00640 1 -- 172.21.15.3:0/3954576643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe9f4058690 con 0x7fea000d08f0 2024-09-10T07:47:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 msgr2=0x7fe9e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 0x7fe9e006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000f60 tx=0x7fe9ec0079b0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 msgr2=0x7fea00072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400c750 tx=0x7fe9f400cc10 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 shutdown_connections 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0015b4f0 0x7fea001f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0015b110 0x7fea00151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 --2- 172.21.15.3:0/3954576643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea000d08f0 0x7fea00072ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 >> 172.21.15.3:0/3954576643 conn(0x7fea0006d000 msgr2=0x7fea000ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 shutdown_connections 2024-09-10T07:47:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:05.732+0000 7fea07a00640 1 -- 172.21.15.3:0/3954576643 wait complete. 2024-09-10T07:47:05.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:47:05.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:05.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T07:47:05.808 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:47:05.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:05.808 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:47:05.858 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:05.858 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:05.898 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:05.898 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:05.977 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:05.977 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:06.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:47:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.212+0000 7fc984c00640 1 Processor -- start 2024-09-10T07:47:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.212+0000 7fc984c00640 1 -- start start 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc984c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc98015b110 0x7fc9800b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc984c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc9800cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc984c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9800b4250 con 0x7fc98015b4f0 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc984c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc9800b44f0 con 0x7fc98015b110 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc984c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9800ccd80 con 0x7fc9800d08f0 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc98015b110 0x7fc9800b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc98015b110 0x7fc9800b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50428/0 (socket says 172.21.15.3:50428) 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97de00640 1 -- 172.21.15.3:0/369949823 learned_addr learned my addr 172.21.15.3:0/369949823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97e800640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc9800cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97de00640 1 -- 172.21.15.3:0/369949823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 msgr2=0x7fc9800cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.214+0000 7fc97de00640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc9800cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.215+0000 7fc97de00640 1 -- 172.21.15.3:0/369949823 >> v1:172.21.15.70:6789/0 conn(0x7fc98015b4f0 legacy=0x7fc9800d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.215+0000 7fc97de00640 1 -- 172.21.15.3:0/369949823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9800cd060 con 0x7fc98015b110 2024-09-10T07:47:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.215+0000 7fc97e800640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc9800cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.215+0000 7fc97de00640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800b3d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9800b4670 tx=0x7fc97002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f11a35c61b2e5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc97d400640 1 -- 172.21.15.3:0/369949823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970037070 con 0x7fc98015b110 2024-09-10T07:47:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc97d400640 1 -- 172.21.15.3:0/369949823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970032440 con 0x7fc98015b110 2024-09-10T07:47:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc97d400640 1 -- 172.21.15.3:0/369949823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970032810 con 0x7fc98015b110 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 -- 172.21.15.3:0/369949823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 msgr2=0x7fc9800b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800b3d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9800b4670 tx=0x7fc97002a250 comp rx=0 tx=0).stop 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 -- 172.21.15.3:0/369949823 shutdown_connections 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc9800cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 --2- 172.21.15.3:0/369949823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800b3d10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.216+0000 7fc984c00640 1 -- 172.21.15.3:0/369949823 >> 172.21.15.3:0/369949823 conn(0x7fc98006d000 msgr2=0x7fc98006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.217+0000 7fc984c00640 1 -- 172.21.15.3:0/369949823 shutdown_connections 2024-09-10T07:47:06.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.217+0000 7fc984c00640 1 -- 172.21.15.3:0/369949823 wait complete. 2024-09-10T07:47:06.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.218+0000 7fc984c00640 1 Processor -- start 2024-09-10T07:47:06.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.218+0000 7fc984c00640 1 -- start start 2024-09-10T07:47:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc980151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98015b4f0 0x7fc98014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc980073d10 con 0x7fc98015b110 2024-09-10T07:47:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9800cd620 con 0x7fc98015b4f0 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc984c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc980073e90 con 0x7fc9800d08f0 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc977e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc977e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50436/0 (socket says 172.21.15.3:50436) 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc977e00640 1 -- 172.21.15.3:0/866429381 learned_addr learned my addr 172.21.15.3:0/866429381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc97e800640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98015b4f0 0x7fc98014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.219+0000 7fc97de00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc980151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 -- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 msgr2=0x7fc980151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc980151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 -- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98015b4f0 msgr2=0x7fc98014bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98015b4f0 0x7fc98014bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 -- 172.21.15.3:0/866429381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc980071180 con 0x7fc98015b110 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc977e00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c470 tx=0x7fc96800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9680052d0 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc968005490 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968007260 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.220+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc980071460 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.221+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc980071a20 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.221+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9800b3d10 con 0x7fc98015b110 2024-09-10T07:47:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.222+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc968007400 con 0x7fc98015b110 2024-09-10T07:47:06.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.224+0000 7fc976000640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 0x7fc95c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.224+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc96808a260 con 0x7fc98015b110 2024-09-10T07:47:06.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.224+0000 7fc97de00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 0x7fc95c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.225+0000 7fc97de00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 0x7fc95c06e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc970034000 tx=0x7fc97002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.228+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc9680585b0 con 0x7fc98015b110 2024-09-10T07:47:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.421+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9800cc3d0 con 0x7fc98015b110 2024-09-10T07:47:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.422+0000 7fc976000640 1 -- 172.21.15.3:0/866429381 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc96805c8b0 con 0x7fc98015b110 2024-09-10T07:47:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 msgr2=0x7fc95c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 0x7fc95c06e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc970034000 tx=0x7fc97002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 msgr2=0x7fc9800aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c470 tx=0x7fc96800c930 comp rx=0 tx=0).stop 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 shutdown_connections 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc95c06be30 0x7fc95c06e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98015b4f0 0x7fc98014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc98015b110 0x7fc9800aeb10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 --2- 172.21.15.3:0/866429381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9800d08f0 0x7fc980151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 >> 172.21.15.3:0/866429381 conn(0x7fc98006d000 msgr2=0x7fc9800b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 shutdown_connections 2024-09-10T07:47:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.424+0000 7fc984c00640 1 -- 172.21.15.3:0/866429381 wait complete. 2024-09-10T07:47:06.440 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T07:47:06.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:06.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T07:47:06.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:47:06.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:06.505 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T07:47:06.549 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:06.549 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:06.586 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:06.586 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:06.660 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:06.660 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:06.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:47:06.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 Processor -- start 2024-09-10T07:47:06.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 -- start start 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18c015e4f0 0x7f18c015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c006bb20 0x7f18c015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c006d160 0x7f18c006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f18c006ccd0 con 0x7f18c006bb20 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f18c006db30 con 0x7f18c015e4f0 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.935+0000 7f18c4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f18c006c0c0 con 0x7f18c006d160 2024-09-10T07:47:06.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c006bb20 0x7f18c015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c006bb20 0x7f18c015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50444/0 (socket says 172.21.15.3:50444) 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 -- 172.21.15.3:0/1468003804 learned_addr learned my addr 172.21.15.3:0/1468003804 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18bde00640 1 --2- 172.21.15.3:0/1468003804 >> v2:172.21.15.70:3300/0 conn(0x7f18c015e4f0 0x7f18c015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18be800640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c006d160 0x7f18c006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 -- 172.21.15.3:0/1468003804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c006d160 msgr2=0x7f18c006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c006d160 0x7f18c006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 -- 172.21.15.3:0/1468003804 >> v2:172.21.15.70:3300/0 conn(0x7f18c015e4f0 msgr2=0x7f18c015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 --2- 172.21.15.3:0/1468003804 >> v2:172.21.15.70:3300/0 conn(0x7f18c015e4f0 0x7f18c015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.936+0000 7f18b7e00640 1 -- 172.21.15.3:0/1468003804 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c006c3a0 con 0x7f18c006bb20 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.937+0000 7f18b7e00640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c015fcb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18c00741e0 tx=0x7f18a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fd2e33d3d460ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18bd400640 1 -- 172.21.15.3:0/1468003804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8037070 con 0x7f18c006bb20 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18bd400640 1 -- 172.21.15.3:0/1468003804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a8032440 con 0x7f18c006bb20 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18bd400640 1 -- 172.21.15.3:0/1468003804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8032810 con 0x7f18c006bb20 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 -- 172.21.15.3:0/1468003804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 msgr2=0x7f18c015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c015fcb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18c00741e0 tx=0x7f18a802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 -- 172.21.15.3:0/1468003804 shutdown_connections 2024-09-10T07:47:06.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c006d160 0x7f18c006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 --2- 172.21.15.3:0/1468003804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c015fcb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 --2- 172.21.15.3:0/1468003804 >> v2:172.21.15.70:3300/0 conn(0x7f18c015e4f0 0x7f18c015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 -- 172.21.15.3:0/1468003804 >> 172.21.15.3:0/1468003804 conn(0x7f18c006e5b0 msgr2=0x7f18c006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 -- 172.21.15.3:0/1468003804 shutdown_connections 2024-09-10T07:47:06.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.938+0000 7f18c4a00640 1 -- 172.21.15.3:0/1468003804 wait complete. 2024-09-10T07:47:06.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.940+0000 7f18c4a00640 1 Processor -- start 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.940+0000 7f18c4a00640 1 -- start start 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 0x7f18c015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c015e4f0 0x7f18c0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18c015e8d0 con 0x7f18c006bb20 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f18c006c960 con 0x7f18c006d160 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18c4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f18c01601f0 con 0x7f18c015e4f0 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18bde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18bde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50450/0 (socket says 172.21.15.3:50450) 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18bde00640 1 -- 172.21.15.3:0/4232420930 learned_addr learned my addr 172.21.15.3:0/4232420930 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18b7e00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 0x7f18c015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18be800640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c015e4f0 0x7f18c0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.941+0000 7f18bde00640 1 -- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c015e4f0 msgr2=0x7f18c0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18bde00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c015e4f0 0x7f18c0158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18bde00640 1 -- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 msgr2=0x7f18c015b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18bde00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 0x7f18c015b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18bde00640 1 -- 172.21.15.3:0/4232420930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01f40a0 con 0x7f18c006bb20 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18b7e00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 0x7f18c015b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18bde00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18b000e870 tx=0x7f18b000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017070 con 0x7f18c006bb20 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b000c8e0 con 0x7f18c006bb20 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b000ccd0 con 0x7f18c006bb20 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c01e7360 con 0x7f18c006bb20 2024-09-10T07:47:06.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.942+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01e7920 con 0x7f18c006bb20 2024-09-10T07:47:06.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.943+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c015fcb0 con 0x7f18c006bb20 2024-09-10T07:47:06.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.944+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18b001a070 con 0x7f18c006bb20 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.945+0000 7f18b6000640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 0x7f189c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.945+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f18b008ac20 con 0x7f18c006bb20 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.945+0000 7f18b7e00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 0x7f189c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.946+0000 7f18b7e00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 0x7f189c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18a8034000 tx=0x7f18a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:06.949+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18b00595c0 con 0x7f18c006bb20 2024-09-10T07:47:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.144+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18c006f300 con 0x7f18c006bb20 2024-09-10T07:47:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.144+0000 7f18b6000640 1 -- 172.21.15.3:0/4232420930 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f18b0058ec0 con 0x7f18c006bb20 2024-09-10T07:47:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 msgr2=0x7f189c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 0x7f189c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18a8034000 tx=0x7f18a802e000 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 msgr2=0x7f18c01e6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18b000e870 tx=0x7f18b000ed30 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 shutdown_connections 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f189c06be30 0x7f189c06e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c015e4f0 0x7f18c0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c006d160 0x7f18c015b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 --2- 172.21.15.3:0/4232420930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c006bb20 0x7f18c01e6d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 >> 172.21.15.3:0/4232420930 conn(0x7f18c006e5b0 msgr2=0x7f18c00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 shutdown_connections 2024-09-10T07:47:07.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.147+0000 7f18c4a00640 1 -- 172.21.15.3:0/4232420930 wait complete. 2024-09-10T07:47:07.162 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T07:47:07.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:07.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T07:47:07.222 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:47:07.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:07.222 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T07:47:07.258 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:07.258 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:07.298 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:07.299 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:07.377 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:47:07.377 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:07.453 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:47:07.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.611+0000 7fbbe7c00640 1 Processor -- start 2024-09-10T07:47:07.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.611+0000 7fbbe7c00640 1 -- start start 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.612+0000 7fbbe7c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 0x7fbbe00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe7c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbe0160af0 0x7fbbe00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe7c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbe00cf800 0x7fbbe00d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe7c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe00cfdd0 con 0x7fbbe0160af0 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe7c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe015cc90 con 0x7fbbe00cf800 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe7c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe015cf70 con 0x7fbbe0160710 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbe0160af0 0x7fbbe00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbe0160af0 0x7fbbe00d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50462/0 (socket says 172.21.15.3:50462) 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 -- 172.21.15.3:0/3395330634 learned_addr learned my addr 172.21.15.3:0/3395330634 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe5200640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 0x7fbbe00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe5c00640 1 --2- 172.21.15.3:0/3395330634 >> v2:172.21.15.70:3300/0 conn(0x7fbbe00cf800 0x7fbbe00d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 -- 172.21.15.3:0/3395330634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 msgr2=0x7fbbe00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 0x7fbbe00cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 -- 172.21.15.3:0/3395330634 >> v2:172.21.15.70:3300/0 conn(0x7fbbe00cf800 msgr2=0x7fbbe00d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/3395330634 >> v2:172.21.15.70:3300/0 conn(0x7fbbe00cf800 0x7fbbe00d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbdfe00640 1 -- 172.21.15.3:0/3395330634 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe015d250 con 0x7fbbe0160af0 2024-09-10T07:47:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.613+0000 7fbbe5200640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 0x7fbbe00cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.614+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe0160af0 0x7fbbe00d00b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006df70 tx=0x7fbbcc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44324a06726c05ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.614+0000 7fbbdf400640 1 -- 172.21.15.3:0/3395330634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc037070 con 0x7fbbe0160af0 2024-09-10T07:47:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbdf400640 1 -- 172.21.15.3:0/3395330634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbcc032440 con 0x7fbbe0160af0 2024-09-10T07:47:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbdf400640 1 -- 172.21.15.3:0/3395330634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc032810 con 0x7fbbe0160af0 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 -- 172.21.15.3:0/3395330634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe0160af0 msgr2=0x7fbbe00d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe0160af0 0x7fbbe00d00b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006df70 tx=0x7fbbcc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 -- 172.21.15.3:0/3395330634 shutdown_connections 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/3395330634 >> v2:172.21.15.70:3300/0 conn(0x7fbbe00cf800 0x7fbbe00d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe0160af0 0x7fbbe00d00b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/3395330634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160710 0x7fbbe00cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 -- 172.21.15.3:0/3395330634 >> 172.21.15.3:0/3395330634 conn(0x7fbbe006d000 msgr2=0x7fbbe006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 -- 172.21.15.3:0/3395330634 shutdown_connections 2024-09-10T07:47:07.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.615+0000 7fbbe7c00640 1 -- 172.21.15.3:0/3395330634 wait complete. 2024-09-10T07:47:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.618+0000 7fbbe7c00640 1 Processor -- start 2024-09-10T07:47:07.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.618+0000 7fbbe7c00640 1 -- start start 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe0160710 0x7fbbe014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160af0 0x7fbbe0150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe015d770 con 0x7fbbe00cf800 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe006bb20 con 0x7fbbe0160710 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe0150ba0 con 0x7fbbe0160af0 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50470/0 (socket says 172.21.15.3:50470) 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 -- 172.21.15.3:0/2375510384 learned_addr learned my addr 172.21.15.3:0/2375510384 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe0160710 0x7fbbe014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160af0 0x7fbbe0150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 -- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160af0 msgr2=0x7fbbe0150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160af0 0x7fbbe0150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 -- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe0160710 msgr2=0x7fbbe014fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe0160710 0x7fbbe014fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 -- 172.21.15.3:0/2375510384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0150e80 con 0x7fbbe00cf800 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe5200640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e730 tx=0x7fbbd400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400f040 con 0x7fbbe00cf800 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd4004230 con 0x7fbbe00cf800 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe0151160 con 0x7fbbe00cf800 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400ee40 con 0x7fbbe00cf800 2024-09-10T07:47:07.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.619+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe0151720 con 0x7fbbe00cf800 2024-09-10T07:47:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.620+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe014e880 con 0x7fbbe00cf800 2024-09-10T07:47:07.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.621+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbd400c6d0 con 0x7fbbe00cf800 2024-09-10T07:47:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.622+0000 7fbbdcc00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 0x7fbbc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.623+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbbd408ab80 con 0x7fbbe00cf800 2024-09-10T07:47:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.623+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 0x7fbbc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:07.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.623+0000 7fbbdfe00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 0x7fbbc006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc034000 tx=0x7fbbcc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.627+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbbd4017020 con 0x7fbbe00cf800 2024-09-10T07:47:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.828+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbe00afd50 con 0x7fbbe00cf800 2024-09-10T07:47:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.828+0000 7fbbdcc00640 1 -- 172.21.15.3:0/2375510384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fbbd4058e40 con 0x7fbbe00cf800 2024-09-10T07:47:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:47:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.830+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 msgr2=0x7fbbc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.830+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 0x7fbbc006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc034000 tx=0x7fbbcc02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 msgr2=0x7fbbe014e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e730 tx=0x7fbbd400ebf0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 shutdown_connections 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbc006be30 0x7fbbc006e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbe0160af0 0x7fbbe0150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe0160710 0x7fbbe014fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 --2- 172.21.15.3:0/2375510384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbe00cf800 0x7fbbe014e2e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 >> 172.21.15.3:0/2375510384 conn(0x7fbbe006d000 msgr2=0x7fbbe006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 shutdown_connections 2024-09-10T07:47:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:07.832+0000 7fbbe7c00640 1 -- 172.21.15.3:0/2375510384 wait complete. 2024-09-10T07:47:07.846 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T07:47:07.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:07.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T07:47:07.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:47:07.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:07.910 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T07:47:07.946 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:47:07.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.103+0000 7fcab5a00640 1 Processor -- start 2024-09-10T07:47:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.103+0000 7fcab5a00640 1 -- start start 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcab015e780 0x7fcab00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcab015eb60 con 0x7fcab00d3680 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcab00d3b80 con 0x7fcab00d1b60 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcab00d1fd0 con 0x7fcab015e780 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcaafe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcab015e780 0x7fcab00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcab00d3680 0x7fcab00b7ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59646/0 (socket says 172.21.15.3:59646) 2024-09-10T07:47:08.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.104+0000 7fcab5000640 1 -- 172.21.15.3:0/268405538 learned_addr learned my addr 172.21.15.3:0/268405538 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.105+0000 7fcaafe00640 1 -- 172.21.15.3:0/268405538 >> v1:172.21.15.70:6789/0 conn(0x7fcab00d1b60 legacy=0x7fcab00d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.105+0000 7fcaafe00640 1 -- 172.21.15.3:0/268405538 >> v1:172.21.15.3:6789/0 conn(0x7fcab00d3680 legacy=0x7fcab00b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.105+0000 7fcaafe00640 1 -- 172.21.15.3:0/268405538 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab00d3e60 con 0x7fcab015e780 2024-09-10T07:47:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.105+0000 7fcaaea00640 1 -- 172.21.15.3:0/268405538 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628059694 0 0) 0x7fcab015eb60 con 0x7fcab00d3680 2024-09-10T07:47:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.106+0000 7fcaafe00640 1 --2- 172.21.15.3:0/268405538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab015e780 0x7fcab00d1620 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00097a0 tx=0x7fcaa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd84e21b0820a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.106+0000 7fcaaea00640 1 -- 172.21.15.3:0/268405538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0037070 con 0x7fcab015e780 2024-09-10T07:47:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.106+0000 7fcaaea00640 1 -- 172.21.15.3:0/268405538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa0032440 con 0x7fcab015e780 2024-09-10T07:47:08.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.106+0000 7fcaaea00640 1 -- 172.21.15.3:0/268405538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0032810 con 0x7fcab015e780 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 -- 172.21.15.3:0/268405538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab015e780 msgr2=0x7fcab00d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 --2- 172.21.15.3:0/268405538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab015e780 0x7fcab00d1620 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00097a0 tx=0x7fcaa002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 -- 172.21.15.3:0/268405538 shutdown_connections 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 --2- 172.21.15.3:0/268405538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab015e780 0x7fcab00d1620 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 -- 172.21.15.3:0/268405538 >> 172.21.15.3:0/268405538 conn(0x7fcab006bf40 msgr2=0x7fcab006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 -- 172.21.15.3:0/268405538 shutdown_connections 2024-09-10T07:47:08.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.107+0000 7fcab5a00640 1 -- 172.21.15.3:0/268405538 wait complete. 2024-09-10T07:47:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.109+0000 7fcab5a00640 1 Processor -- start 2024-09-10T07:47:08.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.109+0000 7fcab5a00640 1 -- start start 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 0x7fcab014eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 0x7fcab01e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00b7a40 con 0x7fcab00d3680 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00d4420 con 0x7fcab015e780 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcab01503e0 con 0x7fcab00d1b60 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50480/0 (socket says 172.21.15.3:50480) 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 -- 172.21.15.3:0/3190708229 learned_addr learned my addr 172.21.15.3:0/3190708229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaafe00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 0x7fcab014eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 -- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 msgr2=0x7fcab014eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 0x7fcab014eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5000640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 0x7fcab01e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 -- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 msgr2=0x7fcab01e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 0x7fcab01e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaaf400640 1 -- 172.21.15.3:0/3190708229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab014f7d0 con 0x7fcab00d3680 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcaafe00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 0x7fcab014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.110+0000 7fcab5000640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 0x7fcab01e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcaaf400640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c3f0 tx=0x7fca9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011790 con 0x7fcab00d3680 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9c017070 con 0x7fcab00d3680 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011b90 con 0x7fcab00d3680 2024-09-10T07:47:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab014fab0 con 0x7fcab00d3680 2024-09-10T07:47:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.111+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab0150070 con 0x7fcab00d3680 2024-09-10T07:47:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.112+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab00d1620 con 0x7fcab00d3680 2024-09-10T07:47:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.113+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca9c01c460 con 0x7fcab00d3680 2024-09-10T07:47:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.114+0000 7fcaa7e00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 0x7fca9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.114+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fca9c08ad70 con 0x7fcab00d3680 2024-09-10T07:47:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.114+0000 7fcaafe00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 0x7fca9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.115+0000 7fcaafe00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 0x7fca9006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034000 tx=0x7fcaa002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.119+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca9c059680 con 0x7fcab00d3680 2024-09-10T07:47:08.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.338+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcab0150560 con 0x7fcab00d3680 2024-09-10T07:47:08.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.339+0000 7fcaa7e00640 1 -- 172.21.15.3:0/3190708229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7fca9c058f80 con 0x7fcab00d3680 2024-09-10T07:47:08.338 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:08.338 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:47:08.338 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 msgr2=0x7fca9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 0x7fca9006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034000 tx=0x7fcaa002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 msgr2=0x7fcab00ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c3f0 tx=0x7fca9c00c8b0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 shutdown_connections 2024-09-10T07:47:08.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca9006be30 0x7fca9006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab015e780 0x7fcab01e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab00d3680 0x7fcab00ab160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 --2- 172.21.15.3:0/3190708229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcab00d1b60 0x7fcab014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 >> 172.21.15.3:0/3190708229 conn(0x7fcab006bf40 msgr2=0x7fcab00b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 shutdown_connections 2024-09-10T07:47:08.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.341+0000 7fcab5a00640 1 -- 172.21.15.3:0/3190708229 wait complete. 2024-09-10T07:47:08.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:08.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.513+0000 7fbaf4c00640 1 Processor -- start 2024-09-10T07:47:08.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.514+0000 7fbaf4c00640 1 -- start start 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaf4c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbaf00d0fd0 0x7fbaf0162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaf4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf00b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaf4c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf00b5d80 con 0x7fbaf00b52c0 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaf4c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf00b6060 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaf4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf01621a0 con 0x7fbaf00737f0 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaede00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbaf00d0fd0 0x7fbaf0162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaede00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbaf00d0fd0 0x7fbaf0162c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50484/0 (socket says 172.21.15.3:50484) 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaede00640 1 -- 172.21.15.3:0/3839675768 learned_addr learned my addr 172.21.15.3:0/3839675768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:08.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.515+0000 7fbaee800640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf00b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.516+0000 7fbaede00640 1 -- 172.21.15.3:0/3839675768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 msgr2=0x7fbaf00b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.516+0000 7fbaede00640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf00b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.516+0000 7fbaede00640 1 -- 172.21.15.3:0/3839675768 >> v1:172.21.15.70:6789/0 conn(0x7fbaf00b52c0 legacy=0x7fbaf00730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.516+0000 7fbaede00640 1 -- 172.21.15.3:0/3839675768 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0162480 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.517+0000 7fbaede00640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf0162c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72d151e64da7595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.517+0000 7fbaed400640 1 -- 172.21.15.3:0/3839675768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0037070 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.517+0000 7fbaed400640 1 -- 172.21.15.3:0/3839675768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae0032440 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.517+0000 7fbaed400640 1 -- 172.21.15.3:0/3839675768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0032810 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3839675768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00d0fd0 msgr2=0x7fbaf0162c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf0162c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002a270 comp rx=0 tx=0).stop 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3839675768 shutdown_connections 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf00b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/3839675768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf0162c80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3839675768 >> 172.21.15.3:0/3839675768 conn(0x7fbaf006bb20 msgr2=0x7fbaf006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3839675768 shutdown_connections 2024-09-10T07:47:08.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.518+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3839675768 wait complete. 2024-09-10T07:47:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.519+0000 7fbaf4c00640 1 Processor -- start 2024-09-10T07:47:08.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 -- start start 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf01e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf01f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf00d13b0 con 0x7fbaf00b52c0 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf006f250 con 0x7fbaf00d0fd0 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.520+0000 7fbaf4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf01ea520 con 0x7fbaf00737f0 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50492/0 (socket says 172.21.15.3:50492) 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 -- 172.21.15.3:0/1153812229 learned_addr learned my addr 172.21.15.3:0/1153812229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:08.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbaede00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf01e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 -- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 msgr2=0x7fbaf01e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbaee800640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf01f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf01e9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 -- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 msgr2=0x7fbaf01f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf01f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 -- 172.21.15.3:0/1153812229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf015bc50 con 0x7fbaf00b52c0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbaee800640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf01f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbaede00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf01e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.521+0000 7fbae7e00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c400 tx=0x7fbad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.522+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80052d0 con 0x7fbaf00b52c0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.522+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8005490 con 0x7fbaf00b52c0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.522+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf015bf30 con 0x7fbaf00b52c0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.522+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8007260 con 0x7fbaf00b52c0 2024-09-10T07:47:08.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.522+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf00b1b70 con 0x7fbaf00b52c0 2024-09-10T07:47:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.523+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf015c2d0 con 0x7fbaf00b52c0 2024-09-10T07:47:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.524+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbad8007400 con 0x7fbaf00b52c0 2024-09-10T07:47:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.525+0000 7fbae6000640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.525+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbad808a3b0 con 0x7fbaf00b52c0 2024-09-10T07:47:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.525+0000 7fbaede00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.526+0000 7fbaede00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbae0034000 tx=0x7fbae002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.529+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbad8058d50 con 0x7fbaf00b52c0 2024-09-10T07:47:08.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.751+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf00b5720 con 0x7fbaf00b52c0 2024-09-10T07:47:08.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.751+0000 7fbae6000640 1 -- 172.21.15.3:0/1153812229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7fbad8058650 con 0x7fbaf00b52c0 2024-09-10T07:47:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:08.750 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:47:08.750 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:47:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 msgr2=0x7fbacc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbae0034000 tx=0x7fbae002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 msgr2=0x7fbaf0159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c400 tx=0x7fbad800c8c0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 shutdown_connections 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf00d0fd0 0x7fbaf01f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf00b52c0 0x7fbaf0159230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/1153812229 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaf00737f0 0x7fbaf01e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 >> 172.21.15.3:0/1153812229 conn(0x7fbaf006bb20 msgr2=0x7fbaf00724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 shutdown_connections 2024-09-10T07:47:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.754+0000 7fbaf4c00640 1 -- 172.21.15.3:0/1153812229 wait complete. 2024-09-10T07:47:08.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.926+0000 7f1290800640 1 Processor -- start 2024-09-10T07:47:08.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.927+0000 7f1290800640 1 -- start start 2024-09-10T07:47:08.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f1290800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1288160710 0x7f12880cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f1290800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12880cfdd0 con 0x7f1288160af0 2024-09-10T07:47:08.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f1290800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128815cc90 con 0x7f12880d07c0 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f1290800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f128815cf70 con 0x7f1288160710 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f128d400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1288160af0 0x7f12880d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59662/0 (socket says 172.21.15.3:59662) 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f128d400640 1 -- 172.21.15.3:0/2403624932 learned_addr learned my addr 172.21.15.3:0/2403624932 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.928+0000 7f128de00640 1 --2- 172.21.15.3:0/2403624932 >> v2:172.21.15.70:3300/0 conn(0x7f1288160710 0x7f12880cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.929+0000 7f128ca00640 1 -- 172.21.15.3:0/2403624932 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215580656 0 0) 0x7f12880cfdd0 con 0x7f1288160af0 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.929+0000 7f128ca00640 1 -- 172.21.15.3:0/2403624932 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1270003520 con 0x7f1288160af0 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.929+0000 7f128de00640 1 -- 172.21.15.3:0/2403624932 >> v1:172.21.15.70:6789/0 conn(0x7f12880d07c0 legacy=0x7f1288072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.929+0000 7f128de00640 1 -- 172.21.15.3:0/2403624932 >> v1:172.21.15.3:6789/0 conn(0x7f1288160af0 legacy=0x7f12880d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.929+0000 7f128de00640 1 -- 172.21.15.3:0/2403624932 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128815d250 con 0x7f1288160710 2024-09-10T07:47:08.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.930+0000 7f128de00640 1 --2- 172.21.15.3:0/2403624932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1288160710 0x7f12880cf2c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097a0 tx=0x7f127c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd191cd92308c498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.930+0000 7f128ca00640 1 -- 172.21.15.3:0/2403624932 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c037070 con 0x7f1288160710 2024-09-10T07:47:08.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.930+0000 7f128ca00640 1 -- 172.21.15.3:0/2403624932 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c032440 con 0x7f1288160710 2024-09-10T07:47:08.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.930+0000 7f128ca00640 1 -- 172.21.15.3:0/2403624932 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c032810 con 0x7f1288160710 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 -- 172.21.15.3:0/2403624932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1288160710 msgr2=0x7f12880cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 --2- 172.21.15.3:0/2403624932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1288160710 0x7f12880cf2c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097a0 tx=0x7f127c02a230 comp rx=0 tx=0).stop 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 -- 172.21.15.3:0/2403624932 shutdown_connections 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 --2- 172.21.15.3:0/2403624932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1288160710 0x7f12880cf2c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 -- 172.21.15.3:0/2403624932 >> 172.21.15.3:0/2403624932 conn(0x7f128806d000 msgr2=0x7f128806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 -- 172.21.15.3:0/2403624932 shutdown_connections 2024-09-10T07:47:08.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.931+0000 7f1290800640 1 -- 172.21.15.3:0/2403624932 wait complete. 2024-09-10T07:47:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.932+0000 7f1290800640 1 Processor -- start 2024-09-10T07:47:08.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.933+0000 7f1290800640 1 -- start start 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.933+0000 7f1290800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12880d07c0 0x7f1288147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f1290800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 0x7f1288159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f1290800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f1290800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f128815d770 con 0x7f1288160af0 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f1290800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f12880cf800 con 0x7f12880d07c0 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f1290800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1288158c00 con 0x7f1288160710 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50500/0 (socket says 172.21.15.3:50500) 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 -- 172.21.15.3:0/1960778984 learned_addr learned my addr 172.21.15.3:0/1960778984 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:08.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128d400640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 0x7f1288159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128de00640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12880d07c0 0x7f1288147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 -- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 msgr2=0x7f1288159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 0x7f1288159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 -- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12880d07c0 msgr2=0x7f1288147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12880d07c0 0x7f1288147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128e800640 1 -- 172.21.15.3:0/1960778984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1288157ff0 con 0x7f1288160af0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.934+0000 7f128d400640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 0x7f1288159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f128e800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f128400c7a0 tx=0x7f128400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284003b70 con 0x7f1288160af0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284003d30 con 0x7f1288160af0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128400aa90 con 0x7f1288160af0 2024-09-10T07:47:08.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881582d0 con 0x7f1288160af0 2024-09-10T07:47:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.935+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1288158890 con 0x7f1288160af0 2024-09-10T07:47:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.936+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12881484f0 con 0x7f1288160af0 2024-09-10T07:47:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.937+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f128400ac30 con 0x7f1288160af0 2024-09-10T07:47:08.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.938+0000 7f1282000640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.938+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f128408ad20 con 0x7f1288160af0 2024-09-10T07:47:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.938+0000 7f128de00640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:08.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.939+0000 7f128de00640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f127c034000 tx=0x7f127c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:08.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:08.942+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12881484f0 con 0x7f1288160af0 2024-09-10T07:47:09.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.130+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f128815d8f0 con 0x7f1288160af0 2024-09-10T07:47:09.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.131+0000 7f1282000640 1 -- 172.21.15.3:0/1960778984 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f1284059760 con 0x7f1288160af0 2024-09-10T07:47:09.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:09.131 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.133+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 msgr2=0x7f126c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.133+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f127c034000 tx=0x7f127c02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.133+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 msgr2=0x7f1288157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.133+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f128400c7a0 tx=0x7f128400cc60 comp rx=0 tx=0).stop 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 shutdown_connections 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1288160af0 0x7f1288157610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1288160710 0x7f1288159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 --2- 172.21.15.3:0/1960778984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12880d07c0 0x7f1288147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 >> 172.21.15.3:0/1960778984 conn(0x7f128806d000 msgr2=0x7f12880b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 shutdown_connections 2024-09-10T07:47:09.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.134+0000 7f1290800640 1 -- 172.21.15.3:0/1960778984 wait complete. 2024-09-10T07:47:09.149 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:47:09.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:09.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.308+0000 7f6a4a600640 1 Processor -- start 2024-09-10T07:47:09.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.308+0000 7f6a4a600640 1 -- start start 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a4a600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a440d3c10 0x7f6a44073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a4a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d4050 0x7f6a44073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a4a600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440cfdc0 con 0x7f6a4415e860 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a4a600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440d3780 con 0x7f6a440d3c10 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a4a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6a440cee40 con 0x7f6a440d4050 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a49200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a4415e860 0x7f6a440cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59670/0 (socket says 172.21.15.3:59670) 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a49200640 1 -- 172.21.15.3:0/4179253803 learned_addr learned my addr 172.21.15.3:0/4179253803 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:09.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a43e00640 1 --2- 172.21.15.3:0/4179253803 >> v2:172.21.15.3:3300/0 conn(0x7f6a440d3c10 0x7f6a44073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a49c00640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d4050 0x7f6a44073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a43e00640 1 -- 172.21.15.3:0/4179253803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d4050 msgr2=0x7f6a44073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a43e00640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d4050 0x7f6a44073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.309+0000 7f6a43e00640 1 -- 172.21.15.3:0/4179253803 >> v1:172.21.15.3:6789/0 conn(0x7f6a4415e860 legacy=0x7f6a440cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.310+0000 7f6a43e00640 1 -- 172.21.15.3:0/4179253803 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440cf120 con 0x7f6a440d3c10 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.310+0000 7f6a43400640 1 -- 172.21.15.3:0/4179253803 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1679421817 0 0) 0x7f6a440cfdc0 con 0x7f6a4415e860 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a43e00640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d3c10 0x7f6a44073150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69519ba1b43c5575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a43400640 1 -- 172.21.15.3:0/4179253803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30037070 con 0x7f6a440d3c10 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a43400640 1 -- 172.21.15.3:0/4179253803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30032440 con 0x7f6a440d3c10 2024-09-10T07:47:09.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a43400640 1 -- 172.21.15.3:0/4179253803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30032810 con 0x7f6a440d3c10 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 -- 172.21.15.3:0/4179253803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d3c10 msgr2=0x7f6a44073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d3c10 0x7f6a44073150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a3002a180 comp rx=0 tx=0).stop 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 -- 172.21.15.3:0/4179253803 shutdown_connections 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d4050 0x7f6a44073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 --2- 172.21.15.3:0/4179253803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d3c10 0x7f6a44073150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.311+0000 7f6a4a600640 1 -- 172.21.15.3:0/4179253803 >> 172.21.15.3:0/4179253803 conn(0x7f6a440d4650 msgr2=0x7f6a440d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.312+0000 7f6a4a600640 1 -- 172.21.15.3:0/4179253803 shutdown_connections 2024-09-10T07:47:09.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.312+0000 7f6a4a600640 1 -- 172.21.15.3:0/4179253803 wait complete. 2024-09-10T07:47:09.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.313+0000 7f6a4a600640 1 Processor -- start 2024-09-10T07:47:09.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.313+0000 7f6a4a600640 1 -- start start 2024-09-10T07:47:09.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d3c10 0x7f6a440add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a4415e860 0x7f6a441502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440d2460 con 0x7f6a440d4050 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440cf400 con 0x7f6a4415e860 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a4a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6a440b7880 con 0x7f6a440d3c10 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50508/0 (socket says 172.21.15.3:50508) 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.314+0000 7f6a43e00640 1 -- 172.21.15.3:0/1754496270 learned_addr learned my addr 172.21.15.3:0/1754496270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a49200640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d3c10 0x7f6a440add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a49c00640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a4415e860 0x7f6a441502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 -- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d3c10 msgr2=0x7f6a440add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d3c10 0x7f6a440add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 -- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a4415e860 msgr2=0x7f6a441502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a4415e860 0x7f6a441502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 -- 172.21.15.3:0/1754496270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a44150800 con 0x7f6a440d4050 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a43e00640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30037050 con 0x7f6a440d4050 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30038040 con 0x7f6a440d4050 2024-09-10T07:47:09.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30006e10 con 0x7f6a440d4050 2024-09-10T07:47:09.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.315+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a44160bf0 con 0x7f6a440d4050 2024-09-10T07:47:09.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.316+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441611b0 con 0x7f6a440d4050 2024-09-10T07:47:09.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.317+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a440b8dc0 con 0x7f6a440d4050 2024-09-10T07:47:09.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.318+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a30030980 con 0x7f6a440d4050 2024-09-10T07:47:09.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.318+0000 7f6a40c00640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 0x7f6a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.318+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6a300aba70 con 0x7f6a440d4050 2024-09-10T07:47:09.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.319+0000 7f6a49200640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 0x7f6a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.319+0000 7f6a49200640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 0x7f6a2406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000f60 tx=0x7f6a38007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.323+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a3007a540 con 0x7f6a440d4050 2024-09-10T07:47:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.545+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a440b3790 con 0x7f6a440d4050 2024-09-10T07:47:09.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.546+0000 7f6a40c00640 1 -- 172.21.15.3:0/1754496270 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f6a30079e40 con 0x7f6a440d4050 2024-09-10T07:47:09.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:47:09.545 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 msgr2=0x7f6a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 0x7f6a2406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000f60 tx=0x7f6a38007c00 comp rx=0 tx=0).stop 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 msgr2=0x7f6a4414fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a300046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 shutdown_connections 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6a2406be30 0x7f6a2406e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a4415e860 0x7f6a441502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a440d4050 0x7f6a4414fd80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 --2- 172.21.15.3:0/1754496270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a440d3c10 0x7f6a440add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 >> 172.21.15.3:0/1754496270 conn(0x7f6a440d4650 msgr2=0x7f6a440d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:09.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 shutdown_connections 2024-09-10T07:47:09.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.548+0000 7f6a4a600640 1 -- 172.21.15.3:0/1754496270 wait complete. 2024-09-10T07:47:09.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:09.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.720+0000 7fad5fa00640 1 Processor -- start 2024-09-10T07:47:09.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.720+0000 7fad5fa00640 1 -- start start 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5fa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad58160710 0x7fad580cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5fa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad580cfdd0 con 0x7fad58160af0 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5fa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad5815cc90 con 0x7fad580d07c0 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5fa00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fad5815cf70 con 0x7fad58160710 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad58160710 0x7fad580cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad58160710 0x7fad580cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50512/0 (socket says 172.21.15.3:50512) 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad57e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fad58160af0 0x7fad580d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59682/0 (socket says 172.21.15.3:59682) 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 -- 172.21.15.3:0/1513341865 learned_addr learned my addr 172.21.15.3:0/1513341865 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 -- 172.21.15.3:0/1513341865 >> v1:172.21.15.70:6789/0 conn(0x7fad580d07c0 legacy=0x7fad58072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 -- 172.21.15.3:0/1513341865 >> v1:172.21.15.3:6789/0 conn(0x7fad58160af0 legacy=0x7fad580d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad5d000640 1 -- 172.21.15.3:0/1513341865 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5815d250 con 0x7fad58160710 2024-09-10T07:47:09.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.722+0000 7fad57400640 1 -- 172.21.15.3:0/1513341865 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685143987 0 0) 0x7fad580cfdd0 con 0x7fad58160af0 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad5d000640 1 --2- 172.21.15.3:0/1513341865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad58160710 0x7fad580cf2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3c54cc74d5852d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad57400640 1 -- 172.21.15.3:0/1513341865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c037070 con 0x7fad58160710 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad57400640 1 -- 172.21.15.3:0/1513341865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4c032440 con 0x7fad58160710 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad57400640 1 -- 172.21.15.3:0/1513341865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c032810 con 0x7fad58160710 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad5fa00640 1 -- 172.21.15.3:0/1513341865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad58160710 msgr2=0x7fad580cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.724+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1513341865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad58160710 0x7fad580cf2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c02a2a0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.725+0000 7fad5fa00640 1 -- 172.21.15.3:0/1513341865 shutdown_connections 2024-09-10T07:47:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.725+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1513341865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad58160710 0x7fad580cf2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.725+0000 7fad5fa00640 1 -- 172.21.15.3:0/1513341865 >> 172.21.15.3:0/1513341865 conn(0x7fad5806d000 msgr2=0x7fad5806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.725+0000 7fad5fa00640 1 -- 172.21.15.3:0/1513341865 shutdown_connections 2024-09-10T07:47:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.725+0000 7fad5fa00640 1 -- 172.21.15.3:0/1513341865 wait complete. 2024-09-10T07:47:09.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.726+0000 7fad5fa00640 1 Processor -- start 2024-09-10T07:47:09.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.726+0000 7fad5fa00640 1 -- start start 2024-09-10T07:47:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad58160710 0x7fad581552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58160af0 0x7fad5815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad5815d770 con 0x7fad580d07c0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad580cf800 con 0x7fad58160af0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5fa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad581557e0 con 0x7fad58160710 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50520/0 (socket says 172.21.15.3:50520) 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.727+0000 7fad5d000640 1 -- 172.21.15.3:0/1967273202 learned_addr learned my addr 172.21.15.3:0/1967273202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad57e00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad58160710 0x7fad581552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5da00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58160af0 0x7fad5815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 -- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad58160710 msgr2=0x7fad581552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad58160710 0x7fad581552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 -- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58160af0 msgr2=0x7fad5815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58160af0 0x7fad5815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 -- 172.21.15.3:0/1967273202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5815a890 con 0x7fad580d07c0 2024-09-10T07:47:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5d000640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c037040 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4c004250 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c03c670 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.728+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5815ab70 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.729+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5815b130 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.730+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad58074ed0 con 0x7fad580d07c0 2024-09-10T07:47:09.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.730+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad4c03a070 con 0x7fad580d07c0 2024-09-10T07:47:09.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.731+0000 7fad54c00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:09.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.731+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fad4c0ab7c0 con 0x7fad580d07c0 2024-09-10T07:47:09.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.732+0000 7fad57e00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:09.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.732+0000 7fad57e00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 0x7fad3806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad580b1d10 tx=0x7fad44007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:09.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.735+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad4c041020 con 0x7fad580d07c0 2024-09-10T07:47:09.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.955+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad580afd50 con 0x7fad580d07c0 2024-09-10T07:47:09.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.956+0000 7fad54c00640 1 -- 172.21.15.3:0/1967273202 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7fad4c07a160 con 0x7fad580d07c0 2024-09-10T07:47:09.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:09.955 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-10T07:46:34:408355+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/828208896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":828208896},{"type":"v1","addr":"172.21.15.3:6837","nonce":828208896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3865540053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3865540053},{"type":"v1","addr":"172.21.15.70:6834","nonce":3865540053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3551147375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3551147375},{"type":"v1","addr":"172.21.15.3:6836","nonce":3551147375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:46:30.640638+0000","modified":"2024-09-10T07:46:33.830367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1500431536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":1500431536},{"type":"v1","addr":"172.21.15.70:6835","nonce":1500431536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-09-10T07:47:09.956 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-10T07:47:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 msgr2=0x7fad3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 0x7fad3806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad580b1d10 tx=0x7fad44007c00 comp rx=0 tx=0).stop 2024-09-10T07:47:09.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 msgr2=0x7fad581521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 shutdown_connections 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad3806be30 0x7fad3806e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58160af0 0x7fad5815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad58160710 0x7fad581552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 --2- 172.21.15.3:0/1967273202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad580d07c0 0x7fad581521a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 >> 172.21.15.3:0/1967273202 conn(0x7fad5806d000 msgr2=0x7fad580ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.958+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 shutdown_connections 2024-09-10T07:47:09.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:09.959+0000 7fad5fa00640 1 -- 172.21.15.3:0/1967273202 wait complete. 2024-09-10T07:47:09.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:10.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.131+0000 7f961aa00640 1 Processor -- start 2024-09-10T07:47:10.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.131+0000 7f961aa00640 1 -- start start 2024-09-10T07:47:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f961aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f961415b110 0x7f96140b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f961aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96140d08f0 0x7f96140cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f961aa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96140b4250 con 0x7f961415b4f0 2024-09-10T07:47:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f961aa00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f96140b44f0 con 0x7f96140d08f0 2024-09-10T07:47:10.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f961aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f96140ccd80 con 0x7f961415b110 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9618a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96140d08f0 0x7f96140cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f960b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f961415b4f0 0x7f96140d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59694/0 (socket says 172.21.15.3:59694) 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f960b400640 1 -- 172.21.15.3:0/1396379311 learned_addr learned my addr 172.21.15.3:0/1396379311 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9613e00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f961415b110 0x7f96140b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9618a00640 1 -- 172.21.15.3:0/1396379311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f961415b110 msgr2=0x7f96140b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9618a00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f961415b110 0x7f96140b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9618a00640 1 -- 172.21.15.3:0/1396379311 >> v1:172.21.15.3:6789/0 conn(0x7f961415b4f0 legacy=0x7f96140d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.133+0000 7f9618a00640 1 -- 172.21.15.3:0/1396379311 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96140cd060 con 0x7f96140d08f0 2024-09-10T07:47:10.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.134+0000 7f9618a00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96140d08f0 0x7f96140cdb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f960c009d90 tx=0x7f960c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ddb648f25fcb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f9613400640 1 -- 172.21.15.3:0/1396379311 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c007740 con 0x7f96140d08f0 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f9613400640 1 -- 172.21.15.3:0/1396379311 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c0078e0 con 0x7f96140d08f0 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f9613400640 1 -- 172.21.15.3:0/1396379311 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c007cb0 con 0x7f96140d08f0 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 -- 172.21.15.3:0/1396379311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96140d08f0 msgr2=0x7f96140cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96140d08f0 0x7f96140cdb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f960c009d90 tx=0x7f960c02a660 comp rx=0 tx=0).stop 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 -- 172.21.15.3:0/1396379311 shutdown_connections 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96140d08f0 0x7f96140cdb40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 --2- 172.21.15.3:0/1396379311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f961415b110 0x7f96140b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 -- 172.21.15.3:0/1396379311 >> 172.21.15.3:0/1396379311 conn(0x7f961406d000 msgr2=0x7f961406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 -- 172.21.15.3:0/1396379311 shutdown_connections 2024-09-10T07:47:10.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.135+0000 7f961aa00640 1 -- 172.21.15.3:0/1396379311 wait complete. 2024-09-10T07:47:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.137+0000 7f961aa00640 1 Processor -- start 2024-09-10T07:47:10.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.137+0000 7f961aa00640 1 -- start start 2024-09-10T07:47:10.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96140d08f0 0x7f961414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b110 0x7f96141f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9614073d10 con 0x7f961415b4f0 2024-09-10T07:47:10.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f96140cd620 con 0x7f961415b110 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f961aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9614073e90 con 0x7f96140d08f0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f9618a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f9618a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50538/0 (socket says 172.21.15.3:50538) 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f9618a00640 1 -- 172.21.15.3:0/2993990796 learned_addr learned my addr 172.21.15.3:0/2993990796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f9613e00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96140d08f0 0x7f961414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.138+0000 7f960b400640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b110 0x7f96141f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 -- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96140d08f0 msgr2=0x7f961414aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96140d08f0 0x7f961414aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 -- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b110 msgr2=0x7f96141f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b110 0x7f96141f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 -- 172.21.15.3:0/2993990796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96141f7e40 con 0x7f961415b4f0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9618a00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f960c009eb0 tx=0x7f960c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c03f070 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c031bc0 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c009450 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96141f8120 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.139+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961414b3e0 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.140+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96140b3d10 con 0x7f961415b4f0 2024-09-10T07:47:10.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.141+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f960c03b410 con 0x7f961415b4f0 2024-09-10T07:47:10.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.142+0000 7f9610c00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 0x7f95f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.142+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f960c0ac500 con 0x7f961415b4f0 2024-09-10T07:47:10.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.142+0000 7f9613e00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 0x7f95f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.143+0000 7f9613e00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 0x7f95f006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9604009b40 tx=0x7f9604009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.146+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f960c07aea0 con 0x7f961415b4f0 2024-09-10T07:47:10.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.335+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96141f82f0 con 0x7f961415b4f0 2024-09-10T07:47:10.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.337+0000 7f9610c00640 1 -- 172.21.15.3:0/2993990796 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16597 (secure 0 0 0) 0x7f960c07a7a0 con 0x7f961415b4f0 2024-09-10T07:47:10.336 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:46:32.208062+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:46:28.671217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T07:46:29.630404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 msgr2=0x7f95f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 0x7f95f006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9604009b40 tx=0x7f9604009190 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 msgr2=0x7f9614159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f960c009eb0 tx=0x7f960c004a40 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 shutdown_connections 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95f006be30 0x7f95f006e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f961415b4f0 0x7f9614159a10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b110 0x7f96141f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 --2- 172.21.15.3:0/2993990796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96140d08f0 0x7f961414aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 >> 172.21.15.3:0/2993990796 conn(0x7f961406d000 msgr2=0x7f96140b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:10.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 shutdown_connections 2024-09-10T07:47:10.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.339+0000 7f961aa00640 1 -- 172.21.15.3:0/2993990796 wait complete. 2024-09-10T07:47:10.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:47:10.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.513+0000 7ff733600640 1 Processor -- start 2024-09-10T07:47:10.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.513+0000 7ff733600640 1 -- start start 2024-09-10T07:47:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff733600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff72c06c230 0x7ff72c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff733600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72c0b36a0 con 0x7ff72c161b30 2024-09-10T07:47:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff733600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72c0b3980 con 0x7ff72c15fdb0 2024-09-10T07:47:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff733600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff72c0b3c60 con 0x7ff72c06c230 2024-09-10T07:47:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff730c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff72c161b30 0x7ff72c15f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59706/0 (socket says 172.21.15.3:59706) 2024-09-10T07:47:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff730c00640 1 -- 172.21.15.3:0/3576258536 learned_addr learned my addr 172.21.15.3:0/3576258536 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff731600640 1 --2- 172.21.15.3:0/3576258536 >> v2:172.21.15.70:3300/0 conn(0x7ff72c06c230 0x7ff72c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397964147 0 0) 0x7ff72c0b36a0 con 0x7ff72c161b30 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.515+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff714003520 con 0x7ff72c161b30 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276998402 0 0) 0x7ff72c0b3980 con 0x7ff72c15fdb0 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff72c0b36a0 con 0x7ff72c15fdb0 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329042548 0 0) 0x7ff714003520 con 0x7ff72c161b30 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 >> v2:172.21.15.70:3300/0 conn(0x7ff72c06c230 msgr2=0x7ff72c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 --2- 172.21.15.3:0/3576258536 >> v2:172.21.15.70:3300/0 conn(0x7ff72c06c230 0x7ff72c0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 >> v1:172.21.15.70:6789/0 conn(0x7ff72c15fdb0 legacy=0x7ff72c06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c0b3f40 con 0x7ff72c161b30 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7ff720001670 con 0x7ff72c161b30 2024-09-10T07:47:10.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 >> v1:172.21.15.3:6789/0 conn(0x7ff72c161b30 legacy=0x7ff72c15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c0b3c60 con 0x7ff714004800 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff71400c040 con 0x7ff7140051c0 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff71400c240 con 0x7ff714003b40 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff72be00640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff730c00640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.516+0000 7ff731600640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff72be00640 1 -- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff714003b40 msgr2=0x7ff714003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff72be00640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff72be00640 1 -- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7140051c0 msgr2=0x7ff71400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:10.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff72be00640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff72be00640 1 -- 172.21.15.3:0/3576258536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff714003520 con 0x7ff714004800 2024-09-10T07:47:10.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.517+0000 7ff731600640 1 -- 172.21.15.3:0/3576258536 reap_dead start 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff72be00640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff714004c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7180018d0 tx=0x7ff714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dadb57a78158b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180014f0 con 0x7ff714004800 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7180052d0 con 0x7ff714004800 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff72b400640 1 -- 172.21.15.3:0/3576258536 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71802ac40 con 0x7ff714004800 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff733600640 1 -- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 msgr2=0x7ff714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff733600640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff714004c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7180018d0 tx=0x7ff714005830 comp rx=0 tx=0).stop 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff733600640 1 -- 172.21.15.3:0/3576258536 shutdown_connections 2024-09-10T07:47:10.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff733600640 1 --2- 172.21.15.3:0/3576258536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.518+0000 7ff733600640 1 -- 172.21.15.3:0/3576258536 >> 172.21.15.3:0/3576258536 conn(0x7ff72c06d000 msgr2=0x7ff72c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:10.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.519+0000 7ff733600640 1 -- 172.21.15.3:0/3576258536 shutdown_connections 2024-09-10T07:47:10.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.519+0000 7ff733600640 1 -- 172.21.15.3:0/3576258536 wait complete. 2024-09-10T07:47:10.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.520+0000 7ff733600640 1 Processor -- start 2024-09-10T07:47:10.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.520+0000 7ff733600640 1 -- start start 2024-09-10T07:47:10.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff72c154eb0 0x7ff72c145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff72c071710 0x7ff72c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c0aefa0 con 0x7ff714004800 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c161f10 con 0x7ff72c071710 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff733600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff72c0b3300 con 0x7ff72c154eb0 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff730c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff730c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50564/0 (socket says 172.21.15.3:50564) 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.521+0000 7ff730c00640 1 -- 172.21.15.3:0/2271566504 learned_addr learned my addr 172.21.15.3:0/2271566504 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff731600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff72c071710 0x7ff72c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff72be00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff72c154eb0 0x7ff72c145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 -- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff72c154eb0 msgr2=0x7ff72c145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff72c154eb0 0x7ff72c145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 -- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff72c071710 msgr2=0x7ff72c155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff72c071710 0x7ff72c155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 -- 172.21.15.3:0/2271566504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c0cedf0 con 0x7ff714004800 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff730c00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff720004f20 tx=0x7ff7200074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000f7b0 con 0x7ff714004800 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720018070 con 0x7ff714004800 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000fbb0 con 0x7ff714004800 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c0cf0d0 con 0x7ff714004800 2024-09-10T07:47:10.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.522+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c0cf690 con 0x7ff714004800 2024-09-10T07:47:10.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.524+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72c160220 con 0x7ff714004800 2024-09-10T07:47:10.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.524+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff72000fd50 con 0x7ff714004800 2024-09-10T07:47:10.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.526+0000 7ff728c00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 0x7ff70c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.526+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff72008adb0 con 0x7ff714004800 2024-09-10T07:47:10.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.526+0000 7ff72be00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 0x7ff70c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.527+0000 7ff72be00640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 0x7ff70c06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff718032000 tx=0x7ff718009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.530+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7200597f0 con 0x7ff714004800 2024-09-10T07:47:10.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.753+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff72c15b2b0 con 0x7ff714004800 2024-09-10T07:47:10.752 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:47:10.754+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:47:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.756+0000 7ff728c00640 1 -- 172.21.15.3:0/2271566504 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7ff7200590f0 con 0x7ff714004800 2024-09-10T07:47:10.754 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 msgr2=0x7ff70c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 0x7ff70c06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff718032000 tx=0x7ff718009030 comp rx=0 tx=0).stop 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 msgr2=0x7ff72c158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff720004f20 tx=0x7ff7200074f0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 shutdown_connections 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff70c06be30 0x7ff70c06e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff72c071710 0x7ff72c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff72c154eb0 0x7ff72c145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 --2- 172.21.15.3:0/2271566504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff714004800 0x7ff72c158780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 >> 172.21.15.3:0/2271566504 conn(0x7ff72c06d000 msgr2=0x7ff72c15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 shutdown_connections 2024-09-10T07:47:10.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.759+0000 7ff733600640 1 -- 172.21.15.3:0/2271566504 wait complete. 2024-09-10T07:47:10.769 INFO:tasks.ceph.mds.d.smithi070.stderr: -7098> 2024-09-10T07:46:36.186+0000 7f9fe8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:47:10.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:47:10.822 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:47:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.933+0000 7f5c45000640 1 Processor -- start 2024-09-10T07:47:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.933+0000 7f5c45000640 1 -- start start 2024-09-10T07:47:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.934+0000 7f5c45000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 0x7f5c401692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.934+0000 7f5c45000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4015faf0 con 0x7f5c40147ce0 2024-09-10T07:47:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.934+0000 7f5c45000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4015ed30 con 0x7f5c400aefb0 2024-09-10T07:47:10.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.934+0000 7f5c45000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4015eaf0 con 0x7f5c40072aa0 2024-09-10T07:47:10.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3de00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c40147ce0 0x7f5c400ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59710/0 (socket says 172.21.15.3:59710) 2024-09-10T07:47:10.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3de00640 1 -- 172.21.15.3:0/1745283401 learned_addr learned my addr 172.21.15.3:0/1745283401 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:10.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3e800640 1 --2- 172.21.15.3:0/1745283401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 0x7f5c401692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3e800640 1 -- 172.21.15.3:0/1745283401 >> v1:172.21.15.70:6789/0 conn(0x7f5c400aefb0 legacy=0x7f5c40072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3ca00640 1 -- 172.21.15.3:0/1745283401 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347086910 0 0) 0x7f5c4015faf0 con 0x7f5c40147ce0 2024-09-10T07:47:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3e800640 1 -- 172.21.15.3:0/1745283401 >> v1:172.21.15.3:6789/0 conn(0x7f5c40147ce0 legacy=0x7f5c400ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.935+0000 7f5c3e800640 1 -- 172.21.15.3:0/1745283401 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4015f010 con 0x7f5c40072aa0 2024-09-10T07:47:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.936+0000 7f5c3e800640 1 --2- 172.21.15.3:0/1745283401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 0x7f5c401692b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009dc0 tx=0x7f5c3802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863952c7a1f05964 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.937+0000 7f5c3ca00640 1 -- 172.21.15.3:0/1745283401 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38007740 con 0x7f5c40072aa0 2024-09-10T07:47:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.937+0000 7f5c3ca00640 1 -- 172.21.15.3:0/1745283401 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c380078e0 con 0x7f5c40072aa0 2024-09-10T07:47:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.937+0000 7f5c3ca00640 1 -- 172.21.15.3:0/1745283401 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38007cb0 con 0x7f5c40072aa0 2024-09-10T07:47:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.937+0000 7f5c45000640 1 -- 172.21.15.3:0/1745283401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 msgr2=0x7f5c401692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.937+0000 7f5c45000640 1 --2- 172.21.15.3:0/1745283401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 0x7f5c401692b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009dc0 tx=0x7f5c3802a5f0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.938+0000 7f5c45000640 1 -- 172.21.15.3:0/1745283401 shutdown_connections 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.938+0000 7f5c45000640 1 --2- 172.21.15.3:0/1745283401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c40072aa0 0x7f5c401692b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.938+0000 7f5c45000640 1 -- 172.21.15.3:0/1745283401 >> 172.21.15.3:0/1745283401 conn(0x7f5c400b5cb0 msgr2=0x7f5c4006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.938+0000 7f5c45000640 1 -- 172.21.15.3:0/1745283401 shutdown_connections 2024-09-10T07:47:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.938+0000 7f5c45000640 1 -- 172.21.15.3:0/1745283401 wait complete. 2024-09-10T07:47:10.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.939+0000 7f5c45000640 1 Processor -- start 2024-09-10T07:47:10.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.939+0000 7f5c45000640 1 -- start start 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c400aefb0 0x7f5c400cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c40147ce0 0x7f5c40146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4015f660 con 0x7f5c40072aa0 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400730c0 con 0x7f5c40147ce0 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c45000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5c401480c0 con 0x7f5c400aefb0 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.940+0000 7f5c3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50578/0 (socket says 172.21.15.3:50578) 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c400aefb0 0x7f5c400cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 -- 172.21.15.3:0/1323803 learned_addr learned my addr 172.21.15.3:0/1323803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3e800640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c40147ce0 0x7f5c40146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 -- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c400aefb0 msgr2=0x7f5c400cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c400aefb0 0x7f5c400cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 -- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c40147ce0 msgr2=0x7f5c40146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c40147ce0 0x7f5c40146020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 -- 172.21.15.3:0/1323803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c40145580 con 0x7f5c40072aa0 2024-09-10T07:47:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.941+0000 7f5c3de00640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c420 tx=0x7f5c3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.942+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300074e0 con 0x7f5c40072aa0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.942+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c300047e0 con 0x7f5c40072aa0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.942+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004c10 con 0x7f5c40072aa0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.942+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c400ce1b0 con 0x7f5c40072aa0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.942+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c40145b00 con 0x7f5c40072aa0 2024-09-10T07:47:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.943+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c40072390 con 0x7f5c40072aa0 2024-09-10T07:47:10.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.944+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c30011070 con 0x7f5c40072aa0 2024-09-10T07:47:10.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.945+0000 7f5c36000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 0x7f5c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:10.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.945+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7f5c3008a9e0 con 0x7f5c40072aa0 2024-09-10T07:47:10.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.945+0000 7f5c3d400640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 0x7f5c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.946+0000 7f5c3d400640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 0x7f5c2006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c401f78b0 tx=0x7f5c2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:10.949+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c30058c40 con 0x7f5c40072aa0 2024-09-10T07:47:11.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.170+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5c4015d3c0 con 0x7f5c40072aa0 2024-09-10T07:47:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.761+0000 7f5c36000640 1 -- 172.21.15.3:0/1323803 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f5c3005cf40 con 0x7f5c40072aa0 2024-09-10T07:47:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.764+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 msgr2=0x7f5c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.764+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 0x7f5c2006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c401f78b0 tx=0x7f5c2c009040 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.764+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 msgr2=0x7f5c400b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.764+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c420 tx=0x7f5c3000c8e0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 shutdown_connections 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c2006be30 0x7f5c2006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c40147ce0 0x7f5c40146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c400aefb0 0x7f5c400cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 --2- 172.21.15.3:0/1323803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40072aa0 0x7f5c400b02b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 >> 172.21.15.3:0/1323803 conn(0x7f5c400b5cb0 msgr2=0x7f5c400ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 shutdown_connections 2024-09-10T07:47:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.765+0000 7f5c45000640 1 -- 172.21.15.3:0/1323803 wait complete. 2024-09-10T07:47:11.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:47:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.944+0000 7fed55e00640 1 Processor -- start 2024-09-10T07:47:11.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.944+0000 7fed55e00640 1 -- start start 2024-09-10T07:47:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed55e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed5015e860 0x7fed500cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed55e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed500d3320 0x7fed500d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed55e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed500d38c0 con 0x7fed500d4ea0 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed55e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fed500b3f10 con 0x7fed500d3320 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed55e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed500d3ba0 con 0x7fed5015e860 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed47e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed500d3320 0x7fed500d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed47e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed500d3320 0x7fed500d4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50592/0 (socket says 172.21.15.3:50592) 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed47e00640 1 -- 172.21.15.3:0/2457462719 learned_addr learned my addr 172.21.15.3:0/2457462719 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.945+0000 7fed4ea00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed5015e860 0x7fed500cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.946+0000 7fed47e00640 1 -- 172.21.15.3:0/2457462719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed5015e860 msgr2=0x7fed500cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.946+0000 7fed47e00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed5015e860 0x7fed500cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.946+0000 7fed47e00640 1 -- 172.21.15.3:0/2457462719 >> v1:172.21.15.3:6789/0 conn(0x7fed500d4ea0 legacy=0x7fed500b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.946+0000 7fed47e00640 1 -- 172.21.15.3:0/2457462719 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed500d3e80 con 0x7fed500d3320 2024-09-10T07:47:11.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed47e00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed500d4960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed38009870 tx=0x7fed3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae86d7ab770ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed4e000640 1 -- 172.21.15.3:0/2457462719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed500d3320 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed4e000640 1 -- 172.21.15.3:0/2457462719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38032440 con 0x7fed500d3320 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed4e000640 1 -- 172.21.15.3:0/2457462719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38032810 con 0x7fed500d3320 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed55e00640 1 -- 172.21.15.3:0/2457462719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 msgr2=0x7fed500d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.947+0000 7fed55e00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed500d4960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed38009870 tx=0x7fed3802a250 comp rx=0 tx=0).stop 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 -- 172.21.15.3:0/2457462719 shutdown_connections 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed500d4960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 --2- 172.21.15.3:0/2457462719 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed5015e860 0x7fed500cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 -- 172.21.15.3:0/2457462719 >> 172.21.15.3:0/2457462719 conn(0x7fed50072560 msgr2=0x7fed50072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 -- 172.21.15.3:0/2457462719 shutdown_connections 2024-09-10T07:47:11.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.948+0000 7fed55e00640 1 -- 172.21.15.3:0/2457462719 wait complete. 2024-09-10T07:47:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.949+0000 7fed55e00640 1 Processor -- start 2024-09-10T07:47:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.949+0000 7fed55e00640 1 -- start start 2024-09-10T07:47:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 0x7fed50156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 0x7fed50157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed500cfc60 con 0x7fed500d3320 2024-09-10T07:47:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed500d4440 con 0x7fed5015e860 2024-09-10T07:47:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.950+0000 7fed55e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed500cfde0 con 0x7fed500d4ea0 2024-09-10T07:47:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50596/0 (socket says 172.21.15.3:50596) 2024-09-10T07:47:11.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 -- 172.21.15.3:0/3258496177 learned_addr learned my addr 172.21.15.3:0/3258496177 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed47e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 0x7fed50156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 -- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 msgr2=0x7fed50156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 0x7fed50156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 -- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 msgr2=0x7fed50157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4f400640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 0x7fed50157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 0x7fed50157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4ea00640 1 -- 172.21.15.3:0/3258496177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5015a6f0 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed47e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 0x7fed50156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.951+0000 7fed4f400640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 0x7fed50157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed4ea00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d5f0 tx=0x7fed4000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400125f0 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed40019070 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400129d0 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5015a9d0 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.952+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5015af90 con 0x7fed500d3320 2024-09-10T07:47:11.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.953+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed500d4960 con 0x7fed500d3320 2024-09-10T07:47:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.954+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed40012b70 con 0x7fed500d3320 2024-09-10T07:47:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.955+0000 7fed47400640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 0x7fed2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.955+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7fed4008b240 con 0x7fed500d3320 2024-09-10T07:47:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.955+0000 7fed47e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 0x7fed2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.956+0000 7fed47e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 0x7fed2c06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:11.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:11.959+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed40059b90 con 0x7fed500d3320 2024-09-10T07:47:12.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.145+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed50160500 con 0x7fed500d3320 2024-09-10T07:47:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.763+0000 7fed47400640 1 -- 172.21.15.3:0/3258496177 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fed40059490 con 0x7fed500d3320 2024-09-10T07:47:12.762 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.766+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 msgr2=0x7fed2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.766+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 0x7fed2c06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.766+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 msgr2=0x7fed50155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.766+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d5f0 tx=0x7fed4000dab0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 shutdown_connections 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed2c06be60 0x7fed2c06e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed5015e860 0x7fed50157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed500d4ea0 0x7fed50156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 --2- 172.21.15.3:0/3258496177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed500d3320 0x7fed50155900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 >> 172.21.15.3:0/3258496177 conn(0x7fed50072560 msgr2=0x7fed500b9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 shutdown_connections 2024-09-10T07:47:12.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.767+0000 7fed55e00640 1 -- 172.21.15.3:0/3258496177 wait complete. 2024-09-10T07:47:12.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:47:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.953+0000 7f5288c00640 1 Processor -- start 2024-09-10T07:47:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.953+0000 7f5288c00640 1 -- start start 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5288c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52840b3610 0x7f528415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5288c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52840cf4e0 con 0x7f528415e7e0 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5288c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52840cf7c0 con 0x7f52840ced40 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5288c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f52840d24d0 con 0x7f52840b3610 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5281400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f528415e7e0 0x7f52840b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59736/0 (socket says 172.21.15.3:59736) 2024-09-10T07:47:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.954+0000 7f5281400640 1 -- 172.21.15.3:0/2447779894 learned_addr learned my addr 172.21.15.3:0/2447779894 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5281e00640 1 --2- 172.21.15.3:0/2447779894 >> v2:172.21.15.70:3300/0 conn(0x7f52840b3610 0x7f528415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5280a00640 1 -- 172.21.15.3:0/2447779894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728631528 0 0) 0x7f52840cf4e0 con 0x7f528415e7e0 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5280a00640 1 -- 172.21.15.3:0/2447779894 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5268003520 con 0x7f528415e7e0 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5281e00640 1 -- 172.21.15.3:0/2447779894 >> v1:172.21.15.70:6789/0 conn(0x7f52840ced40 legacy=0x7f52840d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5281e00640 1 -- 172.21.15.3:0/2447779894 >> v1:172.21.15.3:6789/0 conn(0x7f528415e7e0 legacy=0x7f52840b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.955+0000 7f5281e00640 1 -- 172.21.15.3:0/2447779894 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52840cfaa0 con 0x7f52840b3610 2024-09-10T07:47:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.956+0000 7f5281e00640 1 --2- 172.21.15.3:0/2447779894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840b3610 0x7f528415f6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52740097d0 tx=0x7f527402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af0a5f56e7f03f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.956+0000 7f5280a00640 1 -- 172.21.15.3:0/2447779894 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274037070 con 0x7f52840b3610 2024-09-10T07:47:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.956+0000 7f5280a00640 1 -- 172.21.15.3:0/2447779894 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5274032440 con 0x7f52840b3610 2024-09-10T07:47:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.956+0000 7f5280a00640 1 -- 172.21.15.3:0/2447779894 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52740327f0 con 0x7f52840b3610 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 -- 172.21.15.3:0/2447779894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840b3610 msgr2=0x7f528415f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 --2- 172.21.15.3:0/2447779894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840b3610 0x7f528415f6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52740097d0 tx=0x7f527402a250 comp rx=0 tx=0).stop 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 -- 172.21.15.3:0/2447779894 shutdown_connections 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 --2- 172.21.15.3:0/2447779894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840b3610 0x7f528415f6c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 -- 172.21.15.3:0/2447779894 >> 172.21.15.3:0/2447779894 conn(0x7f5284072560 msgr2=0x7f5284072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 -- 172.21.15.3:0/2447779894 shutdown_connections 2024-09-10T07:47:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.957+0000 7f5288c00640 1 -- 172.21.15.3:0/2447779894 wait complete. 2024-09-10T07:47:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.959+0000 7f5288c00640 1 Processor -- start 2024-09-10T07:47:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.959+0000 7f5288c00640 1 -- start start 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52840b3610 0x7f52840cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840ced40 0x7f528414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52840d3070 con 0x7f528415e7e0 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52840cfd80 con 0x7f52840ced40 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5288c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52840d31f0 con 0x7f52840b3610 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50606/0 (socket says 172.21.15.3:50606) 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 -- 172.21.15.3:0/4154870509 learned_addr learned my addr 172.21.15.3:0/4154870509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5281400640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840ced40 0x7f528414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5281e00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52840b3610 0x7f52840cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 -- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52840b3610 msgr2=0x7f52840cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52840b3610 0x7f52840cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 -- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840ced40 msgr2=0x7f528414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840ced40 0x7f528414dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.960+0000 7f5282800640 1 -- 172.21.15.3:0/4154870509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528414e0f0 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f5282800640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c7a0 tx=0x7f527c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c003b70 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f527c003d30 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52841f7b00 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00aa90 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.961+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52841f80c0 con 0x7f528415e7e0 2024-09-10T07:47:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.962+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528415f6a0 con 0x7f528415e7e0 2024-09-10T07:47:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.963+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f527c00ac30 con 0x7f528415e7e0 2024-09-10T07:47:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.964+0000 7f527a000640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 0x7f526406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.965+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7f527c08ab90 con 0x7f528415e7e0 2024-09-10T07:47:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.965+0000 7f5281e00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 0x7f526406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:12.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.965+0000 7f5281e00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 0x7f526406e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5274034000 tx=0x7f527402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:12.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:12.969+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f527c058f90 con 0x7f528415e7e0 2024-09-10T07:47:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.153+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5284161280 con 0x7f528415e7e0 2024-09-10T07:47:13.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.768+0000 7f527a000640 1 -- 172.21.15.3:0/4154870509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f527c05d290 con 0x7f528415e7e0 2024-09-10T07:47:13.767 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:47:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.771+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 msgr2=0x7f526406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.771+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 0x7f526406e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5274034000 tx=0x7f527402e000 comp rx=0 tx=0).stop 2024-09-10T07:47:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.771+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 msgr2=0x7f5284160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.771+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c7a0 tx=0x7f527c00cc60 comp rx=0 tx=0).stop 2024-09-10T07:47:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 shutdown_connections 2024-09-10T07:47:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f526406be60 0x7f526406e320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f528415e7e0 0x7f5284160270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52840ced40 0x7f528414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 --2- 172.21.15.3:0/4154870509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52840b3610 0x7f52840cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.773+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 >> 172.21.15.3:0/4154870509 conn(0x7f5284072560 msgr2=0x7f52840b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.774+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 shutdown_connections 2024-09-10T07:47:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:13.774+0000 7f5288c00640 1 -- 172.21.15.3:0/4154870509 wait complete. 2024-09-10T07:47:13.796 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:47:13.796 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:47:13.797 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:47:13.797 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:47:13.797 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:47:13.797 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:47:13.797 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:47:13.797 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:47:13.798 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:47:13.798 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:47:13.798 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:47:13.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:47:13.798 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:47:13.798 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:47:13.799 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:47:13.799 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:47:13.813 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:47:13.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:47:13.814 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:47:13.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:47:13.816 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:47:13.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:47:13.858 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:47:13.858 DEBUG:teuthology.parallel:result is None 2024-09-10T07:47:13.860 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:47:13.860 DEBUG:teuthology.parallel:result is None 2024-09-10T07:47:13.861 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:47:13.861 DEBUG:teuthology.parallel:result is None 2024-09-10T07:47:13.991 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:47:13.992 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:47:13.993 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:47:16.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T07:47:19.817 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:47:19.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:47:19.822 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:47:19.823 DEBUG:teuthology.parallel:result is None 2024-09-10T07:47:19.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:47:19.956 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:47:19.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.992+0000 7f708f200640 1 Processor -- start 2024-09-10T07:47:19.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.992+0000 7f708f200640 1 -- start start 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f708f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 0x7f7088162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f708f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70880b70b0 con 0x7f70880d1c10 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f708f200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70880b78b0 con 0x7f708815e970 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f708f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70880b3bf0 con 0x7f70880b37a0 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f7087400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f70880d1c10 0x7f70880ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37040/0 (socket says 172.21.15.3:37040) 2024-09-10T07:47:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.993+0000 7f7087400640 1 -- 172.21.15.3:0/2287597974 learned_addr learned my addr 172.21.15.3:0/2287597974 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f708d200640 1 --2- 172.21.15.3:0/2287597974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 0x7f7088162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027163275 0 0) 0x7f7074001110 con 0x7f70880d1c10 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f706c003520 con 0x7f70880d1c10 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021351680 0 0) 0x7f7078001110 con 0x7f708815e970 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7074001110 con 0x7f708815e970 2024-09-10T07:47:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f708d200640 1 -- 172.21.15.3:0/2287597974 >> v1:172.21.15.70:6789/0 conn(0x7f708815e970 legacy=0x7f70880d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:19.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f708d200640 1 -- 172.21.15.3:0/2287597974 >> v1:172.21.15.3:6789/0 conn(0x7f70880d1c10 legacy=0x7f70880ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:19.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.994+0000 7f708d200640 1 -- 172.21.15.3:0/2287597974 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70880b62f0 con 0x7f70880b37a0 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.995+0000 7f708d200640 1 --2- 172.21.15.3:0/2287597974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 0x7f7088162cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7080009dc0 tx=0x7f708002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0fa0eebbe14e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080007740 con 0x7f70880b37a0 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70800078e0 con 0x7f70880b37a0 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f7086a00640 1 -- 172.21.15.3:0/2287597974 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080007cb0 con 0x7f70880b37a0 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 -- 172.21.15.3:0/2287597974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 msgr2=0x7f7088162cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 --2- 172.21.15.3:0/2287597974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 0x7f7088162cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7080009dc0 tx=0x7f708002a6d0 comp rx=0 tx=0).stop 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 -- 172.21.15.3:0/2287597974 shutdown_connections 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 --2- 172.21.15.3:0/2287597974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70880b37a0 0x7f7088162cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:19.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 -- 172.21.15.3:0/2287597974 >> 172.21.15.3:0/2287597974 conn(0x7f708806bd00 msgr2=0x7f708806c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:19.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 -- 172.21.15.3:0/2287597974 shutdown_connections 2024-09-10T07:47:19.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.996+0000 7f708f200640 1 -- 172.21.15.3:0/2287597974 wait complete. 2024-09-10T07:47:19.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.998+0000 7f708f200640 1 Processor -- start 2024-09-10T07:47:19.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.998+0000 7f708f200640 1 -- start start 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70880d1c10 0x7f70881f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f708815e970 0x7f70880cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70880b68b0 con 0x7f70880b37a0 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7088163210 con 0x7f70880d1c10 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f708f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70880b6b90 con 0x7f708815e970 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:19.999+0000 7f7087e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51200/0 (socket says 172.21.15.3:51200) 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 -- 172.21.15.3:0/1829915566 learned_addr learned my addr 172.21.15.3:0/1829915566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f708d200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f708815e970 0x7f70880cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087400640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70880d1c10 0x7f70881f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 -- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f708815e970 msgr2=0x7f70880cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f708815e970 0x7f70880cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 -- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70880d1c10 msgr2=0x7f70881f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70880d1c10 0x7f70881f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 -- 172.21.15.3:0/1829915566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70880cd050 con 0x7f70880b37a0 2024-09-10T07:47:20.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.000+0000 7f7087e00640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f707800c700 tx=0x7f707800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.001+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70780052d0 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.001+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078005470 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.001+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70880c9810 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.001+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800a860 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.001+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70880c9dd0 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.002+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7088162cd0 con 0x7f70880b37a0 2024-09-10T07:47:20.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.003+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f707800aa00 con 0x7f70880b37a0 2024-09-10T07:47:20.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.003+0000 7f707fe00640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:20.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.003+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4943+0+0 (secure 0 0 0) 0x7f707808a6d0 con 0x7f70880b37a0 2024-09-10T07:47:20.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.004+0000 7f7087400640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:20.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.004+0000 7f7087400640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 0x7f706806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70880d5740 tx=0x7f7074007d70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:20.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.008+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70880b78b0 con 0x7f70880b37a0 2024-09-10T07:47:20.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:20.203+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f70880b3510 con 0x7f70880b37a0 2024-09-10T07:47:20.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:47:20.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:47:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.554+0000 7f707fe00640 1 -- 172.21.15.3:0/1829915566 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f707805f070 con 0x7f70880b37a0 2024-09-10T07:47:21.554 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:47:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 msgr2=0x7f706806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 0x7f706806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70880d5740 tx=0x7f7074007d70 comp rx=0 tx=0).stop 2024-09-10T07:47:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 msgr2=0x7f7088151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f707800c700 tx=0x7f707800cbc0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 shutdown_connections 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f708815e970 0x7f70880cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70880d1c10 0x7f70881f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 --2- 172.21.15.3:0/1829915566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70880b37a0 0x7f7088151cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 >> 172.21.15.3:0/1829915566 conn(0x7f708806bd00 msgr2=0x7f70880b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 shutdown_connections 2024-09-10T07:47:21.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.558+0000 7f708f200640 1 -- 172.21.15.3:0/1829915566 wait complete. 2024-09-10T07:47:21.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:47:21.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.738+0000 7faf03400640 1 Processor -- start 2024-09-10T07:47:21.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.738+0000 7faf03400640 1 -- start start 2024-09-10T07:47:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 0x7faefc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faefc160af0 0x7faefc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 0x7faefc0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7faefc0cfdd0 con 0x7faefc160af0 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faefc15cc90 con 0x7faefc0cf800 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.739+0000 7faf03400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faefc15cf70 con 0x7faefc160710 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faefc160af0 0x7faefc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faefc160af0 0x7faefc0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51208/0 (socket says 172.21.15.3:51208) 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 -- 172.21.15.3:0/2774149116 learned_addr learned my addr 172.21.15.3:0/2774149116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:21.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faf01400640 1 --2- 172.21.15.3:0/2774149116 >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 0x7faefc0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 -- 172.21.15.3:0/2774149116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 msgr2=0x7faefc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faf00a00640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 0x7faefc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 0x7faefc0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 -- 172.21.15.3:0/2774149116 >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 msgr2=0x7faefc0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 --2- 172.21.15.3:0/2774149116 >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 0x7faefc0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faefbe00640 1 -- 172.21.15.3:0/2774149116 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc15d250 con 0x7faefc160af0 2024-09-10T07:47:21.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faf00a00640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 0x7faefc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:21.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.740+0000 7faf01400640 1 --2- 172.21.15.3:0/2774149116 >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 0x7faefc0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:21.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.741+0000 7faefbe00640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc160af0 0x7faefc0d00b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faefc06df70 tx=0x7faee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f534a5acd20338aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:21.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.741+0000 7faefb400640 1 -- 172.21.15.3:0/2774149116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8037070 con 0x7faefc160af0 2024-09-10T07:47:21.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faefb400640 1 -- 172.21.15.3:0/2774149116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee8032440 con 0x7faefc160af0 2024-09-10T07:47:21.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faefb400640 1 -- 172.21.15.3:0/2774149116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8032810 con 0x7faefc160af0 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 -- 172.21.15.3:0/2774149116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc160af0 msgr2=0x7faefc0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc160af0 0x7faefc0d00b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faefc06df70 tx=0x7faee802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 -- 172.21.15.3:0/2774149116 shutdown_connections 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 --2- 172.21.15.3:0/2774149116 >> v2:172.21.15.70:3300/0 conn(0x7faefc0cf800 0x7faefc0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc160af0 0x7faefc0d00b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 --2- 172.21.15.3:0/2774149116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160710 0x7faefc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 -- 172.21.15.3:0/2774149116 >> 172.21.15.3:0/2774149116 conn(0x7faefc06d000 msgr2=0x7faefc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 -- 172.21.15.3:0/2774149116 shutdown_connections 2024-09-10T07:47:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.742+0000 7faf03400640 1 -- 172.21.15.3:0/2774149116 wait complete. 2024-09-10T07:47:21.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.744+0000 7faf03400640 1 Processor -- start 2024-09-10T07:47:21.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.744+0000 7faf03400640 1 -- start start 2024-09-10T07:47:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faefc160710 0x7faefc14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160af0 0x7faefc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faefc15d770 con 0x7faefc0cf800 2024-09-10T07:47:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faefc06bb20 con 0x7faefc160710 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf03400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faefc15db10 con 0x7faefc160af0 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf00a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf00a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51222/0 (socket says 172.21.15.3:51222) 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf00a00640 1 -- 172.21.15.3:0/1218490887 learned_addr learned my addr 172.21.15.3:0/1218490887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faefbe00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faefc160710 0x7faefc14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.745+0000 7faf01400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160af0 0x7faefc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 -- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160af0 msgr2=0x7faefc0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160af0 0x7faefc0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 -- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faefc160710 msgr2=0x7faefc14d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faefc160710 0x7faefc14d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 -- 172.21.15.3:0/1218490887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0b4280 con 0x7faefc0cf800 2024-09-10T07:47:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf00a00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faef000c7f0 tx=0x7faef000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0015070 con 0x7faefc0cf800 2024-09-10T07:47:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef0007a30 con 0x7faefc0cf800 2024-09-10T07:47:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc0b4560 con 0x7faefc0cf800 2024-09-10T07:47:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0007e00 con 0x7faefc0cf800 2024-09-10T07:47:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.746+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc156a30 con 0x7faefc0cf800 2024-09-10T07:47:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.747+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc0afd50 con 0x7faefc0cf800 2024-09-10T07:47:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.748+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faef0018070 con 0x7faefc0cf800 2024-09-10T07:47:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.749+0000 7faef8c00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.749+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7faef008aaa0 con 0x7faefc0cf800 2024-09-10T07:47:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.750+0000 7faefbe00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.750+0000 7faefbe00640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 0x7faedc06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee8034000 tx=0x7faee802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:21.754+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faef00590b0 con 0x7faefc0cf800 2024-09-10T07:47:22.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.010+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faefc0b48f0 con 0x7faefc0cf800 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.010+0000 7faef8c00640 1 -- 172.21.15.3:0/1218490887 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7faef005d3b0 con 0x7faefc0cf800 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.010 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:47:22.013 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.014 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:47:22.015 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.018 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:47:22.020 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.021 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9mmSOXGRAA9x8ip98FXEWnsNQ3F0W+Dw==", 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9mES2XGRAA6jU6f6e/t4KZLaI1YfgSZg==", 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9meTWXGRAAABRRp7ixyMLnkpCtaYNCyQ==", 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9mnT2XGRAAffB8ch9E9K+vJYN0kl9WhQ==", 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9m2kWXGRAApf2ueOT9uwBUHLvz2o6iaA==", 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA++d9mU06XGRAAEqrJ3mdyotiFjKV9asqYkA==", 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:47:22.025 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:47:22.026 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 msgr2=0x7faedc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 0x7faedc06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee8034000 tx=0x7faee802e000 comp rx=0 tx=0).stop 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 msgr2=0x7faefc1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faef000c7f0 tx=0x7faef000cbf0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 shutdown_connections 2024-09-10T07:47:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faedc06be30 0x7faedc06e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faefc160af0 0x7faefc0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faefc160710 0x7faefc14d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 --2- 172.21.15.3:0/1218490887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faefc0cf800 0x7faefc1564f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.012+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 >> 172.21.15.3:0/1218490887 conn(0x7faefc06d000 msgr2=0x7faefc150990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.013+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 shutdown_connections 2024-09-10T07:47:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.013+0000 7faf03400640 1 -- 172.21.15.3:0/1218490887 wait complete. 2024-09-10T07:47:22.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T07:47:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.187+0000 7f19c8e00640 1 Processor -- start 2024-09-10T07:47:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.187+0000 7f19c8e00640 1 -- start start 2024-09-10T07:47:22.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.188+0000 7f19c8e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19c4160af0 0x7f19c40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.188+0000 7f19c8e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19c40d05f0 0x7f19c4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.188+0000 7f19c8e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19c40cfdd0 con 0x7f19c4160710 2024-09-10T07:47:22.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.188+0000 7f19c8e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f19c415cc90 con 0x7f19c40d05f0 2024-09-10T07:47:22.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.188+0000 7f19c8e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f19c415cf70 con 0x7f19c4160af0 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c2800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f19c40d05f0 0x7f19c4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c1e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f19c4160710 0x7f19c40cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37052/0 (socket says 172.21.15.3:37052) 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c1e00640 1 -- 172.21.15.3:0/1117999322 learned_addr learned my addr 172.21.15.3:0/1117999322 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19bbe00640 1 --2- 172.21.15.3:0/1117999322 >> v2:172.21.15.70:3300/0 conn(0x7f19c4160af0 0x7f19c40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c2800640 1 -- 172.21.15.3:0/1117999322 >> v2:172.21.15.70:3300/0 conn(0x7f19c4160af0 msgr2=0x7f19c40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:22.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c2800640 1 --2- 172.21.15.3:0/1117999322 >> v2:172.21.15.70:3300/0 conn(0x7f19c4160af0 0x7f19c40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c2800640 1 -- 172.21.15.3:0/1117999322 >> v1:172.21.15.3:6789/0 conn(0x7f19c4160710 legacy=0x7f19c40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.189+0000 7f19c2800640 1 -- 172.21.15.3:0/1117999322 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c415d250 con 0x7f19c40d05f0 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.190+0000 7f19c2800640 1 --2- 172.21.15.3:0/1117999322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d05f0 0x7f19c4072070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19c406df70 tx=0x7f19bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa7440b7e2c526a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.190+0000 7f19c1400640 1 -- 172.21.15.3:0/1117999322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc007740 con 0x7f19c40d05f0 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.190+0000 7f19c1400640 1 -- 172.21.15.3:0/1117999322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19bc0078e0 con 0x7f19c40d05f0 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.190+0000 7f19c1400640 1 -- 172.21.15.3:0/1117999322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc007cb0 con 0x7f19c40d05f0 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 -- 172.21.15.3:0/1117999322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d05f0 msgr2=0x7f19c4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 --2- 172.21.15.3:0/1117999322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d05f0 0x7f19c4072070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19c406df70 tx=0x7f19bc02a680 comp rx=0 tx=0).stop 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 -- 172.21.15.3:0/1117999322 shutdown_connections 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 --2- 172.21.15.3:0/1117999322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c40d05f0 0x7f19c4072070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 --2- 172.21.15.3:0/1117999322 >> v2:172.21.15.70:3300/0 conn(0x7f19c4160af0 0x7f19c40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 -- 172.21.15.3:0/1117999322 >> 172.21.15.3:0/1117999322 conn(0x7f19c406d000 msgr2=0x7f19c406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 -- 172.21.15.3:0/1117999322 shutdown_connections 2024-09-10T07:47:22.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.191+0000 7f19c8e00640 1 -- 172.21.15.3:0/1117999322 wait complete. 2024-09-10T07:47:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.193+0000 7f19c8e00640 1 Processor -- start 2024-09-10T07:47:22.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.193+0000 7f19c8e00640 1 -- start start 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c40d05f0 0x7f19c40ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c4160af0 0x7f19c415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19c415d770 con 0x7f19c4160710 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f19c415c960 con 0x7f19c4160af0 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c8e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f19c41f7ea0 con 0x7f19c40d05f0 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51250/0 (socket says 172.21.15.3:51250) 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 -- 172.21.15.3:0/3392076850 learned_addr learned my addr 172.21.15.3:0/3392076850 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c1e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c40d05f0 0x7f19c40ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19c2800640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c4160af0 0x7f19c415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 -- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c40d05f0 msgr2=0x7f19c40ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c40d05f0 0x7f19c40ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 -- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c4160af0 msgr2=0x7f19c415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c4160af0 0x7f19c415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.194+0000 7f19bbe00640 1 -- 172.21.15.3:0/3392076850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c41f8180 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19bbe00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00d5f0 tx=0x7f19ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0125f0 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ac019070 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0129d0 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c41f8460 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.195+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c41f8a20 con 0x7f19c4160710 2024-09-10T07:47:22.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.196+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c40cfa60 con 0x7f19c4160710 2024-09-10T07:47:22.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.197+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19ac012b70 con 0x7f19c4160710 2024-09-10T07:47:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.198+0000 7f19ba000640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 0x7f19a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.198+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f19ac08ae90 con 0x7f19c4160710 2024-09-10T07:47:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.198+0000 7f19c1e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 0x7f19a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.199+0000 7f19c1e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 0x7f19a006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19b4000f60 tx=0x7f19b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.203+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19ac059c20 con 0x7f19c4160710 2024-09-10T07:47:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.458+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f19c40ab0e0 con 0x7f19c4160710 2024-09-10T07:47:22.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.460+0000 7f19ba000640 1 -- 172.21.15.3:0/3392076850 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f19ac059520 con 0x7f19c4160710 2024-09-10T07:47:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 msgr2=0x7f19a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 0x7f19a006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19b4000f60 tx=0x7f19b4009040 comp rx=0 tx=0).stop 2024-09-10T07:47:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 msgr2=0x7f19c40aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00d5f0 tx=0x7f19ac00dab0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 shutdown_connections 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19a006be30 0x7f19a006e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19c4160af0 0x7f19c415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19c4160710 0x7f19c40aa490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 --2- 172.21.15.3:0/3392076850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19c40d05f0 0x7f19c40ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 >> 172.21.15.3:0/3392076850 conn(0x7f19c406d000 msgr2=0x7f19c40cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 shutdown_connections 2024-09-10T07:47:22.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.462+0000 7f19c8e00640 1 -- 172.21.15.3:0/3392076850 wait complete. 2024-09-10T07:47:22.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T07:47:22.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.636+0000 7fb511600640 1 Processor -- start 2024-09-10T07:47:22.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.636+0000 7fb511600640 1 -- start start 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.637+0000 7fb511600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 0x7fb50c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.637+0000 7fb511600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb50c0cee40 con 0x7fb50c073150 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.637+0000 7fb511600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb50c0cf120 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.637+0000 7fb511600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb50c0d3780 con 0x7fb50c15e860 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb510c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb50c073150 0x7fb50c0b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37058/0 (socket says 172.21.15.3:37058) 2024-09-10T07:47:22.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb510c00640 1 -- 172.21.15.3:0/673842219 learned_addr learned my addr 172.21.15.3:0/673842219 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:22.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50be00640 1 --2- 172.21.15.3:0/673842219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 0x7fb50c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3207747824 0 0) 0x7fb50c0cee40 con 0x7fb50c073150 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f0003520 con 0x7fb50c073150 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 535332019 0 0) 0x7fb50c0cf120 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb50c0cee40 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50be00640 1 -- 172.21.15.3:0/673842219 >> v1:172.21.15.70:6789/0 conn(0x7fb50c0cfbf0 legacy=0x7fb50c0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50be00640 1 -- 172.21.15.3:0/673842219 >> v1:172.21.15.3:6789/0 conn(0x7fb50c073150 legacy=0x7fb50c0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.638+0000 7fb50be00640 1 -- 172.21.15.3:0/673842219 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c0d3410 con 0x7fb50c15e860 2024-09-10T07:47:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.639+0000 7fb50be00640 1 --2- 172.21.15.3:0/673842219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 0x7fb50c0cf6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0097d0 tx=0x7fb4fc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a311e92ccd81e0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.639+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc037070 con 0x7fb50c15e860 2024-09-10T07:47:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc032440 con 0x7fb50c15e860 2024-09-10T07:47:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb50aa00640 1 -- 172.21.15.3:0/673842219 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc032810 con 0x7fb50c15e860 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 -- 172.21.15.3:0/673842219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 msgr2=0x7fb50c0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 --2- 172.21.15.3:0/673842219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 0x7fb50c0cf6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0097d0 tx=0x7fb4fc02a210 comp rx=0 tx=0).stop 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 -- 172.21.15.3:0/673842219 shutdown_connections 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 --2- 172.21.15.3:0/673842219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c15e860 0x7fb50c0cf6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 -- 172.21.15.3:0/673842219 >> 172.21.15.3:0/673842219 conn(0x7fb50c0d4650 msgr2=0x7fb50c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 -- 172.21.15.3:0/673842219 shutdown_connections 2024-09-10T07:47:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.640+0000 7fb511600640 1 -- 172.21.15.3:0/673842219 wait complete. 2024-09-10T07:47:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.642+0000 7fb511600640 1 Processor -- start 2024-09-10T07:47:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.642+0000 7fb511600640 1 -- start start 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c073150 0x7fb50c0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c15e860 0x7fb50c1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c0d2460 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c0cf400 con 0x7fb50c15e860 2024-09-10T07:47:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb511600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb50c0b7f90 con 0x7fb50c073150 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51254/0 (socket says 172.21.15.3:51254) 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 -- 172.21.15.3:0/864966584 learned_addr learned my addr 172.21.15.3:0/864966584 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb510c00640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c15e860 0x7fb50c1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50be00640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c073150 0x7fb50c0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 -- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c073150 msgr2=0x7fb50c0ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c073150 0x7fb50c0ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 -- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c15e860 msgr2=0x7fb50c1f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c15e860 0x7fb50c1f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.643+0000 7fb50b400640 1 -- 172.21.15.3:0/864966584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c1616a0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb50b400640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c3f0 tx=0x7fb4f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8017070 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f80052f0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c0ad900 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8003830 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.644+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c0adec0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.645+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c0cf6b0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.646+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4f801a070 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.647+0000 7fb503e00640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 0x7fb4ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.648+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fb4f8089e40 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.648+0000 7fb50be00640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 0x7fb4ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.648+0000 7fb50be00640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 0x7fb4ec06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc034000 tx=0x7fb4fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:22.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.652+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4f80584e0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.902+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb50c0ae5d0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.904+0000 7fb503e00640 1 -- 172.21.15.3:0/864966584 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fb4f805c7e0 con 0x7fb50c0cfbf0 2024-09-10T07:47:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 msgr2=0x7fb4ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 0x7fb4ec06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc034000 tx=0x7fb4fc02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 msgr2=0x7fb50c1f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c3f0 tx=0x7fb4f800c8b0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 shutdown_connections 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4ec06be30 0x7fb4ec06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb50c15e860 0x7fb50c1f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb50c0cfbf0 0x7fb50c1f75b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 --2- 172.21.15.3:0/864966584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb50c073150 0x7fb50c0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 >> 172.21.15.3:0/864966584 conn(0x7fb50c0d4650 msgr2=0x7fb50c0b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.906+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 shutdown_connections 2024-09-10T07:47:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:22.907+0000 7fb511600640 1 -- 172.21.15.3:0/864966584 wait complete. 2024-09-10T07:47:22.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T07:47:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.079+0000 7fcad3800640 1 Processor -- start 2024-09-10T07:47:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.079+0000 7fcad3800640 1 -- start start 2024-09-10T07:47:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.080+0000 7fcad3800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcacc160710 0x7fcacc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.080+0000 7fcad3800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcacc0d07c0 0x7fcacc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.080+0000 7fcad3800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcacc0cfdd0 con 0x7fcacc160af0 2024-09-10T07:47:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.080+0000 7fcad3800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcacc15cc90 con 0x7fcacc160710 2024-09-10T07:47:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.080+0000 7fcad3800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcacc15cf70 con 0x7fcacc0d07c0 2024-09-10T07:47:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad0e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcacc160710 0x7fcacc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcacbe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcacc160af0 0x7fcacc0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37060/0 (socket says 172.21.15.3:37060) 2024-09-10T07:47:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcacbe00640 1 -- 172.21.15.3:0/3316632507 learned_addr learned my addr 172.21.15.3:0/3316632507 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad1800640 1 --2- 172.21.15.3:0/3316632507 >> v2:172.21.15.70:3300/0 conn(0x7fcacc0d07c0 0x7fcacc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad0e00640 1 -- 172.21.15.3:0/3316632507 >> v2:172.21.15.70:3300/0 conn(0x7fcacc0d07c0 msgr2=0x7fcacc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad0e00640 1 --2- 172.21.15.3:0/3316632507 >> v2:172.21.15.70:3300/0 conn(0x7fcacc0d07c0 0x7fcacc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad0e00640 1 -- 172.21.15.3:0/3316632507 >> v1:172.21.15.3:6789/0 conn(0x7fcacc160af0 legacy=0x7fcacc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcad0e00640 1 -- 172.21.15.3:0/3316632507 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc15d250 con 0x7fcacc160710 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.081+0000 7fcacb400640 1 -- 172.21.15.3:0/3316632507 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450214978 0 0) 0x7fcacc0cfdd0 con 0x7fcacc160af0 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.082+0000 7fcad0e00640 1 --2- 172.21.15.3:0/3316632507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc160710 0x7fcacc0cf2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0718d0 tx=0x7fcac002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c945fb09000368 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.082+0000 7fcacb400640 1 -- 172.21.15.3:0/3316632507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0037070 con 0x7fcacc160710 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.082+0000 7fcacb400640 1 -- 172.21.15.3:0/3316632507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac0032440 con 0x7fcacc160710 2024-09-10T07:47:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.082+0000 7fcacb400640 1 -- 172.21.15.3:0/3316632507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0032810 con 0x7fcacc160710 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 -- 172.21.15.3:0/3316632507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc160710 msgr2=0x7fcacc0cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 --2- 172.21.15.3:0/3316632507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc160710 0x7fcacc0cf2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0718d0 tx=0x7fcac002a1a0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 -- 172.21.15.3:0/3316632507 shutdown_connections 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 --2- 172.21.15.3:0/3316632507 >> v2:172.21.15.70:3300/0 conn(0x7fcacc0d07c0 0x7fcacc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 --2- 172.21.15.3:0/3316632507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc160710 0x7fcacc0cf2c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 -- 172.21.15.3:0/3316632507 >> 172.21.15.3:0/3316632507 conn(0x7fcacc06d000 msgr2=0x7fcacc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 -- 172.21.15.3:0/3316632507 shutdown_connections 2024-09-10T07:47:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.083+0000 7fcad3800640 1 -- 172.21.15.3:0/3316632507 wait complete. 2024-09-10T07:47:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.085+0000 7fcad3800640 1 Processor -- start 2024-09-10T07:47:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.085+0000 7fcad3800640 1 -- start start 2024-09-10T07:47:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcacc160710 0x7fcacc1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcacc160af0 0x7fcacc151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc15d770 con 0x7fcacc0d07c0 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc0cf800 con 0x7fcacc160af0 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad3800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcacc1f6c30 con 0x7fcacc160710 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51278/0 (socket says 172.21.15.3:51278) 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 -- 172.21.15.3:0/1069881892 learned_addr learned my addr 172.21.15.3:0/1069881892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad1800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcacc160af0 0x7fcacc151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcacbe00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcacc160710 0x7fcacc1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 -- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcacc160710 msgr2=0x7fcacc1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcacc160710 0x7fcacc1f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 -- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcacc160af0 msgr2=0x7fcacc151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcacc160af0 0x7fcacc151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.086+0000 7fcad0e00640 1 -- 172.21.15.3:0/1069881892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc1524c0 con 0x7fcacc0d07c0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcad0e00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009890 tx=0x7fcac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0037040 con 0x7fcacc0d07c0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac0042070 con 0x7fcacc0d07c0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0030960 con 0x7fcacc0d07c0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc154e70 con 0x7fcacc0d07c0 2024-09-10T07:47:23.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.087+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc155430 con 0x7fcacc0d07c0 2024-09-10T07:47:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.088+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc070d80 con 0x7fcacc0d07c0 2024-09-10T07:47:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.089+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcac0030b00 con 0x7fcacc0d07c0 2024-09-10T07:47:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.090+0000 7fcac8c00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.090+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fcac00ab380 con 0x7fcacc0d07c0 2024-09-10T07:47:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.090+0000 7fcacbe00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.091+0000 7fcacbe00640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 0x7fcaac06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcacc153560 tx=0x7fcab8007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.094+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcacc070d80 con 0x7fcacc0d07c0 2024-09-10T07:47:23.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.342+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fcacc0afd50 con 0x7fcacc0d07c0 2024-09-10T07:47:23.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.345+0000 7fcac8c00640 1 -- 172.21.15.3:0/1069881892 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fcac007a070 con 0x7fcacc0d07c0 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 msgr2=0x7fcaac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 0x7fcaac06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcacc153560 tx=0x7fcab8007c00 comp rx=0 tx=0).stop 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 msgr2=0x7fcacc154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009890 tx=0x7fcac00046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 shutdown_connections 2024-09-10T07:47:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcacc160af0 0x7fcacc151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcacc160710 0x7fcacc1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 --2- 172.21.15.3:0/1069881892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcacc0d07c0 0x7fcacc154890 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 >> 172.21.15.3:0/1069881892 conn(0x7fcacc06d000 msgr2=0x7fcacc15fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 shutdown_connections 2024-09-10T07:47:23.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.347+0000 7fcad3800640 1 -- 172.21.15.3:0/1069881892 wait complete. 2024-09-10T07:47:23.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T07:47:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.521+0000 7fe842c00640 1 Processor -- start 2024-09-10T07:47:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.521+0000 7fe842c00640 1 -- start start 2024-09-10T07:47:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe842c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe83c15b330 0x7fe83c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe842c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c15bac0 con 0x7fe83c0b2540 2024-09-10T07:47:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe842c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c15bd30 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe842c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe83c0d0100 con 0x7fe83c15b330 2024-09-10T07:47:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe840e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe83c0b2540 0x7fe83c0cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37068/0 (socket says 172.21.15.3:37068) 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe840e00640 1 -- 172.21.15.3:0/3552395712 learned_addr learned my addr 172.21.15.3:0/3552395712 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe841800640 1 --2- 172.21.15.3:0/3552395712 >> v2:172.21.15.70:3300/0 conn(0x7fe83c15b330 0x7fe83c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578618236 0 0) 0x7fe83c15bac0 con 0x7fe83c0b2540 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe824003520 con 0x7fe83c0b2540 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143801720 0 0) 0x7fe83c15bd30 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.523+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe83c15bac0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.524+0000 7fe841800640 1 -- 172.21.15.3:0/3552395712 >> v1:172.21.15.70:6789/0 conn(0x7fe83c0cfcd0 legacy=0x7fe83c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.524+0000 7fe841800640 1 -- 172.21.15.3:0/3552395712 >> v1:172.21.15.3:6789/0 conn(0x7fe83c0b2540 legacy=0x7fe83c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.524+0000 7fe841800640 1 -- 172.21.15.3:0/3552395712 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c15c010 con 0x7fe83c15b330 2024-09-10T07:47:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe841800640 1 --2- 172.21.15.3:0/3552395712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c0b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097a0 tx=0x7fe83002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecc3419a5b1213b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830037070 con 0x7fe83c15b330 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe830032440 con 0x7fe83c15b330 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe83be00640 1 -- 172.21.15.3:0/3552395712 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830032810 con 0x7fe83c15b330 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 -- 172.21.15.3:0/3552395712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 msgr2=0x7fe83c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 --2- 172.21.15.3:0/3552395712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c0b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097a0 tx=0x7fe83002a230 comp rx=0 tx=0).stop 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 -- 172.21.15.3:0/3552395712 shutdown_connections 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 --2- 172.21.15.3:0/3552395712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c0b2000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 -- 172.21.15.3:0/3552395712 >> 172.21.15.3:0/3552395712 conn(0x7fe83c06bb20 msgr2=0x7fe83c06bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.525+0000 7fe842c00640 1 -- 172.21.15.3:0/3552395712 shutdown_connections 2024-09-10T07:47:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.526+0000 7fe842c00640 1 -- 172.21.15.3:0/3552395712 wait complete. 2024-09-10T07:47:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.527+0000 7fe842c00640 1 Processor -- start 2024-09-10T07:47:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.527+0000 7fe842c00640 1 -- start start 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe83c0b2540 0x7fe83c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c0ce1c0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c15c5d0 con 0x7fe83c15b330 2024-09-10T07:47:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.528+0000 7fe842c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c15b710 con 0x7fe83c0b2540 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51294/0 (socket says 172.21.15.3:51294) 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 -- 172.21.15.3:0/3093083652 learned_addr learned my addr 172.21.15.3:0/3093083652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe841800640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe83c0b2540 0x7fe83c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe842200640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 -- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe83c0b2540 msgr2=0x7fe83c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe83c0b2540 0x7fe83c14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 -- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 msgr2=0x7fe83c155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 -- 172.21.15.3:0/3093083652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c15ccf0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe840e00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c500 tx=0x7fe82800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828017070 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.529+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe828005ce0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.530+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8280072f0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.530+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c15cf90 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.530+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c15d550 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.531+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c063610 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.531+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe82801a070 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.532+0000 7fe839600640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 0x7fe81c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.532+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fe828089e30 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.533+0000 7fe841800640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 0x7fe81c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.533+0000 7fe841800640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 0x7fe81c06e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe830034000 tx=0x7fe83002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.536+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8280584d0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T07:47:23.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.787+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fe83c0b2000 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.789+0000 7fe839600640 1 -- 172.21.15.3:0/3093083652 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fe82805c7d0 con 0x7fe83c0cfcd0 2024-09-10T07:47:23.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.791+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 msgr2=0x7fe81c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.791+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 0x7fe81c06e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe830034000 tx=0x7fe83002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:23.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.791+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 msgr2=0x7fe83c154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.791+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c500 tx=0x7fe82800c9c0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 shutdown_connections 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe81c06be30 0x7fe81c06e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe83c15b330 0x7fe83c155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe83c0cfcd0 0x7fe83c154120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 --2- 172.21.15.3:0/3093083652 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe83c0b2540 0x7fe83c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 >> 172.21.15.3:0/3093083652 conn(0x7fe83c06bb20 msgr2=0x7fe83c14b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 shutdown_connections 2024-09-10T07:47:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.792+0000 7fe842c00640 1 -- 172.21.15.3:0/3093083652 wait complete. 2024-09-10T07:47:23.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T07:47:23.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.964+0000 7f2f5da00640 1 Processor -- start 2024-09-10T07:47:23.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.964+0000 7f2f5da00640 1 -- start start 2024-09-10T07:47:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.965+0000 7f2f5da00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 0x7f2f580d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.965+0000 7f2f5da00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f580d1d90 con 0x7f2f580b6690 2024-09-10T07:47:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.965+0000 7f2f5da00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f580ceef0 con 0x7f2f580b6a70 2024-09-10T07:47:23.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.965+0000 7f2f5da00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f580d1b50 con 0x7f2f580d16e0 2024-09-10T07:47:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f580b6690 0x7f2f580cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39106/0 (socket says 172.21.15.3:39106) 2024-09-10T07:47:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56a00640 1 -- 172.21.15.3:0/530047798 learned_addr learned my addr 172.21.15.3:0/530047798 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:23.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f57400640 1 --2- 172.21.15.3:0/530047798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 0x7f2f580d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2311259750 0 0) 0x7f2f580d1d90 con 0x7f2f580b6690 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f3c003520 con 0x7f2f580b6690 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614420005 0 0) 0x7f2f580ceef0 con 0x7f2f580b6a70 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f580d1d90 con 0x7f2f580b6a70 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f57400640 1 -- 172.21.15.3:0/530047798 >> v1:172.21.15.70:6789/0 conn(0x7f2f580b6a70 legacy=0x7f2f580d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f57400640 1 -- 172.21.15.3:0/530047798 >> v1:172.21.15.3:6789/0 conn(0x7f2f580b6690 legacy=0x7f2f580cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.966+0000 7f2f57400640 1 -- 172.21.15.3:0/530047798 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f580cf1d0 con 0x7f2f580d16e0 2024-09-10T07:47:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.967+0000 7f2f57400640 1 --2- 172.21.15.3:0/530047798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 0x7f2f580d2b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009dc0 tx=0x7f2f5002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b97d8872def125 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.967+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50007740 con 0x7f2f580d16e0 2024-09-10T07:47:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f500078e0 con 0x7f2f580d16e0 2024-09-10T07:47:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f56000640 1 -- 172.21.15.3:0/530047798 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50007cb0 con 0x7f2f580d16e0 2024-09-10T07:47:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 -- 172.21.15.3:0/530047798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 msgr2=0x7f2f580d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 --2- 172.21.15.3:0/530047798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 0x7f2f580d2b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009dc0 tx=0x7f2f5002a6d0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 -- 172.21.15.3:0/530047798 shutdown_connections 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 --2- 172.21.15.3:0/530047798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580d16e0 0x7f2f580d2b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 -- 172.21.15.3:0/530047798 >> 172.21.15.3:0/530047798 conn(0x7f2f58072560 msgr2=0x7f2f58072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 -- 172.21.15.3:0/530047798 shutdown_connections 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.968+0000 7f2f5da00640 1 -- 172.21.15.3:0/530047798 wait complete. 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.970+0000 7f2f5da00640 1 Processor -- start 2024-09-10T07:47:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.970+0000 7f2f5da00640 1 -- start start 2024-09-10T07:47:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580b6690 0x7f2f581f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f580d16e0 0x7f2f580abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5815c530 con 0x7f2f580b6a70 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580cf540 con 0x7f2f580d16e0 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f5da00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5815c6b0 con 0x7f2f580b6690 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f4e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f4e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35842/0 (socket says 172.21.15.3:35842) 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f4e000640 1 -- 172.21.15.3:0/1353998474 learned_addr learned my addr 172.21.15.3:0/1353998474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f56a00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580b6690 0x7f2f581f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.971+0000 7f2f57400640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f580d16e0 0x7f2f580abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 -- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580b6690 msgr2=0x7f2f581f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580b6690 0x7f2f581f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 -- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f580d16e0 msgr2=0x7f2f580abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f580d16e0 0x7f2f580abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 -- 172.21.15.3:0/1353998474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f580ac780 con 0x7f2f580b6a70 2024-09-10T07:47:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4e000640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c500 tx=0x7f2f4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400052d0 con 0x7f2f580b6a70 2024-09-10T07:47:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f40005490 con 0x7f2f580b6a70 2024-09-10T07:47:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012870 con 0x7f2f580b6a70 2024-09-10T07:47:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f580cce70 con 0x7f2f580b6a70 2024-09-10T07:47:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.972+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f580cd430 con 0x7f2f580b6a70 2024-09-10T07:47:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.973+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f580cf9d0 con 0x7f2f580b6a70 2024-09-10T07:47:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.974+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f40012a10 con 0x7f2f580b6a70 2024-09-10T07:47:23.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.976+0000 7f2f4f400640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:23.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.976+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f2f4008a740 con 0x7f2f580b6a70 2024-09-10T07:47:23.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.976+0000 7f2f56a00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:23.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.977+0000 7f2f56a00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 0x7f2f3406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f480011e0 tx=0x7f2f48007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:23.980+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f40058d60 con 0x7f2f580b6a70 2024-09-10T07:47:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.230+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f2f58069140 con 0x7f2f580b6a70 2024-09-10T07:47:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.232+0000 7f2f4f400640 1 -- 172.21.15.3:0/1353998474 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f2f4005d060 con 0x7f2f580b6a70 2024-09-10T07:47:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 msgr2=0x7f2f3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 0x7f2f3406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f480011e0 tx=0x7f2f48007d30 comp rx=0 tx=0).stop 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 msgr2=0x7f2f58158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c500 tx=0x7f2f4000c9c0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 shutdown_connections 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f580d16e0 0x7f2f580abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f580b6a70 0x7f2f58158840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 --2- 172.21.15.3:0/1353998474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f580b6690 0x7f2f581f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 >> 172.21.15.3:0/1353998474 conn(0x7f2f58072560 msgr2=0x7f2f5815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 shutdown_connections 2024-09-10T07:47:24.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.235+0000 7f2f5da00640 1 -- 172.21.15.3:0/1353998474 wait complete. 2024-09-10T07:47:24.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T07:47:24.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.409+0000 7f4187800640 1 Processor -- start 2024-09-10T07:47:24.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.409+0000 7f4187800640 1 -- start start 2024-09-10T07:47:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.410+0000 7f4187800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 0x7f41800d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.410+0000 7f4187800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41800cfdd0 con 0x7f41800d05f0 2024-09-10T07:47:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.410+0000 7f4187800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418015cc90 con 0x7f4180160710 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.410+0000 7f4187800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f418015cf70 con 0x7f4180160af0 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f4185800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41800d05f0 0x7f4180072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39108/0 (socket says 172.21.15.3:39108) 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f4185800640 1 -- 172.21.15.3:0/1371747798 learned_addr learned my addr 172.21.15.3:0/1371747798 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417fe00640 1 --2- 172.21.15.3:0/1371747798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 0x7f41800d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2537041973 0 0) 0x7f41800cfdd0 con 0x7f41800d05f0 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4168003520 con 0x7f41800d05f0 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363449570 0 0) 0x7f418015cc90 con 0x7f4180160710 2024-09-10T07:47:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41800cfdd0 con 0x7f4180160710 2024-09-10T07:47:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1052423923 0 0) 0x7f4168003520 con 0x7f41800d05f0 2024-09-10T07:47:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417fe00640 1 -- 172.21.15.3:0/1371747798 >> v1:172.21.15.70:6789/0 conn(0x7f4180160710 legacy=0x7f41800cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417fe00640 1 -- 172.21.15.3:0/1371747798 >> v1:172.21.15.3:6789/0 conn(0x7f41800d05f0 legacy=0x7f4180072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.411+0000 7f417fe00640 1 -- 172.21.15.3:0/1371747798 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418015d250 con 0x7f4180160af0 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.413+0000 7f417fe00640 1 --2- 172.21.15.3:0/1371747798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 0x7f41800d00b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097d0 tx=0x7f416c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93ecb6e0e43d92f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.413+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c037070 con 0x7f4180160af0 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.413+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c02ac30 con 0x7f4180160af0 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.413+0000 7f417f400640 1 -- 172.21.15.3:0/1371747798 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780019f0 con 0x7f4180160af0 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 -- 172.21.15.3:0/1371747798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 msgr2=0x7f41800d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 --2- 172.21.15.3:0/1371747798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 0x7f41800d00b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097d0 tx=0x7f416c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 -- 172.21.15.3:0/1371747798 shutdown_connections 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 --2- 172.21.15.3:0/1371747798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4180160af0 0x7f41800d00b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 -- 172.21.15.3:0/1371747798 >> 172.21.15.3:0/1371747798 conn(0x7f418006d000 msgr2=0x7f418006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 -- 172.21.15.3:0/1371747798 shutdown_connections 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.414+0000 7f4187800640 1 -- 172.21.15.3:0/1371747798 wait complete. 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.415+0000 7f4187800640 1 Processor -- start 2024-09-10T07:47:24.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 -- start start 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f41800d05f0 0x7f4180152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 0x7f41800b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f418015d770 con 0x7f4180160710 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.416+0000 7f4187800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f41800d0a10 con 0x7f4180160af0 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f4187800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4180156fc0 con 0x7f41800d05f0 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35858/0 (socket says 172.21.15.3:35858) 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 -- 172.21.15.3:0/3812165280 learned_addr learned my addr 172.21.15.3:0/3812165280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f4185800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 0x7f41800b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f4184e00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f41800d05f0 0x7f4180152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 -- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f41800d05f0 msgr2=0x7f4180152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f41800d05f0 0x7f4180152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 -- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 msgr2=0x7f41800b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 0x7f41800b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 -- 172.21.15.3:0/3812165280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41801572a0 con 0x7f4180160710 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f4185800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 0x7f41800b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.417+0000 7f417fe00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f416c035040 tx=0x7f416c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.418+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c037050 con 0x7f4180160710 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.418+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c043070 con 0x7f4180160710 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.418+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c006e10 con 0x7f4180160710 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.418+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4180157580 con 0x7f4180160710 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.418+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4180157b40 con 0x7f4180160710 2024-09-10T07:47:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.419+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41800cfaf0 con 0x7f4180160710 2024-09-10T07:47:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.420+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f416c03ca70 con 0x7f4180160710 2024-09-10T07:47:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.421+0000 7f417cc00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 0x7f416006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.421+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f416c0ab520 con 0x7f4180160710 2024-09-10T07:47:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.421+0000 7f4184e00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 0x7f416006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.422+0000 7f4184e00640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 0x7f416006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41740011e0 tx=0x7f4174009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.425+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f416c07a330 con 0x7f4180160710 2024-09-10T07:47:24.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.676+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f418015d8f0 con 0x7f4180160710 2024-09-10T07:47:24.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.679+0000 7f417cc00640 1 -- 172.21.15.3:0/3812165280 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f416c079c30 con 0x7f4180160710 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 msgr2=0x7f416006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 0x7f416006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41740011e0 tx=0x7f4174009040 comp rx=0 tx=0).stop 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 msgr2=0x7f41800b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f416c035040 tx=0x7f416c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 shutdown_connections 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f416006be30 0x7f416006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4180160af0 0x7f41800b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4180160710 0x7f41800b10d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 --2- 172.21.15.3:0/3812165280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f41800d05f0 0x7f4180152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 >> 172.21.15.3:0/3812165280 conn(0x7f418006d000 msgr2=0x7f41800b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 shutdown_connections 2024-09-10T07:47:24.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.681+0000 7f4187800640 1 -- 172.21.15.3:0/3812165280 wait complete. 2024-09-10T07:47:24.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:47:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.853+0000 7f265fa00640 1 Processor -- start 2024-09-10T07:47:24.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.853+0000 7f265fa00640 1 -- start start 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.854+0000 7f265fa00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f265815b410 0x7f265806ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.854+0000 7f265fa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f26580b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.854+0000 7f265fa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580ccd40 con 0x7f26580b4aa0 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f265fa00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f265806f250 con 0x7f265815b410 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f265fa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26580cd020 con 0x7f26580cfac0 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f265d000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f265815b410 0x7f265806ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f2657e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26580b4aa0 0x7f26580cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:49414/0 (socket says 172.21.15.3:49414) 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f265da00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f26580b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.855+0000 7f2657e00640 1 -- 172.21.15.3:0/2053518539 learned_addr learned my addr 172.21.15.3:0/2053518539 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.856+0000 7f265d000640 1 -- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 msgr2=0x7f26580b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.856+0000 7f265d000640 1 --2- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f26580b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.856+0000 7f265d000640 1 -- 172.21.15.3:0/2053518539 >> v1:172.21.15.70:6789/0 conn(0x7f26580b4aa0 legacy=0x7f26580cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.856+0000 7f265d000640 1 -- 172.21.15.3:0/2053518539 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580cd300 con 0x7f265815b410 2024-09-10T07:47:24.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.856+0000 7f2657400640 1 -- 172.21.15.3:0/2053518539 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826155958 0 0) 0x7f26580ccd40 con 0x7f26580b4aa0 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.857+0000 7f265d000640 1 --2- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f265806ed10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f264c0097a0 tx=0x7f264c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f404fff81871b220 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.857+0000 7f2657400640 1 -- 172.21.15.3:0/2053518539 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037070 con 0x7f265815b410 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.857+0000 7f2657400640 1 -- 172.21.15.3:0/2053518539 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c032440 con 0x7f265815b410 2024-09-10T07:47:24.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.857+0000 7f2657400640 1 -- 172.21.15.3:0/2053518539 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c032810 con 0x7f265815b410 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 -- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 msgr2=0x7f265806ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 --2- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f265806ed10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f264c0097a0 tx=0x7f264c02a180 comp rx=0 tx=0).stop 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 -- 172.21.15.3:0/2053518539 shutdown_connections 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 --2- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f26580b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 --2- 172.21.15.3:0/2053518539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f265806ed10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 -- 172.21.15.3:0/2053518539 >> 172.21.15.3:0/2053518539 conn(0x7f265806cfc0 msgr2=0x7f265806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 -- 172.21.15.3:0/2053518539 shutdown_connections 2024-09-10T07:47:24.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.858+0000 7f265fa00640 1 -- 172.21.15.3:0/2053518539 wait complete. 2024-09-10T07:47:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.860+0000 7f265fa00640 1 Processor -- start 2024-09-10T07:47:24.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.860+0000 7f265fa00640 1 -- start start 2024-09-10T07:47:24.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f265814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f26581e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26580cdba0 con 0x7f26580b4aa0 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26580cd8c0 con 0x7f265815b410 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265fa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2658072540 con 0x7f26580cfac0 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35862/0 (socket says 172.21.15.3:35862) 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 -- 172.21.15.3:0/3829732706 learned_addr learned my addr 172.21.15.3:0/3829732706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:24.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f2657e00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f265814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265da00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f26581e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 -- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 msgr2=0x7f265814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f265814ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 -- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 msgr2=0x7f26581e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f26581e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265d000640 1 -- 172.21.15.3:0/3829732706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581e46f0 con 0x7f26580b4aa0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.861+0000 7f265da00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f26581e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f265d000640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f264c000c00 tx=0x7f264c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037040 con 0x7f26580b4aa0 2024-09-10T07:47:24.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c042070 con 0x7f26580b4aa0 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c030940 con 0x7f26580b4aa0 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581e49d0 con 0x7f26580b4aa0 2024-09-10T07:47:24.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.862+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581e4f90 con 0x7f26580b4aa0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.863+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265815b7f0 con 0x7f26580b4aa0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.864+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f264c030ae0 con 0x7f26580b4aa0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.865+0000 7f2654c00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 0x7f263806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.865+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f264c0ab3f0 con 0x7f26580b4aa0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.865+0000 7f2657e00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 0x7f263806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:24.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.866+0000 7f2657e00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 0x7f263806e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f265806f220 tx=0x7f2644007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:24.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:24.869+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f264c07a050 con 0x7f26580b4aa0 2024-09-10T07:47:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.055+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2658075440 con 0x7f26580b4aa0 2024-09-10T07:47:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.056+0000 7f2654c00640 1 -- 172.21.15.3:0/3829732706 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7f264c079950 con 0x7f26580b4aa0 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:25.056 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:47:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 msgr2=0x7f263806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 0x7f263806e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f265806f220 tx=0x7f2644007de0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 msgr2=0x7f26581e5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f264c000c00 tx=0x7f264c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 shutdown_connections 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f263806be60 0x7f263806e320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f265815b410 0x7f26581e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26580cfac0 0x7f265814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 --2- 172.21.15.3:0/3829732706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26580b4aa0 0x7f26581e5790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.058+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 >> 172.21.15.3:0/3829732706 conn(0x7f265806cfc0 msgr2=0x7f26580ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.059+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 shutdown_connections 2024-09-10T07:47:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.059+0000 7f265fa00640 1 -- 172.21.15.3:0/3829732706 wait complete. 2024-09-10T07:47:25.074 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:47:25.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:47:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.235+0000 7fa9cbc00640 1 Processor -- start 2024-09-10T07:47:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.235+0000 7fa9cbc00640 1 -- start start 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9cbc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9c40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9cbc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c40cf1f0 con 0x7fa9c40b26a0 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9cbc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c40ce150 con 0x7fa9c40cd740 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9cbc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c40d00c0 con 0x7fa9c415c530 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9c9c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa9c40b26a0 0x7fa9c40722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39116/0 (socket says 172.21.15.3:39116) 2024-09-10T07:47:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.236+0000 7fa9c9c00640 1 -- 172.21.15.3:0/35723127 learned_addr learned my addr 172.21.15.3:0/35723127 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c9200640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9c40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689389430 0 0) 0x7fa9c40cf1f0 con 0x7fa9c40b26a0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9ac003520 con 0x7fa9c40b26a0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1268081872 0 0) 0x7fa9ac003520 con 0x7fa9c40b26a0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 msgr2=0x7fa9c40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9c40cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 >> v1:172.21.15.70:6789/0 conn(0x7fa9c40cd740 legacy=0x7fa9c40b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c40ce430 con 0x7fa9c40b26a0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fa9bc0019f0 con 0x7fa9c40b26a0 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 >> v1:172.21.15.3:6789/0 conn(0x7fa9c40b26a0 legacy=0x7fa9c40722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00bad0 con 0x7fa9ac004d40 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00bc80 con 0x7fa9ac003fb0 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac00be80 con 0x7fa9c415c530 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.237+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9200640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c3e00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9c00640 1 -- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 msgr2=0x7fa9ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9c415c530 0x7fa9ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9c00640 1 -- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac003fb0 msgr2=0x7fa9ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.238+0000 7fa9c9c00640 1 -- 172.21.15.3:0/35723127 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac003520 con 0x7fa9ac004d40 2024-09-10T07:47:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.239+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00a770 tx=0x7fa9ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbf0d0d1cea7e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.239+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc00a390 con 0x7fa9ac004d40 2024-09-10T07:47:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.239+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9bc02f070 con 0x7fa9ac004d40 2024-09-10T07:47:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.239+0000 7fa9c3400640 1 -- 172.21.15.3:0/35723127 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc02bc20 con 0x7fa9ac004d40 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 -- 172.21.15.3:0/35723127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac004d40 msgr2=0x7fa9ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/35723127 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00a770 tx=0x7fa9ac005830 comp rx=0 tx=0).stop 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9c9c00640 1 -- 172.21.15.3:0/35723127 reap_dead start 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 -- 172.21.15.3:0/35723127 shutdown_connections 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 -- 172.21.15.3:0/35723127 >> 172.21.15.3:0/35723127 conn(0x7fa9c4062cc0 msgr2=0x7fa9c406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 -- 172.21.15.3:0/35723127 shutdown_connections 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.240+0000 7fa9cbc00640 1 -- 172.21.15.3:0/35723127 wait complete. 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.241+0000 7fa9cbc00640 1 Processor -- start 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.242+0000 7fa9cbc00640 1 -- start start 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.242+0000 7fa9cbc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac004d40 0x7fa9c41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.242+0000 7fa9cbc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c41e7ee0 0x7fa9c41e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.242+0000 7fa9cbc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9cbc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c40ce9f0 con 0x7fa9c4072ec0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9cbc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c40cdb80 con 0x7fa9c41e7ee0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9cbc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c40cecd0 con 0x7fa9ac004d40 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35880/0 (socket says 172.21.15.3:35880) 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 -- 172.21.15.3:0/233265001 learned_addr learned my addr 172.21.15.3:0/233265001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9200640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac004d40 0x7fa9c41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c3e00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c41e7ee0 0x7fa9c41e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 -- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac004d40 msgr2=0x7fa9c41581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac004d40 0x7fa9c41581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 -- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c41e7ee0 msgr2=0x7fa9c41e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c41e7ee0 0x7fa9c41e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 -- 172.21.15.3:0/233265001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c40aaf40 con 0x7fa9c4072ec0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.243+0000 7fa9c9c00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00a770 tx=0x7fa9c40ce600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.244+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc001ee0 con 0x7fa9c4072ec0 2024-09-10T07:47:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.244+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9bc02bb90 con 0x7fa9c4072ec0 2024-09-10T07:47:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.244+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c40ab220 con 0x7fa9c4072ec0 2024-09-10T07:47:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.244+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc034470 con 0x7fa9c4072ec0 2024-09-10T07:47:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.244+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c40af3e0 con 0x7fa9c4072ec0 2024-09-10T07:47:25.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.245+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c40cd200 con 0x7fa9c4072ec0 2024-09-10T07:47:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.246+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9bc034610 con 0x7fa9c4072ec0 2024-09-10T07:47:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.247+0000 7fa9c0c00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 0x7fa9a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.247+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa9bc0aabc0 con 0x7fa9c4072ec0 2024-09-10T07:47:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.247+0000 7fa9c9200640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 0x7fa9a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.248+0000 7fa9c9200640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 0x7fa9a406e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9c41e8b00 tx=0x7fa9b8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.251+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9bc079820 con 0x7fa9c4072ec0 2024-09-10T07:47:25.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.435+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa9c40d1040 con 0x7fa9c4072ec0 2024-09-10T07:47:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.684+0000 7fa9c0c00640 1 -- 172.21.15.3:0/233265001 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7fa9bc079120 con 0x7fa9c4072ec0 2024-09-10T07:47:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:47:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 msgr2=0x7fa9a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 0x7fa9a406e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9c41e8b00 tx=0x7fa9b8009190 comp rx=0 tx=0).stop 2024-09-10T07:47:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 msgr2=0x7fa9c4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00a770 tx=0x7fa9c40ce600 comp rx=0 tx=0).stop 2024-09-10T07:47:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 shutdown_connections 2024-09-10T07:47:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa9a406be60 0x7fa9a406e320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9c4072ec0 0x7fa9c4073340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c41e7ee0 0x7fa9c41e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 --2- 172.21.15.3:0/233265001 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac004d40 0x7fa9c41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.687+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 >> 172.21.15.3:0/233265001 conn(0x7fa9c4062cc0 msgr2=0x7fa9c415ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.688+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 shutdown_connections 2024-09-10T07:47:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.688+0000 7fa9cbc00640 1 -- 172.21.15.3:0/233265001 wait complete. 2024-09-10T07:47:25.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:47:25.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.871+0000 7fab22400640 1 Processor -- start 2024-09-10T07:47:25.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.871+0000 7fab22400640 1 -- start start 2024-09-10T07:47:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab22400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c1451d0 0x7fab1c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab22400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab1c0cf270 0x7fab1c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab22400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab1c0cd900 con 0x7fab1c145730 2024-09-10T07:47:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab22400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fab1c06d7c0 con 0x7fab1c0cf270 2024-09-10T07:47:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab22400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c0cdbe0 con 0x7fab1c1451d0 2024-09-10T07:47:25.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab1c0cf270 0x7fab1c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab13e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab1c145730 0x7fab1c0b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39120/0 (socket says 172.21.15.3:39120) 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab13e00640 1 -- 172.21.15.3:0/4268381975 learned_addr learned my addr 172.21.15.3:0/4268381975 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1b400640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c1451d0 0x7fab1c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1be00640 1 -- 172.21.15.3:0/4268381975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c1451d0 msgr2=0x7fab1c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1be00640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c1451d0 0x7fab1c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1be00640 1 -- 172.21.15.3:0/4268381975 >> v1:172.21.15.3:6789/0 conn(0x7fab1c145730 legacy=0x7fab1c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.873+0000 7fab1aa00640 1 -- 172.21.15.3:0/4268381975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037862338 0 0) 0x7fab1c0cd900 con 0x7fab1c145730 2024-09-10T07:47:25.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.874+0000 7fab1be00640 1 -- 172.21.15.3:0/4268381975 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c06da10 con 0x7fab1c0cf270 2024-09-10T07:47:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab1be00640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c0cf270 0x7fab1c0cf720 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab14009d90 tx=0x7fab1402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440e9432a20dfa81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab1aa00640 1 -- 172.21.15.3:0/4268381975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14031600 con 0x7fab1c0cf270 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab1aa00640 1 -- 172.21.15.3:0/4268381975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab140317a0 con 0x7fab1c0cf270 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab1aa00640 1 -- 172.21.15.3:0/4268381975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14031b70 con 0x7fab1c0cf270 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 -- 172.21.15.3:0/4268381975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c0cf270 msgr2=0x7fab1c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c0cf270 0x7fab1c0cf720 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab14009d90 tx=0x7fab1402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 -- 172.21.15.3:0/4268381975 shutdown_connections 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c0cf270 0x7fab1c0cf720 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 --2- 172.21.15.3:0/4268381975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c1451d0 0x7fab1c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 -- 172.21.15.3:0/4268381975 >> 172.21.15.3:0/4268381975 conn(0x7fab1c0b5cb0 msgr2=0x7fab1c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.875+0000 7fab22400640 1 -- 172.21.15.3:0/4268381975 shutdown_connections 2024-09-10T07:47:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.876+0000 7fab22400640 1 -- 172.21.15.3:0/4268381975 wait complete. 2024-09-10T07:47:25.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.877+0000 7fab22400640 1 Processor -- start 2024-09-10T07:47:25.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.877+0000 7fab22400640 1 -- start start 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab1c0cf270 0x7fab1c1efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c145730 0x7fab1c1ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c06df30 con 0x7fab1c1451d0 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c145f10 con 0x7fab1c0cf270 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab22400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c06e0b0 con 0x7fab1c145730 2024-09-10T07:47:25.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35900/0 (socket says 172.21.15.3:35900) 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab13e00640 1 -- 172.21.15.3:0/3111544843 learned_addr learned my addr 172.21.15.3:0/3111544843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab1be00640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c145730 0x7fab1c1ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.878+0000 7fab1b400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab1c0cf270 0x7fab1c1efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 -- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c145730 msgr2=0x7fab1c1ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c145730 0x7fab1c1ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 -- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab1c0cf270 msgr2=0x7fab1c1efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab1c0cf270 0x7fab1c1efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 -- 172.21.15.3:0/3111544843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c1f2fb0 con 0x7fab1c1451d0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13e00640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab0400c410 tx=0x7fab0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040074e0 con 0x7fab1c1451d0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab040047e0 con 0x7fab1c1451d0 2024-09-10T07:47:25.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04004bd0 con 0x7fab1c1451d0 2024-09-10T07:47:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c1f35e0 con 0x7fab1c1451d0 2024-09-10T07:47:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.879+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1f3ba0 con 0x7fab1c1451d0 2024-09-10T07:47:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.880+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c0cf720 con 0x7fab1c1451d0 2024-09-10T07:47:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.881+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab04011070 con 0x7fab1c1451d0 2024-09-10T07:47:25.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.882+0000 7fab13400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 0x7faaf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:25.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.882+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7fab0408a840 con 0x7fab1c1451d0 2024-09-10T07:47:25.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.883+0000 7fab1b400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 0x7faaf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:25.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.883+0000 7fab1b400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 0x7faaf806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009b40 tx=0x7fab0c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:25.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:25.887+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab04058ce0 con 0x7fab1c1451d0 2024-09-10T07:47:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.072+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fab1c06d060 con 0x7fab1c1451d0 2024-09-10T07:47:26.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.687+0000 7fab13400640 1 -- 172.21.15.3:0/3111544843 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7fab0405cfe0 con 0x7fab1c1451d0 2024-09-10T07:47:26.687 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:47:26.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 msgr2=0x7faaf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 0x7faaf806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009b40 tx=0x7fab0c009190 comp rx=0 tx=0).stop 2024-09-10T07:47:26.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 msgr2=0x7fab1c1f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab0400c410 tx=0x7fab0400c8d0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 shutdown_connections 2024-09-10T07:47:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faaf806be30 0x7faaf806e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab1c145730 0x7fab1c1ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab1c1451d0 0x7fab1c1f2a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 --2- 172.21.15.3:0/3111544843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab1c0cf270 0x7fab1c1efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 >> 172.21.15.3:0/3111544843 conn(0x7fab1c0b5cb0 msgr2=0x7fab1c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.690+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 shutdown_connections 2024-09-10T07:47:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.691+0000 7fab22400640 1 -- 172.21.15.3:0/3111544843 wait complete. 2024-09-10T07:47:26.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:47:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.880+0000 7fe429800640 1 Processor -- start 2024-09-10T07:47:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.880+0000 7fe429800640 1 -- start start 2024-09-10T07:47:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe429800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 0x7fe42406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe429800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4240cd900 con 0x7fe4241451d0 2024-09-10T07:47:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe429800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4240cdbe0 con 0x7fe4240cf270 2024-09-10T07:47:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe429800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe42406d7c0 con 0x7fe424145730 2024-09-10T07:47:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe422a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4241451d0 0x7fe4240b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39130/0 (socket says 172.21.15.3:39130) 2024-09-10T07:47:26.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe422a00640 1 -- 172.21.15.3:0/4230871943 learned_addr learned my addr 172.21.15.3:0/4230871943 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe41be00640 1 --2- 172.21.15.3:0/4230871943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 0x7fe42406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.882+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559338352 0 0) 0x7fe4240cd900 con 0x7fe4241451d0 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.883+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe408003520 con 0x7fe4241451d0 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.883+0000 7fe41be00640 1 -- 172.21.15.3:0/4230871943 >> v1:172.21.15.70:6789/0 conn(0x7fe4240cf270 legacy=0x7fe4240cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.883+0000 7fe41be00640 1 -- 172.21.15.3:0/4230871943 >> v1:172.21.15.3:6789/0 conn(0x7fe4241451d0 legacy=0x7fe4240b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.883+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369699402 0 0) 0x7fe4240cdbe0 con 0x7fe4240cf270 2024-09-10T07:47:26.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.883+0000 7fe41be00640 1 -- 172.21.15.3:0/4230871943 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42406da10 con 0x7fe424145730 2024-09-10T07:47:26.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe41be00640 1 --2- 172.21.15.3:0/4230871943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 0x7fe42406cfc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0097d0 tx=0x7fe40c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81001bf458584065 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c037070 con 0x7fe424145730 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c02ae50 con 0x7fe424145730 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe422000640 1 -- 172.21.15.3:0/4230871943 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c032640 con 0x7fe424145730 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe429800640 1 -- 172.21.15.3:0/4230871943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 msgr2=0x7fe42406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.884+0000 7fe429800640 1 --2- 172.21.15.3:0/4230871943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 0x7fe42406cfc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0097d0 tx=0x7fe40c02a100 comp rx=0 tx=0).stop 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.885+0000 7fe429800640 1 -- 172.21.15.3:0/4230871943 shutdown_connections 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.885+0000 7fe429800640 1 --2- 172.21.15.3:0/4230871943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe424145730 0x7fe42406cfc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.885+0000 7fe429800640 1 -- 172.21.15.3:0/4230871943 >> 172.21.15.3:0/4230871943 conn(0x7fe4240b5cb0 msgr2=0x7fe42406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.885+0000 7fe429800640 1 -- 172.21.15.3:0/4230871943 shutdown_connections 2024-09-10T07:47:26.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.885+0000 7fe429800640 1 -- 172.21.15.3:0/4230871943 wait complete. 2024-09-10T07:47:26.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.886+0000 7fe429800640 1 Processor -- start 2024-09-10T07:47:26.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.886+0000 7fe429800640 1 -- start start 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 0x7fe42415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe424145730 0x7fe42415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe42406e0b0 con 0x7fe4241451d0 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe42406df30 con 0x7fe424145730 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.887+0000 7fe429800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe42406e230 con 0x7fe4240cf270 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35906/0 (socket says 172.21.15.3:35906) 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 -- 172.21.15.3:0/1265248821 learned_addr learned my addr 172.21.15.3:0/1265248821 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe422a00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 0x7fe42415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe423400640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe424145730 0x7fe42415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 -- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 msgr2=0x7fe42415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 0x7fe42415d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 -- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe424145730 msgr2=0x7fe42415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe424145730 0x7fe42415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 -- 172.21.15.3:0/1265248821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42415e6b0 con 0x7fe4241451d0 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe422a00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 0x7fe42415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:26.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.888+0000 7fe41be00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe40c035040 tx=0x7fe40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.889+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c042020 con 0x7fe4241451d0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.889+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c03d470 con 0x7fe4241451d0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.889+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c037040 con 0x7fe4241451d0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.889+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42415e990 con 0x7fe4241451d0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.889+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42415ef50 con 0x7fe4241451d0 2024-09-10T07:47:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.890+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42406cfc0 con 0x7fe4241451d0 2024-09-10T07:47:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.891+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe40c045020 con 0x7fe4241451d0 2024-09-10T07:47:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.892+0000 7fe41aa00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 0x7fe40006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.892+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7fe40c0abff0 con 0x7fe4241451d0 2024-09-10T07:47:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.892+0000 7fe422a00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 0x7fe40006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.893+0000 7fe422a00640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 0x7fe40006e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe414007fd0 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:26.896+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe40c04c030 con 0x7fe4241451d0 2024-09-10T07:47:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.119+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe424142390 con 0x7fe4241451d0 2024-09-10T07:47:27.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.696+0000 7fe41aa00640 1 -- 172.21.15.3:0/1265248821 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fe40c07aa70 con 0x7fe4241451d0 2024-09-10T07:47:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.705+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 msgr2=0x7fe40006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.705+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 0x7fe40006e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe414007fd0 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.705+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 msgr2=0x7fe42413ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.705+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe40c035040 tx=0x7fe40c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 shutdown_connections 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40006be30 0x7fe40006e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe424145730 0x7fe42415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4241451d0 0x7fe42413ffe0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe40c035040 tx=0x7fe40c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 --2- 172.21.15.3:0/1265248821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4240cf270 0x7fe42415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.706+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 >> 172.21.15.3:0/1265248821 conn(0x7fe4240b5cb0 msgr2=0x7fe42406e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:27.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.709+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 shutdown_connections 2024-09-10T07:47:27.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.709+0000 7fe429800640 1 -- 172.21.15.3:0/1265248821 wait complete. 2024-09-10T07:47:27.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.898+0000 7fc417e00640 1 Processor -- start 2024-09-10T07:47:27.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.898+0000 7fc417e00640 1 -- start start 2024-09-10T07:47:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc417e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015e800 0x7fc4100cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc417e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4100d2630 0x7fc4100d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc417e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4100b3fa0 con 0x7fc4100b9a90 2024-09-10T07:47:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc417e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc41015ebe0 con 0x7fc41015e800 2024-09-10T07:47:27.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc417e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc4100d3e50 con 0x7fc4100d2630 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc415400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015e800 0x7fc4100cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc415400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41015e800 0x7fc4100cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35920/0 (socket says 172.21.15.3:35920) 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.899+0000 7fc415400640 1 -- 172.21.15.3:0/2190451431 learned_addr learned my addr 172.21.15.3:0/2190451431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.900+0000 7fc414a00640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4100d2630 0x7fc4100d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.900+0000 7fc415400640 1 -- 172.21.15.3:0/2190451431 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4100d2630 msgr2=0x7fc4100d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.900+0000 7fc415400640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4100d2630 0x7fc4100d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.900+0000 7fc415400640 1 -- 172.21.15.3:0/2190451431 >> v1:172.21.15.70:6789/0 conn(0x7fc4100b9a90 legacy=0x7fc4100d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.900+0000 7fc415400640 1 -- 172.21.15.3:0/2190451431 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4100d2b10 con 0x7fc41015e800 2024-09-10T07:47:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.901+0000 7fc415400640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e800 0x7fc4100cf770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc404009870 tx=0x7fc40402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e6eefd350e3ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.901+0000 7fc40fe00640 1 -- 172.21.15.3:0/2190451431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404037070 con 0x7fc41015e800 2024-09-10T07:47:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.901+0000 7fc40fe00640 1 -- 172.21.15.3:0/2190451431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404032440 con 0x7fc41015e800 2024-09-10T07:47:27.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.901+0000 7fc40fe00640 1 -- 172.21.15.3:0/2190451431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404032810 con 0x7fc41015e800 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 -- 172.21.15.3:0/2190451431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e800 msgr2=0x7fc4100cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e800 0x7fc4100cf770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc404009870 tx=0x7fc40402a270 comp rx=0 tx=0).stop 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 -- 172.21.15.3:0/2190451431 shutdown_connections 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4100d2630 0x7fc4100d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 --2- 172.21.15.3:0/2190451431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e800 0x7fc4100cf770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 -- 172.21.15.3:0/2190451431 >> 172.21.15.3:0/2190451431 conn(0x7fc410072560 msgr2=0x7fc410072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 -- 172.21.15.3:0/2190451431 shutdown_connections 2024-09-10T07:47:27.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.902+0000 7fc417e00640 1 -- 172.21.15.3:0/2190451431 wait complete. 2024-09-10T07:47:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.903+0000 7fc417e00640 1 Processor -- start 2024-09-10T07:47:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.904+0000 7fc417e00640 1 -- start start 2024-09-10T07:47:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.904+0000 7fc417e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4100b9a90 0x7fc4101f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.904+0000 7fc417e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc417e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc41015e800 0x7fc4100b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc417e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4100d33b0 con 0x7fc4100d2630 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc417e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc4100d30d0 con 0x7fc4100b9a90 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc417e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc4100d3690 con 0x7fc41015e800 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35930/0 (socket says 172.21.15.3:35930) 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 -- 172.21.15.3:0/3687816498 learned_addr learned my addr 172.21.15.3:0/3687816498 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc415400640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4100b9a90 0x7fc4101f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc415e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc41015e800 0x7fc4100b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 -- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc41015e800 msgr2=0x7fc4100b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc41015e800 0x7fc4100b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 -- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4100b9a90 msgr2=0x7fc4101f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4100b9a90 0x7fc4101f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.905+0000 7fc414a00640 1 -- 172.21.15.3:0/3687816498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4101584d0 con 0x7fc4100d2630 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc414a00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9e0 tx=0x7fc3fc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc017070 con 0x7fc4100d2630 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc00c9d0 con 0x7fc4100d2630 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4100cbc70 con 0x7fc4100d2630 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00cdc0 con 0x7fc4100d2630 2024-09-10T07:47:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.906+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4100cc230 con 0x7fc4100d2630 2024-09-10T07:47:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.907+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4100cf770 con 0x7fc4100d2630 2024-09-10T07:47:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.908+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3fc01a070 con 0x7fc4100d2630 2024-09-10T07:47:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.909+0000 7fc40d600640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 0x7fc3f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.909+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc3fc08b140 con 0x7fc4100d2630 2024-09-10T07:47:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.909+0000 7fc415400640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 0x7fc3f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.910+0000 7fc415400640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 0x7fc3f006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc404034000 tx=0x7fc40402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:27.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:27.913+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3fc059490 con 0x7fc4100d2630 2024-09-10T07:47:28.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.099+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4100b4e60 con 0x7fc4100d2630 2024-09-10T07:47:28.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.100+0000 7fc40d600640 1 -- 172.21.15.3:0/3687816498 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16610 (secure 0 0 0) 0x7fc3fc05d790 con 0x7fc4100d2630 2024-09-10T07:47:28.100 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:28.100 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":35,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:47:27.120823+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:47:25.477493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T07:47:26.113999+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 msgr2=0x7fc3f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 0x7fc3f006e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc404034000 tx=0x7fc40402e000 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 msgr2=0x7fc410157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9e0 tx=0x7fc3fc00eea0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 shutdown_connections 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc3f006be60 0x7fc3f006e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc41015e800 0x7fc4100b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4100d2630 0x7fc410157f90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 --2- 172.21.15.3:0/3687816498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4100b9a90 0x7fc4101f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.102+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 >> 172.21.15.3:0/3687816498 conn(0x7fc410072560 msgr2=0x7fc4100b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.103+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 shutdown_connections 2024-09-10T07:47:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.103+0000 7fc417e00640 1 -- 172.21.15.3:0/3687816498 wait complete. 2024-09-10T07:47:28.118 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.275+0000 7fe4be400640 1 Processor -- start 2024-09-10T07:47:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.275+0000 7fe4be400640 1 -- start start 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4be400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4b80b37a0 0x7fe4b8162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4be400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b80b70b0 con 0x7fe4b80d1c10 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4be400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b80b78b0 con 0x7fe4b815e970 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4be400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b8163210 con 0x7fe4b80b37a0 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4b7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4b80b37a0 0x7fe4b8162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.276+0000 7fe4afe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4b80d1c10 0x7fe4b80ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39142/0 (socket says 172.21.15.3:39142) 2024-09-10T07:47:28.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.277+0000 7fe4afe00640 1 -- 172.21.15.3:0/1518740785 learned_addr learned my addr 172.21.15.3:0/1518740785 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.277+0000 7fe4b7e00640 1 -- 172.21.15.3:0/1518740785 >> v1:172.21.15.70:6789/0 conn(0x7fe4b815e970 legacy=0x7fe4b80d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.277+0000 7fe4b7e00640 1 -- 172.21.15.3:0/1518740785 >> v1:172.21.15.3:6789/0 conn(0x7fe4b80d1c10 legacy=0x7fe4b80ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.277+0000 7fe4b7e00640 1 -- 172.21.15.3:0/1518740785 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b80b62f0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.277+0000 7fe4b6a00640 1 -- 172.21.15.3:0/1518740785 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715552514 0 0) 0x7fe4a0001110 con 0x7fe4b80d1c10 2024-09-10T07:47:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.278+0000 7fe4b7e00640 1 --2- 172.21.15.3:0/1518740785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b8162cd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009d90 tx=0x7fe4b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e01c5c154ca754 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.278+0000 7fe4b6a00640 1 -- 172.21.15.3:0/1518740785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0007740 con 0x7fe4b80b37a0 2024-09-10T07:47:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.278+0000 7fe4b6a00640 1 -- 172.21.15.3:0/1518740785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b00078e0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.278+0000 7fe4b6a00640 1 -- 172.21.15.3:0/1518740785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0007cb0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 -- 172.21.15.3:0/1518740785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 msgr2=0x7fe4b8162cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 --2- 172.21.15.3:0/1518740785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b8162cd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009d90 tx=0x7fe4b002a6f0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 -- 172.21.15.3:0/1518740785 shutdown_connections 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 --2- 172.21.15.3:0/1518740785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b8162cd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 -- 172.21.15.3:0/1518740785 >> 172.21.15.3:0/1518740785 conn(0x7fe4b806bd00 msgr2=0x7fe4b806c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 -- 172.21.15.3:0/1518740785 shutdown_connections 2024-09-10T07:47:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.279+0000 7fe4be400640 1 -- 172.21.15.3:0/1518740785 wait complete. 2024-09-10T07:47:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 Processor -- start 2024-09-10T07:47:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 -- start start 2024-09-10T07:47:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4b80d1c10 0x7fe4b80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4b815e970 0x7fe4b8161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.281+0000 7fe4be400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b68b0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4be400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b3bb0 con 0x7fe4b815e970 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4be400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80b6b90 con 0x7fe4b80d1c10 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35936/0 (socket says 172.21.15.3:35936) 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 -- 172.21.15.3:0/2519075265 learned_addr learned my addr 172.21.15.3:0/2519075265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7e00640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4b815e970 0x7fe4b8161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4afe00640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4b80d1c10 0x7fe4b80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 -- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4b80d1c10 msgr2=0x7fe4b80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4b80d1c10 0x7fe4b80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 -- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4b815e970 msgr2=0x7fe4b8161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4b815e970 0x7fe4b8161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.282+0000 7fe4b7400640 1 -- 172.21.15.3:0/2519075265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b80caaa0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4b7400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800c450 tx=0x7fe4a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80052d0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a8005470 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8012870 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b80cb960 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.283+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b8161840 con 0x7fe4b80b37a0 2024-09-10T07:47:28.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.284+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b8162cd0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.285+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4a8012a10 con 0x7fe4b80b37a0 2024-09-10T07:47:28.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.286+0000 7fe4af400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 0x7fe49406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.286+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe4a808aae0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.286+0000 7fe4afe00640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 0x7fe49406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.287+0000 7fe4afe00640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 0x7fe49406e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe4b81f7e10 tx=0x7fe4a0007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.290+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4b80b78b0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.475+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4b80d2470 con 0x7fe4b80b37a0 2024-09-10T07:47:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.476+0000 7fe4af400640 1 -- 172.21.15.3:0/2519075265 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16610 (secure 0 0 0) 0x7fe4a80593f0 con 0x7fe4b80b37a0 2024-09-10T07:47:28.476 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:28.477 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":35,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:47:27.120823+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:47:25.477493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T07:47:26.113999+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 msgr2=0x7fe49406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 0x7fe49406e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe4b81f7e10 tx=0x7fe4a0007970 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 msgr2=0x7fe4b814ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800c450 tx=0x7fe4a800c910 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 shutdown_connections 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe49406be60 0x7fe49406e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4b815e970 0x7fe4b8161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4b80d1c10 0x7fe4b80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 --2- 172.21.15.3:0/2519075265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4b80b37a0 0x7fe4b814ecc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 >> 172.21.15.3:0/2519075265 conn(0x7fe4b806bd00 msgr2=0x7fe4b815cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 shutdown_connections 2024-09-10T07:47:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.479+0000 7fe4be400640 1 -- 172.21.15.3:0/2519075265 wait complete. 2024-09-10T07:47:28.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:47:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.654+0000 7fe48e800640 1 Processor -- start 2024-09-10T07:47:28.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.654+0000 7fe48e800640 1 -- start start 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe48e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 0x7fe4880d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe48e800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4881620f0 con 0x7fe4880b76c0 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe48e800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4880b7b50 con 0x7fe488161d10 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe48e800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe488072140 con 0x7fe4880cf6a0 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe487e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4880b76c0 0x7fe488072e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39148/0 (socket says 172.21.15.3:39148) 2024-09-10T07:47:28.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.655+0000 7fe487e00640 1 -- 172.21.15.3:0/3859610124 learned_addr learned my addr 172.21.15.3:0/3859610124 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe47fe00640 1 --2- 172.21.15.3:0/3859610124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 0x7fe4880d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089959266 0 0) 0x7fe4881620f0 con 0x7fe4880b76c0 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe46c003520 con 0x7fe4880b76c0 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350880160 0 0) 0x7fe4880b7b50 con 0x7fe488161d10 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4881620f0 con 0x7fe488161d10 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe47fe00640 1 -- 172.21.15.3:0/3859610124 >> v1:172.21.15.70:6789/0 conn(0x7fe488161d10 legacy=0x7fe4880cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe47fe00640 1 -- 172.21.15.3:0/3859610124 >> v1:172.21.15.3:6789/0 conn(0x7fe4880b76c0 legacy=0x7fe488072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.656+0000 7fe47fe00640 1 -- 172.21.15.3:0/3859610124 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4880d4200 con 0x7fe4880cf6a0 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.657+0000 7fe47fe00640 1 --2- 172.21.15.3:0/3859610124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 0x7fe4880d3b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe4700097d0 tx=0x7fe47002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d669c49e1b6d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.657+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470037070 con 0x7fe4880cf6a0 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.657+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe470032440 con 0x7fe4880cf6a0 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.657+0000 7fe486a00640 1 -- 172.21.15.3:0/3859610124 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470032810 con 0x7fe4880cf6a0 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 -- 172.21.15.3:0/3859610124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 msgr2=0x7fe4880d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 --2- 172.21.15.3:0/3859610124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 0x7fe4880d3b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe4700097d0 tx=0x7fe47002a180 comp rx=0 tx=0).stop 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 -- 172.21.15.3:0/3859610124 shutdown_connections 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 --2- 172.21.15.3:0/3859610124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880cf6a0 0x7fe4880d3b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 -- 172.21.15.3:0/3859610124 >> 172.21.15.3:0/3859610124 conn(0x7fe4880d4730 msgr2=0x7fe4880d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 -- 172.21.15.3:0/3859610124 shutdown_connections 2024-09-10T07:47:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.658+0000 7fe48e800640 1 -- 172.21.15.3:0/3859610124 wait complete. 2024-09-10T07:47:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.660+0000 7fe48e800640 1 Processor -- start 2024-09-10T07:47:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.660+0000 7fe48e800640 1 -- start start 2024-09-10T07:47:28.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.660+0000 7fe48e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880b76c0 0x7fe488160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe48e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe48e800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe488161d10 0x7fe48815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe48e800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe488072950 con 0x7fe4880cf6a0 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe48e800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe488072670 con 0x7fe488161d10 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe48e800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4880d3280 con 0x7fe4880b76c0 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35944/0 (socket says 172.21.15.3:35944) 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 -- 172.21.15.3:0/54580844 learned_addr learned my addr 172.21.15.3:0/54580844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe487400640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880b76c0 0x7fe488160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe487e00640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe488161d10 0x7fe48815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 -- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880b76c0 msgr2=0x7fe488160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880b76c0 0x7fe488160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 -- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe488161d10 msgr2=0x7fe48815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe488161d10 0x7fe48815eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.661+0000 7fe47fe00640 1 -- 172.21.15.3:0/54580844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48815ec80 con 0x7fe4880cf6a0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe47fe00640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe470035040 tx=0x7fe4700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470041020 con 0x7fe4880cf6a0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47003d720 con 0x7fe4880cf6a0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470037040 con 0x7fe4880cf6a0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48815dbe0 con 0x7fe4880cf6a0 2024-09-10T07:47:28.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.662+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48815e1a0 con 0x7fe4880cf6a0 2024-09-10T07:47:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.663+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4880d36b0 con 0x7fe4880cf6a0 2024-09-10T07:47:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.664+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe470044020 con 0x7fe4880cf6a0 2024-09-10T07:47:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.665+0000 7fe47f400640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 0x7fe46406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.665+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe4700ab5a0 con 0x7fe4880cf6a0 2024-09-10T07:47:28.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.665+0000 7fe487400640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 0x7fe46406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:28.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.666+0000 7fe487400640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 0x7fe46406e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4780011e0 tx=0x7fe478009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:28.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.669+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe47004b030 con 0x7fe4880cf6a0 2024-09-10T07:47:28.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:28.897+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe4880b8860 con 0x7fe4880cf6a0 2024-09-10T07:47:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.711+0000 7fe47f400640 1 -- 172.21.15.3:0/54580844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe470079ec0 con 0x7fe4880cf6a0 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 msgr2=0x7fe46406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 0x7fe46406e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4780011e0 tx=0x7fe478009040 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 msgr2=0x7fe4880cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe470035040 tx=0x7fe4700046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 shutdown_connections 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe46406be60 0x7fe46406e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe488161d10 0x7fe48815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4880cf6a0 0x7fe4880cdfe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 --2- 172.21.15.3:0/54580844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4880b76c0 0x7fe488160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 >> 172.21.15.3:0/54580844 conn(0x7fe4880d4730 msgr2=0x7fe4880cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 shutdown_connections 2024-09-10T07:47:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.715+0000 7fe48e800640 1 -- 172.21.15.3:0/54580844 wait complete. 2024-09-10T07:47:29.732 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:47:29.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:47:29.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.897+0000 7fdc61c00640 1 Processor -- start 2024-09-10T07:47:29.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.897+0000 7fdc61c00640 1 -- start start 2024-09-10T07:47:29.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc5c161b70 0x7fdc5c0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5c0d3de0 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5c161f50 con 0x7fdc5c161b70 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5c0d40c0 con 0x7fdc5c0d3860 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc5be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc5c161b70 0x7fdc5c0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc5c0d4ae0 0x7fdc5c0d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39162/0 (socket says 172.21.15.3:39162) 2024-09-10T07:47:29.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc61200640 1 -- 172.21.15.3:0/1992365282 learned_addr learned my addr 172.21.15.3:0/1992365282 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:29.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc53e00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc5be00640 1 -- 172.21.15.3:0/1992365282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 msgr2=0x7fdc5c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:29.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc5be00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc5be00640 1 -- 172.21.15.3:0/1992365282 >> v1:172.21.15.3:6789/0 conn(0x7fdc5c0d4ae0 legacy=0x7fdc5c0d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.899+0000 7fdc5be00640 1 -- 172.21.15.3:0/1992365282 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c0cf690 con 0x7fdc5c161b70 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.900+0000 7fdc5be00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0d3320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0097a0 tx=0x7fdc4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599c39e558ca8f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc5b400640 1 -- 172.21.15.3:0/1992365282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c037070 con 0x7fdc5c161b70 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc5b400640 1 -- 172.21.15.3:0/1992365282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4c032440 con 0x7fdc5c161b70 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc5b400640 1 -- 172.21.15.3:0/1992365282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c032810 con 0x7fdc5c161b70 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 -- 172.21.15.3:0/1992365282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c161b70 msgr2=0x7fdc5c0d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0d3320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0097a0 tx=0x7fdc4c02a180 comp rx=0 tx=0).stop 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 -- 172.21.15.3:0/1992365282 shutdown_connections 2024-09-10T07:47:29.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 --2- 172.21.15.3:0/1992365282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0d3320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 -- 172.21.15.3:0/1992365282 >> 172.21.15.3:0/1992365282 conn(0x7fdc5c062630 msgr2=0x7fdc5c062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 -- 172.21.15.3:0/1992365282 shutdown_connections 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.901+0000 7fdc61c00640 1 -- 172.21.15.3:0/1992365282 wait complete. 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.903+0000 7fdc61c00640 1 Processor -- start 2024-09-10T07:47:29.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.903+0000 7fdc61c00640 1 -- start start 2024-09-10T07:47:29.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0cf2d0 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0c9fc0 con 0x7fdc5c161b70 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5c0cf450 con 0x7fdc5c0d3860 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35958/0 (socket says 172.21.15.3:35958) 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc5be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc53e00640 1 -- 172.21.15.3:0/600769316 learned_addr learned my addr 172.21.15.3:0/600769316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.904+0000 7fdc61200640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 -- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 msgr2=0x7fdc5c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 -- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5c161b70 msgr2=0x7fdc5c0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:29.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 -- 172.21.15.3:0/600769316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c0b4bf0 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc53e00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d800 tx=0x7fdc4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400b6c0 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44019070 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400ba80 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c0b4e90 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.905+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c0b5450 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.906+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c0b7d00 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.907+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc4400bc20 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.908+0000 7fdc58c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 0x7fdc3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:29.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.908+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdc4408b2a0 con 0x7fdc5c0d4ae0 2024-09-10T07:47:29.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.908+0000 7fdc5be00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 0x7fdc3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:29.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.909+0000 7fdc5be00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 0x7fdc3806e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c034000 tx=0x7fdc4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:29.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:29.913+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc44059c40 con 0x7fdc5c0d4ae0 2024-09-10T07:47:30.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.133+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdc5c0ca790 con 0x7fdc5c0d4ae0 2024-09-10T07:47:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.713+0000 7fdc58c00640 1 -- 172.21.15.3:0/600769316 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fdc44059540 con 0x7fdc5c0d4ae0 2024-09-10T07:47:30.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.715+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 msgr2=0x7fdc3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.715+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 0x7fdc3806e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c034000 tx=0x7fdc4c02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.715+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 msgr2=0x7fdc5c14fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.715+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d800 tx=0x7fdc4400dcc0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 shutdown_connections 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc3806be30 0x7fdc3806e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5c161b70 0x7fdc5c0cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc5c0d4ae0 0x7fdc5c14fda0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 --2- 172.21.15.3:0/600769316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc5c0d3860 0x7fdc5c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 >> 172.21.15.3:0/600769316 conn(0x7fdc5c062630 msgr2=0x7fdc5c15ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 shutdown_connections 2024-09-10T07:47:30.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.716+0000 7fdc61c00640 1 -- 172.21.15.3:0/600769316 wait complete. 2024-09-10T07:47:30.732 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:47:30.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:30.792 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:47:30.793+0000 7fc186000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:47:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.892+0000 7f12cd800640 1 Processor -- start 2024-09-10T07:47:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.892+0000 7f12cd800640 1 -- start start 2024-09-10T07:47:30.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12cd800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12c815e860 0x7f12c80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12cd800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 0x7f12c80d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12cd800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c80cee40 con 0x7f12c80d4150 2024-09-10T07:47:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12cd800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f12c80d3780 con 0x7f12c815e860 2024-09-10T07:47:30.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12cd800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f12c80cf120 con 0x7f12c80cfbf0 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12c7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12c815e860 0x7f12c80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.893+0000 7f12c7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12c815e860 0x7f12c80cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35970/0 (socket says 172.21.15.3:35970) 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7e00640 1 -- 172.21.15.3:0/1166869598 learned_addr learned my addr 172.21.15.3:0/1166869598 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7400640 1 --2- 172.21.15.3:0/1166869598 >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 0x7f12c80d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7e00640 1 -- 172.21.15.3:0/1166869598 >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 msgr2=0x7f12c80d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7e00640 1 --2- 172.21.15.3:0/1166869598 >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 0x7f12c80d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7e00640 1 -- 172.21.15.3:0/1166869598 >> v1:172.21.15.70:6789/0 conn(0x7f12c80d4150 legacy=0x7f12c80b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7e00640 1 -- 172.21.15.3:0/1166869598 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c80d3410 con 0x7f12c815e860 2024-09-10T07:47:30.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.894+0000 7f12c7400640 1 --2- 172.21.15.3:0/1166869598 >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 0x7f12c80d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:30.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.895+0000 7f12c7e00640 1 --2- 172.21.15.3:0/1166869598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c815e860 0x7f12c80cf6b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f12c80d0ad0 tx=0x7f12b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6426d9d454b42241 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.895+0000 7f12c6a00640 1 -- 172.21.15.3:0/1166869598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037070 con 0x7f12c815e860 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.895+0000 7f12c6a00640 1 -- 172.21.15.3:0/1166869598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8032440 con 0x7f12c815e860 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.895+0000 7f12c6a00640 1 -- 172.21.15.3:0/1166869598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8032810 con 0x7f12c815e860 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 -- 172.21.15.3:0/1166869598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c815e860 msgr2=0x7f12c80cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 --2- 172.21.15.3:0/1166869598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c815e860 0x7f12c80cf6b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f12c80d0ad0 tx=0x7f12b802a270 comp rx=0 tx=0).stop 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 -- 172.21.15.3:0/1166869598 shutdown_connections 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 --2- 172.21.15.3:0/1166869598 >> v2:172.21.15.70:3300/0 conn(0x7f12c80cfbf0 0x7f12c80d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 --2- 172.21.15.3:0/1166869598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c815e860 0x7f12c80cf6b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 -- 172.21.15.3:0/1166869598 >> 172.21.15.3:0/1166869598 conn(0x7f12c80d4650 msgr2=0x7f12c80d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 -- 172.21.15.3:0/1166869598 shutdown_connections 2024-09-10T07:47:30.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.896+0000 7f12cd800640 1 -- 172.21.15.3:0/1166869598 wait complete. 2024-09-10T07:47:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.897+0000 7f12cd800640 1 Processor -- start 2024-09-10T07:47:30.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 -- start start 2024-09-10T07:47:30.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c80d4150 0x7f12c80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12c815e860 0x7f12c814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12c80b7f90 con 0x7f12c80cfbf0 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f12c80d2460 con 0x7f12c80d4150 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.898+0000 7f12cd800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f12c80b8110 con 0x7f12c815e860 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35986/0 (socket says 172.21.15.3:35986) 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 -- 172.21.15.3:0/48848739 learned_addr learned my addr 172.21.15.3:0/48848739 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7400640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c80d4150 0x7f12c80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12cce00640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12c815e860 0x7f12c814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 -- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12c815e860 msgr2=0x7f12c814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12c815e860 0x7f12c814fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 -- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c80d4150 msgr2=0x7f12c80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c80d4150 0x7f12c80cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:30.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 -- 172.21.15.3:0/48848739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c814ec40 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.899+0000 7f12c7e00640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f12b802a2b0 tx=0x7f12b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.900+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8041020 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.900+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b80306c0 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.900+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037040 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.900+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c814ef20 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.900+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c814f7c0 con 0x7f12c80cfbf0 2024-09-10T07:47:30.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.901+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c80d3c10 con 0x7f12c80cfbf0 2024-09-10T07:47:30.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.902+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12b8044020 con 0x7f12c80cfbf0 2024-09-10T07:47:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.903+0000 7f12bfe00640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 0x7f12a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.903+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f12b80ab740 con 0x7f12c80cfbf0 2024-09-10T07:47:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.903+0000 7f12c7400640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 0x7f12a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:30.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.904+0000 7f12c7400640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 0x7f12a806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12c80cde30 tx=0x7f12b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:30.908+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12b804b030 con 0x7f12c80cfbf0 2024-09-10T07:47:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.129+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12c80b5d80 con 0x7f12c80cfbf0 2024-09-10T07:47:31.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.130+0000 7f12bfe00640 1 -- 172.21.15.3:0/48848739 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5214 (secure 0 0 0) 0x7f12b807a050 con 0x7f12c80cfbf0 2024-09-10T07:47:31.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:31.130 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"btime":"2024-09-10T07:47:30:712515+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:30.135093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741]},"id":2}]} 2024-09-10T07:47:31.130 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 msgr2=0x7f12a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 0x7f12a806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12c80cde30 tx=0x7f12b4008040 comp rx=0 tx=0).stop 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 msgr2=0x7f12c80ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f12b802a2b0 tx=0x7f12b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 shutdown_connections 2024-09-10T07:47:31.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12a806be30 0x7f12a806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12c815e860 0x7f12c814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c80d4150 0x7f12c80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 --2- 172.21.15.3:0/48848739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12c80cfbf0 0x7f12c80ab9b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 >> 172.21.15.3:0/48848739 conn(0x7f12c80d4650 msgr2=0x7f12c8073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 shutdown_connections 2024-09-10T07:47:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.132+0000 7f12cd800640 1 -- 172.21.15.3:0/48848739 wait complete. 2024-09-10T07:47:31.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:47:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.304+0000 7f64db200640 1 Processor -- start 2024-09-10T07:47:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.304+0000 7f64db200640 1 -- start start 2024-09-10T07:47:31.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.305+0000 7f64db200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64d415b110 0x7f64d40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.305+0000 7f64db200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 0x7f64d40cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.305+0000 7f64db200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64d40b4250 con 0x7f64d415b4f0 2024-09-10T07:47:31.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.305+0000 7f64db200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f64d40b44f0 con 0x7f64d415b110 2024-09-10T07:47:31.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.305+0000 7f64db200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f64d40ccd80 con 0x7f64d40d08f0 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64d415b110 0x7f64d40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64d415b110 0x7f64d40b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36000/0 (socket says 172.21.15.3:36000) 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 -- 172.21.15.3:0/815060943 learned_addr learned my addr 172.21.15.3:0/815060943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d9200640 1 --2- 172.21.15.3:0/815060943 >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 0x7f64d40cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 -- 172.21.15.3:0/815060943 >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 msgr2=0x7f64d40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:31.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 --2- 172.21.15.3:0/815060943 >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 0x7f64d40cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 -- 172.21.15.3:0/815060943 >> v1:172.21.15.70:6789/0 conn(0x7f64d415b4f0 legacy=0x7f64d40d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d3e00640 1 -- 172.21.15.3:0/815060943 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d40cd060 con 0x7f64d415b110 2024-09-10T07:47:31.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.306+0000 7f64d9200640 1 --2- 172.21.15.3:0/815060943 >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 0x7f64d40cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.307+0000 7f64d3e00640 1 --2- 172.21.15.3:0/815060943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d415b110 0x7f64d40b3d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64d40cfbb0 tx=0x7f64c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4250fc2f1c763d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.307+0000 7f64d3400640 1 -- 172.21.15.3:0/815060943 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4037070 con 0x7f64d415b110 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.307+0000 7f64d3400640 1 -- 172.21.15.3:0/815060943 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c4032440 con 0x7f64d415b110 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.307+0000 7f64d3400640 1 -- 172.21.15.3:0/815060943 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4032810 con 0x7f64d415b110 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 -- 172.21.15.3:0/815060943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d415b110 msgr2=0x7f64d40b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 --2- 172.21.15.3:0/815060943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d415b110 0x7f64d40b3d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f64d40cfbb0 tx=0x7f64c402a270 comp rx=0 tx=0).stop 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 -- 172.21.15.3:0/815060943 shutdown_connections 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 --2- 172.21.15.3:0/815060943 >> v2:172.21.15.70:3300/0 conn(0x7f64d40d08f0 0x7f64d40cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 --2- 172.21.15.3:0/815060943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d415b110 0x7f64d40b3d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 -- 172.21.15.3:0/815060943 >> 172.21.15.3:0/815060943 conn(0x7f64d406d000 msgr2=0x7f64d406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 -- 172.21.15.3:0/815060943 shutdown_connections 2024-09-10T07:47:31.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.308+0000 7f64db200640 1 -- 172.21.15.3:0/815060943 wait complete. 2024-09-10T07:47:31.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.310+0000 7f64db200640 1 Processor -- start 2024-09-10T07:47:31.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.310+0000 7f64db200640 1 -- start start 2024-09-10T07:47:31.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 0x7f64d415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 0x7f64d415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4073d10 con 0x7f64d40d08f0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f64d40cd620 con 0x7f64d415b110 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64db200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f64d4073e90 con 0x7f64d415b4f0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36012/0 (socket says 172.21.15.3:36012) 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d3e00640 1 -- 172.21.15.3:0/2003370260 learned_addr learned my addr 172.21.15.3:0/2003370260 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64cbe00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 0x7f64d415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d9200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 0x7f64d415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d3e00640 1 -- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 msgr2=0x7f64d415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.311+0000 7f64d3e00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 0x7f64d415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d3e00640 1 -- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 msgr2=0x7f64d415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d3e00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 0x7f64d415d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d3e00640 1 -- 172.21.15.3:0/2003370260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d415ddb0 con 0x7f64d40d08f0 2024-09-10T07:47:31.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64cbe00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 0x7f64d415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d9200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 0x7f64d415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d3e00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64c402a2b0 tx=0x7f64c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4041020 con 0x7f64d40d08f0 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c40306c0 con 0x7f64d40d08f0 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4037040 con 0x7f64d40d08f0 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d4154a80 con 0x7f64d40d08f0 2024-09-10T07:47:31.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.312+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d4154d20 con 0x7f64d40d08f0 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.313+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d40b3d10 con 0x7f64d40d08f0 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.314+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64c4044020 con 0x7f64d40d08f0 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.315+0000 7f64d0c00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 0x7f64b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.315+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f64c40ab740 con 0x7f64d40d08f0 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.315+0000 7f64cbe00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 0x7f64b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.316+0000 7f64cbe00640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 0x7f64b006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007fd0 tx=0x7f64bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.319+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64c404b030 con 0x7f64d40d08f0 2024-09-10T07:47:31.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.505+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64d40afb60 con 0x7f64d40d08f0 2024-09-10T07:47:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.507+0000 7f64d0c00640 1 -- 172.21.15.3:0/2003370260 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16597 (secure 0 0 0) 0x7f64c407a050 con 0x7f64d40d08f0 2024-09-10T07:47:31.507 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:31.507 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:47:28.697555+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:47:25.477493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T07:47:26.113999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 msgr2=0x7f64b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 0x7f64b006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007fd0 tx=0x7f64bc009040 comp rx=0 tx=0).stop 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 msgr2=0x7f64d414b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64c402a2b0 tx=0x7f64c40046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 shutdown_connections 2024-09-10T07:47:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f64b006be30 0x7f64b006e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64d415b4f0 0x7f64d415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d415b110 0x7f64d415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 --2- 172.21.15.3:0/2003370260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64d40d08f0 0x7f64d414b5c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 >> 172.21.15.3:0/2003370260 conn(0x7f64d406d000 msgr2=0x7f64d406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 shutdown_connections 2024-09-10T07:47:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.509+0000 7f64db200640 1 -- 172.21.15.3:0/2003370260 wait complete. 2024-09-10T07:47:31.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:47:31.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:47:31.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.682+0000 7f951c200640 1 Processor -- start 2024-09-10T07:47:31.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.683+0000 7f951c200640 1 -- start start 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f951c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 0x7f95140ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f951c200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95140ccd80 con 0x7f951415b4f0 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f951c200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95140cd060 con 0x7f951415b110 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f951c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f95140b44f0 con 0x7f95140d08f0 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f9518e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f951415b4f0 0x7f95140d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39170/0 (socket says 172.21.15.3:39170) 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f9518e00640 1 -- 172.21.15.3:0/1814843680 learned_addr learned my addr 172.21.15.3:0/1814843680 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.684+0000 7f951a200640 1 --2- 172.21.15.3:0/1814843680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 0x7f95140ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497215863 0 0) 0x7f95140ccd80 con 0x7f951415b4f0 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94fc003520 con 0x7f951415b4f0 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186876379 0 0) 0x7f95140cd060 con 0x7f951415b110 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95140ccd80 con 0x7f951415b110 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f951a200640 1 -- 172.21.15.3:0/1814843680 >> v1:172.21.15.70:6789/0 conn(0x7f951415b110 legacy=0x7f95140b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2461743230 0 0) 0x7f94fc003520 con 0x7f951415b4f0 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f951a200640 1 -- 172.21.15.3:0/1814843680 >> v1:172.21.15.3:6789/0 conn(0x7f951415b4f0 legacy=0x7f95140d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.685+0000 7f951a200640 1 -- 172.21.15.3:0/1814843680 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95140cd340 con 0x7f95140d08f0 2024-09-10T07:47:31.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.686+0000 7f951a200640 1 --2- 172.21.15.3:0/1814843680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 0x7f95140ceb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f950c009dc0 tx=0x7f950c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c84979f3828b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.686+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500001670 con 0x7f95140d08f0 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.686+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c007540 con 0x7f95140d08f0 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.686+0000 7f9513e00640 1 -- 172.21.15.3:0/1814843680 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c007910 con 0x7f95140d08f0 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 -- 172.21.15.3:0/1814843680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 msgr2=0x7f95140ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 --2- 172.21.15.3:0/1814843680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 0x7f95140ceb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f950c009dc0 tx=0x7f950c02a660 comp rx=0 tx=0).stop 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 -- 172.21.15.3:0/1814843680 shutdown_connections 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 --2- 172.21.15.3:0/1814843680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95140d08f0 0x7f95140ceb50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 -- 172.21.15.3:0/1814843680 >> 172.21.15.3:0/1814843680 conn(0x7f951406d000 msgr2=0x7f951406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 -- 172.21.15.3:0/1814843680 shutdown_connections 2024-09-10T07:47:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.687+0000 7f951c200640 1 -- 172.21.15.3:0/1814843680 wait complete. 2024-09-10T07:47:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.689+0000 7f951c200640 1 Processor -- start 2024-09-10T07:47:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.689+0000 7f951c200640 1 -- start start 2024-09-10T07:47:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 0x7f951414d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 0x7f951413c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9514073e90 con 0x7f95140d08f0 2024-09-10T07:47:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9514073d10 con 0x7f951415b110 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9514074010 con 0x7f951415b4f0 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36026/0 (socket says 172.21.15.3:36026) 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 -- 172.21.15.3:0/2156001780 learned_addr learned my addr 172.21.15.3:0/2156001780 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951a200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 0x7f951413c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9518e00640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 0x7f951414d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 -- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 msgr2=0x7f951413c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 0x7f951413c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 -- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 msgr2=0x7f951414d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 0x7f951414d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9519800640 1 -- 172.21.15.3:0/2156001780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141f6ab0 con 0x7f95140d08f0 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f951a200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 0x7f951413c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.690+0000 7f9518e00640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 0x7f951414d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f9519800640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f950800c490 tx=0x7f950800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508017070 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95080052d0 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508003830 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951414b5a0 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.691+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951414bb60 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.692+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95140b3d10 con 0x7f95140d08f0 2024-09-10T07:47:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.693+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f950801a070 con 0x7f95140d08f0 2024-09-10T07:47:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.694+0000 7f9511600640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 0x7f94f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.694+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f950808a310 con 0x7f95140d08f0 2024-09-10T07:47:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.694+0000 7f9518e00640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 0x7f94f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.695+0000 7f9518e00640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 0x7f94f406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f951413db00 tx=0x7f9500009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.699+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9508058c20 con 0x7f95140d08f0 2024-09-10T07:47:31.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.884+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95140d0560 con 0x7f94f406be30 2024-09-10T07:47:31.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.890+0000 7f9511600640 1 -- 172.21.15.3:0/2156001780 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249138 (secure 0 0 0) 0x7f95140d0560 con 0x7f94f406be30 2024-09-10T07:47:31.893 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:31.895 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:47:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 msgr2=0x7f94f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 0x7f94f406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f951413db00 tx=0x7f9500009040 comp rx=0 tx=0).stop 2024-09-10T07:47:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 msgr2=0x7f951414c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f950800c490 tx=0x7f950800c950 comp rx=0 tx=0).stop 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 shutdown_connections 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94f406be30 0x7f94f406e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f951415b4f0 0x7f951413c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f951415b110 0x7f951414d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 --2- 172.21.15.3:0/2156001780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95140d08f0 0x7f951414c360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.894+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 >> 172.21.15.3:0/2156001780 conn(0x7f951406d000 msgr2=0x7f95140ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.895+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 shutdown_connections 2024-09-10T07:47:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:31.895+0000 7f951c200640 1 -- 172.21.15.3:0/2156001780 wait complete. 2024-09-10T07:47:31.910 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T07:47:31.501965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236996,"kb_used_data":4116,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737724,"statfs":{"total":773094113280,"available":772851429376,"internally_reserved":0,"allocated":4214784,"data_stored":2330373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815909"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.702272+0000","last_change":"2024-09-10T07:47:26.702272+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.702272+0000","last_undegraded":"2024-09-10T07:47:26.702272+0000","last_fullsized":"2024-09-10T07:47:26.702272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709608+0000","last_change":"2024-09-10T07:47:27.717295+0000","last_active":"2024-09-10T07:47:28.709608+0000","last_peered":"2024-09-10T07:47:28.709608+0000","last_clean":"2024-09-10T07:47:28.709608+0000","last_became_active":"2024-09-10T07:47:27.716979+0000","last_became_peered":"2024-09-10T07:47:27.716979+0000","last_unstale":"2024-09-10T07:47:28.709608+0000","last_undegraded":"2024-09-10T07:47:28.709608+0000","last_fullsized":"2024-09-10T07:47:28.709608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695776+0000","last_change":"2024-09-10T07:47:26.695776+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695776+0000","last_undegraded":"2024-09-10T07:47:26.695776+0000","last_fullsized":"2024-09-10T07:47:26.695776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731683+0000","last_change":"2024-09-10T07:47:27.726560+0000","last_active":"2024-09-10T07:47:27.731683+0000","last_peered":"2024-09-10T07:47:27.731683+0000","last_clean":"2024-09-10T07:47:27.731683+0000","last_became_active":"2024-09-10T07:47:27.726024+0000","last_became_peered":"2024-09-10T07:47:27.726024+0000","last_unstale":"2024-09-10T07:47:27.731683+0000","last_undegraded":"2024-09-10T07:47:27.731683+0000","last_fullsized":"2024-09-10T07:47:27.731683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:50.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709319+0000","last_change":"2024-09-10T07:47:27.710914+0000","last_active":"2024-09-10T07:47:28.709319+0000","last_peered":"2024-09-10T07:47:28.709319+0000","last_clean":"2024-09-10T07:47:28.709319+0000","last_became_active":"2024-09-10T07:47:27.710617+0000","last_became_peered":"2024-09-10T07:47:27.710617+0000","last_unstale":"2024-09-10T07:47:28.709319+0000","last_undegraded":"2024-09-10T07:47:28.709319+0000","last_fullsized":"2024-09-10T07:47:28.709319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706883+0000","last_change":"2024-09-10T07:47:27.721104+0000","last_active":"2024-09-10T07:47:28.706883+0000","last_peered":"2024-09-10T07:47:28.706883+0000","last_clean":"2024-09-10T07:47:28.706883+0000","last_became_active":"2024-09-10T07:47:27.718624+0000","last_became_peered":"2024-09-10T07:47:27.718624+0000","last_unstale":"2024-09-10T07:47:28.706883+0000","last_undegraded":"2024-09-10T07:47:28.706883+0000","last_fullsized":"2024-09-10T07:47:28.706883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697623+0000","last_change":"2024-09-10T07:47:26.697623+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697623+0000","last_undegraded":"2024-09-10T07:47:26.697623+0000","last_fullsized":"2024-09-10T07:47:26.697623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699517+0000","last_change":"2024-09-10T07:47:26.699517+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699517+0000","last_undegraded":"2024-09-10T07:47:26.699517+0000","last_fullsized":"2024-09-10T07:47:26.699517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697854+0000","last_change":"2024-09-10T07:47:26.697854+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697854+0000","last_undegraded":"2024-09-10T07:47:26.697854+0000","last_fullsized":"2024-09-10T07:47:26.697854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728902+0000","last_change":"2024-09-10T07:47:27.717546+0000","last_active":"2024-09-10T07:47:27.728902+0000","last_peered":"2024-09-10T07:47:27.728902+0000","last_clean":"2024-09-10T07:47:27.728902+0000","last_became_active":"2024-09-10T07:47:27.713266+0000","last_became_peered":"2024-09-10T07:47:27.713266+0000","last_unstale":"2024-09-10T07:47:27.728902+0000","last_undegraded":"2024-09-10T07:47:27.728902+0000","last_fullsized":"2024-09-10T07:47:27.728902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:58.170897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697313+0000","last_change":"2024-09-10T07:47:26.697313+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697313+0000","last_undegraded":"2024-09-10T07:47:26.697313+0000","last_fullsized":"2024-09-10T07:47:26.697313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696911+0000","last_change":"2024-09-10T07:47:26.696911+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696911+0000","last_undegraded":"2024-09-10T07:47:26.696911+0000","last_fullsized":"2024-09-10T07:47:26.696911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271853+0000","last_change":"2024-09-10T07:47:27.728693+0000","last_active":"2024-09-10T07:47:29.271853+0000","last_peered":"2024-09-10T07:47:29.271853+0000","last_clean":"2024-09-10T07:47:29.271853+0000","last_became_active":"2024-09-10T07:47:27.728426+0000","last_became_peered":"2024-09-10T07:47:27.728426+0000","last_unstale":"2024-09-10T07:47:29.271853+0000","last_undegraded":"2024-09-10T07:47:29.271853+0000","last_fullsized":"2024-09-10T07:47:29.271853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.466215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.721891+0000","last_change":"2024-09-10T07:47:27.713960+0000","last_active":"2024-09-10T07:47:27.721891+0000","last_peered":"2024-09-10T07:47:27.721891+0000","last_clean":"2024-09-10T07:47:27.721891+0000","last_became_active":"2024-09-10T07:47:27.713692+0000","last_became_peered":"2024-09-10T07:47:27.713692+0000","last_unstale":"2024-09-10T07:47:27.721891+0000","last_undegraded":"2024-09-10T07:47:27.721891+0000","last_fullsized":"2024-09-10T07:47:27.721891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:50.495716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696378+0000","last_change":"2024-09-10T07:47:26.696378+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696378+0000","last_undegraded":"2024-09-10T07:47:26.696378+0000","last_fullsized":"2024-09-10T07:47:26.696378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697490+0000","last_change":"2024-09-10T07:47:26.697490+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697490+0000","last_undegraded":"2024-09-10T07:47:26.697490+0000","last_fullsized":"2024-09-10T07:47:26.697490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698719+0000","last_change":"2024-09-10T07:47:26.698719+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698719+0000","last_undegraded":"2024-09-10T07:47:26.698719+0000","last_fullsized":"2024-09-10T07:47:26.698719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707015+0000","last_change":"2024-09-10T07:47:27.717429+0000","last_active":"2024-09-10T07:47:28.707015+0000","last_peered":"2024-09-10T07:47:28.707015+0000","last_clean":"2024-09-10T07:47:28.707015+0000","last_became_active":"2024-09-10T07:47:27.717064+0000","last_became_peered":"2024-09-10T07:47:27.717064+0000","last_unstale":"2024-09-10T07:47:28.707015+0000","last_undegraded":"2024-09-10T07:47:28.707015+0000","last_fullsized":"2024-09-10T07:47:28.707015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:44.926594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706888+0000","last_change":"2024-09-10T07:47:27.715748+0000","last_active":"2024-09-10T07:47:28.706888+0000","last_peered":"2024-09-10T07:47:28.706888+0000","last_clean":"2024-09-10T07:47:28.706888+0000","last_became_active":"2024-09-10T07:47:27.715354+0000","last_became_peered":"2024-09-10T07:47:27.715354+0000","last_unstale":"2024-09-10T07:47:28.706888+0000","last_undegraded":"2024-09-10T07:47:28.706888+0000","last_fullsized":"2024-09-10T07:47:28.706888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272066+0000","last_change":"2024-09-10T07:47:27.726942+0000","last_active":"2024-09-10T07:47:29.272066+0000","last_peered":"2024-09-10T07:47:29.272066+0000","last_clean":"2024-09-10T07:47:29.272066+0000","last_became_active":"2024-09-10T07:47:27.726536+0000","last_became_peered":"2024-09-10T07:47:27.726536+0000","last_unstale":"2024-09-10T07:47:29.272066+0000","last_undegraded":"2024-09-10T07:47:29.272066+0000","last_fullsized":"2024-09-10T07:47:29.272066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:25.590064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695528+0000","last_change":"2024-09-10T07:47:26.695528+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695528+0000","last_undegraded":"2024-09-10T07:47:26.695528+0000","last_fullsized":"2024-09-10T07:47:26.695528+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.701477+0000","last_change":"2024-09-10T07:47:26.701477+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.701477+0000","last_undegraded":"2024-09-10T07:47:26.701477+0000","last_fullsized":"2024-09-10T07:47:26.701477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271941+0000","last_change":"2024-09-10T07:47:27.721416+0000","last_active":"2024-09-10T07:47:29.271941+0000","last_peered":"2024-09-10T07:47:29.271941+0000","last_clean":"2024-09-10T07:47:29.271941+0000","last_became_active":"2024-09-10T07:47:27.721174+0000","last_became_peered":"2024-09-10T07:47:27.721174+0000","last_unstale":"2024-09-10T07:47:29.271941+0000","last_undegraded":"2024-09-10T07:47:29.271941+0000","last_fullsized":"2024-09-10T07:47:29.271941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:17.894548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699069+0000","last_change":"2024-09-10T07:47:26.699069+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699069+0000","last_undegraded":"2024-09-10T07:47:26.699069+0000","last_fullsized":"2024-09-10T07:47:26.699069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.732178+0000","last_change":"2024-09-10T07:47:27.723555+0000","last_active":"2024-09-10T07:47:27.732178+0000","last_peered":"2024-09-10T07:47:27.732178+0000","last_clean":"2024-09-10T07:47:27.732178+0000","last_became_active":"2024-09-10T07:47:27.723226+0000","last_became_peered":"2024-09-10T07:47:27.723226+0000","last_unstale":"2024-09-10T07:47:27.732178+0000","last_undegraded":"2024-09-10T07:47:27.732178+0000","last_fullsized":"2024-09-10T07:47:27.732178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:55.167875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731804+0000","last_change":"2024-09-10T07:47:27.729136+0000","last_active":"2024-09-10T07:47:27.731804+0000","last_peered":"2024-09-10T07:47:27.731804+0000","last_clean":"2024-09-10T07:47:27.731804+0000","last_became_active":"2024-09-10T07:47:27.724907+0000","last_became_peered":"2024-09-10T07:47:27.724907+0000","last_unstale":"2024-09-10T07:47:27.731804+0000","last_undegraded":"2024-09-10T07:47:27.731804+0000","last_fullsized":"2024-09-10T07:47:27.731804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:12.255456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709383+0000","last_change":"2024-09-10T07:47:27.710661+0000","last_active":"2024-09-10T07:47:28.709383+0000","last_peered":"2024-09-10T07:47:28.709383+0000","last_clean":"2024-09-10T07:47:28.709383+0000","last_became_active":"2024-09-10T07:47:27.710117+0000","last_became_peered":"2024-09-10T07:47:27.710117+0000","last_unstale":"2024-09-10T07:47:28.709383+0000","last_undegraded":"2024-09-10T07:47:28.709383+0000","last_fullsized":"2024-09-10T07:47:28.709383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700878+0000","last_change":"2024-09-10T07:47:26.700878+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700878+0000","last_undegraded":"2024-09-10T07:47:26.700878+0000","last_fullsized":"2024-09-10T07:47:26.700878+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694553+0000","last_change":"2024-09-10T07:47:26.694553+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694553+0000","last_undegraded":"2024-09-10T07:47:26.694553+0000","last_fullsized":"2024-09-10T07:47:26.694553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694437+0000","last_change":"2024-09-10T07:47:26.694437+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694437+0000","last_undegraded":"2024-09-10T07:47:26.694437+0000","last_fullsized":"2024-09-10T07:47:26.694437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695915+0000","last_change":"2024-09-10T07:47:26.695915+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695915+0000","last_undegraded":"2024-09-10T07:47:26.695915+0000","last_fullsized":"2024-09-10T07:47:26.695915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697245+0000","last_change":"2024-09-10T07:47:26.697245+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697245+0000","last_undegraded":"2024-09-10T07:47:26.697245+0000","last_fullsized":"2024-09-10T07:47:26.697245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272289+0000","last_change":"2024-09-10T07:47:27.714639+0000","last_active":"2024-09-10T07:47:29.272289+0000","last_peered":"2024-09-10T07:47:29.272289+0000","last_clean":"2024-09-10T07:47:29.272289+0000","last_became_active":"2024-09-10T07:47:27.714372+0000","last_became_peered":"2024-09-10T07:47:27.714372+0000","last_unstale":"2024-09-10T07:47:29.272289+0000","last_undegraded":"2024-09-10T07:47:29.272289+0000","last_fullsized":"2024-09-10T07:47:29.272289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715183+0000","last_change":"2024-09-10T07:47:26.711007+0000","last_active":"2024-09-10T07:47:26.715183+0000","last_peered":"2024-09-10T07:47:26.715183+0000","last_clean":"2024-09-10T07:47:26.715183+0000","last_became_active":"2024-09-10T07:47:26.710707+0000","last_became_peered":"2024-09-10T07:47:26.710707+0000","last_unstale":"2024-09-10T07:47:26.715183+0000","last_undegraded":"2024-09-10T07:47:26.715183+0000","last_fullsized":"2024-09-10T07:47:26.715183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:11.897007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695191+0000","last_change":"2024-09-10T07:47:26.695191+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695191+0000","last_undegraded":"2024-09-10T07:47:26.695191+0000","last_fullsized":"2024-09-10T07:47:26.695191+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.737966+0000","last_change":"2024-09-10T07:47:26.717356+0000","last_active":"2024-09-10T07:47:27.737966+0000","last_peered":"2024-09-10T07:47:27.737966+0000","last_clean":"2024-09-10T07:47:27.737966+0000","last_became_active":"2024-09-10T07:47:26.717075+0000","last_became_peered":"2024-09-10T07:47:26.717075+0000","last_unstale":"2024-09-10T07:47:27.737966+0000","last_undegraded":"2024-09-10T07:47:27.737966+0000","last_fullsized":"2024-09-10T07:47:27.737966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:23.907465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695001+0000","last_change":"2024-09-10T07:47:26.695001+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695001+0000","last_undegraded":"2024-09-10T07:47:26.695001+0000","last_fullsized":"2024-09-10T07:47:26.695001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707037+0000","last_change":"2024-09-10T07:47:26.709026+0000","last_active":"2024-09-10T07:47:28.707037+0000","last_peered":"2024-09-10T07:47:28.707037+0000","last_clean":"2024-09-10T07:47:28.707037+0000","last_became_active":"2024-09-10T07:47:26.708550+0000","last_became_peered":"2024-09-10T07:47:26.708550+0000","last_unstale":"2024-09-10T07:47:28.707037+0000","last_undegraded":"2024-09-10T07:47:28.707037+0000","last_fullsized":"2024-09-10T07:47:28.707037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730160+0000","last_change":"2024-09-10T07:47:27.722622+0000","last_active":"2024-09-10T07:47:27.730160+0000","last_peered":"2024-09-10T07:47:27.730160+0000","last_clean":"2024-09-10T07:47:27.730160+0000","last_became_active":"2024-09-10T07:47:27.722182+0000","last_became_peered":"2024-09-10T07:47:27.722182+0000","last_unstale":"2024-09-10T07:47:27.730160+0000","last_undegraded":"2024-09-10T07:47:27.730160+0000","last_fullsized":"2024-09-10T07:47:27.730160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:45.005644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.734519+0000","last_change":"2024-09-10T07:47:26.714730+0000","last_active":"2024-09-10T07:47:27.734519+0000","last_peered":"2024-09-10T07:47:27.734519+0000","last_clean":"2024-09-10T07:47:27.734519+0000","last_became_active":"2024-09-10T07:47:26.713780+0000","last_became_peered":"2024-09-10T07:47:26.713780+0000","last_unstale":"2024-09-10T07:47:27.734519+0000","last_undegraded":"2024-09-10T07:47:27.734519+0000","last_fullsized":"2024-09-10T07:47:27.734519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:59.575014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694052+0000","last_change":"2024-09-10T07:47:26.694052+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694052+0000","last_undegraded":"2024-09-10T07:47:26.694052+0000","last_fullsized":"2024-09-10T07:47:26.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716387+0000","last_change":"2024-09-10T07:47:26.711406+0000","last_active":"2024-09-10T07:47:26.716387+0000","last_peered":"2024-09-10T07:47:26.716387+0000","last_clean":"2024-09-10T07:47:26.716387+0000","last_became_active":"2024-09-10T07:47:26.710814+0000","last_became_peered":"2024-09-10T07:47:26.710814+0000","last_unstale":"2024-09-10T07:47:26.716387+0000","last_undegraded":"2024-09-10T07:47:26.716387+0000","last_fullsized":"2024-09-10T07:47:26.716387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696890+0000","last_change":"2024-09-10T07:47:26.696890+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696890+0000","last_undegraded":"2024-09-10T07:47:26.696890+0000","last_fullsized":"2024-09-10T07:47:26.696890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716460+0000","last_change":"2024-09-10T07:47:26.713426+0000","last_active":"2024-09-10T07:47:26.716460+0000","last_peered":"2024-09-10T07:47:26.716460+0000","last_clean":"2024-09-10T07:47:26.716460+0000","last_became_active":"2024-09-10T07:47:26.712028+0000","last_became_peered":"2024-09-10T07:47:26.712028+0000","last_unstale":"2024-09-10T07:47:26.716460+0000","last_undegraded":"2024-09-10T07:47:26.716460+0000","last_fullsized":"2024-09-10T07:47:26.716460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694004+0000","last_change":"2024-09-10T07:47:26.694004+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694004+0000","last_undegraded":"2024-09-10T07:47:26.694004+0000","last_fullsized":"2024-09-10T07:47:26.694004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706730+0000","last_change":"2024-09-10T07:47:26.701393+0000","last_active":"2024-09-10T07:47:28.706730+0000","last_peered":"2024-09-10T07:47:28.706730+0000","last_clean":"2024-09-10T07:47:28.706730+0000","last_became_active":"2024-09-10T07:47:26.701105+0000","last_became_peered":"2024-09-10T07:47:26.701105+0000","last_unstale":"2024-09-10T07:47:28.706730+0000","last_undegraded":"2024-09-10T07:47:28.706730+0000","last_fullsized":"2024-09-10T07:47:28.706730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.714471+0000","last_change":"2024-09-10T07:47:27.708836+0000","last_active":"2024-09-10T07:47:27.714471+0000","last_peered":"2024-09-10T07:47:27.714471+0000","last_clean":"2024-09-10T07:47:27.714471+0000","last_became_active":"2024-09-10T07:47:27.708519+0000","last_became_peered":"2024-09-10T07:47:27.708519+0000","last_unstale":"2024-09-10T07:47:27.714471+0000","last_undegraded":"2024-09-10T07:47:27.714471+0000","last_fullsized":"2024-09-10T07:47:27.714471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:46.304079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716575+0000","last_change":"2024-09-10T07:47:26.713761+0000","last_active":"2024-09-10T07:47:26.716575+0000","last_peered":"2024-09-10T07:47:26.716575+0000","last_clean":"2024-09-10T07:47:26.716575+0000","last_became_active":"2024-09-10T07:47:26.713497+0000","last_became_peered":"2024-09-10T07:47:26.713497+0000","last_unstale":"2024-09-10T07:47:26.716575+0000","last_undegraded":"2024-09-10T07:47:26.716575+0000","last_fullsized":"2024-09-10T07:47:26.716575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:07.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699737+0000","last_change":"2024-09-10T07:47:26.699737+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699737+0000","last_undegraded":"2024-09-10T07:47:26.699737+0000","last_fullsized":"2024-09-10T07:47:26.699737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709924+0000","last_change":"2024-09-10T07:47:26.716721+0000","last_active":"2024-09-10T07:47:28.709924+0000","last_peered":"2024-09-10T07:47:28.709924+0000","last_clean":"2024-09-10T07:47:28.709924+0000","last_became_active":"2024-09-10T07:47:26.716429+0000","last_became_peered":"2024-09-10T07:47:26.716429+0000","last_unstale":"2024-09-10T07:47:28.709924+0000","last_undegraded":"2024-09-10T07:47:28.709924+0000","last_fullsized":"2024-09-10T07:47:28.709924+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.729913+0000","last_change":"2024-09-10T07:47:27.721272+0000","last_active":"2024-09-10T07:47:27.729913+0000","last_peered":"2024-09-10T07:47:27.729913+0000","last_clean":"2024-09-10T07:47:27.729913+0000","last_became_active":"2024-09-10T07:47:27.718319+0000","last_became_peered":"2024-09-10T07:47:27.718319+0000","last_unstale":"2024-09-10T07:47:27.729913+0000","last_undegraded":"2024-09-10T07:47:27.729913+0000","last_fullsized":"2024-09-10T07:47:27.729913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:00.885462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709462+0000","last_change":"2024-09-10T07:47:26.713954+0000","last_active":"2024-09-10T07:47:28.709462+0000","last_peered":"2024-09-10T07:47:28.709462+0000","last_clean":"2024-09-10T07:47:28.709462+0000","last_became_active":"2024-09-10T07:47:26.713378+0000","last_became_peered":"2024-09-10T07:47:26.713378+0000","last_unstale":"2024-09-10T07:47:28.709462+0000","last_undegraded":"2024-09-10T07:47:28.709462+0000","last_fullsized":"2024-09-10T07:47:28.709462+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:03.461580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731659+0000","last_change":"2024-09-10T07:47:27.724717+0000","last_active":"2024-09-10T07:47:27.731659+0000","last_peered":"2024-09-10T07:47:27.731659+0000","last_clean":"2024-09-10T07:47:27.731659+0000","last_became_active":"2024-09-10T07:47:27.724464+0000","last_became_peered":"2024-09-10T07:47:27.724464+0000","last_unstale":"2024-09-10T07:47:27.731659+0000","last_undegraded":"2024-09-10T07:47:27.731659+0000","last_fullsized":"2024-09-10T07:47:27.731659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:41.202808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715090+0000","last_change":"2024-09-10T07:47:26.709280+0000","last_active":"2024-09-10T07:47:26.715090+0000","last_peered":"2024-09-10T07:47:26.715090+0000","last_clean":"2024-09-10T07:47:26.715090+0000","last_became_active":"2024-09-10T07:47:26.708960+0000","last_became_peered":"2024-09-10T07:47:26.708960+0000","last_unstale":"2024-09-10T07:47:26.715090+0000","last_undegraded":"2024-09-10T07:47:26.715090+0000","last_fullsized":"2024-09-10T07:47:26.715090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:40.600902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698390+0000","last_change":"2024-09-10T07:47:26.698390+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698390+0000","last_undegraded":"2024-09-10T07:47:26.698390+0000","last_fullsized":"2024-09-10T07:47:26.698390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272123+0000","last_change":"2024-09-10T07:47:26.713112+0000","last_active":"2024-09-10T07:47:29.272123+0000","last_peered":"2024-09-10T07:47:29.272123+0000","last_clean":"2024-09-10T07:47:29.272123+0000","last_became_active":"2024-09-10T07:47:26.712737+0000","last_became_peered":"2024-09-10T07:47:26.712737+0000","last_unstale":"2024-09-10T07:47:29.272123+0000","last_undegraded":"2024-09-10T07:47:29.272123+0000","last_fullsized":"2024-09-10T07:47:29.272123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271575+0000","last_change":"2024-09-10T07:47:27.722758+0000","last_active":"2024-09-10T07:47:29.271575+0000","last_peered":"2024-09-10T07:47:29.271575+0000","last_clean":"2024-09-10T07:47:29.271575+0000","last_became_active":"2024-09-10T07:47:27.722517+0000","last_became_peered":"2024-09-10T07:47:27.722517+0000","last_unstale":"2024-09-10T07:47:29.271575+0000","last_undegraded":"2024-09-10T07:47:29.271575+0000","last_fullsized":"2024-09-10T07:47:29.271575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:07.008875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716544+0000","last_change":"2024-09-10T07:47:26.713271+0000","last_active":"2024-09-10T07:47:26.716544+0000","last_peered":"2024-09-10T07:47:26.716544+0000","last_clean":"2024-09-10T07:47:26.716544+0000","last_became_active":"2024-09-10T07:47:26.712715+0000","last_became_peered":"2024-09-10T07:47:26.712715+0000","last_unstale":"2024-09-10T07:47:26.716544+0000","last_undegraded":"2024-09-10T07:47:26.716544+0000","last_fullsized":"2024-09-10T07:47:26.716544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730403+0000","last_change":"2024-09-10T07:47:27.725965+0000","last_active":"2024-09-10T07:47:27.730403+0000","last_peered":"2024-09-10T07:47:27.730403+0000","last_clean":"2024-09-10T07:47:27.730403+0000","last_became_active":"2024-09-10T07:47:27.725029+0000","last_became_peered":"2024-09-10T07:47:27.725029+0000","last_unstale":"2024-09-10T07:47:27.730403+0000","last_undegraded":"2024-09-10T07:47:27.730403+0000","last_fullsized":"2024-09-10T07:47:27.730403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:31.812075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699985+0000","last_change":"2024-09-10T07:47:26.711747+0000","last_active":"2024-09-10T07:47:27.699985+0000","last_peered":"2024-09-10T07:47:27.699985+0000","last_clean":"2024-09-10T07:47:27.699985+0000","last_became_active":"2024-09-10T07:47:26.711458+0000","last_became_peered":"2024-09-10T07:47:26.711458+0000","last_unstale":"2024-09-10T07:47:27.699985+0000","last_undegraded":"2024-09-10T07:47:27.699985+0000","last_fullsized":"2024-09-10T07:47:27.699985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:17.886142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694848+0000","last_change":"2024-09-10T07:47:26.694848+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694848+0000","last_undegraded":"2024-09-10T07:47:26.694848+0000","last_fullsized":"2024-09-10T07:47:26.694848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.698363+0000","last_change":"2024-09-10T07:47:26.708509+0000","last_active":"2024-09-10T07:47:27.698363+0000","last_peered":"2024-09-10T07:47:27.698363+0000","last_clean":"2024-09-10T07:47:27.698363+0000","last_became_active":"2024-09-10T07:47:26.708220+0000","last_became_peered":"2024-09-10T07:47:26.708220+0000","last_unstale":"2024-09-10T07:47:27.698363+0000","last_undegraded":"2024-09-10T07:47:27.698363+0000","last_fullsized":"2024-09-10T07:47:27.698363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:55.494668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700387+0000","last_change":"2024-09-10T07:47:26.700387+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700387+0000","last_undegraded":"2024-09-10T07:47:26.700387+0000","last_fullsized":"2024-09-10T07:47:26.700387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706491+0000","last_change":"2024-09-10T07:47:26.706290+0000","last_active":"2024-09-10T07:47:28.706491+0000","last_peered":"2024-09-10T07:47:28.706491+0000","last_clean":"2024-09-10T07:47:28.706491+0000","last_became_active":"2024-09-10T07:47:26.705979+0000","last_became_peered":"2024-09-10T07:47:26.705979+0000","last_unstale":"2024-09-10T07:47:28.706491+0000","last_undegraded":"2024-09-10T07:47:28.706491+0000","last_fullsized":"2024-09-10T07:47:28.706491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699283+0000","last_change":"2024-09-10T07:47:26.699283+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699283+0000","last_undegraded":"2024-09-10T07:47:26.699283+0000","last_fullsized":"2024-09-10T07:47:26.699283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706848+0000","last_change":"2024-09-10T07:47:26.710001+0000","last_active":"2024-09-10T07:47:28.706848+0000","last_peered":"2024-09-10T07:47:28.706848+0000","last_clean":"2024-09-10T07:47:28.706848+0000","last_became_active":"2024-09-10T07:47:26.708545+0000","last_became_peered":"2024-09-10T07:47:26.708545+0000","last_unstale":"2024-09-10T07:47:28.706848+0000","last_undegraded":"2024-09-10T07:47:28.706848+0000","last_fullsized":"2024-09-10T07:47:28.706848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697405+0000","last_change":"2024-09-10T07:47:26.697405+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697405+0000","last_undegraded":"2024-09-10T07:47:26.697405+0000","last_fullsized":"2024-09-10T07:47:26.697405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716435+0000","last_change":"2024-09-10T07:47:26.713070+0000","last_active":"2024-09-10T07:47:26.716435+0000","last_peered":"2024-09-10T07:47:26.716435+0000","last_clean":"2024-09-10T07:47:26.716435+0000","last_became_active":"2024-09-10T07:47:26.712820+0000","last_became_peered":"2024-09-10T07:47:26.712820+0000","last_unstale":"2024-09-10T07:47:26.716435+0000","last_undegraded":"2024-09-10T07:47:26.716435+0000","last_fullsized":"2024-09-10T07:47:26.716435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709356+0000","last_change":"2024-09-10T07:47:27.712460+0000","last_active":"2024-09-10T07:47:28.709356+0000","last_peered":"2024-09-10T07:47:28.709356+0000","last_clean":"2024-09-10T07:47:28.709356+0000","last_became_active":"2024-09-10T07:47:27.712183+0000","last_became_peered":"2024-09-10T07:47:27.712183+0000","last_unstale":"2024-09-10T07:47:28.709356+0000","last_undegraded":"2024-09-10T07:47:28.709356+0000","last_fullsized":"2024-09-10T07:47:28.709356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716233+0000","last_change":"2024-09-10T07:47:26.713128+0000","last_active":"2024-09-10T07:47:26.716233+0000","last_peered":"2024-09-10T07:47:26.716233+0000","last_clean":"2024-09-10T07:47:26.716233+0000","last_became_active":"2024-09-10T07:47:26.712677+0000","last_became_peered":"2024-09-10T07:47:26.712677+0000","last_unstale":"2024-09-10T07:47:26.716233+0000","last_undegraded":"2024-09-10T07:47:26.716233+0000","last_fullsized":"2024-09-10T07:47:26.716233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:06.290962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706999+0000","last_change":"2024-09-10T07:47:27.724138+0000","last_active":"2024-09-10T07:47:28.706999+0000","last_peered":"2024-09-10T07:47:28.706999+0000","last_clean":"2024-09-10T07:47:28.706999+0000","last_became_active":"2024-09-10T07:47:27.723179+0000","last_became_peered":"2024-09-10T07:47:27.723179+0000","last_unstale":"2024-09-10T07:47:28.706999+0000","last_undegraded":"2024-09-10T07:47:28.706999+0000","last_fullsized":"2024-09-10T07:47:28.706999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716153+0000","last_change":"2024-09-10T07:47:26.711949+0000","last_active":"2024-09-10T07:47:26.716153+0000","last_peered":"2024-09-10T07:47:26.716153+0000","last_clean":"2024-09-10T07:47:26.716153+0000","last_became_active":"2024-09-10T07:47:26.711492+0000","last_became_peered":"2024-09-10T07:47:26.711492+0000","last_unstale":"2024-09-10T07:47:26.716153+0000","last_undegraded":"2024-09-10T07:47:26.716153+0000","last_fullsized":"2024-09-10T07:47:26.716153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:31.026752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271616+0000","last_change":"2024-09-10T07:47:27.725128+0000","last_active":"2024-09-10T07:47:29.271616+0000","last_peered":"2024-09-10T07:47:29.271616+0000","last_clean":"2024-09-10T07:47:29.271616+0000","last_became_active":"2024-09-10T07:47:27.724151+0000","last_became_peered":"2024-09-10T07:47:27.724151+0000","last_unstale":"2024-09-10T07:47:29.271616+0000","last_undegraded":"2024-09-10T07:47:29.271616+0000","last_fullsized":"2024-09-10T07:47:29.271616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:38.688543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709386+0000","last_change":"2024-09-10T07:47:26.715573+0000","last_active":"2024-09-10T07:47:28.709386+0000","last_peered":"2024-09-10T07:47:28.709386+0000","last_clean":"2024-09-10T07:47:28.709386+0000","last_became_active":"2024-09-10T07:47:26.715315+0000","last_became_peered":"2024-09-10T07:47:26.715315+0000","last_unstale":"2024-09-10T07:47:28.709386+0000","last_undegraded":"2024-09-10T07:47:28.709386+0000","last_fullsized":"2024-09-10T07:47:28.709386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:37.683908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709341+0000","last_change":"2024-09-10T07:47:27.701024+0000","last_active":"2024-09-10T07:47:28.709341+0000","last_peered":"2024-09-10T07:47:28.709341+0000","last_clean":"2024-09-10T07:47:28.709341+0000","last_became_active":"2024-09-10T07:47:27.700612+0000","last_became_peered":"2024-09-10T07:47:27.700612+0000","last_unstale":"2024-09-10T07:47:28.709341+0000","last_undegraded":"2024-09-10T07:47:28.709341+0000","last_fullsized":"2024-09-10T07:47:28.709341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715646+0000","last_change":"2024-09-10T07:47:26.707503+0000","last_active":"2024-09-10T07:47:26.715646+0000","last_peered":"2024-09-10T07:47:26.715646+0000","last_clean":"2024-09-10T07:47:26.715646+0000","last_became_active":"2024-09-10T07:47:26.706814+0000","last_became_peered":"2024-09-10T07:47:26.706814+0000","last_unstale":"2024-09-10T07:47:26.715646+0000","last_undegraded":"2024-09-10T07:47:26.715646+0000","last_fullsized":"2024-09-10T07:47:26.715646+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:10.209621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271600+0000","last_change":"2024-09-10T07:47:27.722893+0000","last_active":"2024-09-10T07:47:29.271600+0000","last_peered":"2024-09-10T07:47:29.271600+0000","last_clean":"2024-09-10T07:47:29.271600+0000","last_became_active":"2024-09-10T07:47:27.722551+0000","last_became_peered":"2024-09-10T07:47:27.722551+0000","last_unstale":"2024-09-10T07:47:29.271600+0000","last_undegraded":"2024-09-10T07:47:29.271600+0000","last_fullsized":"2024-09-10T07:47:29.271600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:52.315032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716562+0000","last_change":"2024-09-10T07:47:26.714230+0000","last_active":"2024-09-10T07:47:26.716562+0000","last_peered":"2024-09-10T07:47:26.716562+0000","last_clean":"2024-09-10T07:47:26.716562+0000","last_became_active":"2024-09-10T07:47:26.712186+0000","last_became_peered":"2024-09-10T07:47:26.712186+0000","last_unstale":"2024-09-10T07:47:26.716562+0000","last_undegraded":"2024-09-10T07:47:26.716562+0000","last_fullsized":"2024-09-10T07:47:26.716562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:55.243965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694126+0000","last_change":"2024-09-10T07:47:26.694126+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694126+0000","last_undegraded":"2024-09-10T07:47:26.694126+0000","last_fullsized":"2024-09-10T07:47:26.694126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.712412+0000","last_change":"2024-09-10T07:47:26.709027+0000","last_active":"2024-09-10T07:47:26.712412+0000","last_peered":"2024-09-10T07:47:26.712412+0000","last_clean":"2024-09-10T07:47:26.712412+0000","last_became_active":"2024-09-10T07:47:26.708753+0000","last_became_peered":"2024-09-10T07:47:26.708753+0000","last_unstale":"2024-09-10T07:47:26.712412+0000","last_undegraded":"2024-09-10T07:47:26.712412+0000","last_fullsized":"2024-09-10T07:47:26.712412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:13.182619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707048+0000","last_change":"2024-09-10T07:47:27.728171+0000","last_active":"2024-09-10T07:47:28.707048+0000","last_peered":"2024-09-10T07:47:28.707048+0000","last_clean":"2024-09-10T07:47:28.707048+0000","last_became_active":"2024-09-10T07:47:27.727842+0000","last_became_peered":"2024-09-10T07:47:27.727842+0000","last_unstale":"2024-09-10T07:47:28.707048+0000","last_undegraded":"2024-09-10T07:47:28.707048+0000","last_fullsized":"2024-09-10T07:47:28.707048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:45.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272065+0000","last_change":"2024-09-10T07:47:26.707810+0000","last_active":"2024-09-10T07:47:29.272065+0000","last_peered":"2024-09-10T07:47:29.272065+0000","last_clean":"2024-09-10T07:47:29.272065+0000","last_became_active":"2024-09-10T07:47:26.707462+0000","last_became_peered":"2024-09-10T07:47:26.707462+0000","last_unstale":"2024-09-10T07:47:29.272065+0000","last_undegraded":"2024-09-10T07:47:29.272065+0000","last_fullsized":"2024-09-10T07:47:29.272065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698308+0000","last_change":"2024-09-10T07:47:26.698308+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698308+0000","last_undegraded":"2024-09-10T07:47:26.698308+0000","last_fullsized":"2024-09-10T07:47:26.698308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709605+0000","last_change":"2024-09-10T07:47:26.706696+0000","last_active":"2024-09-10T07:47:28.709605+0000","last_peered":"2024-09-10T07:47:28.709605+0000","last_clean":"2024-09-10T07:47:28.709605+0000","last_became_active":"2024-09-10T07:47:26.706383+0000","last_became_peered":"2024-09-10T07:47:26.706383+0000","last_unstale":"2024-09-10T07:47:28.709605+0000","last_undegraded":"2024-09-10T07:47:28.709605+0000","last_fullsized":"2024-09-10T07:47:28.709605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:16.889913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707098+0000","last_change":"2024-09-10T07:47:27.721003+0000","last_active":"2024-09-10T07:47:28.707098+0000","last_peered":"2024-09-10T07:47:28.707098+0000","last_clean":"2024-09-10T07:47:28.707098+0000","last_became_active":"2024-09-10T07:47:27.720702+0000","last_became_peered":"2024-09-10T07:47:27.720702+0000","last_unstale":"2024-09-10T07:47:28.707098+0000","last_undegraded":"2024-09-10T07:47:28.707098+0000","last_fullsized":"2024-09-10T07:47:28.707098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:33.721528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272228+0000","last_change":"2024-09-10T07:47:26.701128+0000","last_active":"2024-09-10T07:47:29.272228+0000","last_peered":"2024-09-10T07:47:29.272228+0000","last_clean":"2024-09-10T07:47:29.272228+0000","last_became_active":"2024-09-10T07:47:26.700042+0000","last_became_peered":"2024-09-10T07:47:26.700042+0000","last_unstale":"2024-09-10T07:47:29.272228+0000","last_undegraded":"2024-09-10T07:47:29.272228+0000","last_fullsized":"2024-09-10T07:47:29.272228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.692995+0000","last_change":"2024-09-10T07:47:26.692995+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.692995+0000","last_undegraded":"2024-09-10T07:47:26.692995+0000","last_fullsized":"2024-09-10T07:47:26.692995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271867+0000","last_change":"2024-09-10T07:47:26.705882+0000","last_active":"2024-09-10T07:47:29.271867+0000","last_peered":"2024-09-10T07:47:29.271867+0000","last_clean":"2024-09-10T07:47:29.271867+0000","last_became_active":"2024-09-10T07:47:26.705616+0000","last_became_peered":"2024-09-10T07:47:26.705616+0000","last_unstale":"2024-09-10T07:47:29.271867+0000","last_undegraded":"2024-09-10T07:47:29.271867+0000","last_fullsized":"2024-09-10T07:47:29.271867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728015+0000","last_change":"2024-09-10T07:47:27.719286+0000","last_active":"2024-09-10T07:47:27.728015+0000","last_peered":"2024-09-10T07:47:27.728015+0000","last_clean":"2024-09-10T07:47:27.728015+0000","last_became_active":"2024-09-10T07:47:27.715311+0000","last_became_peered":"2024-09-10T07:47:27.715311+0000","last_unstale":"2024-09-10T07:47:27.728015+0000","last_undegraded":"2024-09-10T07:47:27.728015+0000","last_fullsized":"2024-09-10T07:47:27.728015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:26:17.625922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271645+0000","last_change":"2024-09-10T07:47:26.710717+0000","last_active":"2024-09-10T07:47:29.271645+0000","last_peered":"2024-09-10T07:47:29.271645+0000","last_clean":"2024-09-10T07:47:29.271645+0000","last_became_active":"2024-09-10T07:47:26.710392+0000","last_became_peered":"2024-09-10T07:47:26.710392+0000","last_unstale":"2024-09-10T07:47:29.271645+0000","last_undegraded":"2024-09-10T07:47:29.271645+0000","last_fullsized":"2024-09-10T07:47:29.271645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730509+0000","last_change":"2024-09-10T07:47:27.727229+0000","last_active":"2024-09-10T07:47:27.730509+0000","last_peered":"2024-09-10T07:47:27.730509+0000","last_clean":"2024-09-10T07:47:27.730509+0000","last_became_active":"2024-09-10T07:47:27.725550+0000","last_became_peered":"2024-09-10T07:47:27.725550+0000","last_unstale":"2024-09-10T07:47:27.730509+0000","last_undegraded":"2024-09-10T07:47:27.730509+0000","last_fullsized":"2024-09-10T07:47:27.730509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:02.762995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716358+0000","last_change":"2024-09-10T07:47:26.710777+0000","last_active":"2024-09-10T07:47:26.716358+0000","last_peered":"2024-09-10T07:47:26.716358+0000","last_clean":"2024-09-10T07:47:26.716358+0000","last_became_active":"2024-09-10T07:47:26.710499+0000","last_became_peered":"2024-09-10T07:47:26.710499+0000","last_unstale":"2024-09-10T07:47:26.716358+0000","last_undegraded":"2024-09-10T07:47:26.716358+0000","last_fullsized":"2024-09-10T07:47:26.716358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:25.822800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699907+0000","last_change":"2024-09-10T07:47:26.699907+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699907+0000","last_undegraded":"2024-09-10T07:47:26.699907+0000","last_fullsized":"2024-09-10T07:47:26.699907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699089+0000","last_change":"2024-09-10T07:47:26.716992+0000","last_active":"2024-09-10T07:47:27.699089+0000","last_peered":"2024-09-10T07:47:27.699089+0000","last_clean":"2024-09-10T07:47:27.699089+0000","last_became_active":"2024-09-10T07:47:26.716690+0000","last_became_peered":"2024-09-10T07:47:26.716690+0000","last_unstale":"2024-09-10T07:47:27.699089+0000","last_undegraded":"2024-09-10T07:47:27.699089+0000","last_fullsized":"2024-09-10T07:47:27.699089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.180750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.693017+0000","last_change":"2024-09-10T07:47:26.693017+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.693017+0000","last_undegraded":"2024-09-10T07:47:26.693017+0000","last_fullsized":"2024-09-10T07:47:26.693017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.710693+0000","last_change":"2024-09-10T07:47:26.707932+0000","last_active":"2024-09-10T07:47:26.710693+0000","last_peered":"2024-09-10T07:47:26.710693+0000","last_clean":"2024-09-10T07:47:26.710693+0000","last_became_active":"2024-09-10T07:47:26.707401+0000","last_became_peered":"2024-09-10T07:47:26.707401+0000","last_unstale":"2024-09-10T07:47:26.710693+0000","last_undegraded":"2024-09-10T07:47:26.710693+0000","last_fullsized":"2024-09-10T07:47:26.710693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.677752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.697935+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:47:26.697935+0000","last_peered":"2024-09-10T07:47:26.697935+0000","last_clean":"2024-09-10T07:47:26.697935+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:47:26.697935+0000","last_undegraded":"2024-09-10T07:47:26.697935+0000","last_fullsized":"2024-09-10T07:47:26.697935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":725755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":154759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":156349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":745687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":372736,"data_stored":141662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:47:31.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:47:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.107+0000 7fdc23a00640 1 Processor -- start 2024-09-10T07:47:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.108+0000 7fdc23a00640 1 -- start start 2024-09-10T07:47:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc23a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc1c160af0 0x7fdc1c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc23a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc1c0d05f0 0x7fdc1c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc23a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1c0cfdd0 con 0x7fdc1c160710 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc23a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1c15cc90 con 0x7fdc1c160af0 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc23a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1c15cf70 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc1be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc1c160af0 0x7fdc1c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc21000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc1c160710 0x7fdc1c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39180/0 (socket says 172.21.15.3:39180) 2024-09-10T07:47:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc21000640 1 -- 172.21.15.3:0/296544181 learned_addr learned my addr 172.21.15.3:0/296544181 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc21a00640 1 --2- 172.21.15.3:0/296544181 >> v2:172.21.15.70:3300/0 conn(0x7fdc1c0d05f0 0x7fdc1c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc1b400640 1 -- 172.21.15.3:0/296544181 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129324717 0 0) 0x7fdc1c0cfdd0 con 0x7fdc1c160710 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc1be00640 1 -- 172.21.15.3:0/296544181 >> v2:172.21.15.70:3300/0 conn(0x7fdc1c0d05f0 msgr2=0x7fdc1c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.109+0000 7fdc1be00640 1 --2- 172.21.15.3:0/296544181 >> v2:172.21.15.70:3300/0 conn(0x7fdc1c0d05f0 0x7fdc1c072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.110+0000 7fdc1be00640 1 -- 172.21.15.3:0/296544181 >> v1:172.21.15.3:6789/0 conn(0x7fdc1c160710 legacy=0x7fdc1c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.110+0000 7fdc1be00640 1 -- 172.21.15.3:0/296544181 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c15d250 con 0x7fdc1c160af0 2024-09-10T07:47:32.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc1be00640 1 --2- 172.21.15.3:0/296544181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c160af0 0x7fdc1c0d00b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0718d0 tx=0x7fdc0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6ba2d9b776118d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc1b400640 1 -- 172.21.15.3:0/296544181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08037070 con 0x7fdc1c160af0 2024-09-10T07:47:32.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc1b400640 1 -- 172.21.15.3:0/296544181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc08032440 con 0x7fdc1c160af0 2024-09-10T07:47:32.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc1b400640 1 -- 172.21.15.3:0/296544181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08032810 con 0x7fdc1c160af0 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc23a00640 1 -- 172.21.15.3:0/296544181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c160af0 msgr2=0x7fdc1c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.111+0000 7fdc23a00640 1 --2- 172.21.15.3:0/296544181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c160af0 0x7fdc1c0d00b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0718d0 tx=0x7fdc0802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 -- 172.21.15.3:0/296544181 shutdown_connections 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 --2- 172.21.15.3:0/296544181 >> v2:172.21.15.70:3300/0 conn(0x7fdc1c0d05f0 0x7fdc1c072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 --2- 172.21.15.3:0/296544181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c160af0 0x7fdc1c0d00b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 -- 172.21.15.3:0/296544181 >> 172.21.15.3:0/296544181 conn(0x7fdc1c06d000 msgr2=0x7fdc1c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 -- 172.21.15.3:0/296544181 shutdown_connections 2024-09-10T07:47:32.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.112+0000 7fdc23a00640 1 -- 172.21.15.3:0/296544181 wait complete. 2024-09-10T07:47:32.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.113+0000 7fdc23a00640 1 Processor -- start 2024-09-10T07:47:32.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.113+0000 7fdc23a00640 1 -- start start 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc1c160710 0x7fdc1c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc1c160af0 0x7fdc1c0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c15d770 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c15c960 con 0x7fdc1c160710 2024-09-10T07:47:32.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc23a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c14c470 con 0x7fdc1c160af0 2024-09-10T07:47:32.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc21000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.114+0000 7fdc21000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36038/0 (socket says 172.21.15.3:36038) 2024-09-10T07:47:32.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 -- 172.21.15.3:0/3594805873 learned_addr learned my addr 172.21.15.3:0/3594805873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 -- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc1c160af0 msgr2=0x7fdc1c0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc1c160af0 0x7fdc1c0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 -- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc1c160710 msgr2=0x7fdc1c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc1c160710 0x7fdc1c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 -- 172.21.15.3:0/3594805873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c154490 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc21000640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c4d0 tx=0x7fdc1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10018070 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10007500 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004a20 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.115+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c154770 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.116+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c154d30 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.117+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c0cfa60 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.117+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc10004bc0 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.118+0000 7fdc18c00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 0x7fdbfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.118+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdc1008a1d0 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.119+0000 7fdc1be00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 0x7fdbfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.119+0000 7fdc1be00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 0x7fdbfc06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc08034000 tx=0x7fdc0802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.123+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc10058ae0 con 0x7fdc1c0d05f0 2024-09-10T07:47:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.304+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc1c14c600 con 0x7fdbfc06be30 2024-09-10T07:47:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.310+0000 7fdc18c00640 1 -- 172.21.15.3:0/3594805873 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249138 (secure 0 0 0) 0x7fdc08032590 con 0x7fdbfc06be30 2024-09-10T07:47:32.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:47:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 msgr2=0x7fdbfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 0x7fdbfc06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc08034000 tx=0x7fdc0802c000 comp rx=0 tx=0).stop 2024-09-10T07:47:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 msgr2=0x7fdc1c155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c4d0 tx=0x7fdc1000c990 comp rx=0 tx=0).stop 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 shutdown_connections 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdbfc06be30 0x7fdbfc06e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc1c160af0 0x7fdc1c0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc1c160710 0x7fdc1c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 --2- 172.21.15.3:0/3594805873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc1c0d05f0 0x7fdc1c155f50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 >> 172.21.15.3:0/3594805873 conn(0x7fdc1c06d000 msgr2=0x7fdc1c0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 shutdown_connections 2024-09-10T07:47:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.315+0000 7fdc23a00640 1 -- 172.21.15.3:0/3594805873 wait complete. 2024-09-10T07:47:32.333 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T07:47:31.501965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236996,"kb_used_data":4116,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737724,"statfs":{"total":773094113280,"available":772851429376,"internally_reserved":0,"allocated":4214784,"data_stored":2330373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815909"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.702272+0000","last_change":"2024-09-10T07:47:26.702272+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.702272+0000","last_undegraded":"2024-09-10T07:47:26.702272+0000","last_fullsized":"2024-09-10T07:47:26.702272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709608+0000","last_change":"2024-09-10T07:47:27.717295+0000","last_active":"2024-09-10T07:47:28.709608+0000","last_peered":"2024-09-10T07:47:28.709608+0000","last_clean":"2024-09-10T07:47:28.709608+0000","last_became_active":"2024-09-10T07:47:27.716979+0000","last_became_peered":"2024-09-10T07:47:27.716979+0000","last_unstale":"2024-09-10T07:47:28.709608+0000","last_undegraded":"2024-09-10T07:47:28.709608+0000","last_fullsized":"2024-09-10T07:47:28.709608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695776+0000","last_change":"2024-09-10T07:47:26.695776+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695776+0000","last_undegraded":"2024-09-10T07:47:26.695776+0000","last_fullsized":"2024-09-10T07:47:26.695776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731683+0000","last_change":"2024-09-10T07:47:27.726560+0000","last_active":"2024-09-10T07:47:27.731683+0000","last_peered":"2024-09-10T07:47:27.731683+0000","last_clean":"2024-09-10T07:47:27.731683+0000","last_became_active":"2024-09-10T07:47:27.726024+0000","last_became_peered":"2024-09-10T07:47:27.726024+0000","last_unstale":"2024-09-10T07:47:27.731683+0000","last_undegraded":"2024-09-10T07:47:27.731683+0000","last_fullsized":"2024-09-10T07:47:27.731683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:50.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709319+0000","last_change":"2024-09-10T07:47:27.710914+0000","last_active":"2024-09-10T07:47:28.709319+0000","last_peered":"2024-09-10T07:47:28.709319+0000","last_clean":"2024-09-10T07:47:28.709319+0000","last_became_active":"2024-09-10T07:47:27.710617+0000","last_became_peered":"2024-09-10T07:47:27.710617+0000","last_unstale":"2024-09-10T07:47:28.709319+0000","last_undegraded":"2024-09-10T07:47:28.709319+0000","last_fullsized":"2024-09-10T07:47:28.709319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706883+0000","last_change":"2024-09-10T07:47:27.721104+0000","last_active":"2024-09-10T07:47:28.706883+0000","last_peered":"2024-09-10T07:47:28.706883+0000","last_clean":"2024-09-10T07:47:28.706883+0000","last_became_active":"2024-09-10T07:47:27.718624+0000","last_became_peered":"2024-09-10T07:47:27.718624+0000","last_unstale":"2024-09-10T07:47:28.706883+0000","last_undegraded":"2024-09-10T07:47:28.706883+0000","last_fullsized":"2024-09-10T07:47:28.706883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697623+0000","last_change":"2024-09-10T07:47:26.697623+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697623+0000","last_undegraded":"2024-09-10T07:47:26.697623+0000","last_fullsized":"2024-09-10T07:47:26.697623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699517+0000","last_change":"2024-09-10T07:47:26.699517+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699517+0000","last_undegraded":"2024-09-10T07:47:26.699517+0000","last_fullsized":"2024-09-10T07:47:26.699517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697854+0000","last_change":"2024-09-10T07:47:26.697854+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697854+0000","last_undegraded":"2024-09-10T07:47:26.697854+0000","last_fullsized":"2024-09-10T07:47:26.697854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728902+0000","last_change":"2024-09-10T07:47:27.717546+0000","last_active":"2024-09-10T07:47:27.728902+0000","last_peered":"2024-09-10T07:47:27.728902+0000","last_clean":"2024-09-10T07:47:27.728902+0000","last_became_active":"2024-09-10T07:47:27.713266+0000","last_became_peered":"2024-09-10T07:47:27.713266+0000","last_unstale":"2024-09-10T07:47:27.728902+0000","last_undegraded":"2024-09-10T07:47:27.728902+0000","last_fullsized":"2024-09-10T07:47:27.728902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:58.170897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697313+0000","last_change":"2024-09-10T07:47:26.697313+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697313+0000","last_undegraded":"2024-09-10T07:47:26.697313+0000","last_fullsized":"2024-09-10T07:47:26.697313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696911+0000","last_change":"2024-09-10T07:47:26.696911+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696911+0000","last_undegraded":"2024-09-10T07:47:26.696911+0000","last_fullsized":"2024-09-10T07:47:26.696911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271853+0000","last_change":"2024-09-10T07:47:27.728693+0000","last_active":"2024-09-10T07:47:29.271853+0000","last_peered":"2024-09-10T07:47:29.271853+0000","last_clean":"2024-09-10T07:47:29.271853+0000","last_became_active":"2024-09-10T07:47:27.728426+0000","last_became_peered":"2024-09-10T07:47:27.728426+0000","last_unstale":"2024-09-10T07:47:29.271853+0000","last_undegraded":"2024-09-10T07:47:29.271853+0000","last_fullsized":"2024-09-10T07:47:29.271853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.466215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.721891+0000","last_change":"2024-09-10T07:47:27.713960+0000","last_active":"2024-09-10T07:47:27.721891+0000","last_peered":"2024-09-10T07:47:27.721891+0000","last_clean":"2024-09-10T07:47:27.721891+0000","last_became_active":"2024-09-10T07:47:27.713692+0000","last_became_peered":"2024-09-10T07:47:27.713692+0000","last_unstale":"2024-09-10T07:47:27.721891+0000","last_undegraded":"2024-09-10T07:47:27.721891+0000","last_fullsized":"2024-09-10T07:47:27.721891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:50.495716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696378+0000","last_change":"2024-09-10T07:47:26.696378+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696378+0000","last_undegraded":"2024-09-10T07:47:26.696378+0000","last_fullsized":"2024-09-10T07:47:26.696378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697490+0000","last_change":"2024-09-10T07:47:26.697490+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697490+0000","last_undegraded":"2024-09-10T07:47:26.697490+0000","last_fullsized":"2024-09-10T07:47:26.697490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698719+0000","last_change":"2024-09-10T07:47:26.698719+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698719+0000","last_undegraded":"2024-09-10T07:47:26.698719+0000","last_fullsized":"2024-09-10T07:47:26.698719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707015+0000","last_change":"2024-09-10T07:47:27.717429+0000","last_active":"2024-09-10T07:47:28.707015+0000","last_peered":"2024-09-10T07:47:28.707015+0000","last_clean":"2024-09-10T07:47:28.707015+0000","last_became_active":"2024-09-10T07:47:27.717064+0000","last_became_peered":"2024-09-10T07:47:27.717064+0000","last_unstale":"2024-09-10T07:47:28.707015+0000","last_undegraded":"2024-09-10T07:47:28.707015+0000","last_fullsized":"2024-09-10T07:47:28.707015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:44.926594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706888+0000","last_change":"2024-09-10T07:47:27.715748+0000","last_active":"2024-09-10T07:47:28.706888+0000","last_peered":"2024-09-10T07:47:28.706888+0000","last_clean":"2024-09-10T07:47:28.706888+0000","last_became_active":"2024-09-10T07:47:27.715354+0000","last_became_peered":"2024-09-10T07:47:27.715354+0000","last_unstale":"2024-09-10T07:47:28.706888+0000","last_undegraded":"2024-09-10T07:47:28.706888+0000","last_fullsized":"2024-09-10T07:47:28.706888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272066+0000","last_change":"2024-09-10T07:47:27.726942+0000","last_active":"2024-09-10T07:47:29.272066+0000","last_peered":"2024-09-10T07:47:29.272066+0000","last_clean":"2024-09-10T07:47:29.272066+0000","last_became_active":"2024-09-10T07:47:27.726536+0000","last_became_peered":"2024-09-10T07:47:27.726536+0000","last_unstale":"2024-09-10T07:47:29.272066+0000","last_undegraded":"2024-09-10T07:47:29.272066+0000","last_fullsized":"2024-09-10T07:47:29.272066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:25.590064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695528+0000","last_change":"2024-09-10T07:47:26.695528+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695528+0000","last_undegraded":"2024-09-10T07:47:26.695528+0000","last_fullsized":"2024-09-10T07:47:26.695528+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.701477+0000","last_change":"2024-09-10T07:47:26.701477+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.701477+0000","last_undegraded":"2024-09-10T07:47:26.701477+0000","last_fullsized":"2024-09-10T07:47:26.701477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271941+0000","last_change":"2024-09-10T07:47:27.721416+0000","last_active":"2024-09-10T07:47:29.271941+0000","last_peered":"2024-09-10T07:47:29.271941+0000","last_clean":"2024-09-10T07:47:29.271941+0000","last_became_active":"2024-09-10T07:47:27.721174+0000","last_became_peered":"2024-09-10T07:47:27.721174+0000","last_unstale":"2024-09-10T07:47:29.271941+0000","last_undegraded":"2024-09-10T07:47:29.271941+0000","last_fullsized":"2024-09-10T07:47:29.271941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:17.894548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699069+0000","last_change":"2024-09-10T07:47:26.699069+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699069+0000","last_undegraded":"2024-09-10T07:47:26.699069+0000","last_fullsized":"2024-09-10T07:47:26.699069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.732178+0000","last_change":"2024-09-10T07:47:27.723555+0000","last_active":"2024-09-10T07:47:27.732178+0000","last_peered":"2024-09-10T07:47:27.732178+0000","last_clean":"2024-09-10T07:47:27.732178+0000","last_became_active":"2024-09-10T07:47:27.723226+0000","last_became_peered":"2024-09-10T07:47:27.723226+0000","last_unstale":"2024-09-10T07:47:27.732178+0000","last_undegraded":"2024-09-10T07:47:27.732178+0000","last_fullsized":"2024-09-10T07:47:27.732178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:55.167875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731804+0000","last_change":"2024-09-10T07:47:27.729136+0000","last_active":"2024-09-10T07:47:27.731804+0000","last_peered":"2024-09-10T07:47:27.731804+0000","last_clean":"2024-09-10T07:47:27.731804+0000","last_became_active":"2024-09-10T07:47:27.724907+0000","last_became_peered":"2024-09-10T07:47:27.724907+0000","last_unstale":"2024-09-10T07:47:27.731804+0000","last_undegraded":"2024-09-10T07:47:27.731804+0000","last_fullsized":"2024-09-10T07:47:27.731804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:12.255456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709383+0000","last_change":"2024-09-10T07:47:27.710661+0000","last_active":"2024-09-10T07:47:28.709383+0000","last_peered":"2024-09-10T07:47:28.709383+0000","last_clean":"2024-09-10T07:47:28.709383+0000","last_became_active":"2024-09-10T07:47:27.710117+0000","last_became_peered":"2024-09-10T07:47:27.710117+0000","last_unstale":"2024-09-10T07:47:28.709383+0000","last_undegraded":"2024-09-10T07:47:28.709383+0000","last_fullsized":"2024-09-10T07:47:28.709383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700878+0000","last_change":"2024-09-10T07:47:26.700878+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700878+0000","last_undegraded":"2024-09-10T07:47:26.700878+0000","last_fullsized":"2024-09-10T07:47:26.700878+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694553+0000","last_change":"2024-09-10T07:47:26.694553+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694553+0000","last_undegraded":"2024-09-10T07:47:26.694553+0000","last_fullsized":"2024-09-10T07:47:26.694553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694437+0000","last_change":"2024-09-10T07:47:26.694437+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694437+0000","last_undegraded":"2024-09-10T07:47:26.694437+0000","last_fullsized":"2024-09-10T07:47:26.694437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695915+0000","last_change":"2024-09-10T07:47:26.695915+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695915+0000","last_undegraded":"2024-09-10T07:47:26.695915+0000","last_fullsized":"2024-09-10T07:47:26.695915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697245+0000","last_change":"2024-09-10T07:47:26.697245+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697245+0000","last_undegraded":"2024-09-10T07:47:26.697245+0000","last_fullsized":"2024-09-10T07:47:26.697245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272289+0000","last_change":"2024-09-10T07:47:27.714639+0000","last_active":"2024-09-10T07:47:29.272289+0000","last_peered":"2024-09-10T07:47:29.272289+0000","last_clean":"2024-09-10T07:47:29.272289+0000","last_became_active":"2024-09-10T07:47:27.714372+0000","last_became_peered":"2024-09-10T07:47:27.714372+0000","last_unstale":"2024-09-10T07:47:29.272289+0000","last_undegraded":"2024-09-10T07:47:29.272289+0000","last_fullsized":"2024-09-10T07:47:29.272289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715183+0000","last_change":"2024-09-10T07:47:26.711007+0000","last_active":"2024-09-10T07:47:26.715183+0000","last_peered":"2024-09-10T07:47:26.715183+0000","last_clean":"2024-09-10T07:47:26.715183+0000","last_became_active":"2024-09-10T07:47:26.710707+0000","last_became_peered":"2024-09-10T07:47:26.710707+0000","last_unstale":"2024-09-10T07:47:26.715183+0000","last_undegraded":"2024-09-10T07:47:26.715183+0000","last_fullsized":"2024-09-10T07:47:26.715183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:11.897007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695191+0000","last_change":"2024-09-10T07:47:26.695191+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695191+0000","last_undegraded":"2024-09-10T07:47:26.695191+0000","last_fullsized":"2024-09-10T07:47:26.695191+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.737966+0000","last_change":"2024-09-10T07:47:26.717356+0000","last_active":"2024-09-10T07:47:27.737966+0000","last_peered":"2024-09-10T07:47:27.737966+0000","last_clean":"2024-09-10T07:47:27.737966+0000","last_became_active":"2024-09-10T07:47:26.717075+0000","last_became_peered":"2024-09-10T07:47:26.717075+0000","last_unstale":"2024-09-10T07:47:27.737966+0000","last_undegraded":"2024-09-10T07:47:27.737966+0000","last_fullsized":"2024-09-10T07:47:27.737966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:23.907465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695001+0000","last_change":"2024-09-10T07:47:26.695001+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695001+0000","last_undegraded":"2024-09-10T07:47:26.695001+0000","last_fullsized":"2024-09-10T07:47:26.695001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707037+0000","last_change":"2024-09-10T07:47:26.709026+0000","last_active":"2024-09-10T07:47:28.707037+0000","last_peered":"2024-09-10T07:47:28.707037+0000","last_clean":"2024-09-10T07:47:28.707037+0000","last_became_active":"2024-09-10T07:47:26.708550+0000","last_became_peered":"2024-09-10T07:47:26.708550+0000","last_unstale":"2024-09-10T07:47:28.707037+0000","last_undegraded":"2024-09-10T07:47:28.707037+0000","last_fullsized":"2024-09-10T07:47:28.707037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730160+0000","last_change":"2024-09-10T07:47:27.722622+0000","last_active":"2024-09-10T07:47:27.730160+0000","last_peered":"2024-09-10T07:47:27.730160+0000","last_clean":"2024-09-10T07:47:27.730160+0000","last_became_active":"2024-09-10T07:47:27.722182+0000","last_became_peered":"2024-09-10T07:47:27.722182+0000","last_unstale":"2024-09-10T07:47:27.730160+0000","last_undegraded":"2024-09-10T07:47:27.730160+0000","last_fullsized":"2024-09-10T07:47:27.730160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:45.005644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.734519+0000","last_change":"2024-09-10T07:47:26.714730+0000","last_active":"2024-09-10T07:47:27.734519+0000","last_peered":"2024-09-10T07:47:27.734519+0000","last_clean":"2024-09-10T07:47:27.734519+0000","last_became_active":"2024-09-10T07:47:26.713780+0000","last_became_peered":"2024-09-10T07:47:26.713780+0000","last_unstale":"2024-09-10T07:47:27.734519+0000","last_undegraded":"2024-09-10T07:47:27.734519+0000","last_fullsized":"2024-09-10T07:47:27.734519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:59.575014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694052+0000","last_change":"2024-09-10T07:47:26.694052+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694052+0000","last_undegraded":"2024-09-10T07:47:26.694052+0000","last_fullsized":"2024-09-10T07:47:26.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716387+0000","last_change":"2024-09-10T07:47:26.711406+0000","last_active":"2024-09-10T07:47:26.716387+0000","last_peered":"2024-09-10T07:47:26.716387+0000","last_clean":"2024-09-10T07:47:26.716387+0000","last_became_active":"2024-09-10T07:47:26.710814+0000","last_became_peered":"2024-09-10T07:47:26.710814+0000","last_unstale":"2024-09-10T07:47:26.716387+0000","last_undegraded":"2024-09-10T07:47:26.716387+0000","last_fullsized":"2024-09-10T07:47:26.716387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696890+0000","last_change":"2024-09-10T07:47:26.696890+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696890+0000","last_undegraded":"2024-09-10T07:47:26.696890+0000","last_fullsized":"2024-09-10T07:47:26.696890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716460+0000","last_change":"2024-09-10T07:47:26.713426+0000","last_active":"2024-09-10T07:47:26.716460+0000","last_peered":"2024-09-10T07:47:26.716460+0000","last_clean":"2024-09-10T07:47:26.716460+0000","last_became_active":"2024-09-10T07:47:26.712028+0000","last_became_peered":"2024-09-10T07:47:26.712028+0000","last_unstale":"2024-09-10T07:47:26.716460+0000","last_undegraded":"2024-09-10T07:47:26.716460+0000","last_fullsized":"2024-09-10T07:47:26.716460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694004+0000","last_change":"2024-09-10T07:47:26.694004+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694004+0000","last_undegraded":"2024-09-10T07:47:26.694004+0000","last_fullsized":"2024-09-10T07:47:26.694004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706730+0000","last_change":"2024-09-10T07:47:26.701393+0000","last_active":"2024-09-10T07:47:28.706730+0000","last_peered":"2024-09-10T07:47:28.706730+0000","last_clean":"2024-09-10T07:47:28.706730+0000","last_became_active":"2024-09-10T07:47:26.701105+0000","last_became_peered":"2024-09-10T07:47:26.701105+0000","last_unstale":"2024-09-10T07:47:28.706730+0000","last_undegraded":"2024-09-10T07:47:28.706730+0000","last_fullsized":"2024-09-10T07:47:28.706730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.714471+0000","last_change":"2024-09-10T07:47:27.708836+0000","last_active":"2024-09-10T07:47:27.714471+0000","last_peered":"2024-09-10T07:47:27.714471+0000","last_clean":"2024-09-10T07:47:27.714471+0000","last_became_active":"2024-09-10T07:47:27.708519+0000","last_became_peered":"2024-09-10T07:47:27.708519+0000","last_unstale":"2024-09-10T07:47:27.714471+0000","last_undegraded":"2024-09-10T07:47:27.714471+0000","last_fullsized":"2024-09-10T07:47:27.714471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:46.304079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716575+0000","last_change":"2024-09-10T07:47:26.713761+0000","last_active":"2024-09-10T07:47:26.716575+0000","last_peered":"2024-09-10T07:47:26.716575+0000","last_clean":"2024-09-10T07:47:26.716575+0000","last_became_active":"2024-09-10T07:47:26.713497+0000","last_became_peered":"2024-09-10T07:47:26.713497+0000","last_unstale":"2024-09-10T07:47:26.716575+0000","last_undegraded":"2024-09-10T07:47:26.716575+0000","last_fullsized":"2024-09-10T07:47:26.716575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:07.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699737+0000","last_change":"2024-09-10T07:47:26.699737+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699737+0000","last_undegraded":"2024-09-10T07:47:26.699737+0000","last_fullsized":"2024-09-10T07:47:26.699737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709924+0000","last_change":"2024-09-10T07:47:26.716721+0000","last_active":"2024-09-10T07:47:28.709924+0000","last_peered":"2024-09-10T07:47:28.709924+0000","last_clean":"2024-09-10T07:47:28.709924+0000","last_became_active":"2024-09-10T07:47:26.716429+0000","last_became_peered":"2024-09-10T07:47:26.716429+0000","last_unstale":"2024-09-10T07:47:28.709924+0000","last_undegraded":"2024-09-10T07:47:28.709924+0000","last_fullsized":"2024-09-10T07:47:28.709924+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.729913+0000","last_change":"2024-09-10T07:47:27.721272+0000","last_active":"2024-09-10T07:47:27.729913+0000","last_peered":"2024-09-10T07:47:27.729913+0000","last_clean":"2024-09-10T07:47:27.729913+0000","last_became_active":"2024-09-10T07:47:27.718319+0000","last_became_peered":"2024-09-10T07:47:27.718319+0000","last_unstale":"2024-09-10T07:47:27.729913+0000","last_undegraded":"2024-09-10T07:47:27.729913+0000","last_fullsized":"2024-09-10T07:47:27.729913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:00.885462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709462+0000","last_change":"2024-09-10T07:47:26.713954+0000","last_active":"2024-09-10T07:47:28.709462+0000","last_peered":"2024-09-10T07:47:28.709462+0000","last_clean":"2024-09-10T07:47:28.709462+0000","last_became_active":"2024-09-10T07:47:26.713378+0000","last_became_peered":"2024-09-10T07:47:26.713378+0000","last_unstale":"2024-09-10T07:47:28.709462+0000","last_undegraded":"2024-09-10T07:47:28.709462+0000","last_fullsized":"2024-09-10T07:47:28.709462+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:03.461580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731659+0000","last_change":"2024-09-10T07:47:27.724717+0000","last_active":"2024-09-10T07:47:27.731659+0000","last_peered":"2024-09-10T07:47:27.731659+0000","last_clean":"2024-09-10T07:47:27.731659+0000","last_became_active":"2024-09-10T07:47:27.724464+0000","last_became_peered":"2024-09-10T07:47:27.724464+0000","last_unstale":"2024-09-10T07:47:27.731659+0000","last_undegraded":"2024-09-10T07:47:27.731659+0000","last_fullsized":"2024-09-10T07:47:27.731659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:41.202808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715090+0000","last_change":"2024-09-10T07:47:26.709280+0000","last_active":"2024-09-10T07:47:26.715090+0000","last_peered":"2024-09-10T07:47:26.715090+0000","last_clean":"2024-09-10T07:47:26.715090+0000","last_became_active":"2024-09-10T07:47:26.708960+0000","last_became_peered":"2024-09-10T07:47:26.708960+0000","last_unstale":"2024-09-10T07:47:26.715090+0000","last_undegraded":"2024-09-10T07:47:26.715090+0000","last_fullsized":"2024-09-10T07:47:26.715090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:40.600902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698390+0000","last_change":"2024-09-10T07:47:26.698390+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698390+0000","last_undegraded":"2024-09-10T07:47:26.698390+0000","last_fullsized":"2024-09-10T07:47:26.698390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272123+0000","last_change":"2024-09-10T07:47:26.713112+0000","last_active":"2024-09-10T07:47:29.272123+0000","last_peered":"2024-09-10T07:47:29.272123+0000","last_clean":"2024-09-10T07:47:29.272123+0000","last_became_active":"2024-09-10T07:47:26.712737+0000","last_became_peered":"2024-09-10T07:47:26.712737+0000","last_unstale":"2024-09-10T07:47:29.272123+0000","last_undegraded":"2024-09-10T07:47:29.272123+0000","last_fullsized":"2024-09-10T07:47:29.272123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271575+0000","last_change":"2024-09-10T07:47:27.722758+0000","last_active":"2024-09-10T07:47:29.271575+0000","last_peered":"2024-09-10T07:47:29.271575+0000","last_clean":"2024-09-10T07:47:29.271575+0000","last_became_active":"2024-09-10T07:47:27.722517+0000","last_became_peered":"2024-09-10T07:47:27.722517+0000","last_unstale":"2024-09-10T07:47:29.271575+0000","last_undegraded":"2024-09-10T07:47:29.271575+0000","last_fullsized":"2024-09-10T07:47:29.271575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:07.008875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716544+0000","last_change":"2024-09-10T07:47:26.713271+0000","last_active":"2024-09-10T07:47:26.716544+0000","last_peered":"2024-09-10T07:47:26.716544+0000","last_clean":"2024-09-10T07:47:26.716544+0000","last_became_active":"2024-09-10T07:47:26.712715+0000","last_became_peered":"2024-09-10T07:47:26.712715+0000","last_unstale":"2024-09-10T07:47:26.716544+0000","last_undegraded":"2024-09-10T07:47:26.716544+0000","last_fullsized":"2024-09-10T07:47:26.716544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730403+0000","last_change":"2024-09-10T07:47:27.725965+0000","last_active":"2024-09-10T07:47:27.730403+0000","last_peered":"2024-09-10T07:47:27.730403+0000","last_clean":"2024-09-10T07:47:27.730403+0000","last_became_active":"2024-09-10T07:47:27.725029+0000","last_became_peered":"2024-09-10T07:47:27.725029+0000","last_unstale":"2024-09-10T07:47:27.730403+0000","last_undegraded":"2024-09-10T07:47:27.730403+0000","last_fullsized":"2024-09-10T07:47:27.730403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:31.812075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699985+0000","last_change":"2024-09-10T07:47:26.711747+0000","last_active":"2024-09-10T07:47:27.699985+0000","last_peered":"2024-09-10T07:47:27.699985+0000","last_clean":"2024-09-10T07:47:27.699985+0000","last_became_active":"2024-09-10T07:47:26.711458+0000","last_became_peered":"2024-09-10T07:47:26.711458+0000","last_unstale":"2024-09-10T07:47:27.699985+0000","last_undegraded":"2024-09-10T07:47:27.699985+0000","last_fullsized":"2024-09-10T07:47:27.699985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:17.886142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694848+0000","last_change":"2024-09-10T07:47:26.694848+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694848+0000","last_undegraded":"2024-09-10T07:47:26.694848+0000","last_fullsized":"2024-09-10T07:47:26.694848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.698363+0000","last_change":"2024-09-10T07:47:26.708509+0000","last_active":"2024-09-10T07:47:27.698363+0000","last_peered":"2024-09-10T07:47:27.698363+0000","last_clean":"2024-09-10T07:47:27.698363+0000","last_became_active":"2024-09-10T07:47:26.708220+0000","last_became_peered":"2024-09-10T07:47:26.708220+0000","last_unstale":"2024-09-10T07:47:27.698363+0000","last_undegraded":"2024-09-10T07:47:27.698363+0000","last_fullsized":"2024-09-10T07:47:27.698363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:55.494668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700387+0000","last_change":"2024-09-10T07:47:26.700387+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700387+0000","last_undegraded":"2024-09-10T07:47:26.700387+0000","last_fullsized":"2024-09-10T07:47:26.700387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706491+0000","last_change":"2024-09-10T07:47:26.706290+0000","last_active":"2024-09-10T07:47:28.706491+0000","last_peered":"2024-09-10T07:47:28.706491+0000","last_clean":"2024-09-10T07:47:28.706491+0000","last_became_active":"2024-09-10T07:47:26.705979+0000","last_became_peered":"2024-09-10T07:47:26.705979+0000","last_unstale":"2024-09-10T07:47:28.706491+0000","last_undegraded":"2024-09-10T07:47:28.706491+0000","last_fullsized":"2024-09-10T07:47:28.706491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699283+0000","last_change":"2024-09-10T07:47:26.699283+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699283+0000","last_undegraded":"2024-09-10T07:47:26.699283+0000","last_fullsized":"2024-09-10T07:47:26.699283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706848+0000","last_change":"2024-09-10T07:47:26.710001+0000","last_active":"2024-09-10T07:47:28.706848+0000","last_peered":"2024-09-10T07:47:28.706848+0000","last_clean":"2024-09-10T07:47:28.706848+0000","last_became_active":"2024-09-10T07:47:26.708545+0000","last_became_peered":"2024-09-10T07:47:26.708545+0000","last_unstale":"2024-09-10T07:47:28.706848+0000","last_undegraded":"2024-09-10T07:47:28.706848+0000","last_fullsized":"2024-09-10T07:47:28.706848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697405+0000","last_change":"2024-09-10T07:47:26.697405+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697405+0000","last_undegraded":"2024-09-10T07:47:26.697405+0000","last_fullsized":"2024-09-10T07:47:26.697405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716435+0000","last_change":"2024-09-10T07:47:26.713070+0000","last_active":"2024-09-10T07:47:26.716435+0000","last_peered":"2024-09-10T07:47:26.716435+0000","last_clean":"2024-09-10T07:47:26.716435+0000","last_became_active":"2024-09-10T07:47:26.712820+0000","last_became_peered":"2024-09-10T07:47:26.712820+0000","last_unstale":"2024-09-10T07:47:26.716435+0000","last_undegraded":"2024-09-10T07:47:26.716435+0000","last_fullsized":"2024-09-10T07:47:26.716435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709356+0000","last_change":"2024-09-10T07:47:27.712460+0000","last_active":"2024-09-10T07:47:28.709356+0000","last_peered":"2024-09-10T07:47:28.709356+0000","last_clean":"2024-09-10T07:47:28.709356+0000","last_became_active":"2024-09-10T07:47:27.712183+0000","last_became_peered":"2024-09-10T07:47:27.712183+0000","last_unstale":"2024-09-10T07:47:28.709356+0000","last_undegraded":"2024-09-10T07:47:28.709356+0000","last_fullsized":"2024-09-10T07:47:28.709356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716233+0000","last_change":"2024-09-10T07:47:26.713128+0000","last_active":"2024-09-10T07:47:26.716233+0000","last_peered":"2024-09-10T07:47:26.716233+0000","last_clean":"2024-09-10T07:47:26.716233+0000","last_became_active":"2024-09-10T07:47:26.712677+0000","last_became_peered":"2024-09-10T07:47:26.712677+0000","last_unstale":"2024-09-10T07:47:26.716233+0000","last_undegraded":"2024-09-10T07:47:26.716233+0000","last_fullsized":"2024-09-10T07:47:26.716233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:06.290962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706999+0000","last_change":"2024-09-10T07:47:27.724138+0000","last_active":"2024-09-10T07:47:28.706999+0000","last_peered":"2024-09-10T07:47:28.706999+0000","last_clean":"2024-09-10T07:47:28.706999+0000","last_became_active":"2024-09-10T07:47:27.723179+0000","last_became_peered":"2024-09-10T07:47:27.723179+0000","last_unstale":"2024-09-10T07:47:28.706999+0000","last_undegraded":"2024-09-10T07:47:28.706999+0000","last_fullsized":"2024-09-10T07:47:28.706999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716153+0000","last_change":"2024-09-10T07:47:26.711949+0000","last_active":"2024-09-10T07:47:26.716153+0000","last_peered":"2024-09-10T07:47:26.716153+0000","last_clean":"2024-09-10T07:47:26.716153+0000","last_became_active":"2024-09-10T07:47:26.711492+0000","last_became_peered":"2024-09-10T07:47:26.711492+0000","last_unstale":"2024-09-10T07:47:26.716153+0000","last_undegraded":"2024-09-10T07:47:26.716153+0000","last_fullsized":"2024-09-10T07:47:26.716153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:31.026752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271616+0000","last_change":"2024-09-10T07:47:27.725128+0000","last_active":"2024-09-10T07:47:29.271616+0000","last_peered":"2024-09-10T07:47:29.271616+0000","last_clean":"2024-09-10T07:47:29.271616+0000","last_became_active":"2024-09-10T07:47:27.724151+0000","last_became_peered":"2024-09-10T07:47:27.724151+0000","last_unstale":"2024-09-10T07:47:29.271616+0000","last_undegraded":"2024-09-10T07:47:29.271616+0000","last_fullsized":"2024-09-10T07:47:29.271616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:38.688543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709386+0000","last_change":"2024-09-10T07:47:26.715573+0000","last_active":"2024-09-10T07:47:28.709386+0000","last_peered":"2024-09-10T07:47:28.709386+0000","last_clean":"2024-09-10T07:47:28.709386+0000","last_became_active":"2024-09-10T07:47:26.715315+0000","last_became_peered":"2024-09-10T07:47:26.715315+0000","last_unstale":"2024-09-10T07:47:28.709386+0000","last_undegraded":"2024-09-10T07:47:28.709386+0000","last_fullsized":"2024-09-10T07:47:28.709386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:37.683908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709341+0000","last_change":"2024-09-10T07:47:27.701024+0000","last_active":"2024-09-10T07:47:28.709341+0000","last_peered":"2024-09-10T07:47:28.709341+0000","last_clean":"2024-09-10T07:47:28.709341+0000","last_became_active":"2024-09-10T07:47:27.700612+0000","last_became_peered":"2024-09-10T07:47:27.700612+0000","last_unstale":"2024-09-10T07:47:28.709341+0000","last_undegraded":"2024-09-10T07:47:28.709341+0000","last_fullsized":"2024-09-10T07:47:28.709341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715646+0000","last_change":"2024-09-10T07:47:26.707503+0000","last_active":"2024-09-10T07:47:26.715646+0000","last_peered":"2024-09-10T07:47:26.715646+0000","last_clean":"2024-09-10T07:47:26.715646+0000","last_became_active":"2024-09-10T07:47:26.706814+0000","last_became_peered":"2024-09-10T07:47:26.706814+0000","last_unstale":"2024-09-10T07:47:26.715646+0000","last_undegraded":"2024-09-10T07:47:26.715646+0000","last_fullsized":"2024-09-10T07:47:26.715646+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:10.209621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271600+0000","last_change":"2024-09-10T07:47:27.722893+0000","last_active":"2024-09-10T07:47:29.271600+0000","last_peered":"2024-09-10T07:47:29.271600+0000","last_clean":"2024-09-10T07:47:29.271600+0000","last_became_active":"2024-09-10T07:47:27.722551+0000","last_became_peered":"2024-09-10T07:47:27.722551+0000","last_unstale":"2024-09-10T07:47:29.271600+0000","last_undegraded":"2024-09-10T07:47:29.271600+0000","last_fullsized":"2024-09-10T07:47:29.271600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:52.315032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716562+0000","last_change":"2024-09-10T07:47:26.714230+0000","last_active":"2024-09-10T07:47:26.716562+0000","last_peered":"2024-09-10T07:47:26.716562+0000","last_clean":"2024-09-10T07:47:26.716562+0000","last_became_active":"2024-09-10T07:47:26.712186+0000","last_became_peered":"2024-09-10T07:47:26.712186+0000","last_unstale":"2024-09-10T07:47:26.716562+0000","last_undegraded":"2024-09-10T07:47:26.716562+0000","last_fullsized":"2024-09-10T07:47:26.716562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:55.243965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694126+0000","last_change":"2024-09-10T07:47:26.694126+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694126+0000","last_undegraded":"2024-09-10T07:47:26.694126+0000","last_fullsized":"2024-09-10T07:47:26.694126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.712412+0000","last_change":"2024-09-10T07:47:26.709027+0000","last_active":"2024-09-10T07:47:26.712412+0000","last_peered":"2024-09-10T07:47:26.712412+0000","last_clean":"2024-09-10T07:47:26.712412+0000","last_became_active":"2024-09-10T07:47:26.708753+0000","last_became_peered":"2024-09-10T07:47:26.708753+0000","last_unstale":"2024-09-10T07:47:26.712412+0000","last_undegraded":"2024-09-10T07:47:26.712412+0000","last_fullsized":"2024-09-10T07:47:26.712412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:13.182619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707048+0000","last_change":"2024-09-10T07:47:27.728171+0000","last_active":"2024-09-10T07:47:28.707048+0000","last_peered":"2024-09-10T07:47:28.707048+0000","last_clean":"2024-09-10T07:47:28.707048+0000","last_became_active":"2024-09-10T07:47:27.727842+0000","last_became_peered":"2024-09-10T07:47:27.727842+0000","last_unstale":"2024-09-10T07:47:28.707048+0000","last_undegraded":"2024-09-10T07:47:28.707048+0000","last_fullsized":"2024-09-10T07:47:28.707048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:45.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272065+0000","last_change":"2024-09-10T07:47:26.707810+0000","last_active":"2024-09-10T07:47:29.272065+0000","last_peered":"2024-09-10T07:47:29.272065+0000","last_clean":"2024-09-10T07:47:29.272065+0000","last_became_active":"2024-09-10T07:47:26.707462+0000","last_became_peered":"2024-09-10T07:47:26.707462+0000","last_unstale":"2024-09-10T07:47:29.272065+0000","last_undegraded":"2024-09-10T07:47:29.272065+0000","last_fullsized":"2024-09-10T07:47:29.272065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698308+0000","last_change":"2024-09-10T07:47:26.698308+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698308+0000","last_undegraded":"2024-09-10T07:47:26.698308+0000","last_fullsized":"2024-09-10T07:47:26.698308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709605+0000","last_change":"2024-09-10T07:47:26.706696+0000","last_active":"2024-09-10T07:47:28.709605+0000","last_peered":"2024-09-10T07:47:28.709605+0000","last_clean":"2024-09-10T07:47:28.709605+0000","last_became_active":"2024-09-10T07:47:26.706383+0000","last_became_peered":"2024-09-10T07:47:26.706383+0000","last_unstale":"2024-09-10T07:47:28.709605+0000","last_undegraded":"2024-09-10T07:47:28.709605+0000","last_fullsized":"2024-09-10T07:47:28.709605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:16.889913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707098+0000","last_change":"2024-09-10T07:47:27.721003+0000","last_active":"2024-09-10T07:47:28.707098+0000","last_peered":"2024-09-10T07:47:28.707098+0000","last_clean":"2024-09-10T07:47:28.707098+0000","last_became_active":"2024-09-10T07:47:27.720702+0000","last_became_peered":"2024-09-10T07:47:27.720702+0000","last_unstale":"2024-09-10T07:47:28.707098+0000","last_undegraded":"2024-09-10T07:47:28.707098+0000","last_fullsized":"2024-09-10T07:47:28.707098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:33.721528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272228+0000","last_change":"2024-09-10T07:47:26.701128+0000","last_active":"2024-09-10T07:47:29.272228+0000","last_peered":"2024-09-10T07:47:29.272228+0000","last_clean":"2024-09-10T07:47:29.272228+0000","last_became_active":"2024-09-10T07:47:26.700042+0000","last_became_peered":"2024-09-10T07:47:26.700042+0000","last_unstale":"2024-09-10T07:47:29.272228+0000","last_undegraded":"2024-09-10T07:47:29.272228+0000","last_fullsized":"2024-09-10T07:47:29.272228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.692995+0000","last_change":"2024-09-10T07:47:26.692995+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.692995+0000","last_undegraded":"2024-09-10T07:47:26.692995+0000","last_fullsized":"2024-09-10T07:47:26.692995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271867+0000","last_change":"2024-09-10T07:47:26.705882+0000","last_active":"2024-09-10T07:47:29.271867+0000","last_peered":"2024-09-10T07:47:29.271867+0000","last_clean":"2024-09-10T07:47:29.271867+0000","last_became_active":"2024-09-10T07:47:26.705616+0000","last_became_peered":"2024-09-10T07:47:26.705616+0000","last_unstale":"2024-09-10T07:47:29.271867+0000","last_undegraded":"2024-09-10T07:47:29.271867+0000","last_fullsized":"2024-09-10T07:47:29.271867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728015+0000","last_change":"2024-09-10T07:47:27.719286+0000","last_active":"2024-09-10T07:47:27.728015+0000","last_peered":"2024-09-10T07:47:27.728015+0000","last_clean":"2024-09-10T07:47:27.728015+0000","last_became_active":"2024-09-10T07:47:27.715311+0000","last_became_peered":"2024-09-10T07:47:27.715311+0000","last_unstale":"2024-09-10T07:47:27.728015+0000","last_undegraded":"2024-09-10T07:47:27.728015+0000","last_fullsized":"2024-09-10T07:47:27.728015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:26:17.625922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271645+0000","last_change":"2024-09-10T07:47:26.710717+0000","last_active":"2024-09-10T07:47:29.271645+0000","last_peered":"2024-09-10T07:47:29.271645+0000","last_clean":"2024-09-10T07:47:29.271645+0000","last_became_active":"2024-09-10T07:47:26.710392+0000","last_became_peered":"2024-09-10T07:47:26.710392+0000","last_unstale":"2024-09-10T07:47:29.271645+0000","last_undegraded":"2024-09-10T07:47:29.271645+0000","last_fullsized":"2024-09-10T07:47:29.271645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730509+0000","last_change":"2024-09-10T07:47:27.727229+0000","last_active":"2024-09-10T07:47:27.730509+0000","last_peered":"2024-09-10T07:47:27.730509+0000","last_clean":"2024-09-10T07:47:27.730509+0000","last_became_active":"2024-09-10T07:47:27.725550+0000","last_became_peered":"2024-09-10T07:47:27.725550+0000","last_unstale":"2024-09-10T07:47:27.730509+0000","last_undegraded":"2024-09-10T07:47:27.730509+0000","last_fullsized":"2024-09-10T07:47:27.730509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:02.762995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716358+0000","last_change":"2024-09-10T07:47:26.710777+0000","last_active":"2024-09-10T07:47:26.716358+0000","last_peered":"2024-09-10T07:47:26.716358+0000","last_clean":"2024-09-10T07:47:26.716358+0000","last_became_active":"2024-09-10T07:47:26.710499+0000","last_became_peered":"2024-09-10T07:47:26.710499+0000","last_unstale":"2024-09-10T07:47:26.716358+0000","last_undegraded":"2024-09-10T07:47:26.716358+0000","last_fullsized":"2024-09-10T07:47:26.716358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:25.822800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699907+0000","last_change":"2024-09-10T07:47:26.699907+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699907+0000","last_undegraded":"2024-09-10T07:47:26.699907+0000","last_fullsized":"2024-09-10T07:47:26.699907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699089+0000","last_change":"2024-09-10T07:47:26.716992+0000","last_active":"2024-09-10T07:47:27.699089+0000","last_peered":"2024-09-10T07:47:27.699089+0000","last_clean":"2024-09-10T07:47:27.699089+0000","last_became_active":"2024-09-10T07:47:26.716690+0000","last_became_peered":"2024-09-10T07:47:26.716690+0000","last_unstale":"2024-09-10T07:47:27.699089+0000","last_undegraded":"2024-09-10T07:47:27.699089+0000","last_fullsized":"2024-09-10T07:47:27.699089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.180750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.693017+0000","last_change":"2024-09-10T07:47:26.693017+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.693017+0000","last_undegraded":"2024-09-10T07:47:26.693017+0000","last_fullsized":"2024-09-10T07:47:26.693017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.710693+0000","last_change":"2024-09-10T07:47:26.707932+0000","last_active":"2024-09-10T07:47:26.710693+0000","last_peered":"2024-09-10T07:47:26.710693+0000","last_clean":"2024-09-10T07:47:26.710693+0000","last_became_active":"2024-09-10T07:47:26.707401+0000","last_became_peered":"2024-09-10T07:47:26.707401+0000","last_unstale":"2024-09-10T07:47:26.710693+0000","last_undegraded":"2024-09-10T07:47:26.710693+0000","last_fullsized":"2024-09-10T07:47:26.710693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.677752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.697935+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:47:26.697935+0000","last_peered":"2024-09-10T07:47:26.697935+0000","last_clean":"2024-09-10T07:47:26.697935+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:47:26.697935+0000","last_undegraded":"2024-09-10T07:47:26.697935+0000","last_fullsized":"2024-09-10T07:47:26.697935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":725755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":154759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":156349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":745687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":372736,"data_stored":141662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:47:32.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:47:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.536+0000 7fb235000640 1 Processor -- start 2024-09-10T07:47:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.536+0000 7fb235000640 1 -- start start 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb235000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb23015e860 0x7fb2300cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb235000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2300cee40 con 0x7fb2300cfbf0 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb235000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2300cf120 con 0x7fb230073150 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb235000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb2300d3780 con 0x7fb23015e860 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb22f400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb23015e860 0x7fb2300cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb22ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb2300cfbf0 0x7fb2300d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39182/0 (socket says 172.21.15.3:39182) 2024-09-10T07:47:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.537+0000 7fb22ea00640 1 -- 172.21.15.3:0/2026326681 learned_addr learned my addr 172.21.15.3:0/2026326681 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.538+0000 7fb22f400640 1 -- 172.21.15.3:0/2026326681 >> v1:172.21.15.70:6789/0 conn(0x7fb230073150 legacy=0x7fb2300b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.538+0000 7fb22f400640 1 -- 172.21.15.3:0/2026326681 >> v1:172.21.15.3:6789/0 conn(0x7fb2300cfbf0 legacy=0x7fb2300d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.538+0000 7fb22f400640 1 -- 172.21.15.3:0/2026326681 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300d3410 con 0x7fb23015e860 2024-09-10T07:47:32.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.538+0000 7fb22e000640 1 -- 172.21.15.3:0/2026326681 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070070904 0 0) 0x7fb2300cee40 con 0x7fb2300cfbf0 2024-09-10T07:47:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.539+0000 7fb22f400640 1 --2- 172.21.15.3:0/2026326681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb2300cf6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2200097a0 tx=0x7fb22002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5549a4ce56a0ad46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.539+0000 7fb22e000640 1 -- 172.21.15.3:0/2026326681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220037070 con 0x7fb23015e860 2024-09-10T07:47:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.539+0000 7fb22e000640 1 -- 172.21.15.3:0/2026326681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb220032440 con 0x7fb23015e860 2024-09-10T07:47:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.539+0000 7fb22e000640 1 -- 172.21.15.3:0/2026326681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220032810 con 0x7fb23015e860 2024-09-10T07:47:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 -- 172.21.15.3:0/2026326681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 msgr2=0x7fb2300cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 --2- 172.21.15.3:0/2026326681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb2300cf6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2200097a0 tx=0x7fb22002a230 comp rx=0 tx=0).stop 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 -- 172.21.15.3:0/2026326681 shutdown_connections 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 --2- 172.21.15.3:0/2026326681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb2300cf6b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 -- 172.21.15.3:0/2026326681 >> 172.21.15.3:0/2026326681 conn(0x7fb2300d4650 msgr2=0x7fb2300d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 -- 172.21.15.3:0/2026326681 shutdown_connections 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.540+0000 7fb235000640 1 -- 172.21.15.3:0/2026326681 wait complete. 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.541+0000 7fb235000640 1 Processor -- start 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.541+0000 7fb235000640 1 -- start start 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb230073150 0x7fb2301f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2300cfbf0 0x7fb2301f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300d2460 con 0x7fb23015e860 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300cf400 con 0x7fb2300cfbf0 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.542+0000 7fb235000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300b7f90 con 0x7fb230073150 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36062/0 (socket says 172.21.15.3:36062) 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 -- 172.21.15.3:0/291553948 learned_addr learned my addr 172.21.15.3:0/291553948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22ea00640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2300cfbf0 0x7fb2301f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22f400640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb230073150 0x7fb2301f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 -- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb230073150 msgr2=0x7fb2301f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb230073150 0x7fb2301f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 -- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2300cfbf0 msgr2=0x7fb2301f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2300cfbf0 0x7fb2301f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 -- 172.21.15.3:0/291553948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300ae8d0 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb22fe00640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c7d0 tx=0x7fb22800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228003b70 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb228003d30 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.543+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22800aa90 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.544+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2300aebb0 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.544+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301f75e0 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.545+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2300cb170 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.546+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb22800ac30 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.546+0000 7fb227400640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 0x7fb21006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.546+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb22808aeb0 con 0x7fb23015e860 2024-09-10T07:47:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.547+0000 7fb22f400640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 0x7fb21006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.547+0000 7fb22f400640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 0x7fb21006e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb220034000 tx=0x7fb22002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.550+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2280597c0 con 0x7fb23015e860 2024-09-10T07:47:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.732+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb230069e20 con 0x7fb21006be60 2024-09-10T07:47:32.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.738+0000 7fb227400640 1 -- 172.21.15.3:0/291553948 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249138 (secure 0 0 0) 0x7fb230069e20 con 0x7fb21006be60 2024-09-10T07:47:32.741 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:32.742 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 msgr2=0x7fb21006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 0x7fb21006e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb220034000 tx=0x7fb22002e000 comp rx=0 tx=0).stop 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 msgr2=0x7fb230160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c7d0 tx=0x7fb22800cc90 comp rx=0 tx=0).stop 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 shutdown_connections 2024-09-10T07:47:32.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb21006be60 0x7fb21006e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb23015e860 0x7fb230160e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.742+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2300cfbf0 0x7fb2301f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.743+0000 7fb235000640 1 --2- 172.21.15.3:0/291553948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb230073150 0x7fb2301f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.743+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 >> 172.21.15.3:0/291553948 conn(0x7fb2300d4650 msgr2=0x7fb2300b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.743+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 shutdown_connections 2024-09-10T07:47:32.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.743+0000 7fb235000640 1 -- 172.21.15.3:0/291553948 wait complete. 2024-09-10T07:47:32.765 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T07:47:31.501965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236996,"kb_used_data":4116,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737724,"statfs":{"total":773094113280,"available":772851429376,"internally_reserved":0,"allocated":4214784,"data_stored":2330373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815909"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.702272+0000","last_change":"2024-09-10T07:47:26.702272+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.702272+0000","last_undegraded":"2024-09-10T07:47:26.702272+0000","last_fullsized":"2024-09-10T07:47:26.702272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709608+0000","last_change":"2024-09-10T07:47:27.717295+0000","last_active":"2024-09-10T07:47:28.709608+0000","last_peered":"2024-09-10T07:47:28.709608+0000","last_clean":"2024-09-10T07:47:28.709608+0000","last_became_active":"2024-09-10T07:47:27.716979+0000","last_became_peered":"2024-09-10T07:47:27.716979+0000","last_unstale":"2024-09-10T07:47:28.709608+0000","last_undegraded":"2024-09-10T07:47:28.709608+0000","last_fullsized":"2024-09-10T07:47:28.709608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695776+0000","last_change":"2024-09-10T07:47:26.695776+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695776+0000","last_undegraded":"2024-09-10T07:47:26.695776+0000","last_fullsized":"2024-09-10T07:47:26.695776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731683+0000","last_change":"2024-09-10T07:47:27.726560+0000","last_active":"2024-09-10T07:47:27.731683+0000","last_peered":"2024-09-10T07:47:27.731683+0000","last_clean":"2024-09-10T07:47:27.731683+0000","last_became_active":"2024-09-10T07:47:27.726024+0000","last_became_peered":"2024-09-10T07:47:27.726024+0000","last_unstale":"2024-09-10T07:47:27.731683+0000","last_undegraded":"2024-09-10T07:47:27.731683+0000","last_fullsized":"2024-09-10T07:47:27.731683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:50.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709319+0000","last_change":"2024-09-10T07:47:27.710914+0000","last_active":"2024-09-10T07:47:28.709319+0000","last_peered":"2024-09-10T07:47:28.709319+0000","last_clean":"2024-09-10T07:47:28.709319+0000","last_became_active":"2024-09-10T07:47:27.710617+0000","last_became_peered":"2024-09-10T07:47:27.710617+0000","last_unstale":"2024-09-10T07:47:28.709319+0000","last_undegraded":"2024-09-10T07:47:28.709319+0000","last_fullsized":"2024-09-10T07:47:28.709319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706883+0000","last_change":"2024-09-10T07:47:27.721104+0000","last_active":"2024-09-10T07:47:28.706883+0000","last_peered":"2024-09-10T07:47:28.706883+0000","last_clean":"2024-09-10T07:47:28.706883+0000","last_became_active":"2024-09-10T07:47:27.718624+0000","last_became_peered":"2024-09-10T07:47:27.718624+0000","last_unstale":"2024-09-10T07:47:28.706883+0000","last_undegraded":"2024-09-10T07:47:28.706883+0000","last_fullsized":"2024-09-10T07:47:28.706883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697623+0000","last_change":"2024-09-10T07:47:26.697623+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697623+0000","last_undegraded":"2024-09-10T07:47:26.697623+0000","last_fullsized":"2024-09-10T07:47:26.697623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699517+0000","last_change":"2024-09-10T07:47:26.699517+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699517+0000","last_undegraded":"2024-09-10T07:47:26.699517+0000","last_fullsized":"2024-09-10T07:47:26.699517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697854+0000","last_change":"2024-09-10T07:47:26.697854+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697854+0000","last_undegraded":"2024-09-10T07:47:26.697854+0000","last_fullsized":"2024-09-10T07:47:26.697854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728902+0000","last_change":"2024-09-10T07:47:27.717546+0000","last_active":"2024-09-10T07:47:27.728902+0000","last_peered":"2024-09-10T07:47:27.728902+0000","last_clean":"2024-09-10T07:47:27.728902+0000","last_became_active":"2024-09-10T07:47:27.713266+0000","last_became_peered":"2024-09-10T07:47:27.713266+0000","last_unstale":"2024-09-10T07:47:27.728902+0000","last_undegraded":"2024-09-10T07:47:27.728902+0000","last_fullsized":"2024-09-10T07:47:27.728902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:58.170897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697313+0000","last_change":"2024-09-10T07:47:26.697313+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697313+0000","last_undegraded":"2024-09-10T07:47:26.697313+0000","last_fullsized":"2024-09-10T07:47:26.697313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696911+0000","last_change":"2024-09-10T07:47:26.696911+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696911+0000","last_undegraded":"2024-09-10T07:47:26.696911+0000","last_fullsized":"2024-09-10T07:47:26.696911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271853+0000","last_change":"2024-09-10T07:47:27.728693+0000","last_active":"2024-09-10T07:47:29.271853+0000","last_peered":"2024-09-10T07:47:29.271853+0000","last_clean":"2024-09-10T07:47:29.271853+0000","last_became_active":"2024-09-10T07:47:27.728426+0000","last_became_peered":"2024-09-10T07:47:27.728426+0000","last_unstale":"2024-09-10T07:47:29.271853+0000","last_undegraded":"2024-09-10T07:47:29.271853+0000","last_fullsized":"2024-09-10T07:47:29.271853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.466215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.721891+0000","last_change":"2024-09-10T07:47:27.713960+0000","last_active":"2024-09-10T07:47:27.721891+0000","last_peered":"2024-09-10T07:47:27.721891+0000","last_clean":"2024-09-10T07:47:27.721891+0000","last_became_active":"2024-09-10T07:47:27.713692+0000","last_became_peered":"2024-09-10T07:47:27.713692+0000","last_unstale":"2024-09-10T07:47:27.721891+0000","last_undegraded":"2024-09-10T07:47:27.721891+0000","last_fullsized":"2024-09-10T07:47:27.721891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:50.495716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696378+0000","last_change":"2024-09-10T07:47:26.696378+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696378+0000","last_undegraded":"2024-09-10T07:47:26.696378+0000","last_fullsized":"2024-09-10T07:47:26.696378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697490+0000","last_change":"2024-09-10T07:47:26.697490+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697490+0000","last_undegraded":"2024-09-10T07:47:26.697490+0000","last_fullsized":"2024-09-10T07:47:26.697490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698719+0000","last_change":"2024-09-10T07:47:26.698719+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698719+0000","last_undegraded":"2024-09-10T07:47:26.698719+0000","last_fullsized":"2024-09-10T07:47:26.698719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707015+0000","last_change":"2024-09-10T07:47:27.717429+0000","last_active":"2024-09-10T07:47:28.707015+0000","last_peered":"2024-09-10T07:47:28.707015+0000","last_clean":"2024-09-10T07:47:28.707015+0000","last_became_active":"2024-09-10T07:47:27.717064+0000","last_became_peered":"2024-09-10T07:47:27.717064+0000","last_unstale":"2024-09-10T07:47:28.707015+0000","last_undegraded":"2024-09-10T07:47:28.707015+0000","last_fullsized":"2024-09-10T07:47:28.707015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:44.926594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706888+0000","last_change":"2024-09-10T07:47:27.715748+0000","last_active":"2024-09-10T07:47:28.706888+0000","last_peered":"2024-09-10T07:47:28.706888+0000","last_clean":"2024-09-10T07:47:28.706888+0000","last_became_active":"2024-09-10T07:47:27.715354+0000","last_became_peered":"2024-09-10T07:47:27.715354+0000","last_unstale":"2024-09-10T07:47:28.706888+0000","last_undegraded":"2024-09-10T07:47:28.706888+0000","last_fullsized":"2024-09-10T07:47:28.706888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272066+0000","last_change":"2024-09-10T07:47:27.726942+0000","last_active":"2024-09-10T07:47:29.272066+0000","last_peered":"2024-09-10T07:47:29.272066+0000","last_clean":"2024-09-10T07:47:29.272066+0000","last_became_active":"2024-09-10T07:47:27.726536+0000","last_became_peered":"2024-09-10T07:47:27.726536+0000","last_unstale":"2024-09-10T07:47:29.272066+0000","last_undegraded":"2024-09-10T07:47:29.272066+0000","last_fullsized":"2024-09-10T07:47:29.272066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:25.590064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695528+0000","last_change":"2024-09-10T07:47:26.695528+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695528+0000","last_undegraded":"2024-09-10T07:47:26.695528+0000","last_fullsized":"2024-09-10T07:47:26.695528+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.701477+0000","last_change":"2024-09-10T07:47:26.701477+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.701477+0000","last_undegraded":"2024-09-10T07:47:26.701477+0000","last_fullsized":"2024-09-10T07:47:26.701477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271941+0000","last_change":"2024-09-10T07:47:27.721416+0000","last_active":"2024-09-10T07:47:29.271941+0000","last_peered":"2024-09-10T07:47:29.271941+0000","last_clean":"2024-09-10T07:47:29.271941+0000","last_became_active":"2024-09-10T07:47:27.721174+0000","last_became_peered":"2024-09-10T07:47:27.721174+0000","last_unstale":"2024-09-10T07:47:29.271941+0000","last_undegraded":"2024-09-10T07:47:29.271941+0000","last_fullsized":"2024-09-10T07:47:29.271941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:17.894548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699069+0000","last_change":"2024-09-10T07:47:26.699069+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699069+0000","last_undegraded":"2024-09-10T07:47:26.699069+0000","last_fullsized":"2024-09-10T07:47:26.699069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.732178+0000","last_change":"2024-09-10T07:47:27.723555+0000","last_active":"2024-09-10T07:47:27.732178+0000","last_peered":"2024-09-10T07:47:27.732178+0000","last_clean":"2024-09-10T07:47:27.732178+0000","last_became_active":"2024-09-10T07:47:27.723226+0000","last_became_peered":"2024-09-10T07:47:27.723226+0000","last_unstale":"2024-09-10T07:47:27.732178+0000","last_undegraded":"2024-09-10T07:47:27.732178+0000","last_fullsized":"2024-09-10T07:47:27.732178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:55.167875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731804+0000","last_change":"2024-09-10T07:47:27.729136+0000","last_active":"2024-09-10T07:47:27.731804+0000","last_peered":"2024-09-10T07:47:27.731804+0000","last_clean":"2024-09-10T07:47:27.731804+0000","last_became_active":"2024-09-10T07:47:27.724907+0000","last_became_peered":"2024-09-10T07:47:27.724907+0000","last_unstale":"2024-09-10T07:47:27.731804+0000","last_undegraded":"2024-09-10T07:47:27.731804+0000","last_fullsized":"2024-09-10T07:47:27.731804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:12.255456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709383+0000","last_change":"2024-09-10T07:47:27.710661+0000","last_active":"2024-09-10T07:47:28.709383+0000","last_peered":"2024-09-10T07:47:28.709383+0000","last_clean":"2024-09-10T07:47:28.709383+0000","last_became_active":"2024-09-10T07:47:27.710117+0000","last_became_peered":"2024-09-10T07:47:27.710117+0000","last_unstale":"2024-09-10T07:47:28.709383+0000","last_undegraded":"2024-09-10T07:47:28.709383+0000","last_fullsized":"2024-09-10T07:47:28.709383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700878+0000","last_change":"2024-09-10T07:47:26.700878+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700878+0000","last_undegraded":"2024-09-10T07:47:26.700878+0000","last_fullsized":"2024-09-10T07:47:26.700878+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694553+0000","last_change":"2024-09-10T07:47:26.694553+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694553+0000","last_undegraded":"2024-09-10T07:47:26.694553+0000","last_fullsized":"2024-09-10T07:47:26.694553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694437+0000","last_change":"2024-09-10T07:47:26.694437+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694437+0000","last_undegraded":"2024-09-10T07:47:26.694437+0000","last_fullsized":"2024-09-10T07:47:26.694437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695915+0000","last_change":"2024-09-10T07:47:26.695915+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695915+0000","last_undegraded":"2024-09-10T07:47:26.695915+0000","last_fullsized":"2024-09-10T07:47:26.695915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697245+0000","last_change":"2024-09-10T07:47:26.697245+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697245+0000","last_undegraded":"2024-09-10T07:47:26.697245+0000","last_fullsized":"2024-09-10T07:47:26.697245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272289+0000","last_change":"2024-09-10T07:47:27.714639+0000","last_active":"2024-09-10T07:47:29.272289+0000","last_peered":"2024-09-10T07:47:29.272289+0000","last_clean":"2024-09-10T07:47:29.272289+0000","last_became_active":"2024-09-10T07:47:27.714372+0000","last_became_peered":"2024-09-10T07:47:27.714372+0000","last_unstale":"2024-09-10T07:47:29.272289+0000","last_undegraded":"2024-09-10T07:47:29.272289+0000","last_fullsized":"2024-09-10T07:47:29.272289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715183+0000","last_change":"2024-09-10T07:47:26.711007+0000","last_active":"2024-09-10T07:47:26.715183+0000","last_peered":"2024-09-10T07:47:26.715183+0000","last_clean":"2024-09-10T07:47:26.715183+0000","last_became_active":"2024-09-10T07:47:26.710707+0000","last_became_peered":"2024-09-10T07:47:26.710707+0000","last_unstale":"2024-09-10T07:47:26.715183+0000","last_undegraded":"2024-09-10T07:47:26.715183+0000","last_fullsized":"2024-09-10T07:47:26.715183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:11.897007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695191+0000","last_change":"2024-09-10T07:47:26.695191+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695191+0000","last_undegraded":"2024-09-10T07:47:26.695191+0000","last_fullsized":"2024-09-10T07:47:26.695191+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.737966+0000","last_change":"2024-09-10T07:47:26.717356+0000","last_active":"2024-09-10T07:47:27.737966+0000","last_peered":"2024-09-10T07:47:27.737966+0000","last_clean":"2024-09-10T07:47:27.737966+0000","last_became_active":"2024-09-10T07:47:26.717075+0000","last_became_peered":"2024-09-10T07:47:26.717075+0000","last_unstale":"2024-09-10T07:47:27.737966+0000","last_undegraded":"2024-09-10T07:47:27.737966+0000","last_fullsized":"2024-09-10T07:47:27.737966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:23.907465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.695001+0000","last_change":"2024-09-10T07:47:26.695001+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.695001+0000","last_undegraded":"2024-09-10T07:47:26.695001+0000","last_fullsized":"2024-09-10T07:47:26.695001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707037+0000","last_change":"2024-09-10T07:47:26.709026+0000","last_active":"2024-09-10T07:47:28.707037+0000","last_peered":"2024-09-10T07:47:28.707037+0000","last_clean":"2024-09-10T07:47:28.707037+0000","last_became_active":"2024-09-10T07:47:26.708550+0000","last_became_peered":"2024-09-10T07:47:26.708550+0000","last_unstale":"2024-09-10T07:47:28.707037+0000","last_undegraded":"2024-09-10T07:47:28.707037+0000","last_fullsized":"2024-09-10T07:47:28.707037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730160+0000","last_change":"2024-09-10T07:47:27.722622+0000","last_active":"2024-09-10T07:47:27.730160+0000","last_peered":"2024-09-10T07:47:27.730160+0000","last_clean":"2024-09-10T07:47:27.730160+0000","last_became_active":"2024-09-10T07:47:27.722182+0000","last_became_peered":"2024-09-10T07:47:27.722182+0000","last_unstale":"2024-09-10T07:47:27.730160+0000","last_undegraded":"2024-09-10T07:47:27.730160+0000","last_fullsized":"2024-09-10T07:47:27.730160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:45.005644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.734519+0000","last_change":"2024-09-10T07:47:26.714730+0000","last_active":"2024-09-10T07:47:27.734519+0000","last_peered":"2024-09-10T07:47:27.734519+0000","last_clean":"2024-09-10T07:47:27.734519+0000","last_became_active":"2024-09-10T07:47:26.713780+0000","last_became_peered":"2024-09-10T07:47:26.713780+0000","last_unstale":"2024-09-10T07:47:27.734519+0000","last_undegraded":"2024-09-10T07:47:27.734519+0000","last_fullsized":"2024-09-10T07:47:27.734519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:59.575014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694052+0000","last_change":"2024-09-10T07:47:26.694052+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694052+0000","last_undegraded":"2024-09-10T07:47:26.694052+0000","last_fullsized":"2024-09-10T07:47:26.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716387+0000","last_change":"2024-09-10T07:47:26.711406+0000","last_active":"2024-09-10T07:47:26.716387+0000","last_peered":"2024-09-10T07:47:26.716387+0000","last_clean":"2024-09-10T07:47:26.716387+0000","last_became_active":"2024-09-10T07:47:26.710814+0000","last_became_peered":"2024-09-10T07:47:26.710814+0000","last_unstale":"2024-09-10T07:47:26.716387+0000","last_undegraded":"2024-09-10T07:47:26.716387+0000","last_fullsized":"2024-09-10T07:47:26.716387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.696890+0000","last_change":"2024-09-10T07:47:26.696890+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.696890+0000","last_undegraded":"2024-09-10T07:47:26.696890+0000","last_fullsized":"2024-09-10T07:47:26.696890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716460+0000","last_change":"2024-09-10T07:47:26.713426+0000","last_active":"2024-09-10T07:47:26.716460+0000","last_peered":"2024-09-10T07:47:26.716460+0000","last_clean":"2024-09-10T07:47:26.716460+0000","last_became_active":"2024-09-10T07:47:26.712028+0000","last_became_peered":"2024-09-10T07:47:26.712028+0000","last_unstale":"2024-09-10T07:47:26.716460+0000","last_undegraded":"2024-09-10T07:47:26.716460+0000","last_fullsized":"2024-09-10T07:47:26.716460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694004+0000","last_change":"2024-09-10T07:47:26.694004+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694004+0000","last_undegraded":"2024-09-10T07:47:26.694004+0000","last_fullsized":"2024-09-10T07:47:26.694004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706730+0000","last_change":"2024-09-10T07:47:26.701393+0000","last_active":"2024-09-10T07:47:28.706730+0000","last_peered":"2024-09-10T07:47:28.706730+0000","last_clean":"2024-09-10T07:47:28.706730+0000","last_became_active":"2024-09-10T07:47:26.701105+0000","last_became_peered":"2024-09-10T07:47:26.701105+0000","last_unstale":"2024-09-10T07:47:28.706730+0000","last_undegraded":"2024-09-10T07:47:28.706730+0000","last_fullsized":"2024-09-10T07:47:28.706730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.714471+0000","last_change":"2024-09-10T07:47:27.708836+0000","last_active":"2024-09-10T07:47:27.714471+0000","last_peered":"2024-09-10T07:47:27.714471+0000","last_clean":"2024-09-10T07:47:27.714471+0000","last_became_active":"2024-09-10T07:47:27.708519+0000","last_became_peered":"2024-09-10T07:47:27.708519+0000","last_unstale":"2024-09-10T07:47:27.714471+0000","last_undegraded":"2024-09-10T07:47:27.714471+0000","last_fullsized":"2024-09-10T07:47:27.714471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:46.304079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716575+0000","last_change":"2024-09-10T07:47:26.713761+0000","last_active":"2024-09-10T07:47:26.716575+0000","last_peered":"2024-09-10T07:47:26.716575+0000","last_clean":"2024-09-10T07:47:26.716575+0000","last_became_active":"2024-09-10T07:47:26.713497+0000","last_became_peered":"2024-09-10T07:47:26.713497+0000","last_unstale":"2024-09-10T07:47:26.716575+0000","last_undegraded":"2024-09-10T07:47:26.716575+0000","last_fullsized":"2024-09-10T07:47:26.716575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:07.278730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699737+0000","last_change":"2024-09-10T07:47:26.699737+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699737+0000","last_undegraded":"2024-09-10T07:47:26.699737+0000","last_fullsized":"2024-09-10T07:47:26.699737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709924+0000","last_change":"2024-09-10T07:47:26.716721+0000","last_active":"2024-09-10T07:47:28.709924+0000","last_peered":"2024-09-10T07:47:28.709924+0000","last_clean":"2024-09-10T07:47:28.709924+0000","last_became_active":"2024-09-10T07:47:26.716429+0000","last_became_peered":"2024-09-10T07:47:26.716429+0000","last_unstale":"2024-09-10T07:47:28.709924+0000","last_undegraded":"2024-09-10T07:47:28.709924+0000","last_fullsized":"2024-09-10T07:47:28.709924+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.729913+0000","last_change":"2024-09-10T07:47:27.721272+0000","last_active":"2024-09-10T07:47:27.729913+0000","last_peered":"2024-09-10T07:47:27.729913+0000","last_clean":"2024-09-10T07:47:27.729913+0000","last_became_active":"2024-09-10T07:47:27.718319+0000","last_became_peered":"2024-09-10T07:47:27.718319+0000","last_unstale":"2024-09-10T07:47:27.729913+0000","last_undegraded":"2024-09-10T07:47:27.729913+0000","last_fullsized":"2024-09-10T07:47:27.729913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:00.885462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709462+0000","last_change":"2024-09-10T07:47:26.713954+0000","last_active":"2024-09-10T07:47:28.709462+0000","last_peered":"2024-09-10T07:47:28.709462+0000","last_clean":"2024-09-10T07:47:28.709462+0000","last_became_active":"2024-09-10T07:47:26.713378+0000","last_became_peered":"2024-09-10T07:47:26.713378+0000","last_unstale":"2024-09-10T07:47:28.709462+0000","last_undegraded":"2024-09-10T07:47:28.709462+0000","last_fullsized":"2024-09-10T07:47:28.709462+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:03.461580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.731659+0000","last_change":"2024-09-10T07:47:27.724717+0000","last_active":"2024-09-10T07:47:27.731659+0000","last_peered":"2024-09-10T07:47:27.731659+0000","last_clean":"2024-09-10T07:47:27.731659+0000","last_became_active":"2024-09-10T07:47:27.724464+0000","last_became_peered":"2024-09-10T07:47:27.724464+0000","last_unstale":"2024-09-10T07:47:27.731659+0000","last_undegraded":"2024-09-10T07:47:27.731659+0000","last_fullsized":"2024-09-10T07:47:27.731659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:41.202808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715090+0000","last_change":"2024-09-10T07:47:26.709280+0000","last_active":"2024-09-10T07:47:26.715090+0000","last_peered":"2024-09-10T07:47:26.715090+0000","last_clean":"2024-09-10T07:47:26.715090+0000","last_became_active":"2024-09-10T07:47:26.708960+0000","last_became_peered":"2024-09-10T07:47:26.708960+0000","last_unstale":"2024-09-10T07:47:26.715090+0000","last_undegraded":"2024-09-10T07:47:26.715090+0000","last_fullsized":"2024-09-10T07:47:26.715090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:40.600902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698390+0000","last_change":"2024-09-10T07:47:26.698390+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698390+0000","last_undegraded":"2024-09-10T07:47:26.698390+0000","last_fullsized":"2024-09-10T07:47:26.698390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272123+0000","last_change":"2024-09-10T07:47:26.713112+0000","last_active":"2024-09-10T07:47:29.272123+0000","last_peered":"2024-09-10T07:47:29.272123+0000","last_clean":"2024-09-10T07:47:29.272123+0000","last_became_active":"2024-09-10T07:47:26.712737+0000","last_became_peered":"2024-09-10T07:47:26.712737+0000","last_unstale":"2024-09-10T07:47:29.272123+0000","last_undegraded":"2024-09-10T07:47:29.272123+0000","last_fullsized":"2024-09-10T07:47:29.272123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271575+0000","last_change":"2024-09-10T07:47:27.722758+0000","last_active":"2024-09-10T07:47:29.271575+0000","last_peered":"2024-09-10T07:47:29.271575+0000","last_clean":"2024-09-10T07:47:29.271575+0000","last_became_active":"2024-09-10T07:47:27.722517+0000","last_became_peered":"2024-09-10T07:47:27.722517+0000","last_unstale":"2024-09-10T07:47:29.271575+0000","last_undegraded":"2024-09-10T07:47:29.271575+0000","last_fullsized":"2024-09-10T07:47:29.271575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:07.008875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716544+0000","last_change":"2024-09-10T07:47:26.713271+0000","last_active":"2024-09-10T07:47:26.716544+0000","last_peered":"2024-09-10T07:47:26.716544+0000","last_clean":"2024-09-10T07:47:26.716544+0000","last_became_active":"2024-09-10T07:47:26.712715+0000","last_became_peered":"2024-09-10T07:47:26.712715+0000","last_unstale":"2024-09-10T07:47:26.716544+0000","last_undegraded":"2024-09-10T07:47:26.716544+0000","last_fullsized":"2024-09-10T07:47:26.716544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730403+0000","last_change":"2024-09-10T07:47:27.725965+0000","last_active":"2024-09-10T07:47:27.730403+0000","last_peered":"2024-09-10T07:47:27.730403+0000","last_clean":"2024-09-10T07:47:27.730403+0000","last_became_active":"2024-09-10T07:47:27.725029+0000","last_became_peered":"2024-09-10T07:47:27.725029+0000","last_unstale":"2024-09-10T07:47:27.730403+0000","last_undegraded":"2024-09-10T07:47:27.730403+0000","last_fullsized":"2024-09-10T07:47:27.730403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:31.812075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699985+0000","last_change":"2024-09-10T07:47:26.711747+0000","last_active":"2024-09-10T07:47:27.699985+0000","last_peered":"2024-09-10T07:47:27.699985+0000","last_clean":"2024-09-10T07:47:27.699985+0000","last_became_active":"2024-09-10T07:47:26.711458+0000","last_became_peered":"2024-09-10T07:47:26.711458+0000","last_unstale":"2024-09-10T07:47:27.699985+0000","last_undegraded":"2024-09-10T07:47:27.699985+0000","last_fullsized":"2024-09-10T07:47:27.699985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:17.886142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694848+0000","last_change":"2024-09-10T07:47:26.694848+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694848+0000","last_undegraded":"2024-09-10T07:47:26.694848+0000","last_fullsized":"2024-09-10T07:47:26.694848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.698363+0000","last_change":"2024-09-10T07:47:26.708509+0000","last_active":"2024-09-10T07:47:27.698363+0000","last_peered":"2024-09-10T07:47:27.698363+0000","last_clean":"2024-09-10T07:47:27.698363+0000","last_became_active":"2024-09-10T07:47:26.708220+0000","last_became_peered":"2024-09-10T07:47:26.708220+0000","last_unstale":"2024-09-10T07:47:27.698363+0000","last_undegraded":"2024-09-10T07:47:27.698363+0000","last_fullsized":"2024-09-10T07:47:27.698363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:55.494668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.700387+0000","last_change":"2024-09-10T07:47:26.700387+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.700387+0000","last_undegraded":"2024-09-10T07:47:26.700387+0000","last_fullsized":"2024-09-10T07:47:26.700387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706491+0000","last_change":"2024-09-10T07:47:26.706290+0000","last_active":"2024-09-10T07:47:28.706491+0000","last_peered":"2024-09-10T07:47:28.706491+0000","last_clean":"2024-09-10T07:47:28.706491+0000","last_became_active":"2024-09-10T07:47:26.705979+0000","last_became_peered":"2024-09-10T07:47:26.705979+0000","last_unstale":"2024-09-10T07:47:28.706491+0000","last_undegraded":"2024-09-10T07:47:28.706491+0000","last_fullsized":"2024-09-10T07:47:28.706491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699283+0000","last_change":"2024-09-10T07:47:26.699283+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699283+0000","last_undegraded":"2024-09-10T07:47:26.699283+0000","last_fullsized":"2024-09-10T07:47:26.699283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706848+0000","last_change":"2024-09-10T07:47:26.710001+0000","last_active":"2024-09-10T07:47:28.706848+0000","last_peered":"2024-09-10T07:47:28.706848+0000","last_clean":"2024-09-10T07:47:28.706848+0000","last_became_active":"2024-09-10T07:47:26.708545+0000","last_became_peered":"2024-09-10T07:47:26.708545+0000","last_unstale":"2024-09-10T07:47:28.706848+0000","last_undegraded":"2024-09-10T07:47:28.706848+0000","last_fullsized":"2024-09-10T07:47:28.706848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.697405+0000","last_change":"2024-09-10T07:47:26.697405+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.697405+0000","last_undegraded":"2024-09-10T07:47:26.697405+0000","last_fullsized":"2024-09-10T07:47:26.697405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716435+0000","last_change":"2024-09-10T07:47:26.713070+0000","last_active":"2024-09-10T07:47:26.716435+0000","last_peered":"2024-09-10T07:47:26.716435+0000","last_clean":"2024-09-10T07:47:26.716435+0000","last_became_active":"2024-09-10T07:47:26.712820+0000","last_became_peered":"2024-09-10T07:47:26.712820+0000","last_unstale":"2024-09-10T07:47:26.716435+0000","last_undegraded":"2024-09-10T07:47:26.716435+0000","last_fullsized":"2024-09-10T07:47:26.716435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709356+0000","last_change":"2024-09-10T07:47:27.712460+0000","last_active":"2024-09-10T07:47:28.709356+0000","last_peered":"2024-09-10T07:47:28.709356+0000","last_clean":"2024-09-10T07:47:28.709356+0000","last_became_active":"2024-09-10T07:47:27.712183+0000","last_became_peered":"2024-09-10T07:47:27.712183+0000","last_unstale":"2024-09-10T07:47:28.709356+0000","last_undegraded":"2024-09-10T07:47:28.709356+0000","last_fullsized":"2024-09-10T07:47:28.709356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716233+0000","last_change":"2024-09-10T07:47:26.713128+0000","last_active":"2024-09-10T07:47:26.716233+0000","last_peered":"2024-09-10T07:47:26.716233+0000","last_clean":"2024-09-10T07:47:26.716233+0000","last_became_active":"2024-09-10T07:47:26.712677+0000","last_became_peered":"2024-09-10T07:47:26.712677+0000","last_unstale":"2024-09-10T07:47:26.716233+0000","last_undegraded":"2024-09-10T07:47:26.716233+0000","last_fullsized":"2024-09-10T07:47:26.716233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:06.290962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706999+0000","last_change":"2024-09-10T07:47:27.724138+0000","last_active":"2024-09-10T07:47:28.706999+0000","last_peered":"2024-09-10T07:47:28.706999+0000","last_clean":"2024-09-10T07:47:28.706999+0000","last_became_active":"2024-09-10T07:47:27.723179+0000","last_became_peered":"2024-09-10T07:47:27.723179+0000","last_unstale":"2024-09-10T07:47:28.706999+0000","last_undegraded":"2024-09-10T07:47:28.706999+0000","last_fullsized":"2024-09-10T07:47:28.706999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716153+0000","last_change":"2024-09-10T07:47:26.711949+0000","last_active":"2024-09-10T07:47:26.716153+0000","last_peered":"2024-09-10T07:47:26.716153+0000","last_clean":"2024-09-10T07:47:26.716153+0000","last_became_active":"2024-09-10T07:47:26.711492+0000","last_became_peered":"2024-09-10T07:47:26.711492+0000","last_unstale":"2024-09-10T07:47:26.716153+0000","last_undegraded":"2024-09-10T07:47:26.716153+0000","last_fullsized":"2024-09-10T07:47:26.716153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:31.026752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271616+0000","last_change":"2024-09-10T07:47:27.725128+0000","last_active":"2024-09-10T07:47:29.271616+0000","last_peered":"2024-09-10T07:47:29.271616+0000","last_clean":"2024-09-10T07:47:29.271616+0000","last_became_active":"2024-09-10T07:47:27.724151+0000","last_became_peered":"2024-09-10T07:47:27.724151+0000","last_unstale":"2024-09-10T07:47:29.271616+0000","last_undegraded":"2024-09-10T07:47:29.271616+0000","last_fullsized":"2024-09-10T07:47:29.271616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:38.688543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709386+0000","last_change":"2024-09-10T07:47:26.715573+0000","last_active":"2024-09-10T07:47:28.709386+0000","last_peered":"2024-09-10T07:47:28.709386+0000","last_clean":"2024-09-10T07:47:28.709386+0000","last_became_active":"2024-09-10T07:47:26.715315+0000","last_became_peered":"2024-09-10T07:47:26.715315+0000","last_unstale":"2024-09-10T07:47:28.709386+0000","last_undegraded":"2024-09-10T07:47:28.709386+0000","last_fullsized":"2024-09-10T07:47:28.709386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:37.683908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709341+0000","last_change":"2024-09-10T07:47:27.701024+0000","last_active":"2024-09-10T07:47:28.709341+0000","last_peered":"2024-09-10T07:47:28.709341+0000","last_clean":"2024-09-10T07:47:28.709341+0000","last_became_active":"2024-09-10T07:47:27.700612+0000","last_became_peered":"2024-09-10T07:47:27.700612+0000","last_unstale":"2024-09-10T07:47:28.709341+0000","last_undegraded":"2024-09-10T07:47:28.709341+0000","last_fullsized":"2024-09-10T07:47:28.709341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.715646+0000","last_change":"2024-09-10T07:47:26.707503+0000","last_active":"2024-09-10T07:47:26.715646+0000","last_peered":"2024-09-10T07:47:26.715646+0000","last_clean":"2024-09-10T07:47:26.715646+0000","last_became_active":"2024-09-10T07:47:26.706814+0000","last_became_peered":"2024-09-10T07:47:26.706814+0000","last_unstale":"2024-09-10T07:47:26.715646+0000","last_undegraded":"2024-09-10T07:47:26.715646+0000","last_fullsized":"2024-09-10T07:47:26.715646+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:10.209621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271600+0000","last_change":"2024-09-10T07:47:27.722893+0000","last_active":"2024-09-10T07:47:29.271600+0000","last_peered":"2024-09-10T07:47:29.271600+0000","last_clean":"2024-09-10T07:47:29.271600+0000","last_became_active":"2024-09-10T07:47:27.722551+0000","last_became_peered":"2024-09-10T07:47:27.722551+0000","last_unstale":"2024-09-10T07:47:29.271600+0000","last_undegraded":"2024-09-10T07:47:29.271600+0000","last_fullsized":"2024-09-10T07:47:29.271600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:52.315032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716562+0000","last_change":"2024-09-10T07:47:26.714230+0000","last_active":"2024-09-10T07:47:26.716562+0000","last_peered":"2024-09-10T07:47:26.716562+0000","last_clean":"2024-09-10T07:47:26.716562+0000","last_became_active":"2024-09-10T07:47:26.712186+0000","last_became_peered":"2024-09-10T07:47:26.712186+0000","last_unstale":"2024-09-10T07:47:26.716562+0000","last_undegraded":"2024-09-10T07:47:26.716562+0000","last_fullsized":"2024-09-10T07:47:26.716562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:55.243965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.694126+0000","last_change":"2024-09-10T07:47:26.694126+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.694126+0000","last_undegraded":"2024-09-10T07:47:26.694126+0000","last_fullsized":"2024-09-10T07:47:26.694126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.712412+0000","last_change":"2024-09-10T07:47:26.709027+0000","last_active":"2024-09-10T07:47:26.712412+0000","last_peered":"2024-09-10T07:47:26.712412+0000","last_clean":"2024-09-10T07:47:26.712412+0000","last_became_active":"2024-09-10T07:47:26.708753+0000","last_became_peered":"2024-09-10T07:47:26.708753+0000","last_unstale":"2024-09-10T07:47:26.712412+0000","last_undegraded":"2024-09-10T07:47:26.712412+0000","last_fullsized":"2024-09-10T07:47:26.712412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:13.182619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707048+0000","last_change":"2024-09-10T07:47:27.728171+0000","last_active":"2024-09-10T07:47:28.707048+0000","last_peered":"2024-09-10T07:47:28.707048+0000","last_clean":"2024-09-10T07:47:28.707048+0000","last_became_active":"2024-09-10T07:47:27.727842+0000","last_became_peered":"2024-09-10T07:47:27.727842+0000","last_unstale":"2024-09-10T07:47:28.707048+0000","last_undegraded":"2024-09-10T07:47:28.707048+0000","last_fullsized":"2024-09-10T07:47:28.707048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:45.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272065+0000","last_change":"2024-09-10T07:47:26.707810+0000","last_active":"2024-09-10T07:47:29.272065+0000","last_peered":"2024-09-10T07:47:29.272065+0000","last_clean":"2024-09-10T07:47:29.272065+0000","last_became_active":"2024-09-10T07:47:26.707462+0000","last_became_peered":"2024-09-10T07:47:26.707462+0000","last_unstale":"2024-09-10T07:47:29.272065+0000","last_undegraded":"2024-09-10T07:47:29.272065+0000","last_fullsized":"2024-09-10T07:47:29.272065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.698308+0000","last_change":"2024-09-10T07:47:26.698308+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.698308+0000","last_undegraded":"2024-09-10T07:47:26.698308+0000","last_fullsized":"2024-09-10T07:47:26.698308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709605+0000","last_change":"2024-09-10T07:47:26.706696+0000","last_active":"2024-09-10T07:47:28.709605+0000","last_peered":"2024-09-10T07:47:28.709605+0000","last_clean":"2024-09-10T07:47:28.709605+0000","last_became_active":"2024-09-10T07:47:26.706383+0000","last_became_peered":"2024-09-10T07:47:26.706383+0000","last_unstale":"2024-09-10T07:47:28.709605+0000","last_undegraded":"2024-09-10T07:47:28.709605+0000","last_fullsized":"2024-09-10T07:47:28.709605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:16.889913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707098+0000","last_change":"2024-09-10T07:47:27.721003+0000","last_active":"2024-09-10T07:47:28.707098+0000","last_peered":"2024-09-10T07:47:28.707098+0000","last_clean":"2024-09-10T07:47:28.707098+0000","last_became_active":"2024-09-10T07:47:27.720702+0000","last_became_peered":"2024-09-10T07:47:27.720702+0000","last_unstale":"2024-09-10T07:47:28.707098+0000","last_undegraded":"2024-09-10T07:47:28.707098+0000","last_fullsized":"2024-09-10T07:47:28.707098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:33.721528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272228+0000","last_change":"2024-09-10T07:47:26.701128+0000","last_active":"2024-09-10T07:47:29.272228+0000","last_peered":"2024-09-10T07:47:29.272228+0000","last_clean":"2024-09-10T07:47:29.272228+0000","last_became_active":"2024-09-10T07:47:26.700042+0000","last_became_peered":"2024-09-10T07:47:26.700042+0000","last_unstale":"2024-09-10T07:47:29.272228+0000","last_undegraded":"2024-09-10T07:47:29.272228+0000","last_fullsized":"2024-09-10T07:47:29.272228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.692995+0000","last_change":"2024-09-10T07:47:26.692995+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.692995+0000","last_undegraded":"2024-09-10T07:47:26.692995+0000","last_fullsized":"2024-09-10T07:47:26.692995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271867+0000","last_change":"2024-09-10T07:47:26.705882+0000","last_active":"2024-09-10T07:47:29.271867+0000","last_peered":"2024-09-10T07:47:29.271867+0000","last_clean":"2024-09-10T07:47:29.271867+0000","last_became_active":"2024-09-10T07:47:26.705616+0000","last_became_peered":"2024-09-10T07:47:26.705616+0000","last_unstale":"2024-09-10T07:47:29.271867+0000","last_undegraded":"2024-09-10T07:47:29.271867+0000","last_fullsized":"2024-09-10T07:47:29.271867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.728015+0000","last_change":"2024-09-10T07:47:27.719286+0000","last_active":"2024-09-10T07:47:27.728015+0000","last_peered":"2024-09-10T07:47:27.728015+0000","last_clean":"2024-09-10T07:47:27.728015+0000","last_became_active":"2024-09-10T07:47:27.715311+0000","last_became_peered":"2024-09-10T07:47:27.715311+0000","last_unstale":"2024-09-10T07:47:27.728015+0000","last_undegraded":"2024-09-10T07:47:27.728015+0000","last_fullsized":"2024-09-10T07:47:27.728015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:26:17.625922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271645+0000","last_change":"2024-09-10T07:47:26.710717+0000","last_active":"2024-09-10T07:47:29.271645+0000","last_peered":"2024-09-10T07:47:29.271645+0000","last_clean":"2024-09-10T07:47:29.271645+0000","last_became_active":"2024-09-10T07:47:26.710392+0000","last_became_peered":"2024-09-10T07:47:26.710392+0000","last_unstale":"2024-09-10T07:47:29.271645+0000","last_undegraded":"2024-09-10T07:47:29.271645+0000","last_fullsized":"2024-09-10T07:47:29.271645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.730509+0000","last_change":"2024-09-10T07:47:27.727229+0000","last_active":"2024-09-10T07:47:27.730509+0000","last_peered":"2024-09-10T07:47:27.730509+0000","last_clean":"2024-09-10T07:47:27.730509+0000","last_became_active":"2024-09-10T07:47:27.725550+0000","last_became_peered":"2024-09-10T07:47:27.725550+0000","last_unstale":"2024-09-10T07:47:27.730509+0000","last_undegraded":"2024-09-10T07:47:27.730509+0000","last_fullsized":"2024-09-10T07:47:27.730509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:02.762995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.716358+0000","last_change":"2024-09-10T07:47:26.710777+0000","last_active":"2024-09-10T07:47:26.716358+0000","last_peered":"2024-09-10T07:47:26.716358+0000","last_clean":"2024-09-10T07:47:26.716358+0000","last_became_active":"2024-09-10T07:47:26.710499+0000","last_became_peered":"2024-09-10T07:47:26.710499+0000","last_unstale":"2024-09-10T07:47:26.716358+0000","last_undegraded":"2024-09-10T07:47:26.716358+0000","last_fullsized":"2024-09-10T07:47:26.716358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:25.822800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.699907+0000","last_change":"2024-09-10T07:47:26.699907+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.699907+0000","last_undegraded":"2024-09-10T07:47:26.699907+0000","last_fullsized":"2024-09-10T07:47:26.699907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-10T07:47:27.699089+0000","last_change":"2024-09-10T07:47:26.716992+0000","last_active":"2024-09-10T07:47:27.699089+0000","last_peered":"2024-09-10T07:47:27.699089+0000","last_clean":"2024-09-10T07:47:27.699089+0000","last_became_active":"2024-09-10T07:47:26.716690+0000","last_became_peered":"2024-09-10T07:47:26.716690+0000","last_unstale":"2024-09-10T07:47:27.699089+0000","last_undegraded":"2024-09-10T07:47:27.699089+0000","last_fullsized":"2024-09-10T07:47:27.699089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.180750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-10T07:47:26.693017+0000","last_change":"2024-09-10T07:47:26.693017+0000","last_active":"2024-09-10T07:47:26.685267+0000","last_peered":"2024-09-10T07:47:26.685267+0000","last_clean":"2024-09-10T07:47:26.685267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:47:26.693017+0000","last_undegraded":"2024-09-10T07:47:26.693017+0000","last_fullsized":"2024-09-10T07:47:26.693017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.710693+0000","last_change":"2024-09-10T07:47:26.707932+0000","last_active":"2024-09-10T07:47:26.710693+0000","last_peered":"2024-09-10T07:47:26.710693+0000","last_clean":"2024-09-10T07:47:26.710693+0000","last_became_active":"2024-09-10T07:47:26.707401+0000","last_became_peered":"2024-09-10T07:47:26.707401+0000","last_unstale":"2024-09-10T07:47:26.710693+0000","last_undegraded":"2024-09-10T07:47:26.710693+0000","last_fullsized":"2024-09-10T07:47:26.710693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.677752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T07:47:26.697935+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:47:26.697935+0000","last_peered":"2024-09-10T07:47:26.697935+0000","last_clean":"2024-09-10T07:47:26.697935+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:47:26.697935+0000","last_undegraded":"2024-09-10T07:47:26.697935+0000","last_fullsized":"2024-09-10T07:47:26.697935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":725755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":135387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":154759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":156349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":745687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":372736,"data_stored":141662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.702272+0000', 'last_change': '2024-09-10T07:47:26.702272+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.702272+0000', 'last_undegraded': '2024-09-10T07:47:26.702272+0000', 'last_fullsized': '2024-09-10T07:47:26.702272+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.695776+0000', 'last_change': '2024-09-10T07:47:26.695776+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.695776+0000', 'last_undegraded': '2024-09-10T07:47:26.695776+0000', 'last_fullsized': '2024-09-10T07:47:26.695776+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697623+0000', 'last_change': '2024-09-10T07:47:26.697623+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697623+0000', 'last_undegraded': '2024-09-10T07:47:26.697623+0000', 'last_fullsized': '2024-09-10T07:47:26.697623+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-10T07:47:32.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.699517+0000', 'last_change': '2024-09-10T07:47:26.699517+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.699517+0000', 'last_undegraded': '2024-09-10T07:47:26.699517+0000', 'last_fullsized': '2024-09-10T07:47:26.699517+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697854+0000', 'last_change': '2024-09-10T07:47:26.697854+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697854+0000', 'last_undegraded': '2024-09-10T07:47:26.697854+0000', 'last_fullsized': '2024-09-10T07:47:26.697854+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697313+0000', 'last_change': '2024-09-10T07:47:26.697313+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697313+0000', 'last_undegraded': '2024-09-10T07:47:26.697313+0000', 'last_fullsized': '2024-09-10T07:47:26.697313+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.696911+0000', 'last_change': '2024-09-10T07:47:26.696911+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.696911+0000', 'last_undegraded': '2024-09-10T07:47:26.696911+0000', 'last_fullsized': '2024-09-10T07:47:26.696911+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.696378+0000', 'last_change': '2024-09-10T07:47:26.696378+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.696378+0000', 'last_undegraded': '2024-09-10T07:47:26.696378+0000', 'last_fullsized': '2024-09-10T07:47:26.696378+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697490+0000', 'last_change': '2024-09-10T07:47:26.697490+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697490+0000', 'last_undegraded': '2024-09-10T07:47:26.697490+0000', 'last_fullsized': '2024-09-10T07:47:26.697490+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-10T07:47:32.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.698719+0000', 'last_change': '2024-09-10T07:47:26.698719+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.698719+0000', 'last_undegraded': '2024-09-10T07:47:26.698719+0000', 'last_fullsized': '2024-09-10T07:47:26.698719+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.695528+0000', 'last_change': '2024-09-10T07:47:26.695528+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.695528+0000', 'last_undegraded': '2024-09-10T07:47:26.695528+0000', 'last_fullsized': '2024-09-10T07:47:26.695528+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.701477+0000', 'last_change': '2024-09-10T07:47:26.701477+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.701477+0000', 'last_undegraded': '2024-09-10T07:47:26.701477+0000', 'last_fullsized': '2024-09-10T07:47:26.701477+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.699069+0000', 'last_change': '2024-09-10T07:47:26.699069+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.699069+0000', 'last_undegraded': '2024-09-10T07:47:26.699069+0000', 'last_fullsized': '2024-09-10T07:47:26.699069+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.700878+0000', 'last_change': '2024-09-10T07:47:26.700878+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.700878+0000', 'last_undegraded': '2024-09-10T07:47:26.700878+0000', 'last_fullsized': '2024-09-10T07:47:26.700878+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-10T07:47:32.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694553+0000', 'last_change': '2024-09-10T07:47:26.694553+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694553+0000', 'last_undegraded': '2024-09-10T07:47:26.694553+0000', 'last_fullsized': '2024-09-10T07:47:26.694553+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694437+0000', 'last_change': '2024-09-10T07:47:26.694437+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694437+0000', 'last_undegraded': '2024-09-10T07:47:26.694437+0000', 'last_fullsized': '2024-09-10T07:47:26.694437+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.695915+0000', 'last_change': '2024-09-10T07:47:26.695915+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.695915+0000', 'last_undegraded': '2024-09-10T07:47:26.695915+0000', 'last_fullsized': '2024-09-10T07:47:26.695915+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697245+0000', 'last_change': '2024-09-10T07:47:26.697245+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697245+0000', 'last_undegraded': '2024-09-10T07:47:26.697245+0000', 'last_fullsized': '2024-09-10T07:47:26.697245+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.695191+0000', 'last_change': '2024-09-10T07:47:26.695191+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.695191+0000', 'last_undegraded': '2024-09-10T07:47:26.695191+0000', 'last_fullsized': '2024-09-10T07:47:26.695191+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.695001+0000', 'last_change': '2024-09-10T07:47:26.695001+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.695001+0000', 'last_undegraded': '2024-09-10T07:47:26.695001+0000', 'last_fullsized': '2024-09-10T07:47:26.695001+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.772 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694052+0000', 'last_change': '2024-09-10T07:47:26.694052+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694052+0000', 'last_undegraded': '2024-09-10T07:47:26.694052+0000', 'last_fullsized': '2024-09-10T07:47:26.694052+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.696890+0000', 'last_change': '2024-09-10T07:47:26.696890+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.696890+0000', 'last_undegraded': '2024-09-10T07:47:26.696890+0000', 'last_fullsized': '2024-09-10T07:47:26.696890+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694004+0000', 'last_change': '2024-09-10T07:47:26.694004+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694004+0000', 'last_undegraded': '2024-09-10T07:47:26.694004+0000', 'last_fullsized': '2024-09-10T07:47:26.694004+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.699737+0000', 'last_change': '2024-09-10T07:47:26.699737+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.699737+0000', 'last_undegraded': '2024-09-10T07:47:26.699737+0000', 'last_fullsized': '2024-09-10T07:47:26.699737+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.698390+0000', 'last_change': '2024-09-10T07:47:26.698390+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.698390+0000', 'last_undegraded': '2024-09-10T07:47:26.698390+0000', 'last_fullsized': '2024-09-10T07:47:26.698390+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-10T07:47:32.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694848+0000', 'last_change': '2024-09-10T07:47:26.694848+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694848+0000', 'last_undegraded': '2024-09-10T07:47:26.694848+0000', 'last_fullsized': '2024-09-10T07:47:26.694848+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.700387+0000', 'last_change': '2024-09-10T07:47:26.700387+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.700387+0000', 'last_undegraded': '2024-09-10T07:47:26.700387+0000', 'last_fullsized': '2024-09-10T07:47:26.700387+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.699283+0000', 'last_change': '2024-09-10T07:47:26.699283+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.699283+0000', 'last_undegraded': '2024-09-10T07:47:26.699283+0000', 'last_fullsized': '2024-09-10T07:47:26.699283+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.697405+0000', 'last_change': '2024-09-10T07:47:26.697405+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.697405+0000', 'last_undegraded': '2024-09-10T07:47:26.697405+0000', 'last_fullsized': '2024-09-10T07:47:26.697405+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.694126+0000', 'last_change': '2024-09-10T07:47:26.694126+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.694126+0000', 'last_undegraded': '2024-09-10T07:47:26.694126+0000', 'last_fullsized': '2024-09-10T07:47:26.694126+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.698308+0000', 'last_change': '2024-09-10T07:47:26.698308+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.698308+0000', 'last_undegraded': '2024-09-10T07:47:26.698308+0000', 'last_fullsized': '2024-09-10T07:47:26.698308+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.774 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-10T07:47:32.775 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.692995+0000', 'last_change': '2024-09-10T07:47:26.692995+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.692995+0000', 'last_undegraded': '2024-09-10T07:47:26.692995+0000', 'last_fullsized': '2024-09-10T07:47:26.692995+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.775 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-10T07:47:32.775 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.699907+0000', 'last_change': '2024-09-10T07:47:26.699907+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.699907+0000', 'last_undegraded': '2024-09-10T07:47:26.699907+0000', 'last_fullsized': '2024-09-10T07:47:26.699907+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T07:47:32.775 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-10T07:47:32.775 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:47:26.693017+0000', 'last_change': '2024-09-10T07:47:26.693017+0000', 'last_active': '2024-09-10T07:47:26.685267+0000', 'last_peered': '2024-09-10T07:47:26.685267+0000', 'last_clean': '2024-09-10T07:47:26.685267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:47:26.693017+0000', 'last_undegraded': '2024-09-10T07:47:26.693017+0000', 'last_fullsized': '2024-09-10T07:47:26.693017+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'last_clean_scrub_stamp': '2024-09-10T07:47:26.685267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T07:47:32.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:47:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.964+0000 7f95dee00640 1 Processor -- start 2024-09-10T07:47:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.964+0000 7f95dee00640 1 -- start start 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dee00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95d806bb20 0x7f95d806bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d815e800 0x7f95d815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dee00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d80c9800 con 0x7f95d815e420 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dee00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f95d806c4a0 con 0x7f95d806bb20 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f95d80d3300 con 0x7f95d815e800 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dd000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95d806bb20 0x7f95d806bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dd000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95d806bb20 0x7f95d806bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36076/0 (socket says 172.21.15.3:36076) 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95dd000640 1 -- 172.21.15.3:0/3453312547 learned_addr learned my addr 172.21.15.3:0/3453312547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.965+0000 7f95de400640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d815e800 0x7f95d815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.966+0000 7f95dd000640 1 -- 172.21.15.3:0/3453312547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d815e800 msgr2=0x7f95d815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.966+0000 7f95dd000640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d815e800 0x7f95d815ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.966+0000 7f95dd000640 1 -- 172.21.15.3:0/3453312547 >> v1:172.21.15.70:6789/0 conn(0x7f95d815e420 legacy=0x7f95d80d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.966+0000 7f95dd000640 1 -- 172.21.15.3:0/3453312547 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d80d35e0 con 0x7f95d806bb20 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95dd000640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d806bb20 0x7f95d806bf60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009870 tx=0x7f95c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f380592f10da03b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95d7e00640 1 -- 172.21.15.3:0/3453312547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037070 con 0x7f95d806bb20 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95d7e00640 1 -- 172.21.15.3:0/3453312547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4032440 con 0x7f95d806bb20 2024-09-10T07:47:32.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95d7e00640 1 -- 172.21.15.3:0/3453312547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4032810 con 0x7f95d806bb20 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95dee00640 1 -- 172.21.15.3:0/3453312547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d806bb20 msgr2=0x7f95d806bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.967+0000 7f95dee00640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d806bb20 0x7f95d806bf60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009870 tx=0x7f95c402a270 comp rx=0 tx=0).stop 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 -- 172.21.15.3:0/3453312547 shutdown_connections 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d815e800 0x7f95d815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 --2- 172.21.15.3:0/3453312547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d806bb20 0x7f95d806bf60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 -- 172.21.15.3:0/3453312547 >> 172.21.15.3:0/3453312547 conn(0x7f95d80732e0 msgr2=0x7f95d8062630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 -- 172.21.15.3:0/3453312547 shutdown_connections 2024-09-10T07:47:32.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.968+0000 7f95dee00640 1 -- 172.21.15.3:0/3453312547 wait complete. 2024-09-10T07:47:32.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.969+0000 7f95dee00640 1 Processor -- start 2024-09-10T07:47:32.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 -- start start 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d806bb20 0x7f95d80b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d815e800 0x7f95d8152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95d80d3e80 con 0x7f95d815e420 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f95d80d3ba0 con 0x7f95d815e800 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.970+0000 7f95dee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f95d80d2ea0 con 0x7f95d806bb20 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36090/0 (socket says 172.21.15.3:36090) 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 -- 172.21.15.3:0/3064133979 learned_addr learned my addr 172.21.15.3:0/3064133979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:32.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dda00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d806bb20 0x7f95d80b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95de400640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d815e800 0x7f95d8152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 -- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d806bb20 msgr2=0x7f95d80b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d806bb20 0x7f95d80b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 -- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d815e800 msgr2=0x7f95d8152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d815e800 0x7f95d8152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:32.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.971+0000 7f95dd000640 1 -- 172.21.15.3:0/3064133979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81f58e0 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95dd000640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4041020 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c40306c0 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037040 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81f82d0 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.972+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81f8b10 con 0x7f95d815e420 2024-09-10T07:47:32.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.973+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d806bf60 con 0x7f95d815e420 2024-09-10T07:47:32.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.974+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95c4044020 con 0x7f95d815e420 2024-09-10T07:47:32.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.975+0000 7f95d5600640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 0x7f95b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:32.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.976+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f95c40ab580 con 0x7f95d815e420 2024-09-10T07:47:32.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.976+0000 7f95dda00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 0x7f95b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:32.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.976+0000 7f95dda00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 0x7f95b806e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95cc000f60 tx=0x7f95cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:32.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:32.980+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95c404b030 con 0x7f95d815e420 2024-09-10T07:47:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.228+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f95d80ca6f0 con 0x7f95d815e420 2024-09-10T07:47:33.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.229+0000 7f95d5600640 1 -- 172.21.15.3:0/3064133979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1686 (secure 0 0 0) 0x7f95d81f82d0 con 0x7f95d815e420 2024-09-10T07:47:33.228 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:33.228 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":63},{"state_name":"creating+peering","count":34}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592002,"bytes_used":242683904,"bytes_avail":772851429376,"bytes_total":773094113280,"inactive_pgs_ratio":0.3505154550075531,"write_bytes_sec":704,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":17,"btime":"2024-09-10T07:47:30:712515+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4741}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T07:47:23.500467+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 msgr2=0x7f95b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 0x7f95b806e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95cc000f60 tx=0x7f95cc009040 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 msgr2=0x7f95d80d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c40046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 shutdown_connections 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95b806be30 0x7f95b806e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d815e800 0x7f95d8152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95d815e420 0x7f95d80d15c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 --2- 172.21.15.3:0/3064133979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f95d806bb20 0x7f95d80b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 >> 172.21.15.3:0/3064133979 conn(0x7f95d80732e0 msgr2=0x7f95d80d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 shutdown_connections 2024-09-10T07:47:33.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.231+0000 7f95dee00640 1 -- 172.21.15.3:0/3064133979 wait complete. 2024-09-10T07:47:33.246 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:47:33.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:47:33.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.402+0000 7fa4c3e00640 1 Processor -- start 2024-09-10T07:47:33.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.402+0000 7fa4c3e00640 1 -- start start 2024-09-10T07:47:33.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.403+0000 7fa4c3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4c415e860 0x7fa4c40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.403+0000 7fa4c3e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c40cee40 con 0x7fa4c4073150 2024-09-10T07:47:33.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.403+0000 7fa4c3e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c40cf120 con 0x7fa4c40cfbf0 2024-09-10T07:47:33.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.403+0000 7fa4c3e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c40d3780 con 0x7fa4c415e860 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa4c415e860 0x7fa4c40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c3400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa4c4073150 0x7fa4c40b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39194/0 (socket says 172.21.15.3:39194) 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c3400640 1 -- 172.21.15.3:0/1846391008 learned_addr learned my addr 172.21.15.3:0/1846391008 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c2a00640 1 -- 172.21.15.3:0/1846391008 >> v1:172.21.15.70:6789/0 conn(0x7fa4c40cfbf0 legacy=0x7fa4c40d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c2a00640 1 -- 172.21.15.3:0/1846391008 >> v1:172.21.15.3:6789/0 conn(0x7fa4c4073150 legacy=0x7fa4c40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c2a00640 1 -- 172.21.15.3:0/1846391008 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40d3410 con 0x7fa4c415e860 2024-09-10T07:47:33.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.404+0000 7fa4c1600640 1 -- 172.21.15.3:0/1846391008 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4205057981 0 0) 0x7fa4c40cee40 con 0x7fa4c4073150 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.405+0000 7fa4c2a00640 1 --2- 172.21.15.3:0/1846391008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c415e860 0x7fa4c40cf6b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40097a0 tx=0x7fa4b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618506c4c2ec2e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.405+0000 7fa4c1600640 1 -- 172.21.15.3:0/1846391008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4037070 con 0x7fa4c415e860 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.405+0000 7fa4c1600640 1 -- 172.21.15.3:0/1846391008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b4032440 con 0x7fa4c415e860 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.405+0000 7fa4c1600640 1 -- 172.21.15.3:0/1846391008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032810 con 0x7fa4c415e860 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 -- 172.21.15.3:0/1846391008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c415e860 msgr2=0x7fa4c40cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/1846391008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c415e860 0x7fa4c40cf6b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40097a0 tx=0x7fa4b402a230 comp rx=0 tx=0).stop 2024-09-10T07:47:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 -- 172.21.15.3:0/1846391008 shutdown_connections 2024-09-10T07:47:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/1846391008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c415e860 0x7fa4c40cf6b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 -- 172.21.15.3:0/1846391008 >> 172.21.15.3:0/1846391008 conn(0x7fa4c40d4650 msgr2=0x7fa4c40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 -- 172.21.15.3:0/1846391008 shutdown_connections 2024-09-10T07:47:33.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.406+0000 7fa4c3e00640 1 -- 172.21.15.3:0/1846391008 wait complete. 2024-09-10T07:47:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.408+0000 7fa4c3e00640 1 Processor -- start 2024-09-10T07:47:33.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.408+0000 7fa4c3e00640 1 -- start start 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c40cfbf0 0x7fa4c41f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4c415e860 0x7fa4c40ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40d2460 con 0x7fa4c4073150 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40cf400 con 0x7fa4c40cfbf0 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c40b7f90 con 0x7fa4c415e860 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36104/0 (socket says 172.21.15.3:36104) 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 -- 172.21.15.3:0/2118796562 learned_addr learned my addr 172.21.15.3:0/2118796562 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2000640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c40cfbf0 0x7fa4c41f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c3400640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4c415e860 0x7fa4c40ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 -- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4c415e860 msgr2=0x7fa4c40ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4c415e860 0x7fa4c40ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 -- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c40cfbf0 msgr2=0x7fa4c41f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c40cfbf0 0x7fa4c41f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.409+0000 7fa4c2a00640 1 -- 172.21.15.3:0/2118796562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40cb850 con 0x7fa4c4073150 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4c2a00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4035040 tx=0x7fa4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4037040 con 0x7fa4c4073150 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b4030770 con 0x7fa4c4073150 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4030b80 con 0x7fa4c4073150 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c40cbb30 con 0x7fa4c4073150 2024-09-10T07:47:33.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.410+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c40cc0f0 con 0x7fa4c4073150 2024-09-10T07:47:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.411+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c40cf6b0 con 0x7fa4c4073150 2024-09-10T07:47:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.412+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4b4044020 con 0x7fa4c4073150 2024-09-10T07:47:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.413+0000 7fa4baa00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 0x7fa4a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:33.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.413+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b40ab450 con 0x7fa4c4073150 2024-09-10T07:47:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.413+0000 7fa4c2000640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 0x7fa4a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:33.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.414+0000 7fa4c2000640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 0x7fa4a406e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4c41f8760 tx=0x7fa4b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:33.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.417+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4b4032e20 con 0x7fa4c4073150 2024-09-10T07:47:33.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.601+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4c40b5720 con 0x7fa4a406be60 2024-09-10T07:47:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.607+0000 7fa4baa00640 1 -- 172.21.15.3:0/2118796562 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7fa4c40b5720 con 0x7fa4a406be60 2024-09-10T07:47:33.608 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:33.611 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.611+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 msgr2=0x7fa4a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.611+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 0x7fa4a406e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4c41f8760 tx=0x7fa4b0009040 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.611+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 msgr2=0x7fa4c41f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.611+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4035040 tx=0x7fa4b40046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 shutdown_connections 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa4a406be60 0x7fa4a406e320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4c415e860 0x7fa4c40ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c40cfbf0 0x7fa4c41f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 --2- 172.21.15.3:0/2118796562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4c4073150 0x7fa4c41f6db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 >> 172.21.15.3:0/2118796562 conn(0x7fa4c40d4650 msgr2=0x7fa4c40d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 shutdown_connections 2024-09-10T07:47:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:33.612+0000 7fa4c3e00640 1 -- 172.21.15.3:0/2118796562 wait complete. 2024-09-10T07:47:33.627 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-10T07:47:33.502347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237208,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737512,"statfs":{"total":773094113280,"available":772851212288,"internally_reserved":0,"allocated":4431872,"data_stored":2423452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.813523"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706604+0000","last_change":"2024-09-10T07:47:27.729428+0000","last_active":"2024-09-10T07:47:28.706604+0000","last_peered":"2024-09-10T07:47:28.706604+0000","last_clean":"2024-09-10T07:47:28.706604+0000","last_became_active":"2024-09-10T07:47:27.728866+0000","last_became_peered":"2024-09-10T07:47:27.728866+0000","last_unstale":"2024-09-10T07:47:28.706604+0000","last_undegraded":"2024-09-10T07:47:28.706604+0000","last_fullsized":"2024-09-10T07:47:28.706604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709608+0000","last_change":"2024-09-10T07:47:27.717295+0000","last_active":"2024-09-10T07:47:28.709608+0000","last_peered":"2024-09-10T07:47:28.709608+0000","last_clean":"2024-09-10T07:47:28.709608+0000","last_became_active":"2024-09-10T07:47:27.716979+0000","last_became_peered":"2024-09-10T07:47:27.716979+0000","last_unstale":"2024-09-10T07:47:28.709608+0000","last_undegraded":"2024-09-10T07:47:28.709608+0000","last_fullsized":"2024-09-10T07:47:28.709608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271803+0000","last_change":"2024-09-10T07:47:27.721625+0000","last_active":"2024-09-10T07:47:29.271803+0000","last_peered":"2024-09-10T07:47:29.271803+0000","last_clean":"2024-09-10T07:47:29.271803+0000","last_became_active":"2024-09-10T07:47:27.720961+0000","last_became_peered":"2024-09-10T07:47:27.720961+0000","last_unstale":"2024-09-10T07:47:29.271803+0000","last_undegraded":"2024-09-10T07:47:29.271803+0000","last_fullsized":"2024-09-10T07:47:29.271803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272644+0000","last_change":"2024-09-10T07:47:27.726560+0000","last_active":"2024-09-10T07:47:29.272644+0000","last_peered":"2024-09-10T07:47:29.272644+0000","last_clean":"2024-09-10T07:47:29.272644+0000","last_became_active":"2024-09-10T07:47:27.726024+0000","last_became_peered":"2024-09-10T07:47:27.726024+0000","last_unstale":"2024-09-10T07:47:29.272644+0000","last_undegraded":"2024-09-10T07:47:29.272644+0000","last_fullsized":"2024-09-10T07:47:29.272644+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:21.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709319+0000","last_change":"2024-09-10T07:47:27.710914+0000","last_active":"2024-09-10T07:47:28.709319+0000","last_peered":"2024-09-10T07:47:28.709319+0000","last_clean":"2024-09-10T07:47:28.709319+0000","last_became_active":"2024-09-10T07:47:27.710617+0000","last_became_peered":"2024-09-10T07:47:27.710617+0000","last_unstale":"2024-09-10T07:47:28.709319+0000","last_undegraded":"2024-09-10T07:47:28.709319+0000","last_fullsized":"2024-09-10T07:47:28.709319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706883+0000","last_change":"2024-09-10T07:47:27.721104+0000","last_active":"2024-09-10T07:47:28.706883+0000","last_peered":"2024-09-10T07:47:28.706883+0000","last_clean":"2024-09-10T07:47:28.706883+0000","last_became_active":"2024-09-10T07:47:27.718624+0000","last_became_peered":"2024-09-10T07:47:27.718624+0000","last_unstale":"2024-09-10T07:47:28.706883+0000","last_undegraded":"2024-09-10T07:47:28.706883+0000","last_fullsized":"2024-09-10T07:47:28.706883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705676+0000","last_change":"2024-09-10T07:47:27.725374+0000","last_active":"2024-09-10T07:47:28.705676+0000","last_peered":"2024-09-10T07:47:28.705676+0000","last_clean":"2024-09-10T07:47:28.705676+0000","last_became_active":"2024-09-10T07:47:27.724325+0000","last_became_peered":"2024-09-10T07:47:27.724325+0000","last_unstale":"2024-09-10T07:47:28.705676+0000","last_undegraded":"2024-09-10T07:47:28.705676+0000","last_fullsized":"2024-09-10T07:47:28.705676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705533+0000","last_change":"2024-09-10T07:47:27.714209+0000","last_active":"2024-09-10T07:47:28.705533+0000","last_peered":"2024-09-10T07:47:28.705533+0000","last_clean":"2024-09-10T07:47:28.705533+0000","last_became_active":"2024-09-10T07:47:27.713549+0000","last_became_peered":"2024-09-10T07:47:27.713549+0000","last_unstale":"2024-09-10T07:47:28.705533+0000","last_undegraded":"2024-09-10T07:47:28.705533+0000","last_fullsized":"2024-09-10T07:47:28.705533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:34.349282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271791+0000","last_change":"2024-09-10T07:47:27.727604+0000","last_active":"2024-09-10T07:47:29.271791+0000","last_peered":"2024-09-10T07:47:29.271791+0000","last_clean":"2024-09-10T07:47:29.271791+0000","last_became_active":"2024-09-10T07:47:27.727343+0000","last_became_peered":"2024-09-10T07:47:27.727343+0000","last_unstale":"2024-09-10T07:47:29.271791+0000","last_undegraded":"2024-09-10T07:47:29.271791+0000","last_fullsized":"2024-09-10T07:47:29.271791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272974+0000","last_change":"2024-09-10T07:47:27.717546+0000","last_active":"2024-09-10T07:47:29.272974+0000","last_peered":"2024-09-10T07:47:29.272974+0000","last_clean":"2024-09-10T07:47:29.272974+0000","last_became_active":"2024-09-10T07:47:27.713266+0000","last_became_peered":"2024-09-10T07:47:27.713266+0000","last_unstale":"2024-09-10T07:47:29.272974+0000","last_undegraded":"2024-09-10T07:47:29.272974+0000","last_fullsized":"2024-09-10T07:47:29.272974+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:36.714001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707274+0000","last_change":"2024-09-10T07:47:27.712020+0000","last_active":"2024-09-10T07:47:28.707274+0000","last_peered":"2024-09-10T07:47:28.707274+0000","last_clean":"2024-09-10T07:47:28.707274+0000","last_became_active":"2024-09-10T07:47:27.711136+0000","last_became_peered":"2024-09-10T07:47:27.711136+0000","last_unstale":"2024-09-10T07:47:28.707274+0000","last_undegraded":"2024-09-10T07:47:28.707274+0000","last_fullsized":"2024-09-10T07:47:28.707274+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:34.125368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271597+0000","last_change":"2024-09-10T07:47:27.712313+0000","last_active":"2024-09-10T07:47:29.271597+0000","last_peered":"2024-09-10T07:47:29.271597+0000","last_clean":"2024-09-10T07:47:29.271597+0000","last_became_active":"2024-09-10T07:47:27.712045+0000","last_became_peered":"2024-09-10T07:47:27.712045+0000","last_unstale":"2024-09-10T07:47:29.271597+0000","last_undegraded":"2024-09-10T07:47:29.271597+0000","last_fullsized":"2024-09-10T07:47:29.271597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:45.223540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271853+0000","last_change":"2024-09-10T07:47:27.728693+0000","last_active":"2024-09-10T07:47:29.271853+0000","last_peered":"2024-09-10T07:47:29.271853+0000","last_clean":"2024-09-10T07:47:29.271853+0000","last_became_active":"2024-09-10T07:47:27.728426+0000","last_became_peered":"2024-09-10T07:47:27.728426+0000","last_unstale":"2024-09-10T07:47:29.271853+0000","last_undegraded":"2024-09-10T07:47:29.271853+0000","last_fullsized":"2024-09-10T07:47:29.271853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.466215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272359+0000","last_change":"2024-09-10T07:47:27.713960+0000","last_active":"2024-09-10T07:47:29.272359+0000","last_peered":"2024-09-10T07:47:29.272359+0000","last_clean":"2024-09-10T07:47:29.272359+0000","last_became_active":"2024-09-10T07:47:27.713692+0000","last_became_peered":"2024-09-10T07:47:27.713692+0000","last_unstale":"2024-09-10T07:47:29.272359+0000","last_undegraded":"2024-09-10T07:47:29.272359+0000","last_fullsized":"2024-09-10T07:47:29.272359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:28.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271162+0000","last_change":"2024-09-10T07:47:27.720587+0000","last_active":"2024-09-10T07:47:29.271162+0000","last_peered":"2024-09-10T07:47:29.271162+0000","last_clean":"2024-09-10T07:47:29.271162+0000","last_became_active":"2024-09-10T07:47:27.720323+0000","last_became_peered":"2024-09-10T07:47:27.720323+0000","last_unstale":"2024-09-10T07:47:29.271162+0000","last_undegraded":"2024-09-10T07:47:29.271162+0000","last_fullsized":"2024-09-10T07:47:29.271162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271457+0000","last_change":"2024-09-10T07:47:27.710943+0000","last_active":"2024-09-10T07:47:29.271457+0000","last_peered":"2024-09-10T07:47:29.271457+0000","last_clean":"2024-09-10T07:47:29.271457+0000","last_became_active":"2024-09-10T07:47:27.710633+0000","last_became_peered":"2024-09-10T07:47:27.710633+0000","last_unstale":"2024-09-10T07:47:29.271457+0000","last_undegraded":"2024-09-10T07:47:29.271457+0000","last_fullsized":"2024-09-10T07:47:29.271457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707339+0000","last_change":"2024-09-10T07:47:27.708746+0000","last_active":"2024-09-10T07:47:28.707339+0000","last_peered":"2024-09-10T07:47:28.707339+0000","last_clean":"2024-09-10T07:47:28.707339+0000","last_became_active":"2024-09-10T07:47:27.708477+0000","last_became_peered":"2024-09-10T07:47:27.708477+0000","last_unstale":"2024-09-10T07:47:28.707339+0000","last_undegraded":"2024-09-10T07:47:28.707339+0000","last_fullsized":"2024-09-10T07:47:28.707339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:11.614613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707015+0000","last_change":"2024-09-10T07:47:27.717429+0000","last_active":"2024-09-10T07:47:28.707015+0000","last_peered":"2024-09-10T07:47:28.707015+0000","last_clean":"2024-09-10T07:47:28.707015+0000","last_became_active":"2024-09-10T07:47:27.717064+0000","last_became_peered":"2024-09-10T07:47:27.717064+0000","last_unstale":"2024-09-10T07:47:28.707015+0000","last_undegraded":"2024-09-10T07:47:28.707015+0000","last_fullsized":"2024-09-10T07:47:28.707015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:44.926594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706888+0000","last_change":"2024-09-10T07:47:27.715748+0000","last_active":"2024-09-10T07:47:28.706888+0000","last_peered":"2024-09-10T07:47:28.706888+0000","last_clean":"2024-09-10T07:47:28.706888+0000","last_became_active":"2024-09-10T07:47:27.715354+0000","last_became_peered":"2024-09-10T07:47:27.715354+0000","last_unstale":"2024-09-10T07:47:28.706888+0000","last_undegraded":"2024-09-10T07:47:28.706888+0000","last_fullsized":"2024-09-10T07:47:28.706888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272066+0000","last_change":"2024-09-10T07:47:27.726942+0000","last_active":"2024-09-10T07:47:29.272066+0000","last_peered":"2024-09-10T07:47:29.272066+0000","last_clean":"2024-09-10T07:47:29.272066+0000","last_became_active":"2024-09-10T07:47:27.726536+0000","last_became_peered":"2024-09-10T07:47:27.726536+0000","last_unstale":"2024-09-10T07:47:29.272066+0000","last_undegraded":"2024-09-10T07:47:29.272066+0000","last_fullsized":"2024-09-10T07:47:29.272066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:25.590064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271579+0000","last_change":"2024-09-10T07:47:27.715018+0000","last_active":"2024-09-10T07:47:29.271579+0000","last_peered":"2024-09-10T07:47:29.271579+0000","last_clean":"2024-09-10T07:47:29.271579+0000","last_became_active":"2024-09-10T07:47:27.712087+0000","last_became_peered":"2024-09-10T07:47:27.712087+0000","last_unstale":"2024-09-10T07:47:29.271579+0000","last_undegraded":"2024-09-10T07:47:29.271579+0000","last_fullsized":"2024-09-10T07:47:29.271579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706943+0000","last_change":"2024-09-10T07:47:27.723476+0000","last_active":"2024-09-10T07:47:28.706943+0000","last_peered":"2024-09-10T07:47:28.706943+0000","last_clean":"2024-09-10T07:47:28.706943+0000","last_became_active":"2024-09-10T07:47:27.723095+0000","last_became_peered":"2024-09-10T07:47:27.723095+0000","last_unstale":"2024-09-10T07:47:28.706943+0000","last_undegraded":"2024-09-10T07:47:28.706943+0000","last_fullsized":"2024-09-10T07:47:28.706943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271941+0000","last_change":"2024-09-10T07:47:27.721416+0000","last_active":"2024-09-10T07:47:29.271941+0000","last_peered":"2024-09-10T07:47:29.271941+0000","last_clean":"2024-09-10T07:47:29.271941+0000","last_became_active":"2024-09-10T07:47:27.721174+0000","last_became_peered":"2024-09-10T07:47:27.721174+0000","last_unstale":"2024-09-10T07:47:29.271941+0000","last_undegraded":"2024-09-10T07:47:29.271941+0000","last_fullsized":"2024-09-10T07:47:29.271941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:17.894548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705499+0000","last_change":"2024-09-10T07:47:27.721201+0000","last_active":"2024-09-10T07:47:28.705499+0000","last_peered":"2024-09-10T07:47:28.705499+0000","last_clean":"2024-09-10T07:47:28.705499+0000","last_became_active":"2024-09-10T07:47:27.720853+0000","last_became_peered":"2024-09-10T07:47:27.720853+0000","last_unstale":"2024-09-10T07:47:28.705499+0000","last_undegraded":"2024-09-10T07:47:28.705499+0000","last_fullsized":"2024-09-10T07:47:28.705499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272874+0000","last_change":"2024-09-10T07:47:27.723555+0000","last_active":"2024-09-10T07:47:29.272874+0000","last_peered":"2024-09-10T07:47:29.272874+0000","last_clean":"2024-09-10T07:47:29.272874+0000","last_became_active":"2024-09-10T07:47:27.723226+0000","last_became_peered":"2024-09-10T07:47:27.723226+0000","last_unstale":"2024-09-10T07:47:29.272874+0000","last_undegraded":"2024-09-10T07:47:29.272874+0000","last_fullsized":"2024-09-10T07:47:29.272874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:50.385784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.273011+0000","last_change":"2024-09-10T07:47:27.729136+0000","last_active":"2024-09-10T07:47:29.273011+0000","last_peered":"2024-09-10T07:47:29.273011+0000","last_clean":"2024-09-10T07:47:29.273011+0000","last_became_active":"2024-09-10T07:47:27.724907+0000","last_became_peered":"2024-09-10T07:47:27.724907+0000","last_unstale":"2024-09-10T07:47:29.273011+0000","last_undegraded":"2024-09-10T07:47:29.273011+0000","last_fullsized":"2024-09-10T07:47:29.273011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:33:27.535922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709383+0000","last_change":"2024-09-10T07:47:27.710661+0000","last_active":"2024-09-10T07:47:28.709383+0000","last_peered":"2024-09-10T07:47:28.709383+0000","last_clean":"2024-09-10T07:47:28.709383+0000","last_became_active":"2024-09-10T07:47:27.710117+0000","last_became_peered":"2024-09-10T07:47:27.710117+0000","last_unstale":"2024-09-10T07:47:28.709383+0000","last_undegraded":"2024-09-10T07:47:28.709383+0000","last_fullsized":"2024-09-10T07:47:28.709383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706404+0000","last_change":"2024-09-10T07:47:27.713860+0000","last_active":"2024-09-10T07:47:28.706404+0000","last_peered":"2024-09-10T07:47:28.706404+0000","last_clean":"2024-09-10T07:47:28.706404+0000","last_became_active":"2024-09-10T07:47:27.713141+0000","last_became_peered":"2024-09-10T07:47:27.713141+0000","last_unstale":"2024-09-10T07:47:28.706404+0000","last_undegraded":"2024-09-10T07:47:28.706404+0000","last_fullsized":"2024-09-10T07:47:28.706404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271665+0000","last_change":"2024-09-10T07:47:27.704751+0000","last_active":"2024-09-10T07:47:29.271665+0000","last_peered":"2024-09-10T07:47:29.271665+0000","last_clean":"2024-09-10T07:47:29.271665+0000","last_became_active":"2024-09-10T07:47:27.704476+0000","last_became_peered":"2024-09-10T07:47:27.704476+0000","last_unstale":"2024-09-10T07:47:29.271665+0000","last_undegraded":"2024-09-10T07:47:29.271665+0000","last_fullsized":"2024-09-10T07:47:29.271665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271185+0000","last_change":"2024-09-10T07:47:27.697345+0000","last_active":"2024-09-10T07:47:29.271185+0000","last_peered":"2024-09-10T07:47:29.271185+0000","last_clean":"2024-09-10T07:47:29.271185+0000","last_became_active":"2024-09-10T07:47:27.697053+0000","last_became_peered":"2024-09-10T07:47:27.697053+0000","last_unstale":"2024-09-10T07:47:29.271185+0000","last_undegraded":"2024-09-10T07:47:29.271185+0000","last_fullsized":"2024-09-10T07:47:29.271185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:21.227039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705720+0000","last_change":"2024-09-10T07:47:27.718500+0000","last_active":"2024-09-10T07:47:28.705720+0000","last_peered":"2024-09-10T07:47:28.705720+0000","last_clean":"2024-09-10T07:47:28.705720+0000","last_became_active":"2024-09-10T07:47:27.718085+0000","last_became_peered":"2024-09-10T07:47:27.718085+0000","last_unstale":"2024-09-10T07:47:28.705720+0000","last_undegraded":"2024-09-10T07:47:28.705720+0000","last_fullsized":"2024-09-10T07:47:28.705720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:21.227039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706435+0000","last_change":"2024-09-10T07:47:27.708748+0000","last_active":"2024-09-10T07:47:28.706435+0000","last_peered":"2024-09-10T07:47:28.706435+0000","last_clean":"2024-09-10T07:47:28.706435+0000","last_became_active":"2024-09-10T07:47:27.708389+0000","last_became_peered":"2024-09-10T07:47:27.708389+0000","last_unstale":"2024-09-10T07:47:28.706435+0000","last_undegraded":"2024-09-10T07:47:28.706435+0000","last_fullsized":"2024-09-10T07:47:28.706435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:23.217293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272289+0000","last_change":"2024-09-10T07:47:27.714639+0000","last_active":"2024-09-10T07:47:29.272289+0000","last_peered":"2024-09-10T07:47:29.272289+0000","last_clean":"2024-09-10T07:47:29.272289+0000","last_became_active":"2024-09-10T07:47:27.714372+0000","last_became_peered":"2024-09-10T07:47:27.714372+0000","last_unstale":"2024-09-10T07:47:29.272289+0000","last_undegraded":"2024-09-10T07:47:29.272289+0000","last_fullsized":"2024-09-10T07:47:29.272289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707444+0000","last_change":"2024-09-10T07:47:26.711007+0000","last_active":"2024-09-10T07:47:28.707444+0000","last_peered":"2024-09-10T07:47:28.707444+0000","last_clean":"2024-09-10T07:47:28.707444+0000","last_became_active":"2024-09-10T07:47:26.710707+0000","last_became_peered":"2024-09-10T07:47:26.710707+0000","last_unstale":"2024-09-10T07:47:28.707444+0000","last_undegraded":"2024-09-10T07:47:28.707444+0000","last_fullsized":"2024-09-10T07:47:28.707444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:53.495126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706686+0000","last_change":"2024-09-10T07:47:27.714896+0000","last_active":"2024-09-10T07:47:28.706686+0000","last_peered":"2024-09-10T07:47:28.706686+0000","last_clean":"2024-09-10T07:47:28.706686+0000","last_became_active":"2024-09-10T07:47:27.714300+0000","last_became_peered":"2024-09-10T07:47:27.714300+0000","last_unstale":"2024-09-10T07:47:28.706686+0000","last_undegraded":"2024-09-10T07:47:28.706686+0000","last_fullsized":"2024-09-10T07:47:28.706686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:26.738356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.273059+0000","last_change":"2024-09-10T07:47:26.717356+0000","last_active":"2024-09-10T07:47:29.273059+0000","last_peered":"2024-09-10T07:47:29.273059+0000","last_clean":"2024-09-10T07:47:29.273059+0000","last_became_active":"2024-09-10T07:47:26.717075+0000","last_became_peered":"2024-09-10T07:47:26.717075+0000","last_unstale":"2024-09-10T07:47:29.273059+0000","last_undegraded":"2024-09-10T07:47:29.273059+0000","last_fullsized":"2024-09-10T07:47:29.273059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:23.907465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706720+0000","last_change":"2024-09-10T07:47:27.723254+0000","last_active":"2024-09-10T07:47:28.706720+0000","last_peered":"2024-09-10T07:47:28.706720+0000","last_clean":"2024-09-10T07:47:28.706720+0000","last_became_active":"2024-09-10T07:47:27.722364+0000","last_became_peered":"2024-09-10T07:47:27.722364+0000","last_unstale":"2024-09-10T07:47:28.706720+0000","last_undegraded":"2024-09-10T07:47:28.706720+0000","last_fullsized":"2024-09-10T07:47:28.706720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707037+0000","last_change":"2024-09-10T07:47:26.709026+0000","last_active":"2024-09-10T07:47:28.707037+0000","last_peered":"2024-09-10T07:47:28.707037+0000","last_clean":"2024-09-10T07:47:28.707037+0000","last_became_active":"2024-09-10T07:47:26.708550+0000","last_became_peered":"2024-09-10T07:47:26.708550+0000","last_unstale":"2024-09-10T07:47:28.707037+0000","last_undegraded":"2024-09-10T07:47:28.707037+0000","last_fullsized":"2024-09-10T07:47:28.707037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272862+0000","last_change":"2024-09-10T07:47:27.722622+0000","last_active":"2024-09-10T07:47:29.272862+0000","last_peered":"2024-09-10T07:47:29.272862+0000","last_clean":"2024-09-10T07:47:29.272862+0000","last_became_active":"2024-09-10T07:47:27.722182+0000","last_became_peered":"2024-09-10T07:47:27.722182+0000","last_unstale":"2024-09-10T07:47:29.272862+0000","last_undegraded":"2024-09-10T07:47:29.272862+0000","last_fullsized":"2024-09-10T07:47:29.272862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:45.898938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272862+0000","last_change":"2024-09-10T07:47:26.714730+0000","last_active":"2024-09-10T07:47:29.272862+0000","last_peered":"2024-09-10T07:47:29.272862+0000","last_clean":"2024-09-10T07:47:29.272862+0000","last_became_active":"2024-09-10T07:47:26.713780+0000","last_became_peered":"2024-09-10T07:47:26.713780+0000","last_unstale":"2024-09-10T07:47:29.272862+0000","last_undegraded":"2024-09-10T07:47:29.272862+0000","last_fullsized":"2024-09-10T07:47:29.272862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:59.575014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271288+0000","last_change":"2024-09-10T07:47:27.716983+0000","last_active":"2024-09-10T07:47:29.271288+0000","last_peered":"2024-09-10T07:47:29.271288+0000","last_clean":"2024-09-10T07:47:29.271288+0000","last_became_active":"2024-09-10T07:47:27.716698+0000","last_became_peered":"2024-09-10T07:47:27.716698+0000","last_unstale":"2024-09-10T07:47:29.271288+0000","last_undegraded":"2024-09-10T07:47:29.271288+0000","last_fullsized":"2024-09-10T07:47:29.271288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:56.241795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705902+0000","last_change":"2024-09-10T07:47:26.711406+0000","last_active":"2024-09-10T07:47:28.705902+0000","last_peered":"2024-09-10T07:47:28.705902+0000","last_clean":"2024-09-10T07:47:28.705902+0000","last_became_active":"2024-09-10T07:47:26.710814+0000","last_became_peered":"2024-09-10T07:47:26.710814+0000","last_unstale":"2024-09-10T07:47:28.705902+0000","last_undegraded":"2024-09-10T07:47:28.705902+0000","last_fullsized":"2024-09-10T07:47:28.705902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705811+0000","last_change":"2024-09-10T07:47:27.714326+0000","last_active":"2024-09-10T07:47:28.705811+0000","last_peered":"2024-09-10T07:47:28.705811+0000","last_clean":"2024-09-10T07:47:28.705811+0000","last_became_active":"2024-09-10T07:47:27.714018+0000","last_became_peered":"2024-09-10T07:47:27.714018+0000","last_unstale":"2024-09-10T07:47:28.705811+0000","last_undegraded":"2024-09-10T07:47:28.705811+0000","last_fullsized":"2024-09-10T07:47:28.705811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706826+0000","last_change":"2024-09-10T07:47:26.713426+0000","last_active":"2024-09-10T07:47:28.706826+0000","last_peered":"2024-09-10T07:47:28.706826+0000","last_clean":"2024-09-10T07:47:28.706826+0000","last_became_active":"2024-09-10T07:47:26.712028+0000","last_became_peered":"2024-09-10T07:47:26.712028+0000","last_unstale":"2024-09-10T07:47:28.706826+0000","last_undegraded":"2024-09-10T07:47:28.706826+0000","last_fullsized":"2024-09-10T07:47:28.706826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271344+0000","last_change":"2024-09-10T07:47:27.730588+0000","last_active":"2024-09-10T07:47:29.271344+0000","last_peered":"2024-09-10T07:47:29.271344+0000","last_clean":"2024-09-10T07:47:29.271344+0000","last_became_active":"2024-09-10T07:47:27.730242+0000","last_became_peered":"2024-09-10T07:47:27.730242+0000","last_unstale":"2024-09-10T07:47:29.271344+0000","last_undegraded":"2024-09-10T07:47:29.271344+0000","last_fullsized":"2024-09-10T07:47:29.271344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:14.309793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706730+0000","last_change":"2024-09-10T07:47:26.701393+0000","last_active":"2024-09-10T07:47:28.706730+0000","last_peered":"2024-09-10T07:47:28.706730+0000","last_clean":"2024-09-10T07:47:28.706730+0000","last_became_active":"2024-09-10T07:47:26.701105+0000","last_became_peered":"2024-09-10T07:47:26.701105+0000","last_unstale":"2024-09-10T07:47:28.706730+0000","last_undegraded":"2024-09-10T07:47:28.706730+0000","last_fullsized":"2024-09-10T07:47:28.706730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272044+0000","last_change":"2024-09-10T07:47:27.708836+0000","last_active":"2024-09-10T07:47:29.272044+0000","last_peered":"2024-09-10T07:47:29.272044+0000","last_clean":"2024-09-10T07:47:29.272044+0000","last_became_active":"2024-09-10T07:47:27.708519+0000","last_became_peered":"2024-09-10T07:47:27.708519+0000","last_unstale":"2024-09-10T07:47:29.272044+0000","last_undegraded":"2024-09-10T07:47:29.272044+0000","last_fullsized":"2024-09-10T07:47:29.272044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:37.104237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705632+0000","last_change":"2024-09-10T07:47:26.713761+0000","last_active":"2024-09-10T07:47:28.705632+0000","last_peered":"2024-09-10T07:47:28.705632+0000","last_clean":"2024-09-10T07:47:28.705632+0000","last_became_active":"2024-09-10T07:47:26.713497+0000","last_became_peered":"2024-09-10T07:47:26.713497+0000","last_unstale":"2024-09-10T07:47:28.705632+0000","last_undegraded":"2024-09-10T07:47:28.705632+0000","last_fullsized":"2024-09-10T07:47:28.705632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707056+0000","last_change":"2024-09-10T07:47:27.717917+0000","last_active":"2024-09-10T07:47:28.707056+0000","last_peered":"2024-09-10T07:47:28.707056+0000","last_clean":"2024-09-10T07:47:28.707056+0000","last_became_active":"2024-09-10T07:47:27.717135+0000","last_became_peered":"2024-09-10T07:47:27.717135+0000","last_unstale":"2024-09-10T07:47:28.707056+0000","last_undegraded":"2024-09-10T07:47:28.707056+0000","last_fullsized":"2024-09-10T07:47:28.707056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:35.220570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709924+0000","last_change":"2024-09-10T07:47:26.716721+0000","last_active":"2024-09-10T07:47:28.709924+0000","last_peered":"2024-09-10T07:47:28.709924+0000","last_clean":"2024-09-10T07:47:28.709924+0000","last_became_active":"2024-09-10T07:47:26.716429+0000","last_became_peered":"2024-09-10T07:47:26.716429+0000","last_unstale":"2024-09-10T07:47:28.709924+0000","last_undegraded":"2024-09-10T07:47:28.709924+0000","last_fullsized":"2024-09-10T07:47:28.709924+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272981+0000","last_change":"2024-09-10T07:47:27.721272+0000","last_active":"2024-09-10T07:47:29.272981+0000","last_peered":"2024-09-10T07:47:29.272981+0000","last_clean":"2024-09-10T07:47:29.272981+0000","last_became_active":"2024-09-10T07:47:27.718319+0000","last_became_peered":"2024-09-10T07:47:27.718319+0000","last_unstale":"2024-09-10T07:47:29.272981+0000","last_undegraded":"2024-09-10T07:47:29.272981+0000","last_fullsized":"2024-09-10T07:47:29.272981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:45.917831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709462+0000","last_change":"2024-09-10T07:47:26.713954+0000","last_active":"2024-09-10T07:47:28.709462+0000","last_peered":"2024-09-10T07:47:28.709462+0000","last_clean":"2024-09-10T07:47:28.709462+0000","last_became_active":"2024-09-10T07:47:26.713378+0000","last_became_peered":"2024-09-10T07:47:26.713378+0000","last_unstale":"2024-09-10T07:47:28.709462+0000","last_undegraded":"2024-09-10T07:47:28.709462+0000","last_fullsized":"2024-09-10T07:47:28.709462+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:03.461580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272722+0000","last_change":"2024-09-10T07:47:27.724717+0000","last_active":"2024-09-10T07:47:29.272722+0000","last_peered":"2024-09-10T07:47:29.272722+0000","last_clean":"2024-09-10T07:47:29.272722+0000","last_became_active":"2024-09-10T07:47:27.724464+0000","last_became_peered":"2024-09-10T07:47:27.724464+0000","last_unstale":"2024-09-10T07:47:29.272722+0000","last_undegraded":"2024-09-10T07:47:29.272722+0000","last_fullsized":"2024-09-10T07:47:29.272722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:34.310092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707179+0000","last_change":"2024-09-10T07:47:26.709280+0000","last_active":"2024-09-10T07:47:28.707179+0000","last_peered":"2024-09-10T07:47:28.707179+0000","last_clean":"2024-09-10T07:47:28.707179+0000","last_became_active":"2024-09-10T07:47:26.708960+0000","last_became_peered":"2024-09-10T07:47:26.708960+0000","last_unstale":"2024-09-10T07:47:28.707179+0000","last_undegraded":"2024-09-10T07:47:28.707179+0000","last_fullsized":"2024-09-10T07:47:28.707179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:03.869247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707230+0000","last_change":"2024-09-10T07:47:27.718510+0000","last_active":"2024-09-10T07:47:28.707230+0000","last_peered":"2024-09-10T07:47:28.707230+0000","last_clean":"2024-09-10T07:47:28.707230+0000","last_became_active":"2024-09-10T07:47:27.717326+0000","last_became_peered":"2024-09-10T07:47:27.717326+0000","last_unstale":"2024-09-10T07:47:28.707230+0000","last_undegraded":"2024-09-10T07:47:28.707230+0000","last_fullsized":"2024-09-10T07:47:28.707230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.464775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272123+0000","last_change":"2024-09-10T07:47:26.713112+0000","last_active":"2024-09-10T07:47:29.272123+0000","last_peered":"2024-09-10T07:47:29.272123+0000","last_clean":"2024-09-10T07:47:29.272123+0000","last_became_active":"2024-09-10T07:47:26.712737+0000","last_became_peered":"2024-09-10T07:47:26.712737+0000","last_unstale":"2024-09-10T07:47:29.272123+0000","last_undegraded":"2024-09-10T07:47:29.272123+0000","last_fullsized":"2024-09-10T07:47:29.272123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271575+0000","last_change":"2024-09-10T07:47:27.722758+0000","last_active":"2024-09-10T07:47:29.271575+0000","last_peered":"2024-09-10T07:47:29.271575+0000","last_clean":"2024-09-10T07:47:29.271575+0000","last_became_active":"2024-09-10T07:47:27.722517+0000","last_became_peered":"2024-09-10T07:47:27.722517+0000","last_unstale":"2024-09-10T07:47:29.271575+0000","last_undegraded":"2024-09-10T07:47:29.271575+0000","last_fullsized":"2024-09-10T07:47:29.271575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:07.008875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706000+0000","last_change":"2024-09-10T07:47:26.713271+0000","last_active":"2024-09-10T07:47:28.706000+0000","last_peered":"2024-09-10T07:47:28.706000+0000","last_clean":"2024-09-10T07:47:28.706000+0000","last_became_active":"2024-09-10T07:47:26.712715+0000","last_became_peered":"2024-09-10T07:47:26.712715+0000","last_unstale":"2024-09-10T07:47:28.706000+0000","last_undegraded":"2024-09-10T07:47:28.706000+0000","last_fullsized":"2024-09-10T07:47:28.706000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:47.147053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272354+0000","last_change":"2024-09-10T07:47:27.725965+0000","last_active":"2024-09-10T07:47:29.272354+0000","last_peered":"2024-09-10T07:47:29.272354+0000","last_clean":"2024-09-10T07:47:29.272354+0000","last_became_active":"2024-09-10T07:47:27.725029+0000","last_became_peered":"2024-09-10T07:47:27.725029+0000","last_unstale":"2024-09-10T07:47:29.272354+0000","last_undegraded":"2024-09-10T07:47:29.272354+0000","last_fullsized":"2024-09-10T07:47:29.272354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:23.318762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272347+0000","last_change":"2024-09-10T07:47:26.711747+0000","last_active":"2024-09-10T07:47:29.272347+0000","last_peered":"2024-09-10T07:47:29.272347+0000","last_clean":"2024-09-10T07:47:29.272347+0000","last_became_active":"2024-09-10T07:47:26.711458+0000","last_became_peered":"2024-09-10T07:47:26.711458+0000","last_unstale":"2024-09-10T07:47:29.272347+0000","last_undegraded":"2024-09-10T07:47:29.272347+0000","last_fullsized":"2024-09-10T07:47:29.272347+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:17.886142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705655+0000","last_change":"2024-09-10T07:47:27.720915+0000","last_active":"2024-09-10T07:47:28.705655+0000","last_peered":"2024-09-10T07:47:28.705655+0000","last_clean":"2024-09-10T07:47:28.705655+0000","last_became_active":"2024-09-10T07:47:27.720654+0000","last_became_peered":"2024-09-10T07:47:27.720654+0000","last_unstale":"2024-09-10T07:47:28.705655+0000","last_undegraded":"2024-09-10T07:47:28.705655+0000","last_fullsized":"2024-09-10T07:47:28.705655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:07:55.153398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272060+0000","last_change":"2024-09-10T07:47:26.708509+0000","last_active":"2024-09-10T07:47:29.272060+0000","last_peered":"2024-09-10T07:47:29.272060+0000","last_clean":"2024-09-10T07:47:29.272060+0000","last_became_active":"2024-09-10T07:47:26.708220+0000","last_became_peered":"2024-09-10T07:47:26.708220+0000","last_unstale":"2024-09-10T07:47:29.272060+0000","last_undegraded":"2024-09-10T07:47:29.272060+0000","last_fullsized":"2024-09-10T07:47:29.272060+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:55.494668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706893+0000","last_change":"2024-09-10T07:47:27.719995+0000","last_active":"2024-09-10T07:47:28.706893+0000","last_peered":"2024-09-10T07:47:28.706893+0000","last_clean":"2024-09-10T07:47:28.706893+0000","last_became_active":"2024-09-10T07:47:27.719604+0000","last_became_peered":"2024-09-10T07:47:27.719604+0000","last_unstale":"2024-09-10T07:47:28.706893+0000","last_undegraded":"2024-09-10T07:47:28.706893+0000","last_fullsized":"2024-09-10T07:47:28.706893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.659389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706491+0000","last_change":"2024-09-10T07:47:26.706290+0000","last_active":"2024-09-10T07:47:28.706491+0000","last_peered":"2024-09-10T07:47:28.706491+0000","last_clean":"2024-09-10T07:47:28.706491+0000","last_became_active":"2024-09-10T07:47:26.705979+0000","last_became_peered":"2024-09-10T07:47:26.705979+0000","last_unstale":"2024-09-10T07:47:28.706491+0000","last_undegraded":"2024-09-10T07:47:28.706491+0000","last_fullsized":"2024-09-10T07:47:28.706491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707203+0000","last_change":"2024-09-10T07:47:27.717619+0000","last_active":"2024-09-10T07:47:28.707203+0000","last_peered":"2024-09-10T07:47:28.707203+0000","last_clean":"2024-09-10T07:47:28.707203+0000","last_became_active":"2024-09-10T07:47:27.716747+0000","last_became_peered":"2024-09-10T07:47:27.716747+0000","last_unstale":"2024-09-10T07:47:28.707203+0000","last_undegraded":"2024-09-10T07:47:28.707203+0000","last_fullsized":"2024-09-10T07:47:28.707203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:01.440358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706848+0000","last_change":"2024-09-10T07:47:26.710001+0000","last_active":"2024-09-10T07:47:28.706848+0000","last_peered":"2024-09-10T07:47:28.706848+0000","last_clean":"2024-09-10T07:47:28.706848+0000","last_became_active":"2024-09-10T07:47:26.708545+0000","last_became_peered":"2024-09-10T07:47:26.708545+0000","last_unstale":"2024-09-10T07:47:28.706848+0000","last_undegraded":"2024-09-10T07:47:28.706848+0000","last_fullsized":"2024-09-10T07:47:28.706848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271880+0000","last_change":"2024-09-10T07:47:27.733598+0000","last_active":"2024-09-10T07:47:29.271880+0000","last_peered":"2024-09-10T07:47:29.271880+0000","last_clean":"2024-09-10T07:47:29.271880+0000","last_became_active":"2024-09-10T07:47:27.733342+0000","last_became_peered":"2024-09-10T07:47:27.733342+0000","last_unstale":"2024-09-10T07:47:29.271880+0000","last_undegraded":"2024-09-10T07:47:29.271880+0000","last_fullsized":"2024-09-10T07:47:29.271880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706292+0000","last_change":"2024-09-10T07:47:26.713070+0000","last_active":"2024-09-10T07:47:28.706292+0000","last_peered":"2024-09-10T07:47:28.706292+0000","last_clean":"2024-09-10T07:47:28.706292+0000","last_became_active":"2024-09-10T07:47:26.712820+0000","last_became_peered":"2024-09-10T07:47:26.712820+0000","last_unstale":"2024-09-10T07:47:28.706292+0000","last_undegraded":"2024-09-10T07:47:28.706292+0000","last_fullsized":"2024-09-10T07:47:28.706292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:32.163532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709356+0000","last_change":"2024-09-10T07:47:27.712460+0000","last_active":"2024-09-10T07:47:28.709356+0000","last_peered":"2024-09-10T07:47:28.709356+0000","last_clean":"2024-09-10T07:47:28.709356+0000","last_became_active":"2024-09-10T07:47:27.712183+0000","last_became_peered":"2024-09-10T07:47:27.712183+0000","last_unstale":"2024-09-10T07:47:28.709356+0000","last_undegraded":"2024-09-10T07:47:28.709356+0000","last_fullsized":"2024-09-10T07:47:28.709356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707336+0000","last_change":"2024-09-10T07:47:26.713128+0000","last_active":"2024-09-10T07:47:28.707336+0000","last_peered":"2024-09-10T07:47:28.707336+0000","last_clean":"2024-09-10T07:47:28.707336+0000","last_became_active":"2024-09-10T07:47:26.712677+0000","last_became_peered":"2024-09-10T07:47:26.712677+0000","last_unstale":"2024-09-10T07:47:28.707336+0000","last_undegraded":"2024-09-10T07:47:28.707336+0000","last_fullsized":"2024-09-10T07:47:28.707336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:55.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706999+0000","last_change":"2024-09-10T07:47:27.724138+0000","last_active":"2024-09-10T07:47:28.706999+0000","last_peered":"2024-09-10T07:47:28.706999+0000","last_clean":"2024-09-10T07:47:28.706999+0000","last_became_active":"2024-09-10T07:47:27.723179+0000","last_became_peered":"2024-09-10T07:47:27.723179+0000","last_unstale":"2024-09-10T07:47:28.706999+0000","last_undegraded":"2024-09-10T07:47:28.706999+0000","last_fullsized":"2024-09-10T07:47:28.706999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271786+0000","last_change":"2024-09-10T07:47:26.711949+0000","last_active":"2024-09-10T07:47:29.271786+0000","last_peered":"2024-09-10T07:47:29.271786+0000","last_clean":"2024-09-10T07:47:29.271786+0000","last_became_active":"2024-09-10T07:47:26.711492+0000","last_became_peered":"2024-09-10T07:47:26.711492+0000","last_unstale":"2024-09-10T07:47:29.271786+0000","last_undegraded":"2024-09-10T07:47:29.271786+0000","last_fullsized":"2024-09-10T07:47:29.271786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271616+0000","last_change":"2024-09-10T07:47:27.725128+0000","last_active":"2024-09-10T07:47:29.271616+0000","last_peered":"2024-09-10T07:47:29.271616+0000","last_clean":"2024-09-10T07:47:29.271616+0000","last_became_active":"2024-09-10T07:47:27.724151+0000","last_became_peered":"2024-09-10T07:47:27.724151+0000","last_unstale":"2024-09-10T07:47:29.271616+0000","last_undegraded":"2024-09-10T07:47:29.271616+0000","last_fullsized":"2024-09-10T07:47:29.271616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:38.688543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709386+0000","last_change":"2024-09-10T07:47:26.715573+0000","last_active":"2024-09-10T07:47:28.709386+0000","last_peered":"2024-09-10T07:47:28.709386+0000","last_clean":"2024-09-10T07:47:28.709386+0000","last_became_active":"2024-09-10T07:47:26.715315+0000","last_became_peered":"2024-09-10T07:47:26.715315+0000","last_unstale":"2024-09-10T07:47:28.709386+0000","last_undegraded":"2024-09-10T07:47:28.709386+0000","last_fullsized":"2024-09-10T07:47:28.709386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:37.683908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709341+0000","last_change":"2024-09-10T07:47:27.701024+0000","last_active":"2024-09-10T07:47:28.709341+0000","last_peered":"2024-09-10T07:47:28.709341+0000","last_clean":"2024-09-10T07:47:28.709341+0000","last_became_active":"2024-09-10T07:47:27.700612+0000","last_became_peered":"2024-09-10T07:47:27.700612+0000","last_unstale":"2024-09-10T07:47:28.709341+0000","last_undegraded":"2024-09-10T07:47:28.709341+0000","last_fullsized":"2024-09-10T07:47:28.709341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706925+0000","last_change":"2024-09-10T07:47:26.707503+0000","last_active":"2024-09-10T07:47:28.706925+0000","last_peered":"2024-09-10T07:47:28.706925+0000","last_clean":"2024-09-10T07:47:28.706925+0000","last_became_active":"2024-09-10T07:47:26.706814+0000","last_became_peered":"2024-09-10T07:47:26.706814+0000","last_unstale":"2024-09-10T07:47:28.706925+0000","last_undegraded":"2024-09-10T07:47:28.706925+0000","last_fullsized":"2024-09-10T07:47:28.706925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271600+0000","last_change":"2024-09-10T07:47:27.722893+0000","last_active":"2024-09-10T07:47:29.271600+0000","last_peered":"2024-09-10T07:47:29.271600+0000","last_clean":"2024-09-10T07:47:29.271600+0000","last_became_active":"2024-09-10T07:47:27.722551+0000","last_became_peered":"2024-09-10T07:47:27.722551+0000","last_unstale":"2024-09-10T07:47:29.271600+0000","last_undegraded":"2024-09-10T07:47:29.271600+0000","last_fullsized":"2024-09-10T07:47:29.271600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:52.315032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271331+0000","last_change":"2024-09-10T07:47:26.714230+0000","last_active":"2024-09-10T07:47:29.271331+0000","last_peered":"2024-09-10T07:47:29.271331+0000","last_clean":"2024-09-10T07:47:29.271331+0000","last_became_active":"2024-09-10T07:47:26.712186+0000","last_became_peered":"2024-09-10T07:47:26.712186+0000","last_unstale":"2024-09-10T07:47:29.271331+0000","last_undegraded":"2024-09-10T07:47:29.271331+0000","last_fullsized":"2024-09-10T07:47:29.271331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:12.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271313+0000","last_change":"2024-09-10T07:47:27.720014+0000","last_active":"2024-09-10T07:47:29.271313+0000","last_peered":"2024-09-10T07:47:29.271313+0000","last_clean":"2024-09-10T07:47:29.271313+0000","last_became_active":"2024-09-10T07:47:27.718864+0000","last_became_peered":"2024-09-10T07:47:27.718864+0000","last_unstale":"2024-09-10T07:47:29.271313+0000","last_undegraded":"2024-09-10T07:47:29.271313+0000","last_fullsized":"2024-09-10T07:47:29.271313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:11.146080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706471+0000","last_change":"2024-09-10T07:47:26.709027+0000","last_active":"2024-09-10T07:47:28.706471+0000","last_peered":"2024-09-10T07:47:28.706471+0000","last_clean":"2024-09-10T07:47:28.706471+0000","last_became_active":"2024-09-10T07:47:26.708753+0000","last_became_peered":"2024-09-10T07:47:26.708753+0000","last_unstale":"2024-09-10T07:47:28.706471+0000","last_undegraded":"2024-09-10T07:47:28.706471+0000","last_fullsized":"2024-09-10T07:47:28.706471+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:25.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707048+0000","last_change":"2024-09-10T07:47:27.728171+0000","last_active":"2024-09-10T07:47:28.707048+0000","last_peered":"2024-09-10T07:47:28.707048+0000","last_clean":"2024-09-10T07:47:28.707048+0000","last_became_active":"2024-09-10T07:47:27.727842+0000","last_became_peered":"2024-09-10T07:47:27.727842+0000","last_unstale":"2024-09-10T07:47:28.707048+0000","last_undegraded":"2024-09-10T07:47:28.707048+0000","last_fullsized":"2024-09-10T07:47:28.707048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:45.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272065+0000","last_change":"2024-09-10T07:47:26.707810+0000","last_active":"2024-09-10T07:47:29.272065+0000","last_peered":"2024-09-10T07:47:29.272065+0000","last_clean":"2024-09-10T07:47:29.272065+0000","last_became_active":"2024-09-10T07:47:26.707462+0000","last_became_peered":"2024-09-10T07:47:26.707462+0000","last_unstale":"2024-09-10T07:47:29.272065+0000","last_undegraded":"2024-09-10T07:47:29.272065+0000","last_fullsized":"2024-09-10T07:47:29.272065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707444+0000","last_change":"2024-09-10T07:47:27.707117+0000","last_active":"2024-09-10T07:47:28.707444+0000","last_peered":"2024-09-10T07:47:28.707444+0000","last_clean":"2024-09-10T07:47:28.707444+0000","last_became_active":"2024-09-10T07:47:27.706809+0000","last_became_peered":"2024-09-10T07:47:27.706809+0000","last_unstale":"2024-09-10T07:47:28.707444+0000","last_undegraded":"2024-09-10T07:47:28.707444+0000","last_fullsized":"2024-09-10T07:47:28.707444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:39.128769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709605+0000","last_change":"2024-09-10T07:47:26.706696+0000","last_active":"2024-09-10T07:47:28.709605+0000","last_peered":"2024-09-10T07:47:28.709605+0000","last_clean":"2024-09-10T07:47:28.709605+0000","last_became_active":"2024-09-10T07:47:26.706383+0000","last_became_peered":"2024-09-10T07:47:26.706383+0000","last_unstale":"2024-09-10T07:47:28.709605+0000","last_undegraded":"2024-09-10T07:47:28.709605+0000","last_fullsized":"2024-09-10T07:47:28.709605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:16.889913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707098+0000","last_change":"2024-09-10T07:47:27.721003+0000","last_active":"2024-09-10T07:47:28.707098+0000","last_peered":"2024-09-10T07:47:28.707098+0000","last_clean":"2024-09-10T07:47:28.707098+0000","last_became_active":"2024-09-10T07:47:27.720702+0000","last_became_peered":"2024-09-10T07:47:27.720702+0000","last_unstale":"2024-09-10T07:47:28.707098+0000","last_undegraded":"2024-09-10T07:47:28.707098+0000","last_fullsized":"2024-09-10T07:47:28.707098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:33.721528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272228+0000","last_change":"2024-09-10T07:47:26.701128+0000","last_active":"2024-09-10T07:47:29.272228+0000","last_peered":"2024-09-10T07:47:29.272228+0000","last_clean":"2024-09-10T07:47:29.272228+0000","last_became_active":"2024-09-10T07:47:26.700042+0000","last_became_peered":"2024-09-10T07:47:26.700042+0000","last_unstale":"2024-09-10T07:47:29.272228+0000","last_undegraded":"2024-09-10T07:47:29.272228+0000","last_fullsized":"2024-09-10T07:47:29.272228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271326+0000","last_change":"2024-09-10T07:47:27.711768+0000","last_active":"2024-09-10T07:47:29.271326+0000","last_peered":"2024-09-10T07:47:29.271326+0000","last_clean":"2024-09-10T07:47:29.271326+0000","last_became_active":"2024-09-10T07:47:27.709290+0000","last_became_peered":"2024-09-10T07:47:27.709290+0000","last_unstale":"2024-09-10T07:47:29.271326+0000","last_undegraded":"2024-09-10T07:47:29.271326+0000","last_fullsized":"2024-09-10T07:47:29.271326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271867+0000","last_change":"2024-09-10T07:47:26.705882+0000","last_active":"2024-09-10T07:47:29.271867+0000","last_peered":"2024-09-10T07:47:29.271867+0000","last_clean":"2024-09-10T07:47:29.271867+0000","last_became_active":"2024-09-10T07:47:26.705616+0000","last_became_peered":"2024-09-10T07:47:26.705616+0000","last_unstale":"2024-09-10T07:47:29.271867+0000","last_undegraded":"2024-09-10T07:47:29.271867+0000","last_fullsized":"2024-09-10T07:47:29.271867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272523+0000","last_change":"2024-09-10T07:47:27.719286+0000","last_active":"2024-09-10T07:47:29.272523+0000","last_peered":"2024-09-10T07:47:29.272523+0000","last_clean":"2024-09-10T07:47:29.272523+0000","last_became_active":"2024-09-10T07:47:27.715311+0000","last_became_peered":"2024-09-10T07:47:27.715311+0000","last_unstale":"2024-09-10T07:47:29.272523+0000","last_undegraded":"2024-09-10T07:47:29.272523+0000","last_fullsized":"2024-09-10T07:47:29.272523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:23.609705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271645+0000","last_change":"2024-09-10T07:47:26.710717+0000","last_active":"2024-09-10T07:47:29.271645+0000","last_peered":"2024-09-10T07:47:29.271645+0000","last_clean":"2024-09-10T07:47:29.271645+0000","last_became_active":"2024-09-10T07:47:26.710392+0000","last_became_peered":"2024-09-10T07:47:26.710392+0000","last_unstale":"2024-09-10T07:47:29.271645+0000","last_undegraded":"2024-09-10T07:47:29.271645+0000","last_fullsized":"2024-09-10T07:47:29.271645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271871+0000","last_change":"2024-09-10T07:47:27.727229+0000","last_active":"2024-09-10T07:47:29.271871+0000","last_peered":"2024-09-10T07:47:29.271871+0000","last_clean":"2024-09-10T07:47:29.271871+0000","last_became_active":"2024-09-10T07:47:27.725550+0000","last_became_peered":"2024-09-10T07:47:27.725550+0000","last_unstale":"2024-09-10T07:47:29.271871+0000","last_undegraded":"2024-09-10T07:47:29.271871+0000","last_fullsized":"2024-09-10T07:47:29.271871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:38:25.629006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706045+0000","last_change":"2024-09-10T07:47:26.710777+0000","last_active":"2024-09-10T07:47:28.706045+0000","last_peered":"2024-09-10T07:47:28.706045+0000","last_clean":"2024-09-10T07:47:28.706045+0000","last_became_active":"2024-09-10T07:47:26.710499+0000","last_became_peered":"2024-09-10T07:47:26.710499+0000","last_unstale":"2024-09-10T07:47:28.706045+0000","last_undegraded":"2024-09-10T07:47:28.706045+0000","last_fullsized":"2024-09-10T07:47:28.706045+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706975+0000","last_change":"2024-09-10T07:47:27.705681+0000","last_active":"2024-09-10T07:47:28.706975+0000","last_peered":"2024-09-10T07:47:28.706975+0000","last_clean":"2024-09-10T07:47:28.706975+0000","last_became_active":"2024-09-10T07:47:27.705375+0000","last_became_peered":"2024-09-10T07:47:27.705375+0000","last_unstale":"2024-09-10T07:47:28.706975+0000","last_undegraded":"2024-09-10T07:47:28.706975+0000","last_fullsized":"2024-09-10T07:47:28.706975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:40.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271828+0000","last_change":"2024-09-10T07:47:26.716992+0000","last_active":"2024-09-10T07:47:29.271828+0000","last_peered":"2024-09-10T07:47:29.271828+0000","last_clean":"2024-09-10T07:47:29.271828+0000","last_became_active":"2024-09-10T07:47:26.716690+0000","last_became_peered":"2024-09-10T07:47:26.716690+0000","last_unstale":"2024-09-10T07:47:29.271828+0000","last_undegraded":"2024-09-10T07:47:29.271828+0000","last_fullsized":"2024-09-10T07:47:29.271828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.180750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271802+0000","last_change":"2024-09-10T07:47:27.708470+0000","last_active":"2024-09-10T07:47:29.271802+0000","last_peered":"2024-09-10T07:47:29.271802+0000","last_clean":"2024-09-10T07:47:29.271802+0000","last_became_active":"2024-09-10T07:47:27.708196+0000","last_became_peered":"2024-09-10T07:47:27.708196+0000","last_unstale":"2024-09-10T07:47:29.271802+0000","last_undegraded":"2024-09-10T07:47:29.271802+0000","last_fullsized":"2024-09-10T07:47:29.271802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707326+0000","last_change":"2024-09-10T07:47:26.707932+0000","last_active":"2024-09-10T07:47:28.707326+0000","last_peered":"2024-09-10T07:47:28.707326+0000","last_clean":"2024-09-10T07:47:28.707326+0000","last_became_active":"2024-09-10T07:47:26.707401+0000","last_became_peered":"2024-09-10T07:47:26.707401+0000","last_unstale":"2024-09-10T07:47:28.707326+0000","last_undegraded":"2024-09-10T07:47:28.707326+0000","last_fullsized":"2024-09-10T07:47:28.707326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:37.955939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271441+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:47:29.271441+0000","last_peered":"2024-09-10T07:47:29.271441+0000","last_clean":"2024-09-10T07:47:29.271441+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:47:29.271441+0000","last_undegraded":"2024-09-10T07:47:29.271441+0000","last_fullsized":"2024-09-10T07:47:29.271441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":155513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":154771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":745175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":154827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":154759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":156349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":745687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":413696,"data_stored":156371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:47:36.631 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:47:36.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.799+0000 7f3eb7e00640 1 Processor -- start 2024-09-10T07:47:36.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.799+0000 7f3eb7e00640 1 -- start start 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.800+0000 7f3eb7e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3eb006bb20 0x7f3eb015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.800+0000 7f3eb7e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb01603c0 con 0x7f3eb006d160 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.800+0000 7f3eb7e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb006c0c0 con 0x7f3eb015e4f0 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.800+0000 7f3eb7e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb006ccd0 con 0x7f3eb006bb20 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eb5e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3eb006d160 0x7f3eb006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35328/0 (socket says 172.21.15.3:35328) 2024-09-10T07:47:36.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eb5e00640 1 -- 172.21.15.3:0/888537836 learned_addr learned my addr 172.21.15.3:0/888537836 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:36.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eb4a00640 1 --2- 172.21.15.3:0/888537836 >> v2:172.21.15.70:3300/0 conn(0x7f3eb006bb20 0x7f3eb015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816869177 0 0) 0x7f3eb01603c0 con 0x7f3eb006d160 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e98003520 con 0x7f3eb006d160 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835744767 0 0) 0x7f3eb006c0c0 con 0x7f3eb015e4f0 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.801+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eb01603c0 con 0x7f3eb015e4f0 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.802+0000 7f3eb4a00640 1 -- 172.21.15.3:0/888537836 >> v1:172.21.15.70:6789/0 conn(0x7f3eb015e4f0 legacy=0x7f3eb015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.802+0000 7f3eb4a00640 1 -- 172.21.15.3:0/888537836 >> v1:172.21.15.3:6789/0 conn(0x7f3eb006d160 legacy=0x7f3eb006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:36.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.802+0000 7f3eb4a00640 1 -- 172.21.15.3:0/888537836 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb006c3a0 con 0x7f3eb006bb20 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb4a00640 1 --2- 172.21.15.3:0/888537836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb015f570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097a0 tx=0x7f3e9c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3afe04d70cbd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c037070 con 0x7f3eb006bb20 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c032440 con 0x7f3eb006bb20 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eafe00640 1 -- 172.21.15.3:0/888537836 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c032810 con 0x7f3eb006bb20 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb7e00640 1 -- 172.21.15.3:0/888537836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 msgr2=0x7f3eb015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:36.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/888537836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb015f570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097a0 tx=0x7f3e9c02a230 comp rx=0 tx=0).stop 2024-09-10T07:47:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb7e00640 1 -- 172.21.15.3:0/888537836 shutdown_connections 2024-09-10T07:47:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/888537836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb015f570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.803+0000 7f3eb7e00640 1 -- 172.21.15.3:0/888537836 >> 172.21.15.3:0/888537836 conn(0x7f3eb006e5b0 msgr2=0x7f3eb006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.804+0000 7f3eb7e00640 1 -- 172.21.15.3:0/888537836 shutdown_connections 2024-09-10T07:47:36.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.804+0000 7f3eb7e00640 1 -- 172.21.15.3:0/888537836 wait complete. 2024-09-10T07:47:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.805+0000 7f3eb7e00640 1 Processor -- start 2024-09-10T07:47:36.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.805+0000 7f3eb7e00640 1 -- start start 2024-09-10T07:47:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb01557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:36.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb015e4f0 0x7f3eb00b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb015e8d0 con 0x7f3eb006d160 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb006c960 con 0x7f3eb006bb20 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb00b6dc0 con 0x7f3eb015e4f0 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43434/0 (socket says 172.21.15.3:43434) 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.806+0000 7f3eb4a00640 1 -- 172.21.15.3:0/734540244 learned_addr learned my addr 172.21.15.3:0/734540244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb5e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb015e4f0 0x7f3eb00b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb5400640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb01557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 -- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb015e4f0 msgr2=0x7f3eb00b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb015e4f0 0x7f3eb00b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 -- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 msgr2=0x7f3eb01557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb01557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:36.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 -- 172.21.15.3:0/734540244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb00cf870 con 0x7f3eb006d160 2024-09-10T07:47:36.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb5400640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb01557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:36.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.807+0000 7f3eb4a00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0098c0 tx=0x7f3e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.808+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c041030 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.808+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c03d4e0 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.808+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c037040 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.808+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb00cfb50 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.808+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb00d24d0 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.809+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0068db0 con 0x7f3eb006d160 2024-09-10T07:47:36.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.810+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e9c044070 con 0x7f3eb006d160 2024-09-10T07:47:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.810+0000 7f3ead600640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 0x7f3e9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.810+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3e9c0ab470 con 0x7f3eb006d160 2024-09-10T07:47:36.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.811+0000 7f3eb5400640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 0x7f3e9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:36.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.811+0000 7f3eb5400640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 0x7f3e9006e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4001670 tx=0x7f3ea4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:36.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.815+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e9c079eb0 con 0x7f3eb006d160 2024-09-10T07:47:36.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:36.995+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3eb00cfd50 con 0x7f3e9006be30 2024-09-10T07:47:37.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.002+0000 7f3ead600640 1 -- 172.21.15.3:0/734540244 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f3eb00cfd50 con 0x7f3e9006be30 2024-09-10T07:47:37.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:37.006 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 msgr2=0x7f3e9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 0x7f3e9006e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4001670 tx=0x7f3ea4009040 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 msgr2=0x7f3eb0155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0098c0 tx=0x7f3e9c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 shutdown_connections 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e9006be30 0x7f3e9006e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb015e4f0 0x7f3eb00b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3eb006d160 0x7f3eb0155180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 --2- 172.21.15.3:0/734540244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb006bb20 0x7f3eb01557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 >> 172.21.15.3:0/734540244 conn(0x7f3eb006e5b0 msgr2=0x7f3eb00b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 shutdown_connections 2024-09-10T07:47:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.006+0000 7f3eb7e00640 1 -- 172.21.15.3:0/734540244 wait complete. 2024-09-10T07:47:37.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-10T07:47:35.502835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237208,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737512,"statfs":{"total":773094113280,"available":772851212288,"internally_reserved":0,"allocated":4431872,"data_stored":2423452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001636"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706604+0000","last_change":"2024-09-10T07:47:27.729428+0000","last_active":"2024-09-10T07:47:28.706604+0000","last_peered":"2024-09-10T07:47:28.706604+0000","last_clean":"2024-09-10T07:47:28.706604+0000","last_became_active":"2024-09-10T07:47:27.728866+0000","last_became_peered":"2024-09-10T07:47:27.728866+0000","last_unstale":"2024-09-10T07:47:28.706604+0000","last_undegraded":"2024-09-10T07:47:28.706604+0000","last_fullsized":"2024-09-10T07:47:28.706604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709608+0000","last_change":"2024-09-10T07:47:27.717295+0000","last_active":"2024-09-10T07:47:28.709608+0000","last_peered":"2024-09-10T07:47:28.709608+0000","last_clean":"2024-09-10T07:47:28.709608+0000","last_became_active":"2024-09-10T07:47:27.716979+0000","last_became_peered":"2024-09-10T07:47:27.716979+0000","last_unstale":"2024-09-10T07:47:28.709608+0000","last_undegraded":"2024-09-10T07:47:28.709608+0000","last_fullsized":"2024-09-10T07:47:28.709608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271803+0000","last_change":"2024-09-10T07:47:27.721625+0000","last_active":"2024-09-10T07:47:29.271803+0000","last_peered":"2024-09-10T07:47:29.271803+0000","last_clean":"2024-09-10T07:47:29.271803+0000","last_became_active":"2024-09-10T07:47:27.720961+0000","last_became_peered":"2024-09-10T07:47:27.720961+0000","last_unstale":"2024-09-10T07:47:29.271803+0000","last_undegraded":"2024-09-10T07:47:29.271803+0000","last_fullsized":"2024-09-10T07:47:29.271803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272644+0000","last_change":"2024-09-10T07:47:27.726560+0000","last_active":"2024-09-10T07:47:29.272644+0000","last_peered":"2024-09-10T07:47:29.272644+0000","last_clean":"2024-09-10T07:47:29.272644+0000","last_became_active":"2024-09-10T07:47:27.726024+0000","last_became_peered":"2024-09-10T07:47:27.726024+0000","last_unstale":"2024-09-10T07:47:29.272644+0000","last_undegraded":"2024-09-10T07:47:29.272644+0000","last_fullsized":"2024-09-10T07:47:29.272644+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:21.164681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709319+0000","last_change":"2024-09-10T07:47:27.710914+0000","last_active":"2024-09-10T07:47:28.709319+0000","last_peered":"2024-09-10T07:47:28.709319+0000","last_clean":"2024-09-10T07:47:28.709319+0000","last_became_active":"2024-09-10T07:47:27.710617+0000","last_became_peered":"2024-09-10T07:47:27.710617+0000","last_unstale":"2024-09-10T07:47:28.709319+0000","last_undegraded":"2024-09-10T07:47:28.709319+0000","last_fullsized":"2024-09-10T07:47:28.709319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706883+0000","last_change":"2024-09-10T07:47:27.721104+0000","last_active":"2024-09-10T07:47:28.706883+0000","last_peered":"2024-09-10T07:47:28.706883+0000","last_clean":"2024-09-10T07:47:28.706883+0000","last_became_active":"2024-09-10T07:47:27.718624+0000","last_became_peered":"2024-09-10T07:47:27.718624+0000","last_unstale":"2024-09-10T07:47:28.706883+0000","last_undegraded":"2024-09-10T07:47:28.706883+0000","last_fullsized":"2024-09-10T07:47:28.706883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:33.168167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705676+0000","last_change":"2024-09-10T07:47:27.725374+0000","last_active":"2024-09-10T07:47:28.705676+0000","last_peered":"2024-09-10T07:47:28.705676+0000","last_clean":"2024-09-10T07:47:28.705676+0000","last_became_active":"2024-09-10T07:47:27.724325+0000","last_became_peered":"2024-09-10T07:47:27.724325+0000","last_unstale":"2024-09-10T07:47:28.705676+0000","last_undegraded":"2024-09-10T07:47:28.705676+0000","last_fullsized":"2024-09-10T07:47:28.705676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705533+0000","last_change":"2024-09-10T07:47:27.714209+0000","last_active":"2024-09-10T07:47:28.705533+0000","last_peered":"2024-09-10T07:47:28.705533+0000","last_clean":"2024-09-10T07:47:28.705533+0000","last_became_active":"2024-09-10T07:47:27.713549+0000","last_became_peered":"2024-09-10T07:47:27.713549+0000","last_unstale":"2024-09-10T07:47:28.705533+0000","last_undegraded":"2024-09-10T07:47:28.705533+0000","last_fullsized":"2024-09-10T07:47:28.705533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:34.349282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271791+0000","last_change":"2024-09-10T07:47:27.727604+0000","last_active":"2024-09-10T07:47:29.271791+0000","last_peered":"2024-09-10T07:47:29.271791+0000","last_clean":"2024-09-10T07:47:29.271791+0000","last_became_active":"2024-09-10T07:47:27.727343+0000","last_became_peered":"2024-09-10T07:47:27.727343+0000","last_unstale":"2024-09-10T07:47:29.271791+0000","last_undegraded":"2024-09-10T07:47:29.271791+0000","last_fullsized":"2024-09-10T07:47:29.271791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272974+0000","last_change":"2024-09-10T07:47:27.717546+0000","last_active":"2024-09-10T07:47:29.272974+0000","last_peered":"2024-09-10T07:47:29.272974+0000","last_clean":"2024-09-10T07:47:29.272974+0000","last_became_active":"2024-09-10T07:47:27.713266+0000","last_became_peered":"2024-09-10T07:47:27.713266+0000","last_unstale":"2024-09-10T07:47:29.272974+0000","last_undegraded":"2024-09-10T07:47:29.272974+0000","last_fullsized":"2024-09-10T07:47:29.272974+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:36.714001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707274+0000","last_change":"2024-09-10T07:47:27.712020+0000","last_active":"2024-09-10T07:47:28.707274+0000","last_peered":"2024-09-10T07:47:28.707274+0000","last_clean":"2024-09-10T07:47:28.707274+0000","last_became_active":"2024-09-10T07:47:27.711136+0000","last_became_peered":"2024-09-10T07:47:27.711136+0000","last_unstale":"2024-09-10T07:47:28.707274+0000","last_undegraded":"2024-09-10T07:47:28.707274+0000","last_fullsized":"2024-09-10T07:47:28.707274+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:34.125368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271597+0000","last_change":"2024-09-10T07:47:27.712313+0000","last_active":"2024-09-10T07:47:29.271597+0000","last_peered":"2024-09-10T07:47:29.271597+0000","last_clean":"2024-09-10T07:47:29.271597+0000","last_became_active":"2024-09-10T07:47:27.712045+0000","last_became_peered":"2024-09-10T07:47:27.712045+0000","last_unstale":"2024-09-10T07:47:29.271597+0000","last_undegraded":"2024-09-10T07:47:29.271597+0000","last_fullsized":"2024-09-10T07:47:29.271597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:45.223540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271853+0000","last_change":"2024-09-10T07:47:27.728693+0000","last_active":"2024-09-10T07:47:29.271853+0000","last_peered":"2024-09-10T07:47:29.271853+0000","last_clean":"2024-09-10T07:47:29.271853+0000","last_became_active":"2024-09-10T07:47:27.728426+0000","last_became_peered":"2024-09-10T07:47:27.728426+0000","last_unstale":"2024-09-10T07:47:29.271853+0000","last_undegraded":"2024-09-10T07:47:29.271853+0000","last_fullsized":"2024-09-10T07:47:29.271853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.466215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272359+0000","last_change":"2024-09-10T07:47:27.713960+0000","last_active":"2024-09-10T07:47:29.272359+0000","last_peered":"2024-09-10T07:47:29.272359+0000","last_clean":"2024-09-10T07:47:29.272359+0000","last_became_active":"2024-09-10T07:47:27.713692+0000","last_became_peered":"2024-09-10T07:47:27.713692+0000","last_unstale":"2024-09-10T07:47:29.272359+0000","last_undegraded":"2024-09-10T07:47:29.272359+0000","last_fullsized":"2024-09-10T07:47:29.272359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:28.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271162+0000","last_change":"2024-09-10T07:47:27.720587+0000","last_active":"2024-09-10T07:47:29.271162+0000","last_peered":"2024-09-10T07:47:29.271162+0000","last_clean":"2024-09-10T07:47:29.271162+0000","last_became_active":"2024-09-10T07:47:27.720323+0000","last_became_peered":"2024-09-10T07:47:27.720323+0000","last_unstale":"2024-09-10T07:47:29.271162+0000","last_undegraded":"2024-09-10T07:47:29.271162+0000","last_fullsized":"2024-09-10T07:47:29.271162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271457+0000","last_change":"2024-09-10T07:47:27.710943+0000","last_active":"2024-09-10T07:47:29.271457+0000","last_peered":"2024-09-10T07:47:29.271457+0000","last_clean":"2024-09-10T07:47:29.271457+0000","last_became_active":"2024-09-10T07:47:27.710633+0000","last_became_peered":"2024-09-10T07:47:27.710633+0000","last_unstale":"2024-09-10T07:47:29.271457+0000","last_undegraded":"2024-09-10T07:47:29.271457+0000","last_fullsized":"2024-09-10T07:47:29.271457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707339+0000","last_change":"2024-09-10T07:47:27.708746+0000","last_active":"2024-09-10T07:47:28.707339+0000","last_peered":"2024-09-10T07:47:28.707339+0000","last_clean":"2024-09-10T07:47:28.707339+0000","last_became_active":"2024-09-10T07:47:27.708477+0000","last_became_peered":"2024-09-10T07:47:27.708477+0000","last_unstale":"2024-09-10T07:47:28.707339+0000","last_undegraded":"2024-09-10T07:47:28.707339+0000","last_fullsized":"2024-09-10T07:47:28.707339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:11.614613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707015+0000","last_change":"2024-09-10T07:47:27.717429+0000","last_active":"2024-09-10T07:47:28.707015+0000","last_peered":"2024-09-10T07:47:28.707015+0000","last_clean":"2024-09-10T07:47:28.707015+0000","last_became_active":"2024-09-10T07:47:27.717064+0000","last_became_peered":"2024-09-10T07:47:27.717064+0000","last_unstale":"2024-09-10T07:47:28.707015+0000","last_undegraded":"2024-09-10T07:47:28.707015+0000","last_fullsized":"2024-09-10T07:47:28.707015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:44.926594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706888+0000","last_change":"2024-09-10T07:47:27.715748+0000","last_active":"2024-09-10T07:47:28.706888+0000","last_peered":"2024-09-10T07:47:28.706888+0000","last_clean":"2024-09-10T07:47:28.706888+0000","last_became_active":"2024-09-10T07:47:27.715354+0000","last_became_peered":"2024-09-10T07:47:27.715354+0000","last_unstale":"2024-09-10T07:47:28.706888+0000","last_undegraded":"2024-09-10T07:47:28.706888+0000","last_fullsized":"2024-09-10T07:47:28.706888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:21.599703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272066+0000","last_change":"2024-09-10T07:47:27.726942+0000","last_active":"2024-09-10T07:47:29.272066+0000","last_peered":"2024-09-10T07:47:29.272066+0000","last_clean":"2024-09-10T07:47:29.272066+0000","last_became_active":"2024-09-10T07:47:27.726536+0000","last_became_peered":"2024-09-10T07:47:27.726536+0000","last_unstale":"2024-09-10T07:47:29.272066+0000","last_undegraded":"2024-09-10T07:47:29.272066+0000","last_fullsized":"2024-09-10T07:47:29.272066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:25.590064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271579+0000","last_change":"2024-09-10T07:47:27.715018+0000","last_active":"2024-09-10T07:47:29.271579+0000","last_peered":"2024-09-10T07:47:29.271579+0000","last_clean":"2024-09-10T07:47:29.271579+0000","last_became_active":"2024-09-10T07:47:27.712087+0000","last_became_peered":"2024-09-10T07:47:27.712087+0000","last_unstale":"2024-09-10T07:47:29.271579+0000","last_undegraded":"2024-09-10T07:47:29.271579+0000","last_fullsized":"2024-09-10T07:47:29.271579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706943+0000","last_change":"2024-09-10T07:47:27.723476+0000","last_active":"2024-09-10T07:47:28.706943+0000","last_peered":"2024-09-10T07:47:28.706943+0000","last_clean":"2024-09-10T07:47:28.706943+0000","last_became_active":"2024-09-10T07:47:27.723095+0000","last_became_peered":"2024-09-10T07:47:27.723095+0000","last_unstale":"2024-09-10T07:47:28.706943+0000","last_undegraded":"2024-09-10T07:47:28.706943+0000","last_fullsized":"2024-09-10T07:47:28.706943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271941+0000","last_change":"2024-09-10T07:47:27.721416+0000","last_active":"2024-09-10T07:47:29.271941+0000","last_peered":"2024-09-10T07:47:29.271941+0000","last_clean":"2024-09-10T07:47:29.271941+0000","last_became_active":"2024-09-10T07:47:27.721174+0000","last_became_peered":"2024-09-10T07:47:27.721174+0000","last_unstale":"2024-09-10T07:47:29.271941+0000","last_undegraded":"2024-09-10T07:47:29.271941+0000","last_fullsized":"2024-09-10T07:47:29.271941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:17.894548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705499+0000","last_change":"2024-09-10T07:47:27.721201+0000","last_active":"2024-09-10T07:47:28.705499+0000","last_peered":"2024-09-10T07:47:28.705499+0000","last_clean":"2024-09-10T07:47:28.705499+0000","last_became_active":"2024-09-10T07:47:27.720853+0000","last_became_peered":"2024-09-10T07:47:27.720853+0000","last_unstale":"2024-09-10T07:47:28.705499+0000","last_undegraded":"2024-09-10T07:47:28.705499+0000","last_fullsized":"2024-09-10T07:47:28.705499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272874+0000","last_change":"2024-09-10T07:47:27.723555+0000","last_active":"2024-09-10T07:47:29.272874+0000","last_peered":"2024-09-10T07:47:29.272874+0000","last_clean":"2024-09-10T07:47:29.272874+0000","last_became_active":"2024-09-10T07:47:27.723226+0000","last_became_peered":"2024-09-10T07:47:27.723226+0000","last_unstale":"2024-09-10T07:47:29.272874+0000","last_undegraded":"2024-09-10T07:47:29.272874+0000","last_fullsized":"2024-09-10T07:47:29.272874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:50.385784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.273011+0000","last_change":"2024-09-10T07:47:27.729136+0000","last_active":"2024-09-10T07:47:29.273011+0000","last_peered":"2024-09-10T07:47:29.273011+0000","last_clean":"2024-09-10T07:47:29.273011+0000","last_became_active":"2024-09-10T07:47:27.724907+0000","last_became_peered":"2024-09-10T07:47:27.724907+0000","last_unstale":"2024-09-10T07:47:29.273011+0000","last_undegraded":"2024-09-10T07:47:29.273011+0000","last_fullsized":"2024-09-10T07:47:29.273011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:33:27.535922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709383+0000","last_change":"2024-09-10T07:47:27.710661+0000","last_active":"2024-09-10T07:47:28.709383+0000","last_peered":"2024-09-10T07:47:28.709383+0000","last_clean":"2024-09-10T07:47:28.709383+0000","last_became_active":"2024-09-10T07:47:27.710117+0000","last_became_peered":"2024-09-10T07:47:27.710117+0000","last_unstale":"2024-09-10T07:47:28.709383+0000","last_undegraded":"2024-09-10T07:47:28.709383+0000","last_fullsized":"2024-09-10T07:47:28.709383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706404+0000","last_change":"2024-09-10T07:47:27.713860+0000","last_active":"2024-09-10T07:47:28.706404+0000","last_peered":"2024-09-10T07:47:28.706404+0000","last_clean":"2024-09-10T07:47:28.706404+0000","last_became_active":"2024-09-10T07:47:27.713141+0000","last_became_peered":"2024-09-10T07:47:27.713141+0000","last_unstale":"2024-09-10T07:47:28.706404+0000","last_undegraded":"2024-09-10T07:47:28.706404+0000","last_fullsized":"2024-09-10T07:47:28.706404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271665+0000","last_change":"2024-09-10T07:47:27.704751+0000","last_active":"2024-09-10T07:47:29.271665+0000","last_peered":"2024-09-10T07:47:29.271665+0000","last_clean":"2024-09-10T07:47:29.271665+0000","last_became_active":"2024-09-10T07:47:27.704476+0000","last_became_peered":"2024-09-10T07:47:27.704476+0000","last_unstale":"2024-09-10T07:47:29.271665+0000","last_undegraded":"2024-09-10T07:47:29.271665+0000","last_fullsized":"2024-09-10T07:47:29.271665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271185+0000","last_change":"2024-09-10T07:47:27.697345+0000","last_active":"2024-09-10T07:47:29.271185+0000","last_peered":"2024-09-10T07:47:29.271185+0000","last_clean":"2024-09-10T07:47:29.271185+0000","last_became_active":"2024-09-10T07:47:27.697053+0000","last_became_peered":"2024-09-10T07:47:27.697053+0000","last_unstale":"2024-09-10T07:47:29.271185+0000","last_undegraded":"2024-09-10T07:47:29.271185+0000","last_fullsized":"2024-09-10T07:47:29.271185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:21.227039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705720+0000","last_change":"2024-09-10T07:47:27.718500+0000","last_active":"2024-09-10T07:47:28.705720+0000","last_peered":"2024-09-10T07:47:28.705720+0000","last_clean":"2024-09-10T07:47:28.705720+0000","last_became_active":"2024-09-10T07:47:27.718085+0000","last_became_peered":"2024-09-10T07:47:27.718085+0000","last_unstale":"2024-09-10T07:47:28.705720+0000","last_undegraded":"2024-09-10T07:47:28.705720+0000","last_fullsized":"2024-09-10T07:47:28.705720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:21.227039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706435+0000","last_change":"2024-09-10T07:47:27.708748+0000","last_active":"2024-09-10T07:47:28.706435+0000","last_peered":"2024-09-10T07:47:28.706435+0000","last_clean":"2024-09-10T07:47:28.706435+0000","last_became_active":"2024-09-10T07:47:27.708389+0000","last_became_peered":"2024-09-10T07:47:27.708389+0000","last_unstale":"2024-09-10T07:47:28.706435+0000","last_undegraded":"2024-09-10T07:47:28.706435+0000","last_fullsized":"2024-09-10T07:47:28.706435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:23.217293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272289+0000","last_change":"2024-09-10T07:47:27.714639+0000","last_active":"2024-09-10T07:47:29.272289+0000","last_peered":"2024-09-10T07:47:29.272289+0000","last_clean":"2024-09-10T07:47:29.272289+0000","last_became_active":"2024-09-10T07:47:27.714372+0000","last_became_peered":"2024-09-10T07:47:27.714372+0000","last_unstale":"2024-09-10T07:47:29.272289+0000","last_undegraded":"2024-09-10T07:47:29.272289+0000","last_fullsized":"2024-09-10T07:47:29.272289+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:36.859521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707444+0000","last_change":"2024-09-10T07:47:26.711007+0000","last_active":"2024-09-10T07:47:28.707444+0000","last_peered":"2024-09-10T07:47:28.707444+0000","last_clean":"2024-09-10T07:47:28.707444+0000","last_became_active":"2024-09-10T07:47:26.710707+0000","last_became_peered":"2024-09-10T07:47:26.710707+0000","last_unstale":"2024-09-10T07:47:28.707444+0000","last_undegraded":"2024-09-10T07:47:28.707444+0000","last_fullsized":"2024-09-10T07:47:28.707444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:53.495126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706686+0000","last_change":"2024-09-10T07:47:27.714896+0000","last_active":"2024-09-10T07:47:28.706686+0000","last_peered":"2024-09-10T07:47:28.706686+0000","last_clean":"2024-09-10T07:47:28.706686+0000","last_became_active":"2024-09-10T07:47:27.714300+0000","last_became_peered":"2024-09-10T07:47:27.714300+0000","last_unstale":"2024-09-10T07:47:28.706686+0000","last_undegraded":"2024-09-10T07:47:28.706686+0000","last_fullsized":"2024-09-10T07:47:28.706686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:26.738356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.273059+0000","last_change":"2024-09-10T07:47:26.717356+0000","last_active":"2024-09-10T07:47:29.273059+0000","last_peered":"2024-09-10T07:47:29.273059+0000","last_clean":"2024-09-10T07:47:29.273059+0000","last_became_active":"2024-09-10T07:47:26.717075+0000","last_became_peered":"2024-09-10T07:47:26.717075+0000","last_unstale":"2024-09-10T07:47:29.273059+0000","last_undegraded":"2024-09-10T07:47:29.273059+0000","last_fullsized":"2024-09-10T07:47:29.273059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:23.907465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706720+0000","last_change":"2024-09-10T07:47:27.723254+0000","last_active":"2024-09-10T07:47:28.706720+0000","last_peered":"2024-09-10T07:47:28.706720+0000","last_clean":"2024-09-10T07:47:28.706720+0000","last_became_active":"2024-09-10T07:47:27.722364+0000","last_became_peered":"2024-09-10T07:47:27.722364+0000","last_unstale":"2024-09-10T07:47:28.706720+0000","last_undegraded":"2024-09-10T07:47:28.706720+0000","last_fullsized":"2024-09-10T07:47:28.706720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:12.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707037+0000","last_change":"2024-09-10T07:47:26.709026+0000","last_active":"2024-09-10T07:47:28.707037+0000","last_peered":"2024-09-10T07:47:28.707037+0000","last_clean":"2024-09-10T07:47:28.707037+0000","last_became_active":"2024-09-10T07:47:26.708550+0000","last_became_peered":"2024-09-10T07:47:26.708550+0000","last_unstale":"2024-09-10T07:47:28.707037+0000","last_undegraded":"2024-09-10T07:47:28.707037+0000","last_fullsized":"2024-09-10T07:47:28.707037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:10.813450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272862+0000","last_change":"2024-09-10T07:47:27.722622+0000","last_active":"2024-09-10T07:47:29.272862+0000","last_peered":"2024-09-10T07:47:29.272862+0000","last_clean":"2024-09-10T07:47:29.272862+0000","last_became_active":"2024-09-10T07:47:27.722182+0000","last_became_peered":"2024-09-10T07:47:27.722182+0000","last_unstale":"2024-09-10T07:47:29.272862+0000","last_undegraded":"2024-09-10T07:47:29.272862+0000","last_fullsized":"2024-09-10T07:47:29.272862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:45.898938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272862+0000","last_change":"2024-09-10T07:47:26.714730+0000","last_active":"2024-09-10T07:47:29.272862+0000","last_peered":"2024-09-10T07:47:29.272862+0000","last_clean":"2024-09-10T07:47:29.272862+0000","last_became_active":"2024-09-10T07:47:26.713780+0000","last_became_peered":"2024-09-10T07:47:26.713780+0000","last_unstale":"2024-09-10T07:47:29.272862+0000","last_undegraded":"2024-09-10T07:47:29.272862+0000","last_fullsized":"2024-09-10T07:47:29.272862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:59.575014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271288+0000","last_change":"2024-09-10T07:47:27.716983+0000","last_active":"2024-09-10T07:47:29.271288+0000","last_peered":"2024-09-10T07:47:29.271288+0000","last_clean":"2024-09-10T07:47:29.271288+0000","last_became_active":"2024-09-10T07:47:27.716698+0000","last_became_peered":"2024-09-10T07:47:27.716698+0000","last_unstale":"2024-09-10T07:47:29.271288+0000","last_undegraded":"2024-09-10T07:47:29.271288+0000","last_fullsized":"2024-09-10T07:47:29.271288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:56.241795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705902+0000","last_change":"2024-09-10T07:47:26.711406+0000","last_active":"2024-09-10T07:47:28.705902+0000","last_peered":"2024-09-10T07:47:28.705902+0000","last_clean":"2024-09-10T07:47:28.705902+0000","last_became_active":"2024-09-10T07:47:26.710814+0000","last_became_peered":"2024-09-10T07:47:26.710814+0000","last_unstale":"2024-09-10T07:47:28.705902+0000","last_undegraded":"2024-09-10T07:47:28.705902+0000","last_fullsized":"2024-09-10T07:47:28.705902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705811+0000","last_change":"2024-09-10T07:47:27.714326+0000","last_active":"2024-09-10T07:47:28.705811+0000","last_peered":"2024-09-10T07:47:28.705811+0000","last_clean":"2024-09-10T07:47:28.705811+0000","last_became_active":"2024-09-10T07:47:27.714018+0000","last_became_peered":"2024-09-10T07:47:27.714018+0000","last_unstale":"2024-09-10T07:47:28.705811+0000","last_undegraded":"2024-09-10T07:47:28.705811+0000","last_fullsized":"2024-09-10T07:47:28.705811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706826+0000","last_change":"2024-09-10T07:47:26.713426+0000","last_active":"2024-09-10T07:47:28.706826+0000","last_peered":"2024-09-10T07:47:28.706826+0000","last_clean":"2024-09-10T07:47:28.706826+0000","last_became_active":"2024-09-10T07:47:26.712028+0000","last_became_peered":"2024-09-10T07:47:26.712028+0000","last_unstale":"2024-09-10T07:47:28.706826+0000","last_undegraded":"2024-09-10T07:47:28.706826+0000","last_fullsized":"2024-09-10T07:47:28.706826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271344+0000","last_change":"2024-09-10T07:47:27.730588+0000","last_active":"2024-09-10T07:47:29.271344+0000","last_peered":"2024-09-10T07:47:29.271344+0000","last_clean":"2024-09-10T07:47:29.271344+0000","last_became_active":"2024-09-10T07:47:27.730242+0000","last_became_peered":"2024-09-10T07:47:27.730242+0000","last_unstale":"2024-09-10T07:47:29.271344+0000","last_undegraded":"2024-09-10T07:47:29.271344+0000","last_fullsized":"2024-09-10T07:47:29.271344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:14.309793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706730+0000","last_change":"2024-09-10T07:47:26.701393+0000","last_active":"2024-09-10T07:47:28.706730+0000","last_peered":"2024-09-10T07:47:28.706730+0000","last_clean":"2024-09-10T07:47:28.706730+0000","last_became_active":"2024-09-10T07:47:26.701105+0000","last_became_peered":"2024-09-10T07:47:26.701105+0000","last_unstale":"2024-09-10T07:47:28.706730+0000","last_undegraded":"2024-09-10T07:47:28.706730+0000","last_fullsized":"2024-09-10T07:47:28.706730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:58.899886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272044+0000","last_change":"2024-09-10T07:47:27.708836+0000","last_active":"2024-09-10T07:47:29.272044+0000","last_peered":"2024-09-10T07:47:29.272044+0000","last_clean":"2024-09-10T07:47:29.272044+0000","last_became_active":"2024-09-10T07:47:27.708519+0000","last_became_peered":"2024-09-10T07:47:27.708519+0000","last_unstale":"2024-09-10T07:47:29.272044+0000","last_undegraded":"2024-09-10T07:47:29.272044+0000","last_fullsized":"2024-09-10T07:47:29.272044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:37.104237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705632+0000","last_change":"2024-09-10T07:47:26.713761+0000","last_active":"2024-09-10T07:47:28.705632+0000","last_peered":"2024-09-10T07:47:28.705632+0000","last_clean":"2024-09-10T07:47:28.705632+0000","last_became_active":"2024-09-10T07:47:26.713497+0000","last_became_peered":"2024-09-10T07:47:26.713497+0000","last_unstale":"2024-09-10T07:47:28.705632+0000","last_undegraded":"2024-09-10T07:47:28.705632+0000","last_fullsized":"2024-09-10T07:47:28.705632+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707056+0000","last_change":"2024-09-10T07:47:27.717917+0000","last_active":"2024-09-10T07:47:28.707056+0000","last_peered":"2024-09-10T07:47:28.707056+0000","last_clean":"2024-09-10T07:47:28.707056+0000","last_became_active":"2024-09-10T07:47:27.717135+0000","last_became_peered":"2024-09-10T07:47:27.717135+0000","last_unstale":"2024-09-10T07:47:28.707056+0000","last_undegraded":"2024-09-10T07:47:28.707056+0000","last_fullsized":"2024-09-10T07:47:28.707056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:35.220570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709924+0000","last_change":"2024-09-10T07:47:26.716721+0000","last_active":"2024-09-10T07:47:28.709924+0000","last_peered":"2024-09-10T07:47:28.709924+0000","last_clean":"2024-09-10T07:47:28.709924+0000","last_became_active":"2024-09-10T07:47:26.716429+0000","last_became_peered":"2024-09-10T07:47:26.716429+0000","last_unstale":"2024-09-10T07:47:28.709924+0000","last_undegraded":"2024-09-10T07:47:28.709924+0000","last_fullsized":"2024-09-10T07:47:28.709924+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272981+0000","last_change":"2024-09-10T07:47:27.721272+0000","last_active":"2024-09-10T07:47:29.272981+0000","last_peered":"2024-09-10T07:47:29.272981+0000","last_clean":"2024-09-10T07:47:29.272981+0000","last_became_active":"2024-09-10T07:47:27.718319+0000","last_became_peered":"2024-09-10T07:47:27.718319+0000","last_unstale":"2024-09-10T07:47:29.272981+0000","last_undegraded":"2024-09-10T07:47:29.272981+0000","last_fullsized":"2024-09-10T07:47:29.272981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:45.917831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709462+0000","last_change":"2024-09-10T07:47:26.713954+0000","last_active":"2024-09-10T07:47:28.709462+0000","last_peered":"2024-09-10T07:47:28.709462+0000","last_clean":"2024-09-10T07:47:28.709462+0000","last_became_active":"2024-09-10T07:47:26.713378+0000","last_became_peered":"2024-09-10T07:47:26.713378+0000","last_unstale":"2024-09-10T07:47:28.709462+0000","last_undegraded":"2024-09-10T07:47:28.709462+0000","last_fullsized":"2024-09-10T07:47:28.709462+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:03.461580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272722+0000","last_change":"2024-09-10T07:47:27.724717+0000","last_active":"2024-09-10T07:47:29.272722+0000","last_peered":"2024-09-10T07:47:29.272722+0000","last_clean":"2024-09-10T07:47:29.272722+0000","last_became_active":"2024-09-10T07:47:27.724464+0000","last_became_peered":"2024-09-10T07:47:27.724464+0000","last_unstale":"2024-09-10T07:47:29.272722+0000","last_undegraded":"2024-09-10T07:47:29.272722+0000","last_fullsized":"2024-09-10T07:47:29.272722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:34.310092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707179+0000","last_change":"2024-09-10T07:47:26.709280+0000","last_active":"2024-09-10T07:47:28.707179+0000","last_peered":"2024-09-10T07:47:28.707179+0000","last_clean":"2024-09-10T07:47:28.707179+0000","last_became_active":"2024-09-10T07:47:26.708960+0000","last_became_peered":"2024-09-10T07:47:26.708960+0000","last_unstale":"2024-09-10T07:47:28.707179+0000","last_undegraded":"2024-09-10T07:47:28.707179+0000","last_fullsized":"2024-09-10T07:47:28.707179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:03.869247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707230+0000","last_change":"2024-09-10T07:47:27.718510+0000","last_active":"2024-09-10T07:47:28.707230+0000","last_peered":"2024-09-10T07:47:28.707230+0000","last_clean":"2024-09-10T07:47:28.707230+0000","last_became_active":"2024-09-10T07:47:27.717326+0000","last_became_peered":"2024-09-10T07:47:27.717326+0000","last_unstale":"2024-09-10T07:47:28.707230+0000","last_undegraded":"2024-09-10T07:47:28.707230+0000","last_fullsized":"2024-09-10T07:47:28.707230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.464775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272123+0000","last_change":"2024-09-10T07:47:26.713112+0000","last_active":"2024-09-10T07:47:29.272123+0000","last_peered":"2024-09-10T07:47:29.272123+0000","last_clean":"2024-09-10T07:47:29.272123+0000","last_became_active":"2024-09-10T07:47:26.712737+0000","last_became_peered":"2024-09-10T07:47:26.712737+0000","last_unstale":"2024-09-10T07:47:29.272123+0000","last_undegraded":"2024-09-10T07:47:29.272123+0000","last_fullsized":"2024-09-10T07:47:29.272123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:55.237159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271575+0000","last_change":"2024-09-10T07:47:27.722758+0000","last_active":"2024-09-10T07:47:29.271575+0000","last_peered":"2024-09-10T07:47:29.271575+0000","last_clean":"2024-09-10T07:47:29.271575+0000","last_became_active":"2024-09-10T07:47:27.722517+0000","last_became_peered":"2024-09-10T07:47:27.722517+0000","last_unstale":"2024-09-10T07:47:29.271575+0000","last_undegraded":"2024-09-10T07:47:29.271575+0000","last_fullsized":"2024-09-10T07:47:29.271575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:07.008875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706000+0000","last_change":"2024-09-10T07:47:26.713271+0000","last_active":"2024-09-10T07:47:28.706000+0000","last_peered":"2024-09-10T07:47:28.706000+0000","last_clean":"2024-09-10T07:47:28.706000+0000","last_became_active":"2024-09-10T07:47:26.712715+0000","last_became_peered":"2024-09-10T07:47:26.712715+0000","last_unstale":"2024-09-10T07:47:28.706000+0000","last_undegraded":"2024-09-10T07:47:28.706000+0000","last_fullsized":"2024-09-10T07:47:28.706000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:47.147053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272354+0000","last_change":"2024-09-10T07:47:27.725965+0000","last_active":"2024-09-10T07:47:29.272354+0000","last_peered":"2024-09-10T07:47:29.272354+0000","last_clean":"2024-09-10T07:47:29.272354+0000","last_became_active":"2024-09-10T07:47:27.725029+0000","last_became_peered":"2024-09-10T07:47:27.725029+0000","last_unstale":"2024-09-10T07:47:29.272354+0000","last_undegraded":"2024-09-10T07:47:29.272354+0000","last_fullsized":"2024-09-10T07:47:29.272354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:23.318762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272347+0000","last_change":"2024-09-10T07:47:26.711747+0000","last_active":"2024-09-10T07:47:29.272347+0000","last_peered":"2024-09-10T07:47:29.272347+0000","last_clean":"2024-09-10T07:47:29.272347+0000","last_became_active":"2024-09-10T07:47:26.711458+0000","last_became_peered":"2024-09-10T07:47:26.711458+0000","last_unstale":"2024-09-10T07:47:29.272347+0000","last_undegraded":"2024-09-10T07:47:29.272347+0000","last_fullsized":"2024-09-10T07:47:29.272347+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:17.886142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.705655+0000","last_change":"2024-09-10T07:47:27.720915+0000","last_active":"2024-09-10T07:47:28.705655+0000","last_peered":"2024-09-10T07:47:28.705655+0000","last_clean":"2024-09-10T07:47:28.705655+0000","last_became_active":"2024-09-10T07:47:27.720654+0000","last_became_peered":"2024-09-10T07:47:27.720654+0000","last_unstale":"2024-09-10T07:47:28.705655+0000","last_undegraded":"2024-09-10T07:47:28.705655+0000","last_fullsized":"2024-09-10T07:47:28.705655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:07:55.153398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272060+0000","last_change":"2024-09-10T07:47:26.708509+0000","last_active":"2024-09-10T07:47:29.272060+0000","last_peered":"2024-09-10T07:47:29.272060+0000","last_clean":"2024-09-10T07:47:29.272060+0000","last_became_active":"2024-09-10T07:47:26.708220+0000","last_became_peered":"2024-09-10T07:47:26.708220+0000","last_unstale":"2024-09-10T07:47:29.272060+0000","last_undegraded":"2024-09-10T07:47:29.272060+0000","last_fullsized":"2024-09-10T07:47:29.272060+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:55.494668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706893+0000","last_change":"2024-09-10T07:47:27.719995+0000","last_active":"2024-09-10T07:47:28.706893+0000","last_peered":"2024-09-10T07:47:28.706893+0000","last_clean":"2024-09-10T07:47:28.706893+0000","last_became_active":"2024-09-10T07:47:27.719604+0000","last_became_peered":"2024-09-10T07:47:27.719604+0000","last_unstale":"2024-09-10T07:47:28.706893+0000","last_undegraded":"2024-09-10T07:47:28.706893+0000","last_fullsized":"2024-09-10T07:47:28.706893+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:47:23.659389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706491+0000","last_change":"2024-09-10T07:47:26.706290+0000","last_active":"2024-09-10T07:47:28.706491+0000","last_peered":"2024-09-10T07:47:28.706491+0000","last_clean":"2024-09-10T07:47:28.706491+0000","last_became_active":"2024-09-10T07:47:26.705979+0000","last_became_peered":"2024-09-10T07:47:26.705979+0000","last_unstale":"2024-09-10T07:47:28.706491+0000","last_undegraded":"2024-09-10T07:47:28.706491+0000","last_fullsized":"2024-09-10T07:47:28.706491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.059130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707203+0000","last_change":"2024-09-10T07:47:27.717619+0000","last_active":"2024-09-10T07:47:28.707203+0000","last_peered":"2024-09-10T07:47:28.707203+0000","last_clean":"2024-09-10T07:47:28.707203+0000","last_became_active":"2024-09-10T07:47:27.716747+0000","last_became_peered":"2024-09-10T07:47:27.716747+0000","last_unstale":"2024-09-10T07:47:28.707203+0000","last_undegraded":"2024-09-10T07:47:28.707203+0000","last_fullsized":"2024-09-10T07:47:28.707203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:01.440358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706848+0000","last_change":"2024-09-10T07:47:26.710001+0000","last_active":"2024-09-10T07:47:28.706848+0000","last_peered":"2024-09-10T07:47:28.706848+0000","last_clean":"2024-09-10T07:47:28.706848+0000","last_became_active":"2024-09-10T07:47:26.708545+0000","last_became_peered":"2024-09-10T07:47:26.708545+0000","last_unstale":"2024-09-10T07:47:28.706848+0000","last_undegraded":"2024-09-10T07:47:28.706848+0000","last_fullsized":"2024-09-10T07:47:28.706848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:18.788954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271880+0000","last_change":"2024-09-10T07:47:27.733598+0000","last_active":"2024-09-10T07:47:29.271880+0000","last_peered":"2024-09-10T07:47:29.271880+0000","last_clean":"2024-09-10T07:47:29.271880+0000","last_became_active":"2024-09-10T07:47:27.733342+0000","last_became_peered":"2024-09-10T07:47:27.733342+0000","last_unstale":"2024-09-10T07:47:29.271880+0000","last_undegraded":"2024-09-10T07:47:29.271880+0000","last_fullsized":"2024-09-10T07:47:29.271880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706292+0000","last_change":"2024-09-10T07:47:26.713070+0000","last_active":"2024-09-10T07:47:28.706292+0000","last_peered":"2024-09-10T07:47:28.706292+0000","last_clean":"2024-09-10T07:47:28.706292+0000","last_became_active":"2024-09-10T07:47:26.712820+0000","last_became_peered":"2024-09-10T07:47:26.712820+0000","last_unstale":"2024-09-10T07:47:28.706292+0000","last_undegraded":"2024-09-10T07:47:28.706292+0000","last_fullsized":"2024-09-10T07:47:28.706292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:32.163532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709356+0000","last_change":"2024-09-10T07:47:27.712460+0000","last_active":"2024-09-10T07:47:28.709356+0000","last_peered":"2024-09-10T07:47:28.709356+0000","last_clean":"2024-09-10T07:47:28.709356+0000","last_became_active":"2024-09-10T07:47:27.712183+0000","last_became_peered":"2024-09-10T07:47:27.712183+0000","last_unstale":"2024-09-10T07:47:28.709356+0000","last_undegraded":"2024-09-10T07:47:28.709356+0000","last_fullsized":"2024-09-10T07:47:28.709356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.605537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707336+0000","last_change":"2024-09-10T07:47:26.713128+0000","last_active":"2024-09-10T07:47:28.707336+0000","last_peered":"2024-09-10T07:47:28.707336+0000","last_clean":"2024-09-10T07:47:28.707336+0000","last_became_active":"2024-09-10T07:47:26.712677+0000","last_became_peered":"2024-09-10T07:47:26.712677+0000","last_unstale":"2024-09-10T07:47:28.707336+0000","last_undegraded":"2024-09-10T07:47:28.707336+0000","last_fullsized":"2024-09-10T07:47:28.707336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:55.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706999+0000","last_change":"2024-09-10T07:47:27.724138+0000","last_active":"2024-09-10T07:47:28.706999+0000","last_peered":"2024-09-10T07:47:28.706999+0000","last_clean":"2024-09-10T07:47:28.706999+0000","last_became_active":"2024-09-10T07:47:27.723179+0000","last_became_peered":"2024-09-10T07:47:27.723179+0000","last_unstale":"2024-09-10T07:47:28.706999+0000","last_undegraded":"2024-09-10T07:47:28.706999+0000","last_fullsized":"2024-09-10T07:47:28.706999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.151689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271786+0000","last_change":"2024-09-10T07:47:26.711949+0000","last_active":"2024-09-10T07:47:29.271786+0000","last_peered":"2024-09-10T07:47:29.271786+0000","last_clean":"2024-09-10T07:47:29.271786+0000","last_became_active":"2024-09-10T07:47:26.711492+0000","last_became_peered":"2024-09-10T07:47:26.711492+0000","last_unstale":"2024-09-10T07:47:29.271786+0000","last_undegraded":"2024-09-10T07:47:29.271786+0000","last_fullsized":"2024-09-10T07:47:29.271786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:51.310397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271616+0000","last_change":"2024-09-10T07:47:27.725128+0000","last_active":"2024-09-10T07:47:29.271616+0000","last_peered":"2024-09-10T07:47:29.271616+0000","last_clean":"2024-09-10T07:47:29.271616+0000","last_became_active":"2024-09-10T07:47:27.724151+0000","last_became_peered":"2024-09-10T07:47:27.724151+0000","last_unstale":"2024-09-10T07:47:29.271616+0000","last_undegraded":"2024-09-10T07:47:29.271616+0000","last_fullsized":"2024-09-10T07:47:29.271616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:38.688543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709386+0000","last_change":"2024-09-10T07:47:26.715573+0000","last_active":"2024-09-10T07:47:28.709386+0000","last_peered":"2024-09-10T07:47:28.709386+0000","last_clean":"2024-09-10T07:47:28.709386+0000","last_became_active":"2024-09-10T07:47:26.715315+0000","last_became_peered":"2024-09-10T07:47:26.715315+0000","last_unstale":"2024-09-10T07:47:28.709386+0000","last_undegraded":"2024-09-10T07:47:28.709386+0000","last_fullsized":"2024-09-10T07:47:28.709386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:34:37.683908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709341+0000","last_change":"2024-09-10T07:47:27.701024+0000","last_active":"2024-09-10T07:47:28.709341+0000","last_peered":"2024-09-10T07:47:28.709341+0000","last_clean":"2024-09-10T07:47:28.709341+0000","last_became_active":"2024-09-10T07:47:27.700612+0000","last_became_peered":"2024-09-10T07:47:27.700612+0000","last_unstale":"2024-09-10T07:47:28.709341+0000","last_undegraded":"2024-09-10T07:47:28.709341+0000","last_fullsized":"2024-09-10T07:47:28.709341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:14.187254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706925+0000","last_change":"2024-09-10T07:47:26.707503+0000","last_active":"2024-09-10T07:47:28.706925+0000","last_peered":"2024-09-10T07:47:28.706925+0000","last_clean":"2024-09-10T07:47:28.706925+0000","last_became_active":"2024-09-10T07:47:26.706814+0000","last_became_peered":"2024-09-10T07:47:26.706814+0000","last_unstale":"2024-09-10T07:47:28.706925+0000","last_undegraded":"2024-09-10T07:47:28.706925+0000","last_fullsized":"2024-09-10T07:47:28.706925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271600+0000","last_change":"2024-09-10T07:47:27.722893+0000","last_active":"2024-09-10T07:47:29.271600+0000","last_peered":"2024-09-10T07:47:29.271600+0000","last_clean":"2024-09-10T07:47:29.271600+0000","last_became_active":"2024-09-10T07:47:27.722551+0000","last_became_peered":"2024-09-10T07:47:27.722551+0000","last_unstale":"2024-09-10T07:47:29.271600+0000","last_undegraded":"2024-09-10T07:47:29.271600+0000","last_fullsized":"2024-09-10T07:47:29.271600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:52.315032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271331+0000","last_change":"2024-09-10T07:47:26.714230+0000","last_active":"2024-09-10T07:47:29.271331+0000","last_peered":"2024-09-10T07:47:29.271331+0000","last_clean":"2024-09-10T07:47:29.271331+0000","last_became_active":"2024-09-10T07:47:26.712186+0000","last_became_peered":"2024-09-10T07:47:26.712186+0000","last_unstale":"2024-09-10T07:47:29.271331+0000","last_undegraded":"2024-09-10T07:47:29.271331+0000","last_fullsized":"2024-09-10T07:47:29.271331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:12.558388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271313+0000","last_change":"2024-09-10T07:47:27.720014+0000","last_active":"2024-09-10T07:47:29.271313+0000","last_peered":"2024-09-10T07:47:29.271313+0000","last_clean":"2024-09-10T07:47:29.271313+0000","last_became_active":"2024-09-10T07:47:27.718864+0000","last_became_peered":"2024-09-10T07:47:27.718864+0000","last_unstale":"2024-09-10T07:47:29.271313+0000","last_undegraded":"2024-09-10T07:47:29.271313+0000","last_fullsized":"2024-09-10T07:47:29.271313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:11.146080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706471+0000","last_change":"2024-09-10T07:47:26.709027+0000","last_active":"2024-09-10T07:47:28.706471+0000","last_peered":"2024-09-10T07:47:28.706471+0000","last_clean":"2024-09-10T07:47:28.706471+0000","last_became_active":"2024-09-10T07:47:26.708753+0000","last_became_peered":"2024-09-10T07:47:26.708753+0000","last_unstale":"2024-09-10T07:47:28.706471+0000","last_undegraded":"2024-09-10T07:47:28.706471+0000","last_fullsized":"2024-09-10T07:47:28.706471+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:25.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707048+0000","last_change":"2024-09-10T07:47:27.728171+0000","last_active":"2024-09-10T07:47:28.707048+0000","last_peered":"2024-09-10T07:47:28.707048+0000","last_clean":"2024-09-10T07:47:28.707048+0000","last_became_active":"2024-09-10T07:47:27.727842+0000","last_became_peered":"2024-09-10T07:47:27.727842+0000","last_unstale":"2024-09-10T07:47:28.707048+0000","last_undegraded":"2024-09-10T07:47:28.707048+0000","last_fullsized":"2024-09-10T07:47:28.707048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:45.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272065+0000","last_change":"2024-09-10T07:47:26.707810+0000","last_active":"2024-09-10T07:47:29.272065+0000","last_peered":"2024-09-10T07:47:29.272065+0000","last_clean":"2024-09-10T07:47:29.272065+0000","last_became_active":"2024-09-10T07:47:26.707462+0000","last_became_peered":"2024-09-10T07:47:26.707462+0000","last_unstale":"2024-09-10T07:47:29.272065+0000","last_undegraded":"2024-09-10T07:47:29.272065+0000","last_fullsized":"2024-09-10T07:47:29.272065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:44.218905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707444+0000","last_change":"2024-09-10T07:47:27.707117+0000","last_active":"2024-09-10T07:47:28.707444+0000","last_peered":"2024-09-10T07:47:28.707444+0000","last_clean":"2024-09-10T07:47:28.707444+0000","last_became_active":"2024-09-10T07:47:27.706809+0000","last_became_peered":"2024-09-10T07:47:27.706809+0000","last_unstale":"2024-09-10T07:47:28.707444+0000","last_undegraded":"2024-09-10T07:47:28.707444+0000","last_fullsized":"2024-09-10T07:47:28.707444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:39.128769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.709605+0000","last_change":"2024-09-10T07:47:26.706696+0000","last_active":"2024-09-10T07:47:28.709605+0000","last_peered":"2024-09-10T07:47:28.709605+0000","last_clean":"2024-09-10T07:47:28.709605+0000","last_became_active":"2024-09-10T07:47:26.706383+0000","last_became_peered":"2024-09-10T07:47:26.706383+0000","last_unstale":"2024-09-10T07:47:28.709605+0000","last_undegraded":"2024-09-10T07:47:28.709605+0000","last_fullsized":"2024-09-10T07:47:28.709605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:16.889913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707098+0000","last_change":"2024-09-10T07:47:27.721003+0000","last_active":"2024-09-10T07:47:28.707098+0000","last_peered":"2024-09-10T07:47:28.707098+0000","last_clean":"2024-09-10T07:47:28.707098+0000","last_became_active":"2024-09-10T07:47:27.720702+0000","last_became_peered":"2024-09-10T07:47:27.720702+0000","last_unstale":"2024-09-10T07:47:28.707098+0000","last_undegraded":"2024-09-10T07:47:28.707098+0000","last_fullsized":"2024-09-10T07:47:28.707098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:33.721528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272228+0000","last_change":"2024-09-10T07:47:26.701128+0000","last_active":"2024-09-10T07:47:29.272228+0000","last_peered":"2024-09-10T07:47:29.272228+0000","last_clean":"2024-09-10T07:47:29.272228+0000","last_became_active":"2024-09-10T07:47:26.700042+0000","last_became_peered":"2024-09-10T07:47:26.700042+0000","last_unstale":"2024-09-10T07:47:29.272228+0000","last_undegraded":"2024-09-10T07:47:29.272228+0000","last_fullsized":"2024-09-10T07:47:29.272228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:13.305158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271326+0000","last_change":"2024-09-10T07:47:27.711768+0000","last_active":"2024-09-10T07:47:29.271326+0000","last_peered":"2024-09-10T07:47:29.271326+0000","last_clean":"2024-09-10T07:47:29.271326+0000","last_became_active":"2024-09-10T07:47:27.709290+0000","last_became_peered":"2024-09-10T07:47:27.709290+0000","last_unstale":"2024-09-10T07:47:29.271326+0000","last_undegraded":"2024-09-10T07:47:29.271326+0000","last_fullsized":"2024-09-10T07:47:29.271326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:39.682387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271867+0000","last_change":"2024-09-10T07:47:26.705882+0000","last_active":"2024-09-10T07:47:29.271867+0000","last_peered":"2024-09-10T07:47:29.271867+0000","last_clean":"2024-09-10T07:47:29.271867+0000","last_became_active":"2024-09-10T07:47:26.705616+0000","last_became_peered":"2024-09-10T07:47:26.705616+0000","last_unstale":"2024-09-10T07:47:29.271867+0000","last_undegraded":"2024-09-10T07:47:29.271867+0000","last_fullsized":"2024-09-10T07:47:29.271867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:20.595068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.272523+0000","last_change":"2024-09-10T07:47:27.719286+0000","last_active":"2024-09-10T07:47:29.272523+0000","last_peered":"2024-09-10T07:47:29.272523+0000","last_clean":"2024-09-10T07:47:29.272523+0000","last_became_active":"2024-09-10T07:47:27.715311+0000","last_became_peered":"2024-09-10T07:47:27.715311+0000","last_unstale":"2024-09-10T07:47:29.272523+0000","last_undegraded":"2024-09-10T07:47:29.272523+0000","last_fullsized":"2024-09-10T07:47:29.272523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:23.609705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271645+0000","last_change":"2024-09-10T07:47:26.710717+0000","last_active":"2024-09-10T07:47:29.271645+0000","last_peered":"2024-09-10T07:47:29.271645+0000","last_clean":"2024-09-10T07:47:29.271645+0000","last_became_active":"2024-09-10T07:47:26.710392+0000","last_became_peered":"2024-09-10T07:47:26.710392+0000","last_unstale":"2024-09-10T07:47:29.271645+0000","last_undegraded":"2024-09-10T07:47:29.271645+0000","last_fullsized":"2024-09-10T07:47:29.271645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271871+0000","last_change":"2024-09-10T07:47:27.727229+0000","last_active":"2024-09-10T07:47:29.271871+0000","last_peered":"2024-09-10T07:47:29.271871+0000","last_clean":"2024-09-10T07:47:29.271871+0000","last_became_active":"2024-09-10T07:47:27.725550+0000","last_became_peered":"2024-09-10T07:47:27.725550+0000","last_unstale":"2024-09-10T07:47:29.271871+0000","last_undegraded":"2024-09-10T07:47:29.271871+0000","last_fullsized":"2024-09-10T07:47:29.271871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:38:25.629006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706045+0000","last_change":"2024-09-10T07:47:26.710777+0000","last_active":"2024-09-10T07:47:28.706045+0000","last_peered":"2024-09-10T07:47:28.706045+0000","last_clean":"2024-09-10T07:47:28.706045+0000","last_became_active":"2024-09-10T07:47:26.710499+0000","last_became_peered":"2024-09-10T07:47:26.710499+0000","last_unstale":"2024-09-10T07:47:28.706045+0000","last_undegraded":"2024-09-10T07:47:28.706045+0000","last_fullsized":"2024-09-10T07:47:28.706045+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.141444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.706975+0000","last_change":"2024-09-10T07:47:27.705681+0000","last_active":"2024-09-10T07:47:28.706975+0000","last_peered":"2024-09-10T07:47:28.706975+0000","last_clean":"2024-09-10T07:47:28.706975+0000","last_became_active":"2024-09-10T07:47:27.705375+0000","last_became_peered":"2024-09-10T07:47:27.705375+0000","last_unstale":"2024-09-10T07:47:28.706975+0000","last_undegraded":"2024-09-10T07:47:28.706975+0000","last_fullsized":"2024-09-10T07:47:28.706975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:40.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271828+0000","last_change":"2024-09-10T07:47:26.716992+0000","last_active":"2024-09-10T07:47:29.271828+0000","last_peered":"2024-09-10T07:47:29.271828+0000","last_clean":"2024-09-10T07:47:29.271828+0000","last_became_active":"2024-09-10T07:47:26.716690+0000","last_became_peered":"2024-09-10T07:47:26.716690+0000","last_unstale":"2024-09-10T07:47:29.271828+0000","last_undegraded":"2024-09-10T07:47:29.271828+0000","last_fullsized":"2024-09-10T07:47:29.271828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.180750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271802+0000","last_change":"2024-09-10T07:47:27.708470+0000","last_active":"2024-09-10T07:47:29.271802+0000","last_peered":"2024-09-10T07:47:29.271802+0000","last_clean":"2024-09-10T07:47:29.271802+0000","last_became_active":"2024-09-10T07:47:27.708196+0000","last_became_peered":"2024-09-10T07:47:27.708196+0000","last_unstale":"2024-09-10T07:47:29.271802+0000","last_undegraded":"2024-09-10T07:47:29.271802+0000","last_fullsized":"2024-09-10T07:47:29.271802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:26.685267+0000","last_clean_scrub_stamp":"2024-09-10T07:47:26.685267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:07.295597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:28.707326+0000","last_change":"2024-09-10T07:47:26.707932+0000","last_active":"2024-09-10T07:47:28.707326+0000","last_peered":"2024-09-10T07:47:28.707326+0000","last_clean":"2024-09-10T07:47:28.707326+0000","last_became_active":"2024-09-10T07:47:26.707401+0000","last_became_peered":"2024-09-10T07:47:26.707401+0000","last_unstale":"2024-09-10T07:47:28.707326+0000","last_undegraded":"2024-09-10T07:47:28.707326+0000","last_fullsized":"2024-09-10T07:47:28.707326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:47:25.680631+0000","last_clean_scrub_stamp":"2024-09-10T07:47:25.680631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:37.955939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T07:47:29.271441+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:47:29.271441+0000","last_peered":"2024-09-10T07:47:29.271441+0000","last_clean":"2024-09-10T07:47:29.271441+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:47:29.271441+0000","last_undegraded":"2024-09-10T07:47:29.271441+0000","last_fullsized":"2024-09-10T07:47:29.271441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":155513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":154771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":745175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":154827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":154759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":409600,"data_stored":156349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":745687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":413696,"data_stored":156371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:47:37.025 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:47:37.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:47:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.225+0000 7fc268e00640 1 Processor -- start 2024-09-10T07:47:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.225+0000 7fc268e00640 1 -- start start 2024-09-10T07:47:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.226+0000 7fc268e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 0x7fc26415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc268e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc26415eff0 con 0x7fc26415b230 2024-09-10T07:47:37.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc268e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc26415f2b0 con 0x7fc2640b4c90 2024-09-10T07:47:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc268e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc26415fbe0 con 0x7fc264072c50 2024-09-10T07:47:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc26415b230 0x7fc2640b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35334/0 (socket says 172.21.15.3:35334) 2024-09-10T07:47:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261e00640 1 -- 172.21.15.3:0/1780189427 learned_addr learned my addr 172.21.15.3:0/1780189427 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc262800640 1 --2- 172.21.15.3:0/1780189427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 0x7fc26415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982685100 0 0) 0x7fc26415eff0 con 0x7fc26415b230 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc248005730 con 0x7fc26415b230 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615089057 0 0) 0x7fc26415f2b0 con 0x7fc2640b4c90 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc26415eff0 con 0x7fc2640b4c90 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc262800640 1 -- 172.21.15.3:0/1780189427 >> v1:172.21.15.70:6789/0 conn(0x7fc2640b4c90 legacy=0x7fc264071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc262800640 1 -- 172.21.15.3:0/1780189427 >> v1:172.21.15.3:6789/0 conn(0x7fc26415b230 legacy=0x7fc2640b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.227+0000 7fc262800640 1 -- 172.21.15.3:0/1780189427 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26415f590 con 0x7fc264072c50 2024-09-10T07:47:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.228+0000 7fc262800640 1 --2- 172.21.15.3:0/1780189427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 0x7fc26415c390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009dc0 tx=0x7fc25c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed04e12382a20405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c007740 con 0x7fc264072c50 2024-09-10T07:47:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25c0078e0 con 0x7fc264072c50 2024-09-10T07:47:37.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc261400640 1 -- 172.21.15.3:0/1780189427 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c007cb0 con 0x7fc264072c50 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc268e00640 1 -- 172.21.15.3:0/1780189427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 msgr2=0x7fc26415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc268e00640 1 --2- 172.21.15.3:0/1780189427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 0x7fc26415c390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009dc0 tx=0x7fc25c02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc268e00640 1 -- 172.21.15.3:0/1780189427 shutdown_connections 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc268e00640 1 --2- 172.21.15.3:0/1780189427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc264072c50 0x7fc26415c390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.229+0000 7fc268e00640 1 -- 172.21.15.3:0/1780189427 >> 172.21.15.3:0/1780189427 conn(0x7fc2640b5cb0 msgr2=0x7fc26406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.230+0000 7fc268e00640 1 -- 172.21.15.3:0/1780189427 shutdown_connections 2024-09-10T07:47:37.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.230+0000 7fc268e00640 1 -- 172.21.15.3:0/1780189427 wait complete. 2024-09-10T07:47:37.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.231+0000 7fc268e00640 1 Processor -- start 2024-09-10T07:47:37.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.231+0000 7fc268e00640 1 -- start start 2024-09-10T07:47:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 0x7fc26414acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 0x7fc2640b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc26415b610 con 0x7fc264072c50 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc26415c8d0 con 0x7fc26415b230 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc268e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc26415b790 con 0x7fc2640b4c90 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc261e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc261e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43448/0 (socket says 172.21.15.3:43448) 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.232+0000 7fc261e00640 1 -- 172.21.15.3:0/2196649535 learned_addr learned my addr 172.21.15.3:0/2196649535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 -- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 msgr2=0x7fc26414acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc259400640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 0x7fc26414acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc262800640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 0x7fc2640b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 0x7fc26414acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 -- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 msgr2=0x7fc2640b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 0x7fc2640b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 -- 172.21.15.3:0/2196649535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2640b1990 con 0x7fc264072c50 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc259400640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 0x7fc26414acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc262800640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 0x7fc2640b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc261e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c530 tx=0x7fc25400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017070 con 0x7fc264072c50 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc254005ce0 con 0x7fc264072c50 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.233+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254007330 con 0x7fc264072c50 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.234+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2640b1c70 con 0x7fc264072c50 2024-09-10T07:47:37.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.234+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26414b200 con 0x7fc264072c50 2024-09-10T07:47:37.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.235+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26406e2c0 con 0x7fc264072c50 2024-09-10T07:47:37.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.236+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc25401a070 con 0x7fc264072c50 2024-09-10T07:47:37.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.237+0000 7fc25aa00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 0x7fc24006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.237+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc25408a270 con 0x7fc264072c50 2024-09-10T07:47:37.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.237+0000 7fc259400640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 0x7fc24006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.238+0000 7fc259400640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 0x7fc24006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2640afd80 tx=0x7fc24c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.241+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc254058c10 con 0x7fc264072c50 2024-09-10T07:47:37.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.491+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc264156270 con 0x7fc264072c50 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.493+0000 7fc25aa00640 1 -- 172.21.15.3:0/2196649535 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fc254058510 con 0x7fc264072c50 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:47:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:47:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 msgr2=0x7fc24006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 0x7fc24006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2640afd80 tx=0x7fc24c009040 comp rx=0 tx=0).stop 2024-09-10T07:47:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 msgr2=0x7fc2640b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c530 tx=0x7fc25400c9f0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 shutdown_connections 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc24006be30 0x7fc24006e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc26415b230 0x7fc2640b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc2640b4c90 0x7fc26414acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 --2- 172.21.15.3:0/2196649535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc264072c50 0x7fc2640b2ad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 >> 172.21.15.3:0/2196649535 conn(0x7fc2640b5cb0 msgr2=0x7fc2640cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.495+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 shutdown_connections 2024-09-10T07:47:37.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.496+0000 7fc268e00640 1 -- 172.21.15.3:0/2196649535 wait complete. 2024-09-10T07:47:37.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.669+0000 7f6065200640 1 Processor -- start 2024-09-10T07:47:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.669+0000 7f6065200640 1 -- start start 2024-09-10T07:47:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f6065200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6060160710 0x7f60600cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f6065200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600cfdd0 con 0x7f60600d07c0 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f6065200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606015cc90 con 0x7f6060160af0 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f6065200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f606015cf70 con 0x7f6060160710 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f605e800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60600d07c0 0x7f6060072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35346/0 (socket says 172.21.15.3:35346) 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f605e800640 1 -- 172.21.15.3:0/370467376 learned_addr learned my addr 172.21.15.3:0/370467376 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f605de00640 1 --2- 172.21.15.3:0/370467376 >> v2:172.21.15.3:3300/0 conn(0x7f6060160710 0x7f60600cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f605de00640 1 -- 172.21.15.3:0/370467376 >> v1:172.21.15.70:6789/0 conn(0x7f6060160af0 legacy=0x7f60600d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:37.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.671+0000 7f605de00640 1 -- 172.21.15.3:0/370467376 >> v1:172.21.15.3:6789/0 conn(0x7f60600d07c0 legacy=0x7f6060072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.672+0000 7f605de00640 1 -- 172.21.15.3:0/370467376 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606015d250 con 0x7f6060160710 2024-09-10T07:47:37.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.672+0000 7f605de00640 1 --2- 172.21.15.3:0/370467376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6060160710 0x7f60600cf2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7436862c7e6d4dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f605d400640 1 -- 172.21.15.3:0/370467376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050037070 con 0x7f6060160710 2024-09-10T07:47:37.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f605d400640 1 -- 172.21.15.3:0/370467376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6050032440 con 0x7f6060160710 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f605d400640 1 -- 172.21.15.3:0/370467376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050032810 con 0x7f6060160710 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 -- 172.21.15.3:0/370467376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6060160710 msgr2=0x7f60600cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 --2- 172.21.15.3:0/370467376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6060160710 0x7f60600cf2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 -- 172.21.15.3:0/370467376 shutdown_connections 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 --2- 172.21.15.3:0/370467376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6060160710 0x7f60600cf2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 -- 172.21.15.3:0/370467376 >> 172.21.15.3:0/370467376 conn(0x7f606006d000 msgr2=0x7f606006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 -- 172.21.15.3:0/370467376 shutdown_connections 2024-09-10T07:47:37.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.673+0000 7f6065200640 1 -- 172.21.15.3:0/370467376 wait complete. 2024-09-10T07:47:37.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.675+0000 7f6065200640 1 Processor -- start 2024-09-10T07:47:37.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.675+0000 7f6065200640 1 -- start start 2024-09-10T07:47:37.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6060160710 0x7f606014a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060160af0 0x7f606015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f606015d770 con 0x7f60600d07c0 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f60600cf800 con 0x7f6060160af0 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f6065200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f60600b3f30 con 0x7f6060160710 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f605de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f605de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43470/0 (socket says 172.21.15.3:43470) 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.676+0000 7f605de00640 1 -- 172.21.15.3:0/3063930906 learned_addr learned my addr 172.21.15.3:0/3063930906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 -- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6060160710 msgr2=0x7f606014a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6060160710 0x7f606014a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 -- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060160af0 msgr2=0x7f606015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060160af0 0x7f606015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 -- 172.21.15.3:0/3063930906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606015f990 con 0x7f60600d07c0 2024-09-10T07:47:37.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f605de00640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f60500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050037040 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6050042070 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050030a70 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.677+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606015fc70 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.678+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606015f680 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.678+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606015d8f0 con 0x7f60600d07c0 2024-09-10T07:47:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.679+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6050030c10 con 0x7f60600d07c0 2024-09-10T07:47:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.680+0000 7f6056000640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 0x7f603c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.680+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f60500ab900 con 0x7f60600d07c0 2024-09-10T07:47:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.680+0000 7f6057e00640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 0x7f603c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.681+0000 7f6057e00640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 0x7f603c06e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6048006fd0 tx=0x7f6048006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:37.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.685+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6050045360 con 0x7f60600d07c0 2024-09-10T07:47:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.908+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60600afd50 con 0x7f60600d07c0 2024-09-10T07:47:37.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.909+0000 7f6056000640 1 -- 172.21.15.3:0/3063930906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5214 (secure 0 0 0) 0x7f60500ab030 con 0x7f60600d07c0 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"btime":"2024-09-10T07:47:30:712515+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:30.135093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741]},"id":2}]} 2024-09-10T07:47:37.909 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 msgr2=0x7f603c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 0x7f603c06e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6048006fd0 tx=0x7f6048006bc0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 msgr2=0x7f606014ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f60500046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 shutdown_connections 2024-09-10T07:47:37.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f603c06be60 0x7f603c06e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060160af0 0x7f606015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6060160710 0x7f606014a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 --2- 172.21.15.3:0/3063930906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60600d07c0 0x7f606014ada0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 >> 172.21.15.3:0/3063930906 conn(0x7f606006d000 msgr2=0x7f60600b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.911+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 shutdown_connections 2024-09-10T07:47:37.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:37.912+0000 7f6065200640 1 -- 172.21.15.3:0/3063930906 wait complete. 2024-09-10T07:47:38.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:47:27.121193+0000', 'modified': '2024-09-10T07:47:30.135093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4741}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4741': {'gid': 4741, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6836/2294972972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2294972972}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 2294972972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4741, 'qdb_cluster': [4741]} 2024-09-10T07:47:38.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:47:38.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:47:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.493+0000 7fd371200640 1 Processor -- start 2024-09-10T07:47:38.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.493+0000 7fd371200640 1 -- start start 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd371200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd36c0d0fd0 0x7fd36c162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd371200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c0b5d80 con 0x7fd36c0b52c0 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd371200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c0b6060 con 0x7fd36c0737f0 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd371200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd36c1621a0 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd361400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd36c0b52c0 0x7fd36c0730e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35352/0 (socket says 172.21.15.3:35352) 2024-09-10T07:47:38.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.494+0000 7fd361400640 1 -- 172.21.15.3:0/2625578934 learned_addr learned my addr 172.21.15.3:0/2625578934 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.495+0000 7fd369e00640 1 --2- 172.21.15.3:0/2625578934 >> v2:172.21.15.70:3300/0 conn(0x7fd36c0d0fd0 0x7fd36c162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.495+0000 7fd369e00640 1 -- 172.21.15.3:0/2625578934 >> v1:172.21.15.70:6789/0 conn(0x7fd36c0737f0 legacy=0x7fd36c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.495+0000 7fd369e00640 1 -- 172.21.15.3:0/2625578934 >> v1:172.21.15.3:6789/0 conn(0x7fd36c0b52c0 legacy=0x7fd36c0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.495+0000 7fd369e00640 1 -- 172.21.15.3:0/2625578934 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c162480 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.497+0000 7fd369e00640 1 --2- 172.21.15.3:0/2625578934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0d0fd0 0x7fd36c162c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1dbf0123575a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.497+0000 7fd369400640 1 -- 172.21.15.3:0/2625578934 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037070 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.497+0000 7fd369400640 1 -- 172.21.15.3:0/2625578934 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c032440 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.497+0000 7fd369400640 1 -- 172.21.15.3:0/2625578934 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c032810 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 -- 172.21.15.3:0/2625578934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0d0fd0 msgr2=0x7fd36c162c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 --2- 172.21.15.3:0/2625578934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0d0fd0 0x7fd36c162c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097a0 tx=0x7fd35c02a230 comp rx=0 tx=0).stop 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 -- 172.21.15.3:0/2625578934 shutdown_connections 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 --2- 172.21.15.3:0/2625578934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0d0fd0 0x7fd36c162c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 -- 172.21.15.3:0/2625578934 >> 172.21.15.3:0/2625578934 conn(0x7fd36c06bb20 msgr2=0x7fd36c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 -- 172.21.15.3:0/2625578934 shutdown_connections 2024-09-10T07:47:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.498+0000 7fd371200640 1 -- 172.21.15.3:0/2625578934 wait complete. 2024-09-10T07:47:38.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.500+0000 7fd371200640 1 Processor -- start 2024-09-10T07:47:38.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.500+0000 7fd371200640 1 -- start start 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 0x7fd36c09de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0d0fd0 0x7fd36c1465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c0d13b0 con 0x7fd36c0737f0 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c06f250 con 0x7fd36c0b52c0 2024-09-10T07:47:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd371200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c15b540 con 0x7fd36c0d0fd0 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43484/0 (socket says 172.21.15.3:43484) 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 -- 172.21.15.3:0/1752404340 learned_addr learned my addr 172.21.15.3:0/1752404340 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd361400640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 0x7fd36c09de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd36a800640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0d0fd0 0x7fd36c1465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 -- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0d0fd0 msgr2=0x7fd36c1465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0d0fd0 0x7fd36c1465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 -- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 msgr2=0x7fd36c09de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 0x7fd36c09de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd369e00640 1 -- 172.21.15.3:0/1752404340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c09edb0 con 0x7fd36c0737f0 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.501+0000 7fd361400640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 0x7fd36c09de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd369e00640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd35c035040 tx=0x7fd35c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd362a00640 1 -- 172.21.15.3:0/1752404340 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037040 con 0x7fd36c0737f0 2024-09-10T07:47:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd362a00640 1 -- 172.21.15.3:0/1752404340 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c042070 con 0x7fd36c0737f0 2024-09-10T07:47:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd362a00640 1 -- 172.21.15.3:0/1752404340 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c030bb0 con 0x7fd36c0737f0 2024-09-10T07:47:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c0b0ef0 con 0x7fd36c0737f0 2024-09-10T07:47:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.502+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c09e7f0 con 0x7fd36c0737f0 2024-09-10T07:47:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.505+0000 7fd362a00640 1 -- 172.21.15.3:0/1752404340 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd35c047460 con 0x7fd36c0737f0 2024-09-10T07:47:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.506+0000 7fd362a00640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 0x7fd34806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.506+0000 7fd362a00640 1 -- 172.21.15.3:0/1752404340 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd35c0abb00 con 0x7fd36c0737f0 2024-09-10T07:47:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.506+0000 7fd361400640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 0x7fd34806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.507+0000 7fd361400640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 0x7fd34806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06edf0 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.507+0000 7fd37421b740 1 Processor -- start 2024-09-10T07:47:38.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.507+0000 7fd37421b740 1 -- start start 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e45490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2dfa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5592f2d5e3b0 con 0x5592f2dfd4b0 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5592f2d63410 con 0x5592f2dffcf0 2024-09-10T07:47:38.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5592f2d0b620 con 0x5592f2e01480 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43486/0 (socket says 172.21.15.3:43486) 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 -- 172.21.15.3:0/2968434497 learned_addr learned my addr 172.21.15.3:0/2968434497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd36a800640 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2dfa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd369e00640 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e45490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 -- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 msgr2=0x5592f2dfa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2dfa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 -- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 msgr2=0x5592f2e45490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e45490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.508+0000 7fd361400640 1 -- 172.21.15.3:0/2968434497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f2a762d0 con 0x5592f2dfd4b0 2024-09-10T07:47:38.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd361400640 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd3540058e0 tx=0x7fd354011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18f5ebd50ef70aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd35aa00640 1 -- 172.21.15.3:0/2968434497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354005e60 con 0x5592f2dfd4b0 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd35aa00640 1 -- 172.21.15.3:0/2968434497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35400e7b0 con 0x5592f2dfd4b0 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd35aa00640 1 -- 172.21.15.3:0/2968434497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35400eb60 con 0x5592f2dfd4b0 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd37421b740 1 -- 172.21.15.3:0/2968434497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 msgr2=0x5592f2e000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.509+0000 7fd37421b740 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd3540058e0 tx=0x7fd354011000 comp rx=0 tx=0).stop 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 -- 172.21.15.3:0/2968434497 shutdown_connections 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2dfa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e000d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 --2- 172.21.15.3:0/2968434497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e45490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 -- 172.21.15.3:0/2968434497 >> 172.21.15.3:0/2968434497 conn(0x5592f2dfc740 msgr2=0x5592f2e44960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 -- 172.21.15.3:0/2968434497 shutdown_connections 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 -- 172.21.15.3:0/2968434497 wait complete. 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.510+0000 7fd37421b740 1 -- start start 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.511+0000 7fd37421b740 1 Processor -- start 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2e01480 0x5592f2e292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e29820 0x5592f2e15d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5592f2d1e780 con 0x5592f2dffcf0 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5592f2d5f3d0 con 0x5592f2e01480 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5592f2dfbbe0 con 0x5592f2e29820 2024-09-10T07:47:38.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd37421b740 10 client.0 _ll_register_callbacks cb 0x5592f2d5c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43498/0 (socket says 172.21.15.3:43498) 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 -- 172.21.15.3:0/973395350 learned_addr learned my addr 172.21.15.3:0/973395350 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 -- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e29820 msgr2=0x5592f2e15d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e29820 0x5592f2e15d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 -- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2e01480 msgr2=0x5592f2e292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2e01480 0x5592f2e292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 -- 172.21.15.3:0/973395350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f2b406b0 con 0x5592f2dffcf0 2024-09-10T07:47:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd369e00640 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd35c02a820 tx=0x7fd35c079da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aa927c8e455878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.512+0000 7fd34f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0a8070 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd35c076410 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35400e7b0 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd35c045360 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 10 client.5007 fetch_fsmap learned FSMap version 17 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5592f2e15570 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35400e7b0 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd35c03a030 con 0x5592f2dffcf0 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 10 client.5007 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T07:47:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 10 client.5007 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 20 client.5007 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e611f0 0x5592f2e635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 4 client.5007 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.513+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592f2d70b40 con 0x5592f2e611f0 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.514+0000 7fd361400640 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e611f0 0x5592f2e635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.514+0000 7fd361400640 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e611f0 0x5592f2e635e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b520f1d9374afc5b server_cookie=10cf25d6dc3e480b in_seq=0 out_seq=0 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.514+0000 7fd34f400640 10 client.5007 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd34f400640 1 -- 172.21.15.3:0/973395350 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592f2d70b40 con 0x5592f2e611f0 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd34f400640 10 client.5007 handle_command_reply: tid=0 2024-09-10T07:47:38.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 client.5007 shutdown 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 msgr2=0x5592f2e3f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd35c02a820 tx=0x7fd35c079da0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 shutdown_connections 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e611f0 0x5592f2e635e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e29820 0x5592f2e15d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2e01480 0x5592f2e292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 --2- 172.21.15.3:0/973395350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dffcf0 0x5592f2e3f510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd34f400640 0 client.5007 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.515+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 >> 172.21.15.3:0/973395350 conn(0x5592f2dfd4b0 msgr2=0x5592f2e429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.516+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 shutdown_connections 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.516+0000 7fd37421b740 1 -- 172.21.15.3:0/973395350 wait complete. 2024-09-10T07:47:38.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.516+0000 7fd37421b740 20 client.5007 trim_cache size 0 max 16384 2024-09-10T07:47:38.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.534+0000 7fd37421b740 1 Processor -- start 2024-09-10T07:47:38.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.534+0000 7fd37421b740 1 -- start start 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e41dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2ebabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1ffc90 con 0x5592f2dfd4b0 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c15a0c0 con 0x5592f2dffcf0 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c0b2110 con 0x5592f2e01480 2024-09-10T07:47:38.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.535+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43506/0 (socket says 172.21.15.3:43506) 2024-09-10T07:47:38.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 -- 172.21.15.3:0/3229480226 learned_addr learned my addr 172.21.15.3:0/3229480226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 -- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 msgr2=0x5592f2ebabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2ebabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 -- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 msgr2=0x5592f2e41dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e41dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 -- 172.21.15.3:0/3229480226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f2e14700 con 0x5592f2dfd4b0 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd369e00640 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd35c078050 tx=0x7fd35c0a5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69920dce414a1c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd35a000640 1 -- 172.21.15.3:0/3229480226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0a8040 con 0x5592f2dfd4b0 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd35a000640 1 -- 172.21.15.3:0/3229480226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c0a3070 con 0x5592f2dfd4b0 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.536+0000 7fd35a000640 1 -- 172.21.15.3:0/3229480226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c076410 con 0x5592f2dfd4b0 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 -- 172.21.15.3:0/3229480226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 msgr2=0x5592f2e00ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd35c078050 tx=0x7fd35c0a5000 comp rx=0 tx=0).stop 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 -- 172.21.15.3:0/3229480226 shutdown_connections 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2e01480 0x5592f2ebabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2e41dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 --2- 172.21.15.3:0/3229480226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2dfd4b0 0x5592f2e00ce0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 -- 172.21.15.3:0/3229480226 >> 172.21.15.3:0/3229480226 conn(0x5592f2e3f670 msgr2=0x5592f2e466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 -- 172.21.15.3:0/3229480226 shutdown_connections 2024-09-10T07:47:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.537+0000 7fd37421b740 1 -- 172.21.15.3:0/3229480226 wait complete. 2024-09-10T07:47:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.538+0000 7fd37421b740 1 -- start start 2024-09-10T07:47:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.538+0000 7fd37421b740 1 Processor -- start 2024-09-10T07:47:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2dfd4b0 0x5592f2e5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2ea4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5592f2e16fb0 con 0x5592f2e01480 2024-09-10T07:47:38.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5592f2e3fa50 con 0x5592f2dffcf0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5592f2e17600 con 0x5592f2dfd4b0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd37421b740 10 client.0 _ll_register_callbacks cb 0x7fd35c079c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd36a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd36a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43518/0 (socket says 172.21.15.3:43518) 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd36a800640 1 -- 172.21.15.3:0/4058832278 learned_addr learned my addr 172.21.15.3:0/4058832278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd369e00640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2dfd4b0 0x5592f2e5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.539+0000 7fd361400640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2ea4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 -- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2dfd4b0 msgr2=0x5592f2e5a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2dfd4b0 0x5592f2e5a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 -- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 msgr2=0x5592f2ea4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2ea4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 -- 172.21.15.3:0/4058832278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f2e58510 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd36a800640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd364005900 tx=0x7fd364001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4408eaf23c9b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd34f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36401b060 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd364016b10 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.540+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c0a3070 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd364016d10 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 10 client.5013 fetch_fsmap learned FSMap version 17 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5592f2e47920 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c0a3070 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd3640143c0 con 0x5592f2e01480 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 10 client.5013 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T07:47:38.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 10 client.5013 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 20 client.5013 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e29820 0x5592f2eb9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 4 client.5013 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "status"}) -- 0x5592f2d6ae90 con 0x5592f2e29820 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.541+0000 7fd369e00640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e29820 0x5592f2eb9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd369e00640 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e29820 0x5592f2eb9750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11def2334062b15f server_cookie=96c3d8e34c9b8274 in_seq=0 out_seq=0 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd34f400640 10 client.5013 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd34f400640 1 -- 172.21.15.3:0/4058832278 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5592f2d6ae90 con 0x5592f2e29820 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd34f400640 10 client.5013 handle_command_reply: tid=0 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 client.5013 shutdown 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 msgr2=0x5592f2ea6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd364005900 tx=0x7fd364001910 comp rx=0 tx=0).stop 2024-09-10T07:47:38.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 shutdown_connections 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5592f2e29820 0x5592f2eb9750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5592f2e01480 0x5592f2ea6b00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5592f2dffcf0 0x5592f2ea4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 --2- 172.21.15.3:0/4058832278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5592f2dfd4b0 0x5592f2e5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 >> 172.21.15.3:0/4058832278 conn(0x5592f2e3f670 msgr2=0x5592f2e13bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd34f400640 0 client.5013 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 shutdown_connections 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.542+0000 7fd37421b740 1 -- 172.21.15.3:0/4058832278 wait complete. 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd37421b740 20 client.5013 trim_cache size 0 max 16384 2024-09-10T07:47:38.546 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4741, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.844335287, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 17, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 36, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 36, 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 24.544964280999999 2024-09-10T07:47:38.547 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 msgr2=0x7fd34806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 0x7fd34806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06edf0 tx=0x7fd354009040 comp rx=0 tx=0).stop 2024-09-10T07:47:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 msgr2=0x7fd36c09ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd35c035040 tx=0x7fd35c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 shutdown_connections 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34806be30 0x7fd34806e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd36c0d0fd0 0x7fd36c1465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd36c0b52c0 0x7fd36c09de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 --2- 172.21.15.3:0/1752404340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd36c0737f0 0x7fd36c09ce10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 >> 172.21.15.3:0/1752404340 conn(0x7fd36c06bb20 msgr2=0x7fd36c072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 shutdown_connections 2024-09-10T07:47:38.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.543+0000 7fd371200640 1 -- 172.21.15.3:0/1752404340 wait complete. 2024-09-10T07:47:38.558 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:47:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.714+0000 7ffb8aa00640 1 Processor -- start 2024-09-10T07:47:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.714+0000 7ffb8aa00640 1 -- start start 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.715+0000 7ffb8aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb84161dd0 0x7ffb8406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.715+0000 7ffb8aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb840b5ea0 0x7ffb840b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.715+0000 7ffb8aa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8406fe10 con 0x7ffb84062630 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.715+0000 7ffb8aa00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffb840b47d0 con 0x7ffb84161dd0 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.715+0000 7ffb8aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8406c870 con 0x7ffb840b5ea0 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb84161dd0 0x7ffb8406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb84161dd0 0x7ffb8406f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43532/0 (socket says 172.21.15.3:43532) 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 -- 172.21.15.3:0/716840365 learned_addr learned my addr 172.21.15.3:0/716840365 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83400640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb840b5ea0 0x7ffb840b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 -- 172.21.15.3:0/716840365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb840b5ea0 msgr2=0x7ffb840b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb840b5ea0 0x7ffb840b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 -- 172.21.15.3:0/716840365 >> v1:172.21.15.70:6789/0 conn(0x7ffb84062630 legacy=0x7ffb84062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.716+0000 7ffb83e00640 1 -- 172.21.15.3:0/716840365 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8406cb50 con 0x7ffb84161dd0 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.717+0000 7ffb83e00640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84161dd0 0x7ffb8406f8d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c448270ba8d985a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.717+0000 7ffb82a00640 1 -- 172.21.15.3:0/716840365 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74037070 con 0x7ffb84161dd0 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.717+0000 7ffb82a00640 1 -- 172.21.15.3:0/716840365 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb74032440 con 0x7ffb84161dd0 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.717+0000 7ffb82a00640 1 -- 172.21.15.3:0/716840365 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74032810 con 0x7ffb84161dd0 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 -- 172.21.15.3:0/716840365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84161dd0 msgr2=0x7ffb8406f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84161dd0 0x7ffb8406f8d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb7402a210 comp rx=0 tx=0).stop 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 -- 172.21.15.3:0/716840365 shutdown_connections 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb840b5ea0 0x7ffb840b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/716840365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84161dd0 0x7ffb8406f8d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 -- 172.21.15.3:0/716840365 >> 172.21.15.3:0/716840365 conn(0x7ffb840b3540 msgr2=0x7ffb840b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 -- 172.21.15.3:0/716840365 shutdown_connections 2024-09-10T07:47:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.718+0000 7ffb8aa00640 1 -- 172.21.15.3:0/716840365 wait complete. 2024-09-10T07:47:38.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.720+0000 7ffb8aa00640 1 Processor -- start 2024-09-10T07:47:38.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.720+0000 7ffb8aa00640 1 -- start start 2024-09-10T07:47:38.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb840b5ea0 0x7ffb84155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb84161dd0 0x7ffb84146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840b6820 con 0x7ffb84062630 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840b06c0 con 0x7ffb840b5ea0 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb8aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8406f3a0 con 0x7ffb84161dd0 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb83e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb83e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43538/0 (socket says 172.21.15.3:43538) 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb83e00640 1 -- 172.21.15.3:0/435133660 learned_addr learned my addr 172.21.15.3:0/435133660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb88a00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb84161dd0 0x7ffb84146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.721+0000 7ffb83400640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb840b5ea0 0x7ffb84155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 -- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb84161dd0 msgr2=0x7ffb84146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb84161dd0 0x7ffb84146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 -- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb840b5ea0 msgr2=0x7ffb84155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb840b5ea0 0x7ffb84155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 -- 172.21.15.3:0/435133660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb84155960 con 0x7ffb84062630 2024-09-10T07:47:38.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb83e00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74041020 con 0x7ffb84062630 2024-09-10T07:47:38.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb740306c0 con 0x7ffb84062630 2024-09-10T07:47:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74037040 con 0x7ffb84062630 2024-09-10T07:47:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb84155c40 con 0x7ffb84062630 2024-09-10T07:47:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.722+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb84156200 con 0x7ffb84062630 2024-09-10T07:47:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.723+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb840b43f0 con 0x7ffb84062630 2024-09-10T07:47:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.724+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb74044020 con 0x7ffb84062630 2024-09-10T07:47:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.725+0000 7ffb7be00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 0x7ffb6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.725+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7ffb740ab540 con 0x7ffb84062630 2024-09-10T07:47:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.725+0000 7ffb83400640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 0x7ffb6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.726+0000 7ffb83400640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 0x7ffb6406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffb84159d80 tx=0x7ffb70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.729+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb7404b030 con 0x7ffb84062630 2024-09-10T07:47:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.947+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffb841615f0 con 0x7ffb84062630 2024-09-10T07:47:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.947+0000 7ffb7be00640 1 -- 172.21.15.3:0/435133660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ffb74079890 con 0x7ffb84062630 2024-09-10T07:47:38.947 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 msgr2=0x7ffb6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 0x7ffb6406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffb84159d80 tx=0x7ffb70008040 comp rx=0 tx=0).stop 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 msgr2=0x7ffb8414fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb740046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 shutdown_connections 2024-09-10T07:47:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb6406be30 0x7ffb6406e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb84161dd0 0x7ffb84146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb840b5ea0 0x7ffb84155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 --2- 172.21.15.3:0/435133660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb84062630 0x7ffb8414fd70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 >> 172.21.15.3:0/435133660 conn(0x7ffb840b3540 msgr2=0x7ffb840afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 shutdown_connections 2024-09-10T07:47:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:38.950+0000 7ffb8aa00640 1 -- 172.21.15.3:0/435133660 wait complete. 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:47:38.965 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:47:38.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:38.966 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:47:39.002 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42637sec preferred_lft 42637sec 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:47:39.003 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:47:39.004 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:47:39.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.005 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:47:39.096 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:47:39.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.099 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:47:39.135 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:47:39.135 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:47:39.136 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:47:39.136 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:47:39.136 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:47:39.136 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:47:39.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:47:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:39.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.333 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:47:39.368 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:47:39.368 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:47:39.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:47:39.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:47:39.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:47:39.465 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:47:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:47:39.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:47:39.501 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:47:39.577 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:47:39.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:47:39.705 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:47:39.706 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:47:39.706 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:47:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:47:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:47:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:47:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:47:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:47:39.744 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:47:39.744 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:47:39.744 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:47:39.744 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:47:39.745 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:47:39.745 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:47:39.745 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:47:39.745 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:47:39.745 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:47:39.746 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:47:39.746 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:47:39.746 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:47:39.746 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:47:39.748 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:47:39.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:39.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:47:39.805 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:47:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:47:39.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:39.901 INFO:teuthology.orchestra.run.smithi003.stdout:client5022 2024-09-10T07:47:39.901 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:47:39.901 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:47:39.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:47:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.135+0000 7fcaac800640 1 Processor -- start 2024-09-10T07:47:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.135+0000 7fcaac800640 1 -- start start 2024-09-10T07:47:40.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaac800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa4070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaac800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaa406fa30 0x7fcaa40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaac800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa40aeb40 con 0x7fcaa415e970 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaac800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa406f5a0 con 0x7fcaa406fa30 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaac800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa406ec70 con 0x7fcaa4062d40 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaa406fa30 0x7fcaa40afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcaa406fa30 0x7fcaa40afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43548/0 (socket says 172.21.15.3:43548) 2024-09-10T07:47:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 -- 172.21.15.3:0/3076482265 learned_addr learned my addr 172.21.15.3:0/3076482265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:40.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaa9400640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 -- 172.21.15.3:0/3076482265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 msgr2=0x7fcaa4070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:40.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa4070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 -- 172.21.15.3:0/3076482265 >> v1:172.21.15.70:6789/0 conn(0x7fcaa415e970 legacy=0x7fcaa4062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.137+0000 7fcaaa800640 1 -- 172.21.15.3:0/3076482265 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa406ef50 con 0x7fcaa406fa30 2024-09-10T07:47:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.138+0000 7fcaaa800640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa40afea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009e50 tx=0x7fcaa002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c15db7a8dd673d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaa8a00640 1 -- 172.21.15.3:0/3076482265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0036070 con 0x7fcaa406fa30 2024-09-10T07:47:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaa8a00640 1 -- 172.21.15.3:0/3076482265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa0031600 con 0x7fcaa406fa30 2024-09-10T07:47:40.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaa8a00640 1 -- 172.21.15.3:0/3076482265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00319d0 con 0x7fcaa406fa30 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 -- 172.21.15.3:0/3076482265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 msgr2=0x7fcaa40afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa40afea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009e50 tx=0x7fcaa002a680 comp rx=0 tx=0).stop 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 -- 172.21.15.3:0/3076482265 shutdown_connections 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa40afea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 --2- 172.21.15.3:0/3076482265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa4070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 -- 172.21.15.3:0/3076482265 >> 172.21.15.3:0/3076482265 conn(0x7fcaa4073060 msgr2=0x7fcaa406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 -- 172.21.15.3:0/3076482265 shutdown_connections 2024-09-10T07:47:40.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.139+0000 7fcaac800640 1 -- 172.21.15.3:0/3076482265 wait complete. 2024-09-10T07:47:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.141+0000 7fcaac800640 1 Processor -- start 2024-09-10T07:47:40.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.141+0000 7fcaac800640 1 -- start start 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa415e970 0x7fcaa415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40b44d0 con 0x7fcaa406fa30 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa406e920 con 0x7fcaa415e970 2024-09-10T07:47:40.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaac800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa40b4650 con 0x7fcaa4062d40 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43550/0 (socket says 172.21.15.3:43550) 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaa9400640 1 -- 172.21.15.3:0/3256825122 learned_addr learned my addr 172.21.15.3:0/3256825122 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaa9e00640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.142+0000 7fcaaa800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa415e970 0x7fcaa415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 -- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 msgr2=0x7fcaa414ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa414ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 -- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa415e970 msgr2=0x7fcaa415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa415e970 0x7fcaa415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 -- 172.21.15.3:0/3256825122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa414b2e0 con 0x7fcaa406fa30 2024-09-10T07:47:40.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaa9400640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d600 tx=0x7fca9400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940125f0 con 0x7fcaa406fa30 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca94019070 con 0x7fcaa406fa30 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940129f0 con 0x7fcaa406fa30 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa415cd30 con 0x7fcaa406fa30 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.143+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa415aa00 con 0x7fcaa406fa30 2024-09-10T07:47:40.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.144+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa4070190 con 0x7fcaa406fa30 2024-09-10T07:47:40.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.145+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca94012b90 con 0x7fcaa406fa30 2024-09-10T07:47:40.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.147+0000 7fca9e000640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 0x7fca8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.147+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fca9408b220 con 0x7fcaa406fa30 2024-09-10T07:47:40.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.147+0000 7fcaa9e00640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 0x7fca8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.148+0000 7fcaa9e00640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 0x7fca8806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca98000f60 tx=0x7fca98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.151+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcaa414b2e0 con 0x7fcaa406fa30 2024-09-10T07:47:40.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.374+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fcaa4063310 con 0x7fcaa406fa30 2024-09-10T07:47:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.571+0000 7fca9e000640 1 -- 172.21.15.3:0/3256825122 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fca940594e0 con 0x7fcaa406fa30 2024-09-10T07:47:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 msgr2=0x7fca8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 0x7fca8806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca98000f60 tx=0x7fca98009040 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 msgr2=0x7fcaa415b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d600 tx=0x7fca9400dac0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 shutdown_connections 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fca8806be30 0x7fca8806e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa415e970 0x7fcaa415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcaa406fa30 0x7fcaa415b4e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 --2- 172.21.15.3:0/3256825122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcaa4062d40 0x7fcaa414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.573+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 >> 172.21.15.3:0/3256825122 conn(0x7fcaa4073060 msgr2=0x7fcaa415a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.574+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 shutdown_connections 2024-09-10T07:47:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.574+0000 7fcaac800640 1 -- 172.21.15.3:0/3256825122 wait complete. 2024-09-10T07:47:40.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.749+0000 7f912a800640 1 Processor -- start 2024-09-10T07:47:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.749+0000 7f912a800640 1 -- start start 2024-09-10T07:47:40.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f912a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9124160710 0x7f91240cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f912a800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91240d07c0 0x7f9124072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f912a800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91240cfdd0 con 0x7f9124160af0 2024-09-10T07:47:40.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f912a800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f912415cc90 con 0x7f9124160710 2024-09-10T07:47:40.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f912a800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f912415cf70 con 0x7f91240d07c0 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f9123400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9124160710 0x7f91240cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f9123400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9124160710 0x7f91240cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43564/0 (socket says 172.21.15.3:43564) 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f9123400640 1 -- 172.21.15.3:0/3859832809 learned_addr learned my addr 172.21.15.3:0/3859832809 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.751+0000 7f9123e00640 1 --2- 172.21.15.3:0/3859832809 >> v2:172.21.15.70:3300/0 conn(0x7f91240d07c0 0x7f9124072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.752+0000 7f9123400640 1 -- 172.21.15.3:0/3859832809 >> v2:172.21.15.70:3300/0 conn(0x7f91240d07c0 msgr2=0x7f9124072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.752+0000 7f9123400640 1 --2- 172.21.15.3:0/3859832809 >> v2:172.21.15.70:3300/0 conn(0x7f91240d07c0 0x7f9124072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.752+0000 7f9123400640 1 -- 172.21.15.3:0/3859832809 >> v1:172.21.15.3:6789/0 conn(0x7f9124160af0 legacy=0x7f91240d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.752+0000 7f9123400640 1 -- 172.21.15.3:0/3859832809 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912415d250 con 0x7f9124160710 2024-09-10T07:47:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.752+0000 7f9122a00640 1 -- 172.21.15.3:0/3859832809 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807685592 0 0) 0x7f91240cfdd0 con 0x7f9124160af0 2024-09-10T07:47:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f9123400640 1 --2- 172.21.15.3:0/3859832809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160710 0x7f91240cf2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f91240718d0 tx=0x7f911402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a974a20b2aa53330 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f9122a00640 1 -- 172.21.15.3:0/3859832809 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114037070 con 0x7f9124160710 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f9122a00640 1 -- 172.21.15.3:0/3859832809 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9114032440 con 0x7f9124160710 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f9122a00640 1 -- 172.21.15.3:0/3859832809 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114032810 con 0x7f9124160710 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 -- 172.21.15.3:0/3859832809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160710 msgr2=0x7f91240cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 --2- 172.21.15.3:0/3859832809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160710 0x7f91240cf2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f91240718d0 tx=0x7f911402a270 comp rx=0 tx=0).stop 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 -- 172.21.15.3:0/3859832809 shutdown_connections 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 --2- 172.21.15.3:0/3859832809 >> v2:172.21.15.70:3300/0 conn(0x7f91240d07c0 0x7f9124072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 --2- 172.21.15.3:0/3859832809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160710 0x7f91240cf2c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.753+0000 7f912a800640 1 -- 172.21.15.3:0/3859832809 >> 172.21.15.3:0/3859832809 conn(0x7f912406d000 msgr2=0x7f912406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.754+0000 7f912a800640 1 -- 172.21.15.3:0/3859832809 shutdown_connections 2024-09-10T07:47:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.754+0000 7f912a800640 1 -- 172.21.15.3:0/3859832809 wait complete. 2024-09-10T07:47:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.755+0000 7f912a800640 1 Processor -- start 2024-09-10T07:47:40.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.755+0000 7f912a800640 1 -- start start 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91240d07c0 0x7f912414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9124160710 0x7f912414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f912415d770 con 0x7f9124160af0 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91240cf800 con 0x7f9124160710 2024-09-10T07:47:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f912a800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9124145470 con 0x7f91240d07c0 2024-09-10T07:47:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.756+0000 7f9123e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91240d07c0 0x7f912414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43570/0 (socket says 172.21.15.3:43570) 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 -- 172.21.15.3:0/2043860277 learned_addr learned my addr 172.21.15.3:0/2043860277 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f911be00640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9124160710 0x7f912414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 -- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91240d07c0 msgr2=0x7f912414a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91240d07c0 0x7f912414a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 -- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9124160710 msgr2=0x7f912414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9124160710 0x7f912414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 -- 172.21.15.3:0/2043860277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124145750 con 0x7f9124160af0 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.757+0000 7f9123e00640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f911c00efc0 tx=0x7f911c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.758+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c018020 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.758+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911c014480 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.758+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c014870 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.758+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9124145a30 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.758+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912415f0b0 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.759+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91240afd50 con 0x7f9124160af0 2024-09-10T07:47:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.760+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f911c01b020 con 0x7f9124160af0 2024-09-10T07:47:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.761+0000 7f911b400640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 0x7f910006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.761+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f911c08bbe0 con 0x7f9124160af0 2024-09-10T07:47:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.761+0000 7f9123400640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 0x7f910006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.762+0000 7f9123400640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 0x7f910006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9114034000 tx=0x7f911402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.765+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f911c022020 con 0x7f9124160af0 2024-09-10T07:47:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.984+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9124073240 con 0x7f9124160af0 2024-09-10T07:47:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.985+0000 7f911b400640 1 -- 172.21.15.3:0/2043860277 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5230 (secure 0 0 0) 0x7f911c010040 con 0x7f9124160af0 2024-09-10T07:47:40.984 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:40.985 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"btime":"2024-09-10T07:47:40:574183+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4601,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:40.574173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4741,"mds_1":4607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741]},"id":2}]} 2024-09-10T07:47:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 19 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 msgr2=0x7f910006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 0x7f910006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9114034000 tx=0x7f911402e000 comp rx=0 tx=0).stop 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 msgr2=0x7f912415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f911c00efc0 tx=0x7f911c00c490 comp rx=0 tx=0).stop 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 shutdown_connections 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f910006be30 0x7f910006e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9124160af0 0x7f912415eb70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9124160710 0x7f912414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 --2- 172.21.15.3:0/2043860277 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91240d07c0 0x7f912414a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 >> 172.21.15.3:0/2043860277 conn(0x7f912406d000 msgr2=0x7f912415b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 shutdown_connections 2024-09-10T07:47:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:40.987+0000 7f912a800640 1 -- 172.21.15.3:0/2043860277 wait complete. 2024-09-10T07:47:41.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:47:27.121193+0000', 'modified': '2024-09-10T07:47:40.574173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 4741, 'mds_1': 4607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4607': {'gid': 4607, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.70:6835/2716956690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 2716956690}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 2716956690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4741': {'gid': 4741, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6836/2294972972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2294972972}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 2294972972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4741, 'qdb_cluster': [4741]} 2024-09-10T07:47:41.406 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4607:up:creating 2024-09-10T07:47:42.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.576+0000 7f260e600640 1 Processor -- start 2024-09-10T07:47:42.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.576+0000 7f260e600640 1 -- start start 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.577+0000 7f260e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 0x7f2608073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.577+0000 7f260e600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 0x7f2608073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.577+0000 7f260e600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26080cfdc0 con 0x7f260815e860 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.577+0000 7f260e600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f26080d3780 con 0x7f26080d4050 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.577+0000 7f260e600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26080cee40 con 0x7f26080d3c10 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f260dc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 0x7f2608073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f2607e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 0x7f2608073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f260d200640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f260815e860 0x7f26080cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:51910/0 (socket says 172.21.15.3:51910) 2024-09-10T07:47:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f260d200640 1 -- 172.21.15.3:0/215337802 learned_addr learned my addr 172.21.15.3:0/215337802 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f2607e00640 1 -- 172.21.15.3:0/215337802 >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 msgr2=0x7f2608073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f2607e00640 1 --2- 172.21.15.3:0/215337802 >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 0x7f2608073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f2607e00640 1 -- 172.21.15.3:0/215337802 >> v1:172.21.15.70:6789/0 conn(0x7f260815e860 legacy=0x7f26080cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f2607e00640 1 -- 172.21.15.3:0/215337802 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26080cf120 con 0x7f26080d3c10 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.578+0000 7f260dc00640 1 --2- 172.21.15.3:0/215337802 >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 0x7f2608073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.579+0000 7f2607e00640 1 --2- 172.21.15.3:0/215337802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 0x7f2608073150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca50a20bbe7851 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f2607400640 1 -- 172.21.15.3:0/215337802 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4037070 con 0x7f26080d3c10 2024-09-10T07:47:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f2607400640 1 -- 172.21.15.3:0/215337802 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f4032440 con 0x7f26080d3c10 2024-09-10T07:47:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f2607400640 1 -- 172.21.15.3:0/215337802 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4032810 con 0x7f26080d3c10 2024-09-10T07:47:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 -- 172.21.15.3:0/215337802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 msgr2=0x7f2608073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 --2- 172.21.15.3:0/215337802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 0x7f2608073150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a180 comp rx=0 tx=0).stop 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 -- 172.21.15.3:0/215337802 shutdown_connections 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 --2- 172.21.15.3:0/215337802 >> v2:172.21.15.70:3300/0 conn(0x7f26080d4050 0x7f2608073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 --2- 172.21.15.3:0/215337802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26080d3c10 0x7f2608073150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.580+0000 7f260e600640 1 -- 172.21.15.3:0/215337802 >> 172.21.15.3:0/215337802 conn(0x7f26080d4650 msgr2=0x7f26080d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.581+0000 7f260e600640 1 -- 172.21.15.3:0/215337802 shutdown_connections 2024-09-10T07:47:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.581+0000 7f260e600640 1 -- 172.21.15.3:0/215337802 wait complete. 2024-09-10T07:47:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.582+0000 7f260e600640 1 Processor -- start 2024-09-10T07:47:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.582+0000 7f260e600640 1 -- start start 2024-09-10T07:47:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26080d4050 0x7f260814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f260815e860 0x7f26080cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26080d2460 con 0x7f26080d3c10 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26080cf400 con 0x7f26080d4050 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.583+0000 7f260e600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26080b7880 con 0x7f260815e860 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43584/0 (socket says 172.21.15.3:43584) 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 -- 172.21.15.3:0/1513819310 learned_addr learned my addr 172.21.15.3:0/1513819310 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260dc00640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f260815e860 0x7f26080cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f2607e00640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26080d4050 0x7f260814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 -- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f260815e860 msgr2=0x7f26080cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f260815e860 0x7f26080cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 -- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26080d4050 msgr2=0x7f260814d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26080d4050 0x7f260814d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.584+0000 7f260d200640 1 -- 172.21.15.3:0/1513819310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26080cdaa0 con 0x7f26080d3c10 2024-09-10T07:47:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.585+0000 7f260d200640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c480 tx=0x7f25fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.586+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc017070 con 0x7f26080d3c10 2024-09-10T07:47:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.586+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25fc005ce0 con 0x7f26080d3c10 2024-09-10T07:47:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.586+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0072d0 con 0x7f26080d3c10 2024-09-10T07:47:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.587+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26080ce5b0 con 0x7f26080d3c10 2024-09-10T07:47:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.587+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26080cdff0 con 0x7f26080d3c10 2024-09-10T07:47:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.588+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26080cf6b0 con 0x7f26080d3c10 2024-09-10T07:47:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.589+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25fc01a070 con 0x7f26080d3c10 2024-09-10T07:47:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.590+0000 7f2604c00640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 0x7f25e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.590+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f25fc089ff0 con 0x7f26080d3c10 2024-09-10T07:47:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.590+0000 7f2607e00640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 0x7f25e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:42.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.591+0000 7f2607e00640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 0x7f25e806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.594+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25fc058360 con 0x7f26080d3c10 2024-09-10T07:47:42.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.817+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26080b87c0 con 0x7f26080d3c10 2024-09-10T07:47:42.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.818+0000 7f2604c00640 1 -- 172.21.15.3:0/1513819310 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5252 (secure 0 0 0) 0x7f25fc05c660 con 0x7f26080d3c10 2024-09-10T07:47:42.817 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:42.817 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"btime":"2024-09-10T07:47:42:583951+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:42.583947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4601,4607,4741]},"id":2}]} 2024-09-10T07:47:42.818 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 msgr2=0x7f25e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 0x7f25e806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 msgr2=0x7f260815b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c480 tx=0x7f25fc00c880 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 shutdown_connections 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f25e806be30 0x7f25e806e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f260815e860 0x7f26080cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26080d4050 0x7f260814d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 --2- 172.21.15.3:0/1513819310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26080d3c10 0x7f260815b140 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 >> 172.21.15.3:0/1513819310 conn(0x7f26080d4650 msgr2=0x7f26080b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:42.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 shutdown_connections 2024-09-10T07:47:42.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:42.820+0000 7f260e600640 1 -- 172.21.15.3:0/1513819310 wait complete. 2024-09-10T07:47:43.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:47:27.121193+0000', 'modified': '2024-09-10T07:47:42.583947+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4741, 'mds_1': 4607, 'mds_2': 4601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4601': {'gid': 4601, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6837/708626559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 708626559}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 708626559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4607': {'gid': 4607, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.70:6835/2716956690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 2716956690}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 2716956690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4741': {'gid': 4741, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6836/2294972972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2294972972}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 2294972972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4741, 'qdb_cluster': [4601, 4607, 4741]} 2024-09-10T07:47:43.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:47:43.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:47:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.399+0000 7f85d8400640 1 Processor -- start 2024-09-10T07:47:43.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.399+0000 7f85d8400640 1 -- start start 2024-09-10T07:47:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d8400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0bb0 0x7f85d0071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d8400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85d00d0f90 0x7f85d015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d8400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d0156f80 con 0x7f85d015d040 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d8400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f85d015d780 con 0x7f85d00d0f90 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d8400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f85d0071450 con 0x7f85d00d0bb0 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.400+0000 7f85d5000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85d00d0f90 0x7f85d015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d6400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f85d015d040 0x7f85d00af4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35374/0 (socket says 172.21.15.3:35374) 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d6400640 1 -- 172.21.15.3:0/3389038557 learned_addr learned my addr 172.21.15.3:0/3389038557 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d5a00640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0bb0 0x7f85d0071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d5000640 1 -- 172.21.15.3:0/3389038557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0bb0 msgr2=0x7f85d0071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d5000640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0bb0 0x7f85d0071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d5000640 1 -- 172.21.15.3:0/3389038557 >> v1:172.21.15.3:6789/0 conn(0x7f85d015d040 legacy=0x7f85d00af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.401+0000 7f85d5000640 1 -- 172.21.15.3:0/3389038557 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d015da60 con 0x7f85d00d0f90 2024-09-10T07:47:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.402+0000 7f85d5000640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0f90 0x7f85d015cb00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0097a0 tx=0x7f85bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9b106fbe51870 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.402+0000 7f85cfe00640 1 -- 172.21.15.3:0/3389038557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc037070 con 0x7f85d00d0f90 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.402+0000 7f85cfe00640 1 -- 172.21.15.3:0/3389038557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85bc032440 con 0x7f85d00d0f90 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.402+0000 7f85cfe00640 1 -- 172.21.15.3:0/3389038557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc032810 con 0x7f85d00d0f90 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 -- 172.21.15.3:0/3389038557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0f90 msgr2=0x7f85d015cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0f90 0x7f85d015cb00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0097a0 tx=0x7f85bc02a180 comp rx=0 tx=0).stop 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 -- 172.21.15.3:0/3389038557 shutdown_connections 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0f90 0x7f85d015cb00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 --2- 172.21.15.3:0/3389038557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0bb0 0x7f85d0071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 -- 172.21.15.3:0/3389038557 >> 172.21.15.3:0/3389038557 conn(0x7f85d006bf40 msgr2=0x7f85d006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 -- 172.21.15.3:0/3389038557 shutdown_connections 2024-09-10T07:47:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.403+0000 7f85d8400640 1 -- 172.21.15.3:0/3389038557 wait complete. 2024-09-10T07:47:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.405+0000 7f85d8400640 1 Processor -- start 2024-09-10T07:47:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.405+0000 7f85d8400640 1 -- start start 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0f90 0x7f85d0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85d015d040 0x7f85d0159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85d015d450 con 0x7f85d00d0bb0 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0071a10 con 0x7f85d015d040 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d8400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00d13f0 con 0x7f85d00d0f90 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43602/0 (socket says 172.21.15.3:43602) 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 -- 172.21.15.3:0/1130777956 learned_addr learned my addr 172.21.15.3:0/1130777956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5000640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0f90 0x7f85d0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d6400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85d015d040 0x7f85d0159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 -- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0f90 msgr2=0x7f85d0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0f90 0x7f85d0157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 -- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85d015d040 msgr2=0x7f85d0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85d015d040 0x7f85d0159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.406+0000 7f85d5a00640 1 -- 172.21.15.3:0/1130777956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d0158cb0 con 0x7f85d00d0bb0 2024-09-10T07:47:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9b0 tx=0x7f85c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85cd600640 1 -- 172.21.15.3:0/1130777956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4017070 con 0x7f85d00d0bb0 2024-09-10T07:47:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85cd600640 1 -- 172.21.15.3:0/1130777956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c400c9a0 con 0x7f85d00d0bb0 2024-09-10T07:47:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85cd600640 1 -- 172.21.15.3:0/1130777956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c400cd90 con 0x7f85d00d0bb0 2024-09-10T07:47:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d0157c10 con 0x7f85d00d0bb0 2024-09-10T07:47:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.407+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01581d0 con 0x7f85d00d0bb0 2024-09-10T07:47:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.409+0000 7f85cd600640 1 -- 172.21.15.3:0/1130777956 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85c401a070 con 0x7f85d00d0bb0 2024-09-10T07:47:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.410+0000 7f85cd600640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 0x7f85b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.410+0000 7f85cd600640 1 -- 172.21.15.3:0/1130777956 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f85c408ae30 con 0x7f85d00d0bb0 2024-09-10T07:47:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.410+0000 7f85d5000640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 0x7f85b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.410+0000 7f85db3c7740 1 Processor -- start 2024-09-10T07:47:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.411+0000 7f85db3c7740 1 -- start start 2024-09-10T07:47:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.411+0000 7f85d5000640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 0x7f85b006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85bc034000 tx=0x7f85bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.411+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f550490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.411+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 0x557b4f50b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85db3c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557b4f4693b0 con 0x557b4f50c480 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557b4f46e410 con 0x557b4f50acf0 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557b4f416620 con 0x557b4f5084b0 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43616/0 (socket says 172.21.15.3:43616) 2024-09-10T07:47:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 -- 172.21.15.3:0/1014611835 learned_addr learned my addr 172.21.15.3:0/1014611835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d5000640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 0x557b4f50b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f550490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 -- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 msgr2=0x557b4f50b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 0x557b4f50b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 -- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 msgr2=0x557b4f550490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f550490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 -- 172.21.15.3:0/1014611835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4f1812d0 con 0x557b4f50c480 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d5000640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 0x557b4f50b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f550490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.412+0000 7f85d6400640 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85c800f230 tx=0x7f85c8001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b636fd02cf60f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85c2000640 1 -- 172.21.15.3:0/1014611835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80072e0 con 0x557b4f50c480 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85c2000640 1 -- 172.21.15.3:0/1014611835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8018070 con 0x557b4f50c480 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85c2000640 1 -- 172.21.15.3:0/1014611835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8007680 con 0x557b4f50c480 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 -- 172.21.15.3:0/1014611835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 msgr2=0x557b4f505570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85c800f230 tx=0x7f85c8001950 comp rx=0 tx=0).stop 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 -- 172.21.15.3:0/1014611835 shutdown_connections 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f505570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f5084b0 0x557b4f50b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1014611835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f550490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 -- 172.21.15.3:0/1014611835 >> 172.21.15.3:0/1014611835 conn(0x557b4f507740 msgr2=0x557b4f54f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.413+0000 7f85db3c7740 1 -- 172.21.15.3:0/1014611835 shutdown_connections 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.414+0000 7f85db3c7740 1 -- 172.21.15.3:0/1014611835 wait complete. 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.414+0000 7f85db3c7740 1 -- start start 2024-09-10T07:47:43.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.414+0000 7f85db3c7740 1 Processor -- start 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f560490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f5298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557b4f429780 con 0x557b4f560b90 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557b4f46a3d0 con 0x557b4f50acf0 2024-09-10T07:47:43.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557b4f506be0 con 0x557b4f50c480 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.415+0000 7f85db3c7740 10 client.0 _ll_register_callbacks cb 0x557b4f4675d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43622/0 (socket says 172.21.15.3:43622) 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 -- 172.21.15.3:0/1476654840 learned_addr learned my addr 172.21.15.3:0/1476654840 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d5000640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f5298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f560490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 -- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 msgr2=0x557b4f5298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f5298c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 -- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 msgr2=0x557b4f560490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f560490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 -- 172.21.15.3:0/1476654840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4f24b6b0 con 0x557b4f560b90 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d5000640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f5298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:43.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85d6400640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85c80012e0 tx=0x7f85c8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2484473a3ee3ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8027430 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.416+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f85c8028c80 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c8018070 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85c802f050 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 10 client.5055 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x557b4f534300 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8018070 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f85c8032020 con 0x557b4f560b90 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 10 client.5055 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 10 client.5055 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 20 client.5055 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f52a7a0 0x557b4f52ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 4 client.5055 mds_command: new command op to 4601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.417+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 --> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4f47bb40 con 0x557b4f52a7a0 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.418+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f52a7a0 0x557b4f52ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.418+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f52a7a0 0x557b4f52ab80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af6374d0978936dd server_cookie=a4e20e1656900aad in_seq=0 out_seq=0 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.418+0000 7f85b6a00640 10 client.5055 ms_handle_connect on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85b6a00640 1 -- 172.21.15.3:0/1476654840 <== mds.2 v2:172.21.15.3:6835/708626559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b4f47bb40 con 0x557b4f52a7a0 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85b6a00640 10 client.5055 handle_command_reply: tid=0 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 client.5055 shutdown 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 msgr2=0x557b4f5092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85c80012e0 tx=0x7f85c8001250 comp rx=0 tx=0).stop 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 shutdown_connections 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f52a7a0 0x557b4f52ab80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f5092c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f5298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85b6a00640 0 client.5055 ms_handle_reset on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1476654840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f560490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 >> 172.21.15.3:0/1476654840 conn(0x557b4f5084b0 msgr2=0x557b4f54ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 shutdown_connections 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 1 -- 172.21.15.3:0/1476654840 wait complete. 2024-09-10T07:47:43.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.419+0000 7f85db3c7740 20 client.5055 trim_cache size 0 max 16384 2024-09-10T07:47:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.438+0000 7f85db3c7740 1 Processor -- start 2024-09-10T07:47:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.438+0000 7f85db3c7740 1 -- start start 2024-09-10T07:47:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f506f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f50a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01ffee0 con 0x557b4f560b90 2024-09-10T07:47:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f85d02005b0 con 0x557b4f50acf0 2024-09-10T07:47:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f85d01ff080 con 0x557b4f50c480 2024-09-10T07:47:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43630/0 (socket says 172.21.15.3:43630) 2024-09-10T07:47:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 -- 172.21.15.3:0/1846445062 learned_addr learned my addr 172.21.15.3:0/1846445062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d5000640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f50a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f506f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 -- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 msgr2=0x557b4f50a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f50a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 -- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 msgr2=0x557b4f506f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f506f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d6400640 1 -- 172.21.15.3:0/1846445062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4f52aef0 con 0x557b4f560b90 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.439+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f506f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85d6400640 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85c8002050 tx=0x7f85c8020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46215192526098bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85c1600640 1 -- 172.21.15.3:0/1846445062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8038070 con 0x557b4f560b90 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85c1600640 1 -- 172.21.15.3:0/1846445062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8018070 con 0x557b4f560b90 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85c1600640 1 -- 172.21.15.3:0/1846445062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8027850 con 0x557b4f560b90 2024-09-10T07:47:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85db3c7740 1 -- 172.21.15.3:0/1846445062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 msgr2=0x557b4f573600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.440+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85c8002050 tx=0x7f85c8020070 comp rx=0 tx=0).stop 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 -- 172.21.15.3:0/1846445062 shutdown_connections 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f560b90 0x557b4f573600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50c480 0x557b4f50a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1846445062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f50acf0 0x557b4f506f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 -- 172.21.15.3:0/1846445062 >> 172.21.15.3:0/1846445062 conn(0x557b4f5084b0 msgr2=0x557b4f5090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 -- 172.21.15.3:0/1846445062 shutdown_connections 2024-09-10T07:47:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 -- 172.21.15.3:0/1846445062 wait complete. 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 -- start start 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.441+0000 7f85db3c7740 1 Processor -- start 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50acf0 0x557b4f5cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f560b90 0x557b4f573910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557b4f529f20 con 0x557b4f50c480 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557b4f560fd0 con 0x557b4f560b90 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557b4f5350f0 con 0x557b4f50acf0 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.442+0000 7f85db3c7740 10 client.0 _ll_register_callbacks cb 0x7f85c8002110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43646/0 (socket says 172.21.15.3:43646) 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 -- 172.21.15.3:0/1548597998 learned_addr learned my addr 172.21.15.3:0/1548597998 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50acf0 0x557b4f5cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d6400640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f560b90 0x557b4f573910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 -- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50acf0 msgr2=0x557b4f5cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50acf0 0x557b4f5cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 -- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f560b90 msgr2=0x557b4f573910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f560b90 0x557b4f573910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.443+0000 7f85d5000640 1 -- 172.21.15.3:0/1548597998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4f54e050 con 0x557b4f50c480 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85d5000640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85bc000c00 tx=0x7f85bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e9ec345d7de154 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc04f070 con 0x557b4f50c480 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f85bc03c910 con 0x557b4f50c480 2024-09-10T07:47:43.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c8018070 con 0x557b4f50c480 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85bc037040 con 0x557b4f50c480 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 10 client.5061 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x557b4f527230 con 0x557b4f50c480 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8018070 con 0x557b4f50c480 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f85bc03cb10 con 0x557b4f50c480 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 10 client.5061 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 10 client.5061 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 20 client.5061 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f551e70 0x557b4f528590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 4 client.5061 mds_command: new command op to 4601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.444+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 --> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] -- command(tid 0: {"prefix": "status"}) -- 0x557b4f475e90 con 0x557b4f551e70 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f551e70 0x557b4f528590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85d5a00640 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f551e70 0x557b4f528590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5892610101f4eebd server_cookie=f643906e42741a2f in_seq=0 out_seq=0 2024-09-10T07:47:43.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85b6a00640 10 client.5061 ms_handle_connect on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85b6a00640 1 -- 172.21.15.3:0/1548597998 <== mds.2 v2:172.21.15.3:6835/708626559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x557b4f475e90 con 0x557b4f551e70 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85b6a00640 10 client.5061 handle_command_reply: tid=0 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.445+0000 7f85db3c7740 1 client.5061 shutdown 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 msgr2=0x557b4f5733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85bc000c00 tx=0x7f85bc004730 comp rx=0 tx=0).stop 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 shutdown_connections 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x557b4f551e70 0x557b4f528590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557b4f560b90 0x557b4f573910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b4f50c480 0x557b4f5733d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 --2- 172.21.15.3:0/1548597998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557b4f50acf0 0x557b4f5cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85b6a00640 0 client.5061 ms_handle_reset on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 >> 172.21.15.3:0/1548597998 conn(0x557b4f5084b0 msgr2=0x557b4f506f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 shutdown_connections 2024-09-10T07:47:43.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 1 -- 172.21.15.3:0/1548597998 wait complete. 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.446+0000 7f85db3c7740 20 client.5061 trim_cache size 0 max 16384 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4601, 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.8625024809999999, 2024-09-10T07:47:43.450 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-09-10T07:47:43.451 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 36, 2024-09-10T07:47:43.451 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 36, 2024-09-10T07:47:43.451 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 29.447703288 2024-09-10T07:47:43.451 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:43.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 msgr2=0x7f85b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 0x7f85b006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85bc034000 tx=0x7f85bc02e000 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 msgr2=0x7f85d01e8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9b0 tx=0x7f85c400ee70 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 shutdown_connections 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f85b006be30 0x7f85b006e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85d015d040 0x7f85d0159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f85d00d0f90 0x7f85d0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 --2- 172.21.15.3:0/1130777956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d00d0bb0 0x7f85d01e8c20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 >> 172.21.15.3:0/1130777956 conn(0x7f85d006bf40 msgr2=0x7f85d00b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 shutdown_connections 2024-09-10T07:47:43.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.447+0000 7f85d8400640 1 -- 172.21.15.3:0/1130777956 wait complete. 2024-09-10T07:47:43.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T07:47:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.620+0000 7fa20b000640 1 Processor -- start 2024-09-10T07:47:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.620+0000 7fa20b000640 1 -- start start 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa20b000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 0x7fa204167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa20b000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2040cfdd0 con 0x7fa204160710 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa20b000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20415cc90 con 0x7fa204160af0 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa20b000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa20415cf70 con 0x7fa2040d07c0 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa203e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa204160710 0x7fa2040cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35386/0 (socket says 172.21.15.3:35386) 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.621+0000 7fa203e00640 1 -- 172.21.15.3:0/590713562 learned_addr learned my addr 172.21.15.3:0/590713562 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa209000640 1 --2- 172.21.15.3:0/590713562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 0x7fa204167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085093839 0 0) 0x7fa2040cfdd0 con 0x7fa204160710 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e8003520 con 0x7fa204160710 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816195803 0 0) 0x7fa20415cc90 con 0x7fa204160af0 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa209000640 1 -- 172.21.15.3:0/590713562 >> v1:172.21.15.70:6789/0 conn(0x7fa204160af0 legacy=0x7fa2040d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa209000640 1 -- 172.21.15.3:0/590713562 >> v1:172.21.15.3:6789/0 conn(0x7fa204160710 legacy=0x7fa2040cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.622+0000 7fa209000640 1 -- 172.21.15.3:0/590713562 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20415d250 con 0x7fa2040d07c0 2024-09-10T07:47:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.623+0000 7fa209000640 1 --2- 172.21.15.3:0/590713562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 0x7fa204167090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009dc0 tx=0x7fa1fc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726b0fcedc0b3c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.623+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc007740 con 0x7fa2040d07c0 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.623+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1fc0078e0 con 0x7fa2040d07c0 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.623+0000 7fa202a00640 1 -- 172.21.15.3:0/590713562 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc007cb0 con 0x7fa2040d07c0 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 -- 172.21.15.3:0/590713562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 msgr2=0x7fa204167090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 --2- 172.21.15.3:0/590713562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 0x7fa204167090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009dc0 tx=0x7fa1fc02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 -- 172.21.15.3:0/590713562 shutdown_connections 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 --2- 172.21.15.3:0/590713562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2040d07c0 0x7fa204167090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 -- 172.21.15.3:0/590713562 >> 172.21.15.3:0/590713562 conn(0x7fa20406d000 msgr2=0x7fa20406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 -- 172.21.15.3:0/590713562 shutdown_connections 2024-09-10T07:47:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.624+0000 7fa20b000640 1 -- 172.21.15.3:0/590713562 wait complete. 2024-09-10T07:47:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.626+0000 7fa20b000640 1 Processor -- start 2024-09-10T07:47:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.626+0000 7fa20b000640 1 -- start start 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2040d07c0 0x7fa204146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa204160af0 0x7fa20415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa20415d770 con 0x7fa204160710 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa204073080 con 0x7fa2040d07c0 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa20b000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa2041f84c0 con 0x7fa204160af0 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43656/0 (socket says 172.21.15.3:43656) 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 -- 172.21.15.3:0/108981543 learned_addr learned my addr 172.21.15.3:0/108981543 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203e00640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2040d07c0 0x7fa204146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa209000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa204160af0 0x7fa20415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 -- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa204160af0 msgr2=0x7fa20415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa204160af0 0x7fa20415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 -- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2040d07c0 msgr2=0x7fa204146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2040d07c0 0x7fa204146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.627+0000 7fa203400640 1 -- 172.21.15.3:0/108981543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2041f87a0 con 0x7fa204160710 2024-09-10T07:47:43.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa203400640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c410 tx=0x7fa1f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa1fbe00640 1 -- 172.21.15.3:0/108981543 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0017070 con 0x7fa204160710 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa1fbe00640 1 -- 172.21.15.3:0/108981543 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f00035f0 con 0x7fa204160710 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa1fbe00640 1 -- 172.21.15.3:0/108981543 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00039e0 con 0x7fa204160710 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041543d0 con 0x7fa204160710 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.628+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041473a0 con 0x7fa204160710 2024-09-10T07:47:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.630+0000 7fa1fbe00640 1 -- 172.21.15.3:0/108981543 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1f001a070 con 0x7fa204160710 2024-09-10T07:47:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.631+0000 7fa1fbe00640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.631+0000 7fa1fbe00640 1 -- 172.21.15.3:0/108981543 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f008a2b0 con 0x7fa204160710 2024-09-10T07:47:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.631+0000 7fa203e00640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.632+0000 7fa20ded1740 1 Processor -- start 2024-09-10T07:47:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.632+0000 7fa203e00640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 0x7fa1e406e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005880 tx=0x7fa1f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.632+0000 7fa20ded1740 1 -- start start 2024-09-10T07:47:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822563cf0 0x5558225a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582255e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5558224c23b0 con 0x5558225614b0 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5558224c7410 con 0x555822563cf0 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55582246f620 con 0x555822565480 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43666/0 (socket says 172.21.15.3:43666) 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 -- 172.21.15.3:0/1508538776 learned_addr learned my addr 172.21.15.3:0/1508538776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203e00640 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822563cf0 0x5558225a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa209000640 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582255e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 -- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 msgr2=0x55582255e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582255e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 -- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822563cf0 msgr2=0x5558225a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822563cf0 0x5558225a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 -- 172.21.15.3:0/1508538776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558221da2d0 con 0x5558225614b0 2024-09-10T07:47:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.633+0000 7fa203400640 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0060b70 tx=0x7fa1f005c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb8001deb40233a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa1f8c00640 1 -- 172.21.15.3:0/1508538776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0056870 con 0x5558225614b0 2024-09-10T07:47:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa1f8c00640 1 -- 172.21.15.3:0/1508538776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0054be0 con 0x5558225614b0 2024-09-10T07:47:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa1f8c00640 1 -- 172.21.15.3:0/1508538776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0088de0 con 0x5558225614b0 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 -- 172.21.15.3:0/1508538776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 msgr2=0x5558225640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0060b70 tx=0x7fa1f005c870 comp rx=0 tx=0).stop 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 -- 172.21.15.3:0/1508538776 shutdown_connections 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582255e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225614b0 0x5558225640d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 --2- 172.21.15.3:0/1508538776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822563cf0 0x5558225a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.634+0000 7fa20ded1740 1 -- 172.21.15.3:0/1508538776 >> 172.21.15.3:0/1508538776 conn(0x555822560740 msgr2=0x5558225a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.635+0000 7fa20ded1740 1 -- 172.21.15.3:0/1508538776 shutdown_connections 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.635+0000 7fa20ded1740 1 -- 172.21.15.3:0/1508538776 wait complete. 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.635+0000 7fa20ded1740 1 -- start start 2024-09-10T07:47:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.635+0000 7fa20ded1740 1 Processor -- start 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822563cf0 0x5558225a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555822482780 con 0x5558225a5930 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5558224c33d0 con 0x555822565480 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.636+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55582255fbe0 con 0x555822563cf0 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa20ded1740 10 client.0 _ll_register_callbacks cb 0x5558224c05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43670/0 (socket says 172.21.15.3:43670) 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 -- 172.21.15.3:0/2325151753 learned_addr learned my addr 172.21.15.3:0/2325151753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa203e00640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822563cf0 0x5558225a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa203400640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 -- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822563cf0 msgr2=0x5558225a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822563cf0 0x5558225a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 -- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 msgr2=0x55582258bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.637+0000 7fa209000640 1 -- 172.21.15.3:0/2325151753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558222a46b0 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa209000640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009eb0 tx=0x7fa1fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c826a705a2cb1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc009450 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f0054be0 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc004530 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc004780 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 10 client.5073 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55582258cab0 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0054be0 con 0x5558225a5930 2024-09-10T07:47:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fa1fc007540 con 0x5558225a5930 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 10 client.5073 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 10 client.5073 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.638+0000 7fa20ded1740 20 client.5073 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.639+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225bc920 0x5558225bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.639+0000 7fa20ded1740 4 client.5073 mds_command: new command op to 4607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.639+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 --> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558224d4b40 con 0x5558225bc920 2024-09-10T07:47:43.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.639+0000 7fa203e00640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225bc920 0x5558225bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.639+0000 7fa203e00640 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225bc920 0x5558225bcd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8ffcb599f03c87 server_cookie=2dde5f178cf135a4 in_seq=0 out_seq=0 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.640+0000 7fa1ed600640 10 client.5073 ms_handle_connect on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.640+0000 7fa1ed600640 1 -- 172.21.15.3:0/2325151753 <== mds.1 v2:172.21.15.70:6834/2716956690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558224d4b40 con 0x5558225bc920 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa1ed600640 10 client.5073 handle_command_reply: tid=0 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 client.5073 shutdown 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 msgr2=0x5558225a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009eb0 tx=0x7fa1fc004ab0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 shutdown_connections 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225bc920 0x5558225bcd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558225a5930 0x5558225a5db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 --2- 172.21.15.3:0/2325151753 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822563cf0 0x5558225a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa1ed600640 0 client.5073 ms_handle_reset on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 >> 172.21.15.3:0/2325151753 conn(0x5558225614b0 msgr2=0x5558225a69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 shutdown_connections 2024-09-10T07:47:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 1 -- 172.21.15.3:0/2325151753 wait complete. 2024-09-10T07:47:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.641+0000 7fa20ded1740 20 client.5073 trim_cache size 0 max 16384 2024-09-10T07:47:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.660+0000 7fa20ded1740 1 Processor -- start 2024-09-10T07:47:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.660+0000 7fa20ded1740 1 -- start start 2024-09-10T07:47:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582258bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 0x5558225b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2042008d0 con 0x555822563cf0 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041ffb30 con 0x5558225a5930 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa204200200 con 0x555822565480 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43678/0 (socket says 172.21.15.3:43678) 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 -- 172.21.15.3:0/4175434015 learned_addr learned my addr 172.21.15.3:0/4175434015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa209000640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 0x5558225b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 -- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 msgr2=0x55582258bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203400640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582258bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582258bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 -- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 msgr2=0x5558225b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 0x5558225b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203e00640 1 -- 172.21.15.3:0/4175434015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558224d2a20 con 0x555822563cf0 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa209000640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 0x5558225b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:43.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.661+0000 7fa203400640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582258bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa203e00640 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000f60 tx=0x7fa1f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceca36354047ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa1efe00640 1 -- 172.21.15.3:0/4175434015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40161d0 con 0x555822563cf0 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa1efe00640 1 -- 172.21.15.3:0/4175434015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f4016970 con 0x555822563cf0 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa1efe00640 1 -- 172.21.15.3:0/4175434015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4016b20 con 0x555822563cf0 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 -- 172.21.15.3:0/4175434015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 msgr2=0x55582258b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000f60 tx=0x7fa1f4000ed0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 -- 172.21.15.3:0/4175434015 shutdown_connections 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558225a5930 0x5558225b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555822565480 0x55582258bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 --2- 172.21.15.3:0/4175434015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258b6d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.662+0000 7fa20ded1740 1 -- 172.21.15.3:0/4175434015 >> 172.21.15.3:0/4175434015 conn(0x5558225614b0 msgr2=0x555822559df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.663+0000 7fa20ded1740 1 -- 172.21.15.3:0/4175434015 shutdown_connections 2024-09-10T07:47:43.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.663+0000 7fa20ded1740 1 -- 172.21.15.3:0/4175434015 wait complete. 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.663+0000 7fa20ded1740 1 -- start start 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.663+0000 7fa20ded1740 1 Processor -- start 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558225a5930 0x555822626850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55582258d350 con 0x555822563cf0 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041fec40 con 0x555822565480 2024-09-10T07:47:43.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55582258c2a0 con 0x5558225a5930 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa20ded1740 10 client.0 _ll_register_callbacks cb 0x7fa1fc009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.664+0000 7fa203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43690/0 (socket says 172.21.15.3:43690) 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 -- 172.21.15.3:0/254457596 learned_addr learned my addr 172.21.15.3:0/254457596 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 -- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558225a5930 msgr2=0x555822626850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558225a5930 0x555822626850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 -- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 msgr2=0x55582258eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 -- 172.21.15.3:0/254457596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558225a7670 con 0x555822563cf0 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa203e00640 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40162a0 tx=0x7fa1f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3cf89892d8505c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40049e0 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f4004b80 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.665+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f4016970 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa1f4016e70 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 10 client.5079 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5558225a6220 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f4016970 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7fa1f402b020 con 0x555822563cf0 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 10 client.5079 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 10 client.5079 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 20 client.5079 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225cbef0 0x5558225cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 4 client.5079 mds_command: new command op to 4607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:47:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 --> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] -- command(tid 0: {"prefix": "status"}) -- 0x5558224cee90 con 0x5558225cbef0 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.666+0000 7fa203400640 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225cbef0 0x5558225cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa203400640 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225cbef0 0x5558225cc2d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6d6b1a996fcd66a server_cookie=968f2aeec85d1c11 in_seq=0 out_seq=0 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa1ed600640 10 client.5079 ms_handle_connect on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa1ed600640 1 -- 172.21.15.3:0/254457596 <== mds.1 v2:172.21.15.70:6834/2716956690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5558224cee90 con 0x5558225cbef0 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa1ed600640 10 client.5079 handle_command_reply: tid=0 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa20ded1740 1 client.5079 shutdown 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 msgr2=0x55582258e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.667+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40162a0 tx=0x7fa1f4000bc0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 shutdown_connections 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x5558225cbef0 0x5558225cc2d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558225a5930 0x555822626850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555822565480 0x55582258eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 --2- 172.21.15.3:0/254457596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555822563cf0 0x55582258e560 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa1ed600640 0 client.5079 ms_handle_reset on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 >> 172.21.15.3:0/254457596 conn(0x5558225614b0 msgr2=0x5558225aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 shutdown_connections 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 1 -- 172.21.15.3:0/254457596 wait complete. 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20ded1740 20 client.5079 trim_cache size 0 max 16384 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4607, 2024-09-10T07:47:43.672 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.0909111509999998, 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 36, 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 36, 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 23.707042058999999 2024-09-10T07:47:43.673 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 msgr2=0x7fa1e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 0x7fa1e406e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005880 tx=0x7fa1f4009040 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 msgr2=0x7fa2041f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c410 tx=0x7fa1f000c8d0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 shutdown_connections 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa1e406be30 0x7fa1e406e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa204160af0 0x7fa20415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa204160710 0x7fa2041f7ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 --2- 172.21.15.3:0/108981543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2040d07c0 0x7fa204146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.668+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 >> 172.21.15.3:0/108981543 conn(0x7fa20406d000 msgr2=0x7fa204074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.669+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 shutdown_connections 2024-09-10T07:47:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.669+0000 7fa20b000640 1 -- 172.21.15.3:0/108981543 wait complete. 2024-09-10T07:47:43.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:47:43.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.846+0000 7f7dbb200640 1 Processor -- start 2024-09-10T07:47:43.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.846+0000 7f7dbb200640 1 -- start start 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 0x7f7dbc0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0d4150 0x7f7dbc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc0d3780 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc0cee40 con 0x7f7dbc15e860 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7dbb200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc0cf120 con 0x7f7dbc0d4150 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.847+0000 7f7db9400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54620/0 (socket says 172.21.15.3:54620) 2024-09-10T07:47:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 -- 172.21.15.3:0/4135054518 learned_addr learned my addr 172.21.15.3:0/4135054518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9e00640 1 --2- 172.21.15.3:0/4135054518 >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 0x7f7dbc0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7dba800640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0d4150 0x7f7dbc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 -- 172.21.15.3:0/4135054518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0d4150 msgr2=0x7f7dbc0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0d4150 0x7f7dbc0b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 -- 172.21.15.3:0/4135054518 >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 msgr2=0x7f7dbc0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 --2- 172.21.15.3:0/4135054518 >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 0x7f7dbc0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9400640 1 -- 172.21.15.3:0/4135054518 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc0d3410 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.848+0000 7f7db9e00640 1 --2- 172.21.15.3:0/4135054518 >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 0x7f7dbc0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.849+0000 7f7db9400640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0ba7c0 tx=0x7f7da802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bded3d814d26aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.849+0000 7f7db8a00640 1 -- 172.21.15.3:0/4135054518 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8037070 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7db8a00640 1 -- 172.21.15.3:0/4135054518 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da8032440 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7db8a00640 1 -- 172.21.15.3:0/4135054518 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8032810 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 -- 172.21.15.3:0/4135054518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0cfbf0 msgr2=0x7f7dbc0d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0ba7c0 tx=0x7f7da802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 -- 172.21.15.3:0/4135054518 shutdown_connections 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0d4150 0x7f7dbc0b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 --2- 172.21.15.3:0/4135054518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0cfbf0 0x7f7dbc0d3c10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 --2- 172.21.15.3:0/4135054518 >> v2:172.21.15.70:3300/0 conn(0x7f7dbc15e860 0x7f7dbc0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 -- 172.21.15.3:0/4135054518 >> 172.21.15.3:0/4135054518 conn(0x7f7dbc0d4650 msgr2=0x7f7dbc0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 -- 172.21.15.3:0/4135054518 shutdown_connections 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.850+0000 7f7dbb200640 1 -- 172.21.15.3:0/4135054518 wait complete. 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.852+0000 7f7dbb200640 1 Processor -- start 2024-09-10T07:47:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.852+0000 7f7dbb200640 1 -- start start 2024-09-10T07:47:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0cfbf0 0x7f7dbc149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 0x7f7dbc160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0b7dc0 con 0x7f7dbc0d4150 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0d2460 con 0x7f7dbc15e860 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dbb200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc0b7f40 con 0x7f7dbc0cfbf0 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54626/0 (socket says 172.21.15.3:54626) 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7db9400640 1 -- 172.21.15.3:0/71592798 learned_addr learned my addr 172.21.15.3:0/71592798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7db9e00640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0cfbf0 0x7f7dbc149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.853+0000 7f7dba800640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 0x7f7dbc160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 -- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0cfbf0 msgr2=0x7f7dbc149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0cfbf0 0x7f7dbc149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 -- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 msgr2=0x7f7dbc160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 0x7f7dbc160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 -- 172.21.15.3:0/71592798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc15f6f0 con 0x7f7dbc0d4150 2024-09-10T07:47:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7dba800640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 0x7f7dbc160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db9400640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db2000640 1 -- 172.21.15.3:0/71592798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8037040 con 0x7f7dbc0d4150 2024-09-10T07:47:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db2000640 1 -- 172.21.15.3:0/71592798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da8042070 con 0x7f7dbc0d4150 2024-09-10T07:47:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7db2000640 1 -- 172.21.15.3:0/71592798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8030880 con 0x7f7dbc0d4150 2024-09-10T07:47:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.854+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc15f9d0 con 0x7f7dbc0d4150 2024-09-10T07:47:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.855+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc15ff90 con 0x7f7dbc0d4150 2024-09-10T07:47:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.857+0000 7f7db2000640 1 -- 172.21.15.3:0/71592798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7da8030a20 con 0x7f7dbc0d4150 2024-09-10T07:47:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.857+0000 7f7db2000640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 0x7f7d9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.858+0000 7f7db2000640 1 -- 172.21.15.3:0/71592798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f7da80ab6a0 con 0x7f7dbc0d4150 2024-09-10T07:47:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.858+0000 7f7db9e00640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 0x7f7d9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.858+0000 7f7dc3a84740 1 Processor -- start 2024-09-10T07:47:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.858+0000 7f7db9e00640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 0x7f7d9c06e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7dac006fd0 tx=0x7f7dac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.858+0000 7f7dc3a84740 1 -- start start 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562aece7f3b0 con 0x562aecf1e660 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562aece84410 con 0x562aecf20ea0 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562aece2c620 con 0x562aecf22630 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.859+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54636/0 (socket says 172.21.15.3:54636) 2024-09-10T07:47:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 -- 172.21.15.3:0/1642421660 learned_addr learned my addr 172.21.15.3:0/1642421660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9e00640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7dba800640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 -- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 msgr2=0x562aecf1b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf1b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 -- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 msgr2=0x562aecf66640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 -- 172.21.15.3:0/1642421660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aecb972d0 con 0x562aecf1e660 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9e00640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7db9400640 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7da8079470 tx=0x7f7da8075c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4790d1ffcb7e126 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7da6600640 1 -- 172.21.15.3:0/1642421660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80798b0 con 0x562aecf1e660 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7da6600640 1 -- 172.21.15.3:0/1642421660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da807dc60 con 0x562aecf1e660 2024-09-10T07:47:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.860+0000 7f7da6600640 1 -- 172.21.15.3:0/1642421660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8082490 con 0x562aecf1e660 2024-09-10T07:47:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 -- 172.21.15.3:0/1642421660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 msgr2=0x562aecf21280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7da8079470 tx=0x7f7da8075c80 comp rx=0 tx=0).stop 2024-09-10T07:47:43.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 -- 172.21.15.3:0/1642421660 shutdown_connections 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf1e660 0x562aecf21280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/1642421660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 -- 172.21.15.3:0/1642421660 >> 172.21.15.3:0/1642421660 conn(0x562aecf1d8f0 msgr2=0x562aecf65b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 -- 172.21.15.3:0/1642421660 shutdown_connections 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.861+0000 7f7dc3a84740 1 -- 172.21.15.3:0/1642421660 wait complete. 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.862+0000 7f7dc3a84740 1 -- start start 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.862+0000 7f7dc3a84740 1 Processor -- start 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf43920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562aece3f780 con 0x562aecf5fae0 2024-09-10T07:47:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562aece803d0 con 0x562aecf20ea0 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562aece84410 con 0x562aecf22630 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dc3a84740 10 client.0 _ll_register_callbacks cb 0x562aece7d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dba800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dba800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54652/0 (socket says 172.21.15.3:54652) 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7dba800640 1 -- 172.21.15.3:0/11527164 learned_addr learned my addr 172.21.15.3:0/11527164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7db9e00640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.863+0000 7f7db9400640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf43920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 -- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 msgr2=0x562aecf43920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf43920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 -- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 msgr2=0x562aecf79d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf79d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 -- 172.21.15.3:0/11527164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aecc616b0 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dba800640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007d00 tx=0x7f7db4014610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5606ff74762985 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7d9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db401c070 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da807dc60 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.864+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f7db4018640 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7db4027050 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 10 client.5091 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x562aecf63040 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da807dc60 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f7db402a020 con 0x562aecf5fae0 2024-09-10T07:47:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 10 client.5091 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 10 client.5091 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 20 client.5091 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf1af60 0x562aecf76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 4 client.5091 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.865+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aece91b40 con 0x562aecf1af60 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.866+0000 7f7db9e00640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf1af60 0x562aecf76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.866+0000 7f7db9e00640 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf1af60 0x562aecf76490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f500c73638bbd5 server_cookie=80ca4a0fbe61a6a1 in_seq=0 out_seq=0 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.866+0000 7f7d9aa00640 10 client.5091 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7d9aa00640 1 -- 172.21.15.3:0/11527164 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562aece91b40 con 0x562aecf1af60 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7d9aa00640 10 client.5091 handle_command_reply: tid=0 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 client.5091 shutdown 2024-09-10T07:47:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 msgr2=0x562aecf60140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007d00 tx=0x7f7db4014610 comp rx=0 tx=0).stop 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 shutdown_connections 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf1af60 0x562aecf76490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf5fae0 0x562aecf60140 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf43920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/11527164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7d9aa00640 0 client.5091 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 >> 172.21.15.3:0/11527164 conn(0x562aecf1e660 msgr2=0x562aecf65330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.867+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 shutdown_connections 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.868+0000 7f7dc3a84740 1 -- 172.21.15.3:0/11527164 wait complete. 2024-09-10T07:47:43.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.868+0000 7f7dc3a84740 20 client.5091 trim_cache size 0 max 16384 2024-09-10T07:47:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.886+0000 7f7dc3a84740 1 Processor -- start 2024-09-10T07:47:43.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.886+0000 7f7dc3a84740 1 -- start start 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf5fae0 0x562aecf88d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc1ffdf0 con 0x562aecf20ea0 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc2004c0 con 0x562aecf5fae0 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc1feeb0 con 0x562aecf22630 2024-09-10T07:47:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.887+0000 7f7db9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54668/0 (socket says 172.21.15.3:54668) 2024-09-10T07:47:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 -- 172.21.15.3:0/3629651902 learned_addr learned my addr 172.21.15.3:0/3629651902 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7dba800640 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf5fae0 0x562aecf88d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9400640 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 -- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 msgr2=0x562aecf88820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf88820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 -- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf5fae0 msgr2=0x562aecf88d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf5fae0 0x562aecf88d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 -- 172.21.15.3:0/3629651902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aecf1d690 con 0x562aecf20ea0 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7db9e00640 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x562aecf75e60 tx=0x7f7dac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed3b474fe8de058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7da5c00640 1 -- 172.21.15.3:0/3629651902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac01c040 con 0x562aecf20ea0 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7da5c00640 1 -- 172.21.15.3:0/3629651902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac028030 con 0x562aecf20ea0 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.888+0000 7f7da5c00640 1 -- 172.21.15.3:0/3629651902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac024440 con 0x562aecf20ea0 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 -- 172.21.15.3:0/3629651902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 msgr2=0x562aecf75270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x562aecf75e60 tx=0x7f7dac004610 comp rx=0 tx=0).stop 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 -- 172.21.15.3:0/3629651902 shutdown_connections 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf5fae0 0x562aecf88d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf22630 0x562aecf88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/3629651902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf20ea0 0x562aecf75270 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 -- 172.21.15.3:0/3629651902 >> 172.21.15.3:0/3629651902 conn(0x562aecf1e660 msgr2=0x562aecf66c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 -- 172.21.15.3:0/3629651902 shutdown_connections 2024-09-10T07:47:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.889+0000 7f7dc3a84740 1 -- 172.21.15.3:0/3629651902 wait complete. 2024-09-10T07:47:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.890+0000 7f7dc3a84740 1 -- start start 2024-09-10T07:47:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.890+0000 7f7dc3a84740 1 Processor -- start 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf5fae0 0x562aecfe3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562aecf1ade0 con 0x562aecf22630 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562aece7f3b0 con 0x562aecf20ea0 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562aecf1b430 con 0x562aecf5fae0 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dc3a84740 10 client.0 _ll_register_callbacks cb 0x7f7db4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7db9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54680/0 (socket says 172.21.15.3:54680) 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7db9400640 1 -- 172.21.15.3:0/231993684 learned_addr learned my addr 172.21.15.3:0/231993684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7dba800640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf5fae0 0x562aecfe3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.891+0000 7f7db9e00640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 -- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf5fae0 msgr2=0x562aecfe3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf5fae0 0x562aecfe3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 -- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 msgr2=0x562aecf66c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 -- 172.21.15.3:0/231993684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aecf789b0 con 0x562aecf22630 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7db9400640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7da80795b0 tx=0x7f7da803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fb0ed6cb2dc770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7d9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8082490 con 0x562aecf22630 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f7da8030d10 con 0x562aecf22630 2024-09-10T07:47:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dac028030 con 0x562aecf22630 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7da80798b0 con 0x562aecf22630 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7dc3a84740 10 client.5097 fetch_fsmap learned FSMap version 22 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x562aecf7d700 con 0x562aecf22630 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.892+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac028030 con 0x562aecf22630 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 22) ==== 3542+0+0 (secure 0 0 0) 0x7f7da803a070 con 0x562aecf22630 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 10 client.5097 fetch_fsmap finished waiting for FSMap version 22 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 10 client.5097 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 20 client.5097 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf47720 0x562aecf47b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 4 client.5097 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "status"}) -- 0x562aece8be90 con 0x562aecf47720 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7db9e00640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf47720 0x562aecf47b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7db9e00640 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf47720 0x562aecf47b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c045a4ed3a5851 server_cookie=a2314c961c76acc in_seq=0 out_seq=0 2024-09-10T07:47:43.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.893+0000 7f7d9aa00640 10 client.5097 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7d9aa00640 1 -- 172.21.15.3:0/231993684 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562aece8be90 con 0x562aecf47720 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7d9aa00640 10 client.5097 handle_command_reply: tid=0 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 client.5097 shutdown 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 msgr2=0x562aecf7d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7da80795b0 tx=0x7f7da803f040 comp rx=0 tx=0).stop 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 shutdown_connections 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x562aecf47720 0x562aecf47b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562aecf5fae0 0x562aecfe3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aecf22630 0x562aecf7d1c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 --2- 172.21.15.3:0/231993684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aecf20ea0 0x562aecf66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 >> 172.21.15.3:0/231993684 conn(0x562aecf1e660 msgr2=0x562aecf63e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7d9aa00640 0 client.5097 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 shutdown_connections 2024-09-10T07:47:43.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 1 -- 172.21.15.3:0/231993684 wait complete. 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.894+0000 7f7dc3a84740 20 client.5097 trim_cache size 0 max 16384 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4741, 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 16.195995117999999, 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 22, 2024-09-10T07:47:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 36, 2024-09-10T07:47:43.899 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 36, 2024-09-10T07:47:43.899 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 29.896624665000001 2024-09-10T07:47:43.899 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 msgr2=0x7f7d9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 0x7f7d9c06e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7dac006fd0 tx=0x7f7dac008040 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 msgr2=0x7f7dbc0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da80046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 shutdown_connections 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d9c06be60 0x7f7d9c06e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7dbc15e860 0x7f7dbc160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc0d4150 0x7f7dbc0cdf30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 --2- 172.21.15.3:0/71592798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7dbc0cfbf0 0x7f7dbc149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 >> 172.21.15.3:0/71592798 conn(0x7f7dbc0d4650 msgr2=0x7f7dbc072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 shutdown_connections 2024-09-10T07:47:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:43.895+0000 7f7dbb200640 1 -- 172.21.15.3:0/71592798 wait complete. 2024-09-10T07:47:43.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:47:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.067+0000 7f06cd800640 1 Processor -- start 2024-09-10T07:47:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.067+0000 7f06cd800640 1 -- start start 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c8162ac0 0x7f06c80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06c80cd7d0 0x7f06c8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06c80713c0 0x7f06c80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f06c80cf930 con 0x7f06c80713c0 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f06c80cfc10 con 0x7f06c80cd7d0 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06cd800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f06c80b38d0 con 0x7f06c8162ac0 2024-09-10T07:47:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.068+0000 7f06c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06c80713c0 0x7f06c80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06c80713c0 0x7f06c80b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54692/0 (socket says 172.21.15.3:54692) 2024-09-10T07:47:44.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 -- 172.21.15.3:0/3345510142 learned_addr learned my addr 172.21.15.3:0/3345510142 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c6a00640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c8162ac0 0x7f06c80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c6000640 1 --2- 172.21.15.3:0/3345510142 >> v2:172.21.15.70:3300/0 conn(0x7f06c80cd7d0 0x7f06c8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 -- 172.21.15.3:0/3345510142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c8162ac0 msgr2=0x7f06c80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c8162ac0 0x7f06c80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 -- 172.21.15.3:0/3345510142 >> v2:172.21.15.70:3300/0 conn(0x7f06c80cd7d0 msgr2=0x7f06c8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 --2- 172.21.15.3:0/3345510142 >> v2:172.21.15.70:3300/0 conn(0x7f06c80cd7d0 0x7f06c8070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.069+0000 7f06c7400640 1 -- 172.21.15.3:0/3345510142 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80b3bb0 con 0x7f06c80713c0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.070+0000 7f06c7400640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c80713c0 0x7f06c80b40d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06c806e3b0 tx=0x7f06c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaba78f8c5d1e43d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.070+0000 7f06c5600640 1 -- 172.21.15.3:0/3345510142 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0036070 con 0x7f06c80713c0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.070+0000 7f06c5600640 1 -- 172.21.15.3:0/3345510142 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0031600 con 0x7f06c80713c0 2024-09-10T07:47:44.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.070+0000 7f06c5600640 1 -- 172.21.15.3:0/3345510142 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00319d0 con 0x7f06c80713c0 2024-09-10T07:47:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 -- 172.21.15.3:0/3345510142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c80713c0 msgr2=0x7f06c80b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c80713c0 0x7f06c80b40d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06c806e3b0 tx=0x7f06c002a6a0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 -- 172.21.15.3:0/3345510142 shutdown_connections 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c80713c0 0x7f06c80b40d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 --2- 172.21.15.3:0/3345510142 >> v2:172.21.15.70:3300/0 conn(0x7f06c80cd7d0 0x7f06c8070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 --2- 172.21.15.3:0/3345510142 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c8162ac0 0x7f06c80cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 -- 172.21.15.3:0/3345510142 >> 172.21.15.3:0/3345510142 conn(0x7f06c806d000 msgr2=0x7f06c806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 -- 172.21.15.3:0/3345510142 shutdown_connections 2024-09-10T07:47:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.071+0000 7f06cd800640 1 -- 172.21.15.3:0/3345510142 wait complete. 2024-09-10T07:47:44.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.072+0000 7f06cd800640 1 Processor -- start 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 -- start start 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 0x7f06c8155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c80cd7d0 0x7f06c80aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8163020 con 0x7f06c8162ac0 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8162ea0 con 0x7f06c80cd7d0 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.073+0000 7f06cd800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f06c80cdc10 con 0x7f06c80713c0 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54696/0 (socket says 172.21.15.3:54696) 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 -- 172.21.15.3:0/2665995120 learned_addr learned my addr 172.21.15.3:0/2665995120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:44.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c6a00640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 0x7f06c8155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c6000640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c80cd7d0 0x7f06c80aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 -- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 msgr2=0x7f06c8155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 0x7f06c8155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 -- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c80cd7d0 msgr2=0x7f06c80aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c80cd7d0 0x7f06c80aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 -- 172.21.15.3:0/2665995120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80b00a0 con 0x7f06c8162ac0 2024-09-10T07:47:44.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c6a00640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 0x7f06c8155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.074+0000 7f06c7400640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06c002ac90 tx=0x7f06c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.075+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c003f020 con 0x7f06c8162ac0 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.075+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0037040 con 0x7f06c8162ac0 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.075+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0036070 con 0x7f06c8162ac0 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.075+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c80b0340 con 0x7f06c8162ac0 2024-09-10T07:47:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.075+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c80b0900 con 0x7f06c8162ac0 2024-09-10T07:47:44.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.076+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c8070e80 con 0x7f06c8162ac0 2024-09-10T07:47:44.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.077+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06c0042020 con 0x7f06c8162ac0 2024-09-10T07:47:44.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.078+0000 7f06bea00640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.078+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f06c00ab2d0 con 0x7f06c8162ac0 2024-09-10T07:47:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.078+0000 7f06c6a00640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:44.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.079+0000 7f06c6a00640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 0x7f06a806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.082+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06c0049030 con 0x7f06c8162ac0 2024-09-10T07:47:44.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.303+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c80b0ad0 con 0x7f06c8162ac0 2024-09-10T07:47:44.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.304+0000 7f06bea00640 1 -- 172.21.15.3:0/2665995120 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5252 (secure 0 0 0) 0x7f06c0079d10 con 0x7f06c8162ac0 2024-09-10T07:47:44.304 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:47:44.304 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"btime":"2024-09-10T07:47:42:583951+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:42.583947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4601,4607,4741]},"id":2}]} 2024-09-10T07:47:44.304 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 22 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 msgr2=0x7f06a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 0x7f06a806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 msgr2=0x7f06c80acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06c002ac90 tx=0x7f06c0004a40 comp rx=0 tx=0).stop 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 shutdown_connections 2024-09-10T07:47:44.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06c8162ac0 0x7f06c80acb50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c80cd7d0 0x7f06c80aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 --2- 172.21.15.3:0/2665995120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06c80713c0 0x7f06c8155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.306+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 >> 172.21.15.3:0/2665995120 conn(0x7f06c806d000 msgr2=0x7f06c815f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.307+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 shutdown_connections 2024-09-10T07:47:44.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:44.307+0000 7f06cd800640 1 -- 172.21.15.3:0/2665995120 wait complete. 2024-09-10T07:47:44.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:44.323 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:47:44.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:44.370 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:47:44.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:44.413 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:47:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:44.526 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:47:44.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:44.561 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:47:44.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:44.853 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:47:44.854 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:44.854 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:46.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:47.041 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:47:47.041 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:47.041 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:49.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:49.230 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:47:49.231 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:49.231 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:51.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:47:51.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:47:51.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:51.410 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:47:51.411 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:51.411 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:53.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:53.600 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:47:53.601 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:53.601 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:55.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:47:55.793 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:47:55.793 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:47:55.794 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:47:55.794 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-10T07:47:55.975 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-10T07:47:55.975 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:47:55.976 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:47:55.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:47:56.151 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:47:56.152 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:47:56.152 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:47:56.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2024-09-10T07:47:56.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.311+0000 7f1293e00640 1 Processor -- start 2024-09-10T07:47:56.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.312+0000 7f1293e00640 1 -- start start 2024-09-10T07:47:56.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1293e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1293e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 0x7f128c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1293e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c0cfd40 con 0x7f128c06bb20 2024-09-10T07:47:56.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1293e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f128c0d2c80 con 0x7f128c0cf3b0 2024-09-10T07:47:56.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1293e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0d2ed0 con 0x7f128c15e820 2024-09-10T07:47:56.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1291400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1291400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:48528/0 (socket says 172.21.15.3:48528) 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1291400640 1 -- 172.21.15.3:0/4218288069 learned_addr learned my addr 172.21.15.3:0/4218288069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1291e00640 1 --2- 172.21.15.3:0/4218288069 >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 0x7f128c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.313+0000 7f1290a00640 1 --1- 172.21.15.3:0/4218288069 >> v1:172.21.15.70:6789/0 conn(0x7f128c06bb20 0x7f128c06bf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:52474/0 (socket says 172.21.15.3:52474) 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.314+0000 7f1291400640 1 -- 172.21.15.3:0/4218288069 >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 msgr2=0x7f128c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.314+0000 7f1291400640 1 --2- 172.21.15.3:0/4218288069 >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 0x7f128c0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.314+0000 7f1291400640 1 -- 172.21.15.3:0/4218288069 >> v1:172.21.15.70:6789/0 conn(0x7f128c06bb20 legacy=0x7f128c06bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.314+0000 7f1291400640 1 -- 172.21.15.3:0/4218288069 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0d31b0 con 0x7f128c15e820 2024-09-10T07:47:56.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.314+0000 7f1291e00640 1 --2- 172.21.15.3:0/4218288069 >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 0x7f128c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1291400640 1 --2- 172.21.15.3:0/4218288069 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1280009870 tx=0x7f128002a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e09247cbc573c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f128be00640 1 -- 172.21.15.3:0/4218288069 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280037070 con 0x7f128c15e820 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f128be00640 1 -- 172.21.15.3:0/4218288069 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280032640 con 0x7f128c15e820 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f128be00640 1 -- 172.21.15.3:0/4218288069 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280032a10 con 0x7f128c15e820 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 -- 172.21.15.3:0/4218288069 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 msgr2=0x7f128c0d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 --2- 172.21.15.3:0/4218288069 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1280009870 tx=0x7f128002a2f0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 -- 172.21.15.3:0/4218288069 shutdown_connections 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 --2- 172.21.15.3:0/4218288069 >> v2:172.21.15.70:3300/0 conn(0x7f128c0cf3b0 0x7f128c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 --2- 172.21.15.3:0/4218288069 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c15e820 0x7f128c0d3f70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.315+0000 7f1293e00640 1 -- 172.21.15.3:0/4218288069 >> 172.21.15.3:0/4218288069 conn(0x7f128c072770 msgr2=0x7f128c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.316+0000 7f1293e00640 1 -- 172.21.15.3:0/4218288069 shutdown_connections 2024-09-10T07:47:56.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.316+0000 7f1293e00640 1 -- 172.21.15.3:0/4218288069 wait complete. 2024-09-10T07:47:56.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.317+0000 7f1293e00640 1 Processor -- start 2024-09-10T07:47:56.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.317+0000 7f1293e00640 1 -- start start 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 0x7f128c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f128c15e820 0x7f128c161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0d3a50 con 0x7f128c0cf3b0 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0d3770 con 0x7f128c15e820 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1293e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0d44b0 con 0x7f128c06bb20 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1291400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 0x7f128c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38338/0 (socket says 172.21.15.3:38338) 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1290a00640 1 -- 172.21.15.3:0/2574773409 learned_addr learned my addr 172.21.15.3:0/2574773409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.318+0000 7f1291e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f128c15e820 0x7f128c161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 -- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f128c15e820 msgr2=0x7f128c161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f128c15e820 0x7f128c161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 -- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 msgr2=0x7f128c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 -- 172.21.15.3:0/2574773409 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0c9b10 con 0x7f128c06bb20 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1290a00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1291400640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 0x7f128c0c8be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1280031000 tx=0x7f12800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.319+0000 7f1289600640 1 -- 172.21.15.3:0/2574773409 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280037040 con 0x7f128c06bb20 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.320+0000 7f1289600640 1 -- 172.21.15.3:0/2574773409 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280042070 con 0x7f128c06bb20 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.320+0000 7f1289600640 1 -- 172.21.15.3:0/2574773409 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800309b0 con 0x7f128c06bb20 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.320+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c0c9270 con 0x7f128c06bb20 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.320+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c0c9830 con 0x7f128c06bb20 2024-09-10T07:47:56.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.322+0000 7f1289600640 1 -- 172.21.15.3:0/2574773409 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1280030c10 con 0x7f128c06bb20 2024-09-10T07:47:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.322+0000 7f1289600640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.323+0000 7f1289600640 1 -- 172.21.15.3:0/2574773409 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f12800ab5d0 con 0x7f128c06bb20 2024-09-10T07:47:56.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.323+0000 7f1290a00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.323+0000 7f1296e6b740 1 Processor -- start 2024-09-10T07:47:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.323+0000 7f1296e6b740 1 -- start start 2024-09-10T07:47:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1290a00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1278004e00 tx=0x7f1278007f10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 0x5558a515dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 0x5558a515e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5558a507b660 con 0x5558a512d800 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5558a5023680 con 0x5558a512d420 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5558a5077620 con 0x5558a5113480 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.324+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38346/0 (socket says 172.21.15.3:38346) 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 -- 172.21.15.3:0/201185199 learned_addr learned my addr 172.21.15.3:0/201185199 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1291400640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 0x5558a515dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1291e00640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 0x5558a515e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 -- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 msgr2=0x5558a515e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 0x5558a515e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 -- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 msgr2=0x5558a515dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 0x5558a515dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 -- 172.21.15.3:0/201185199 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a4e587d0 con 0x5558a512d800 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1291400640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 0x5558a515dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1291e00640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 0x5558a515e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f1290a00640 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1278004950 tx=0x7f1278013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459f0d0f83aa02fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.325+0000 7f127dc00640 1 -- 172.21.15.3:0/201185199 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278026070 con 0x5558a512d800 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f127dc00640 1 -- 172.21.15.3:0/201185199 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12780076b0 con 0x5558a512d800 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f127dc00640 1 -- 172.21.15.3:0/201185199 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278012550 con 0x5558a512d800 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 -- 172.21.15.3:0/201185199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 msgr2=0x5558a515e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1278004950 tx=0x7f1278013000 comp rx=0 tx=0).stop 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 -- 172.21.15.3:0/201185199 shutdown_connections 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a5113480 0x5558a515e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a515e200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 --2- 172.21.15.3:0/201185199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d420 0x5558a515dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 -- 172.21.15.3:0/201185199 >> 172.21.15.3:0/201185199 conn(0x5558a5114220 msgr2=0x5558a515c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 -- 172.21.15.3:0/201185199 shutdown_connections 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.326+0000 7f1296e6b740 1 -- 172.21.15.3:0/201185199 wait complete. 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.327+0000 7f1296e6b740 1 -- start start 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.327+0000 7f1296e6b740 1 Processor -- start 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d800 0x5558a5159990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 0x5558a5159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5558a51123e0 con 0x5558a512d420 2024-09-10T07:47:56.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5558a50044c0 con 0x5558a512d800 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5558a51126c0 con 0x5558a51170f0 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1296e6b740 10 client.0 _ll_register_callbacks cb 0x5558a5073e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1291400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1291400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38362/0 (socket says 172.21.15.3:38362) 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1291400640 1 -- 172.21.15.3:0/1288610947 learned_addr learned my addr 172.21.15.3:0/1288610947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1291e00640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 0x5558a5159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.328+0000 7f1290a00640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d800 0x5558a5159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 -- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 msgr2=0x5558a5159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 0x5558a5159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 -- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d800 msgr2=0x5558a5159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d800 0x5558a5159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 -- 172.21.15.3:0/1288610947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a4d8e3f0 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291e00640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 0x5558a5159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1291400640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1280006c80 tx=0x7f1280079470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ea0b4e0372e6f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1272000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280079870 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1280091810 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12780076b0 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f128007dc20 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1296e6b740 10 client.5112 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.329+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5558a4d4e0f0 con 0x5558a512d420 2024-09-10T07:47:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12780076b0 con 0x5558a512d420 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f128003a030 con 0x5558a512d420 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 10 client.5112 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 10 client.5112 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 20 client.5112 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a516ca10 0x5558a516ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 4 client.5112 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a5088d90 con 0x5558a516ca10 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.330+0000 7f1290a00640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a516ca10 0x5558a516ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.331+0000 7f1272000640 10 client.5112 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.331+0000 7f1290a00640 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a516ca10 0x5558a516ce30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0aa1f955f6e4a8 server_cookie=21bd8794bd2ad455 in_seq=0 out_seq=0 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.331+0000 7f1272000640 1 -- 172.21.15.3:0/1288610947 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558a5088d90 con 0x5558a516ca10 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.331+0000 7f1272000640 10 client.5112 handle_command_reply: tid=0 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.331+0000 7f1296e6b740 1 client.5112 shutdown 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 msgr2=0x5558a5116bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1280006c80 tx=0x7f1280079470 comp rx=0 tx=0).stop 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 shutdown_connections 2024-09-10T07:47:56.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a516ca10 0x5558a516ce30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a51170f0 0x5558a5159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a512d800 0x5558a5159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1272000640 0 client.5112 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 --2- 172.21.15.3:0/1288610947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5116bb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 >> 172.21.15.3:0/1288610947 conn(0x5558a5113480 msgr2=0x5558a515aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 shutdown_connections 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 1 -- 172.21.15.3:0/1288610947 wait complete. 2024-09-10T07:47:56.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.332+0000 7f1296e6b740 20 client.5112 trim_cache size 0 max 16384 2024-09-10T07:47:56.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.347+0000 7f1296e6b740 1 Processor -- start 2024-09-10T07:47:56.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.347+0000 7f1296e6b740 1 -- start start 2024-09-10T07:47:56.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a5156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d800 0x5558a512b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f128c200cb0 con 0x5558a512d420 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f128c200580 con 0x5558a51170f0 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f128c1f8800 con 0x5558a512d800 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.348+0000 7f1290a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38376/0 (socket says 172.21.15.3:38376) 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1290a00640 1 -- 172.21.15.3:0/2225601725 learned_addr learned my addr 172.21.15.3:0/2225601725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a5156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291e00640 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d800 0x5558a512b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 -- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d800 msgr2=0x5558a512b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d800 0x5558a512b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 -- 172.21.15.3:0/2225601725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 msgr2=0x5558a5156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 -- 172.21.15.3:0/2225601725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a510e050 con 0x5558a51170f0 2024-09-10T07:47:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f1291400640 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a5156120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1280030740 tx=0x7f1280080050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f4cec954974715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f127d200640 1 -- 172.21.15.3:0/2225601725 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280032640 con 0x5558a51170f0 2024-09-10T07:47:56.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f127d200640 1 -- 172.21.15.3:0/2225601725 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280079870 con 0x5558a51170f0 2024-09-10T07:47:56.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.349+0000 7f127d200640 1 -- 172.21.15.3:0/2225601725 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280081fc0 con 0x5558a51170f0 2024-09-10T07:47:56.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 -- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 msgr2=0x5558a5156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a5156120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1280030740 tx=0x7f1280080050 comp rx=0 tx=0).stop 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 -- 172.21.15.3:0/2225601725 shutdown_connections 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d800 0x5558a512b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d420 0x5558a5156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 --2- 172.21.15.3:0/2225601725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a5156120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 -- 172.21.15.3:0/2225601725 >> 172.21.15.3:0/2225601725 conn(0x5558a5113480 msgr2=0x5558a515aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 -- 172.21.15.3:0/2225601725 shutdown_connections 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.350+0000 7f1296e6b740 1 -- 172.21.15.3:0/2225601725 wait complete. 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.351+0000 7f1296e6b740 1 -- start start 2024-09-10T07:47:56.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.351+0000 7f1296e6b740 1 Processor -- start 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a51c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d420 0x5558a5116580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5558a510da60 con 0x5558a512d800 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5558a5115bd0 con 0x5558a51170f0 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5558a51dc060 con 0x5558a512d420 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1296e6b740 10 client.0 _ll_register_callbacks cb 0x7f1280006cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1291e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1291e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38380/0 (socket says 172.21.15.3:38380) 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1291400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a51c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1291e00640 1 -- 172.21.15.3:0/493128330 learned_addr learned my addr 172.21.15.3:0/493128330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.352+0000 7f1290a00640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d420 0x5558a5116580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 -- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d420 msgr2=0x5558a5116580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d420 0x5558a5116580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 -- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 msgr2=0x5558a51c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a51c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 -- 172.21.15.3:0/493128330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a51c4560 con 0x5558a512d800 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291400640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a51c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1291e00640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12840046e0 tx=0x7f1284017780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfa5faeb5579a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1272000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284021430 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1284022c80 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1280079870 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1284018420 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1296e6b740 10 client.5118 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.353+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5558a515aad0 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280079870 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f1284029050 con 0x5558a512d800 2024-09-10T07:47:56.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 10 client.5118 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 10 client.5118 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 20 client.5118 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a515fd50 0x5558a5160160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 4 client.5118 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558a50830e0 con 0x5558a515fd50 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1291400640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a515fd50 0x5558a5160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.354+0000 7f1291400640 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a515fd50 0x5558a5160160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66aa68f060f6d6ed server_cookie=c61d6feb54caf6a2 in_seq=0 out_seq=0 2024-09-10T07:47:56.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.355+0000 7f1272000640 10 client.5118 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1272000640 1 -- 172.21.15.3:0/493128330 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558a50830e0 con 0x5558a515fd50 2024-09-10T07:47:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1272000640 10 client.5118 handle_command_reply: tid=0 2024-09-10T07:47:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 client.5118 shutdown 2024-09-10T07:47:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 msgr2=0x5558a5116ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12840046e0 tx=0x7f1284017780 comp rx=0 tx=0).stop 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 shutdown_connections 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5558a515fd50 0x5558a5160160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5558a512d800 0x5558a5116ac0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558a512d420 0x5558a5116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 --2- 172.21.15.3:0/493128330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558a51170f0 0x5558a51c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1272000640 0 client.5118 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 >> 172.21.15.3:0/493128330 conn(0x5558a5113480 msgr2=0x5558a5156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.368+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 shutdown_connections 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1296e6b740 1 -- 172.21.15.3:0/493128330 wait complete. 2024-09-10T07:47:56.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1296e6b740 20 client.5118 trim_cache size 0 max 16384 2024-09-10T07:47:56.370 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:56.370 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:47:56.370 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:47:56.370 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 msgr2=0x7f126c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1278004e00 tx=0x7f1278007f10 comp rx=0 tx=0).stop 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 msgr2=0x7f128c0c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 0x7f128c0c8be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1280031000 tx=0x7f12800046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 shutdown_connections 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f128c15e820 0x7f128c161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f128c0cf3b0 0x7f128c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 --2- 172.21.15.3:0/2574773409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f128c06bb20 0x7f128c0c8be0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.369+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 >> 172.21.15.3:0/2574773409 conn(0x7f128c072770 msgr2=0x7f128c0cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.370+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 shutdown_connections 2024-09-10T07:47:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.370+0000 7f1293e00640 1 -- 172.21.15.3:0/2574773409 wait complete. 2024-09-10T07:47:56.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2024-09-10T07:47:56.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.544+0000 7fe011400640 1 Processor -- start 2024-09-10T07:47:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.544+0000 7fe011400640 1 -- start start 2024-09-10T07:47:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe011400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe00c1451d0 0x7fe00c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe011400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe011400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe00c0cd900 con 0x7fe00c0cf270 2024-09-10T07:47:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe011400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe00c06d7c0 con 0x7fe00c1451d0 2024-09-10T07:47:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe011400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe00c0cdbe0 con 0x7fe00c145730 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe003e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe00b200640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe00c0cf270 0x7fe00c0cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:52484/0 (socket says 172.21.15.3:52484) 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe00b200640 1 -- 172.21.15.3:0/3297672227 learned_addr learned my addr 172.21.15.3:0/3297672227 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.545+0000 7fe00a800640 1 --2- 172.21.15.3:0/3297672227 >> v2:172.21.15.70:3300/0 conn(0x7fe00c1451d0 0x7fe00c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe009e00640 1 -- 172.21.15.3:0/3297672227 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362426606 0 0) 0x7fe00c0cd900 con 0x7fe00c0cf270 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe00a800640 1 -- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 msgr2=0x7fe00c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe00a800640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe00a800640 1 -- 172.21.15.3:0/3297672227 >> v1:172.21.15.70:6789/0 conn(0x7fe00c0cf270 legacy=0x7fe00c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe00a800640 1 -- 172.21.15.3:0/3297672227 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c06da10 con 0x7fe00c1451d0 2024-09-10T07:47:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.546+0000 7fe003e00640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.547+0000 7fe00a800640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c1451d0 0x7fe00c06cfc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0097a0 tx=0x7fdffc02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed104047d79b321e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.547+0000 7fe009e00640 1 -- 172.21.15.3:0/3297672227 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc037070 con 0x7fe00c1451d0 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.547+0000 7fe009e00640 1 -- 172.21.15.3:0/3297672227 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdffc02adb0 con 0x7fe00c1451d0 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.547+0000 7fe009e00640 1 -- 172.21.15.3:0/3297672227 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc032660 con 0x7fe00c1451d0 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 -- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c1451d0 msgr2=0x7fe00c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c1451d0 0x7fe00c06cfc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0097a0 tx=0x7fdffc02a060 comp rx=0 tx=0).stop 2024-09-10T07:47:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 -- 172.21.15.3:0/3297672227 shutdown_connections 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 --2- 172.21.15.3:0/3297672227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c1451d0 0x7fe00c06cfc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 -- 172.21.15.3:0/3297672227 >> 172.21.15.3:0/3297672227 conn(0x7fe00c0b5cb0 msgr2=0x7fe00c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 -- 172.21.15.3:0/3297672227 shutdown_connections 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.548+0000 7fe011400640 1 -- 172.21.15.3:0/3297672227 wait complete. 2024-09-10T07:47:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.549+0000 7fe011400640 1 Processor -- start 2024-09-10T07:47:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.550+0000 7fe011400640 1 -- start start 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.550+0000 7fe011400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 0x7fe00c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.550+0000 7fe011400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe011400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe011400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c06df30 con 0x7fe00c1451d0 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe011400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c0b38b0 con 0x7fe00c0cf270 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe011400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe00c06e0b0 con 0x7fe00c145730 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe003e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe003e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38396/0 (socket says 172.21.15.3:38396) 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe003e00640 1 -- 172.21.15.3:0/16080126 learned_addr learned my addr 172.21.15.3:0/16080126 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 0x7fe00c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00b200640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 -- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 msgr2=0x7fe00c143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 -- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 msgr2=0x7fe00c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe00a800640 1 -- 172.21.15.3:0/16080126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c15dfa0 con 0x7fe00c0cf270 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.551+0000 7fe003e00640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe00a800640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 0x7fe00c15bdd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031000 tx=0x7fdffc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe002a00640 1 -- 172.21.15.3:0/16080126 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc041070 con 0x7fe00c0cf270 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe002a00640 1 -- 172.21.15.3:0/16080126 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdffc032e20 con 0x7fe00c0cf270 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe002a00640 1 -- 172.21.15.3:0/16080126 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc037040 con 0x7fe00c0cf270 2024-09-10T07:47:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c15e280 con 0x7fe00c0cf270 2024-09-10T07:47:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.552+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c15e840 con 0x7fe00c0cf270 2024-09-10T07:47:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.555+0000 7fe002a00640 1 -- 172.21.15.3:0/16080126 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdffc03a070 con 0x7fe00c0cf270 2024-09-10T07:47:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.556+0000 7fe002a00640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.556+0000 7fe014285740 1 Processor -- start 2024-09-10T07:47:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.556+0000 7fe002a00640 1 -- 172.21.15.3:0/16080126 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdffc0ab0c0 con 0x7fe00c0cf270 2024-09-10T07:47:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.556+0000 7fe003e00640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.556+0000 7fe014285740 1 -- start start 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe003e00640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 0x7fdfe806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe00c144260 tx=0x7fdff4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a932200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 0x55ef1a932740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef1a84f660 con 0x55ef1a901420 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef1a7f7680 con 0x55ef1a901800 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef1a84b620 con 0x55ef1a8e7480 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe00a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.557+0000 7fe00a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38402/0 (socket says 172.21.15.3:38402) 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 -- 172.21.15.3:0/1315542803 learned_addr learned my addr 172.21.15.3:0/1315542803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe003e00640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a932200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00b200640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 0x55ef1a932740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 -- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 msgr2=0x55ef1a932740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 0x55ef1a932740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 -- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 msgr2=0x55ef1a932200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a932200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 -- 172.21.15.3:0/1315542803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1a62c7d0 con 0x55ef1a901420 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe003e00640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a932200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00b200640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 0x55ef1a932740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:56.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fe00a800640 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdffc075870 tx=0x7fdffc078f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7440546aae8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fdffb000640 1 -- 172.21.15.3:0/1315542803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc07d710 con 0x55ef1a901420 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fdffb000640 1 -- 172.21.15.3:0/1315542803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdffc081f40 con 0x55ef1a901420 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.558+0000 7fdffb000640 1 -- 172.21.15.3:0/1315542803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc075a80 con 0x55ef1a901420 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 -- 172.21.15.3:0/1315542803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 msgr2=0x55ef1a931c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdffc075870 tx=0x7fdffc078f50 comp rx=0 tx=0).stop 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 -- 172.21.15.3:0/1315542803 shutdown_connections 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a8e7480 0x55ef1a932740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a932200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 --2- 172.21.15.3:0/1315542803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a901420 0x55ef1a931c40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 -- 172.21.15.3:0/1315542803 >> 172.21.15.3:0/1315542803 conn(0x55ef1a8e8220 msgr2=0x55ef1a930840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 -- 172.21.15.3:0/1315542803 shutdown_connections 2024-09-10T07:47:56.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.559+0000 7fe014285740 1 -- 172.21.15.3:0/1315542803 wait complete. 2024-09-10T07:47:56.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.560+0000 7fe014285740 1 -- start start 2024-09-10T07:47:56.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.560+0000 7fe014285740 1 Processor -- start 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 0x55ef1a942950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901800 0x55ef1a8f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef1a84f660 con 0x55ef1a92e100 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef1a7d84c0 con 0x55ef1a901420 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef1a8e63e0 con 0x55ef1a901800 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe014285740 10 client.0 _ll_register_callbacks cb 0x55ef1a847e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe00b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe00b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38406/0 (socket says 172.21.15.3:38406) 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe00b200640 1 -- 172.21.15.3:0/2171023249 learned_addr learned my addr 172.21.15.3:0/2171023249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe003e00640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901800 0x55ef1a8f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.561+0000 7fe00a800640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 0x55ef1a942950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 -- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901800 msgr2=0x55ef1a8f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901800 0x55ef1a8f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 -- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 msgr2=0x55ef1a942950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 0x55ef1a942950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 -- 172.21.15.3:0/2171023249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1a5623f0 con 0x55ef1a92e100 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00a800640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 0x55ef1a942950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe00b200640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0040021b0 tx=0x7fe004007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33274837e37ad1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fdfef400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004013570 con 0x55ef1a92e100 2024-09-10T07:47:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe00400c780 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.562+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdffc081f40 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe00400c980 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 10 client.5127 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ef1a5220f0 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdffc081f40 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fe00400cb20 con 0x55ef1a92e100 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 10 client.5127 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 10 client.5127 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.b 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 20 client.5127 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a92c470 0x55ef1a915540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 4 client.5127 mds_command: new command op to 4607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 --> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef1a85cd90 con 0x55ef1a92c470 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.563+0000 7fe00a800640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a92c470 0x55ef1a915540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.564+0000 7fe00a800640 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a92c470 0x55ef1a915540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3f2fdf73c3855f server_cookie=2b239f8c32f17204 in_seq=0 out_seq=0 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.564+0000 7fdfef400640 10 client.5127 ms_handle_connect on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.564+0000 7fdfef400640 1 -- 172.21.15.3:0/2171023249 <== mds.1 v2:172.21.15.70:6834/2716956690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef1a85cd90 con 0x55ef1a92c470 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.564+0000 7fdfef400640 10 client.5127 handle_command_reply: tid=0 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 client.5127 shutdown 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 msgr2=0x55ef1a946460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0040021b0 tx=0x7fe004007430 comp rx=0 tx=0).stop 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 shutdown_connections 2024-09-10T07:47:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a92c470 0x55ef1a915540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a946460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fdfef400640 0 client.5127 ms_handle_reset on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901800 0x55ef1a8f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 --2- 172.21.15.3:0/2171023249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901420 0x55ef1a942950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 >> 172.21.15.3:0/2171023249 conn(0x55ef1a8e7480 msgr2=0x55ef1a92a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 shutdown_connections 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 1 -- 172.21.15.3:0/2171023249 wait complete. 2024-09-10T07:47:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.565+0000 7fe014285740 20 client.5127 trim_cache size 0 max 16384 2024-09-10T07:47:56.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.580+0000 7fe014285740 1 Processor -- start 2024-09-10T07:47:56.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.580+0000 7fe014285740 1 -- start start 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a8e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a8e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c1fc610 con 0x55ef1a92e100 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c072520 con 0x55ef1a901800 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe00c072c20 con 0x55ef1a901420 2024-09-10T07:47:56.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe00b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe00b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38416/0 (socket says 172.21.15.3:38416) 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.581+0000 7fe00b200640 1 -- 172.21.15.3:0/2763246424 learned_addr learned my addr 172.21.15.3:0/2763246424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00a800640 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a8e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 -- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 msgr2=0x55ef1a8e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a8e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 -- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 msgr2=0x55ef1a8e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a8e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 -- 172.21.15.3:0/2763246424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1a92cbc0 con 0x55ef1a92e100 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00a800640 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a8e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fe00b200640 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe004004e10 tx=0x7fe00400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678f33a536d40f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fdffa600640 1 -- 172.21.15.3:0/2763246424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00400cb20 con 0x55ef1a92e100 2024-09-10T07:47:56.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fdffa600640 1 -- 172.21.15.3:0/2763246424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe004019070 con 0x55ef1a92e100 2024-09-10T07:47:56.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.582+0000 7fdffa600640 1 -- 172.21.15.3:0/2763246424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00400c350 con 0x55ef1a92e100 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 -- 172.21.15.3:0/2763246424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 msgr2=0x55ef1a8ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe004004e10 tx=0x7fe00400e000 comp rx=0 tx=0).stop 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 -- 172.21.15.3:0/2763246424 shutdown_connections 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8ffdd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a8e6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 --2- 172.21.15.3:0/2763246424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a8e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 -- 172.21.15.3:0/2763246424 >> 172.21.15.3:0/2763246424 conn(0x55ef1a8e7480 msgr2=0x55ef1a92fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 -- 172.21.15.3:0/2763246424 shutdown_connections 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.583+0000 7fe014285740 1 -- 172.21.15.3:0/2763246424 wait complete. 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.584+0000 7fe014285740 1 -- start start 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.584+0000 7fe014285740 1 Processor -- start 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a9b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef1a8f1ff0 con 0x55ef1a92e100 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c13c4d0 con 0x55ef1a901800 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef1a9b4910 con 0x55ef1a901420 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe014285740 10 client.0 _ll_register_callbacks cb 0x7fe004004be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe003e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a9b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:48624/0 (socket says 172.21.15.3:48624) 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 -- 172.21.15.3:0/952472425 learned_addr learned my addr 172.21.15.3:0/952472425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 -- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 msgr2=0x55ef1a9b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a9b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 -- 172.21.15.3:0/952472425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 msgr2=0x55ef1a8f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00a800640 1 -- 172.21.15.3:0/952472425 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1a8eb870 con 0x55ef1a901420 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe003e00640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a9b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.585+0000 7fe00b200640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fe00a800640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdffc079200 tx=0x7fdffc008070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5280bcc639d1801b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fdfef400640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc077530 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdffc075a80 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004019070 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdffc0911d0 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fe014285740 10 client.4823 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ef1a9b4570 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.586+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe004019070 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fdffc081ab0 con 0x55ef1a901420 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 10 client.4823 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 10 client.4823 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.b 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 20 client.4823 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a95a700 0x55ef1a8e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 4 client.4823 mds_command: new command op to 4607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 --> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef1a8570e0 con 0x55ef1a95a700 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe003e00640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a95a700 0x55ef1a8e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fe003e00640 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a95a700 0x55ef1a8e68d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5761e6e05ba70d5 server_cookie=8ec899596c777e60 in_seq=0 out_seq=0 2024-09-10T07:47:56.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.587+0000 7fdfef400640 10 client.4823 ms_handle_connect on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:56.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fdfef400640 1 -- 172.21.15.3:0/952472425 <== mds.1 v2:172.21.15.70:6834/2716956690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef1a8570e0 con 0x55ef1a95a700 2024-09-10T07:47:56.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fdfef400640 10 client.4823 handle_command_reply: tid=0 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 client.4823 shutdown 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 msgr2=0x55ef1a9acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdffc079200 tx=0x7fdffc008070 comp rx=0 tx=0).stop 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 shutdown_connections 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:6834/2716956690,v1:172.21.15.70:6835/2716956690] conn(0x55ef1a95a700 0x55ef1a8e68d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef1a92e100 0x55ef1a8f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef1a901800 0x55ef1a9b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fdfef400640 0 client.4823 ms_handle_reset on v2:172.21.15.70:6834/2716956690 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 --2- 172.21.15.3:0/952472425 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef1a901420 0x55ef1a9acf30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 >> 172.21.15.3:0/952472425 conn(0x55ef1a8e7480 msgr2=0x55ef1a8e5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 shutdown_connections 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.597+0000 7fe014285740 1 -- 172.21.15.3:0/952472425 wait complete. 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe014285740 20 client.4823 trim_cache size 0 max 16384 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:47:56.599 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:56.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 msgr2=0x7fdfe806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 0x7fdfe806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe00c144260 tx=0x7fdff4009190 comp rx=0 tx=0).stop 2024-09-10T07:47:56.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 msgr2=0x7fe00c15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 0x7fe00c15bdd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031000 tx=0x7fdffc0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 shutdown_connections 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe00c145730 0x7fe00c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe00c1451d0 0x7fe00c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 --2- 172.21.15.3:0/16080126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0cf270 0x7fe00c15bdd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 >> 172.21.15.3:0/16080126 conn(0x7fe00c0b5cb0 msgr2=0x7fe00c0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 shutdown_connections 2024-09-10T07:47:56.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.598+0000 7fe011400640 1 -- 172.21.15.3:0/16080126 wait complete. 2024-09-10T07:47:56.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2024-09-10T07:47:56.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.773+0000 7fb64fe00640 1 Processor -- start 2024-09-10T07:47:56.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.773+0000 7fb64fe00640 1 -- start start 2024-09-10T07:47:56.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6500d45a0 0x7fb6500d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 0x7fb6500d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6500d4f00 con 0x7fb650161b70 2024-09-10T07:47:56.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64fe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb650161f50 con 0x7fb6500d45a0 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64fe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb6500d3de0 con 0x7fb6500cfe90 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb647e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6500d45a0 0x7fb6500d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb647e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb6500d45a0 0x7fb6500d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38438/0 (socket says 172.21.15.3:38438) 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb64ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb650161b70 0x7fb6500d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51098/0 (socket says 172.21.15.3:51098) 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.774+0000 7fb647e00640 1 -- 172.21.15.3:0/1350956545 learned_addr learned my addr 172.21.15.3:0/1350956545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb64f400640 1 --2- 172.21.15.3:0/1350956545 >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 0x7fb6500d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb647e00640 1 -- 172.21.15.3:0/1350956545 >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 msgr2=0x7fb6500d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb647e00640 1 --2- 172.21.15.3:0/1350956545 >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 0x7fb6500d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb647e00640 1 -- 172.21.15.3:0/1350956545 >> v1:172.21.15.3:6789/0 conn(0x7fb650161b70 legacy=0x7fb6500d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb647e00640 1 -- 172.21.15.3:0/1350956545 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6500d40c0 con 0x7fb6500d45a0 2024-09-10T07:47:56.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb64f400640 1 --2- 172.21.15.3:0/1350956545 >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 0x7fb6500d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:56.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.775+0000 7fb64e000640 1 -- 172.21.15.3:0/1350956545 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824832490 0 0) 0x7fb6500d4f00 con 0x7fb650161b70 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.776+0000 7fb647e00640 1 --2- 172.21.15.3:0/1350956545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500d45a0 0x7fb6500d49c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6500b4160 tx=0x7fb63802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdea768be09ed40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.776+0000 7fb64e000640 1 -- 172.21.15.3:0/1350956545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638037070 con 0x7fb6500d45a0 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.776+0000 7fb64e000640 1 -- 172.21.15.3:0/1350956545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb638032440 con 0x7fb6500d45a0 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.776+0000 7fb64e000640 1 -- 172.21.15.3:0/1350956545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638032810 con 0x7fb6500d45a0 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 -- 172.21.15.3:0/1350956545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500d45a0 msgr2=0x7fb6500d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1350956545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500d45a0 0x7fb6500d49c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6500b4160 tx=0x7fb63802a270 comp rx=0 tx=0).stop 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 -- 172.21.15.3:0/1350956545 shutdown_connections 2024-09-10T07:47:56.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1350956545 >> v2:172.21.15.70:3300/0 conn(0x7fb6500cfe90 0x7fb6500d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1350956545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500d45a0 0x7fb6500d49c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 -- 172.21.15.3:0/1350956545 >> 172.21.15.3:0/1350956545 conn(0x7fb650062630 msgr2=0x7fb650062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 -- 172.21.15.3:0/1350956545 shutdown_connections 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.777+0000 7fb64fe00640 1 -- 172.21.15.3:0/1350956545 wait complete. 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.778+0000 7fb64fe00640 1 Processor -- start 2024-09-10T07:47:56.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 -- start start 2024-09-10T07:47:56.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6500d45a0 0x7fb6500cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 0x7fb6500cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6500c9fc0 con 0x7fb6500cfe90 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb6500cf970 con 0x7fb650161b70 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.779+0000 7fb64fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb6500d3a30 con 0x7fb6500d45a0 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38448/0 (socket says 172.21.15.3:38448) 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 -- 172.21.15.3:0/1741118839 learned_addr learned my addr 172.21.15.3:0/1741118839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64f400640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 0x7fb6500cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb647e00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6500d45a0 0x7fb6500cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 -- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6500d45a0 msgr2=0x7fb6500cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6500d45a0 0x7fb6500cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 -- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 msgr2=0x7fb6500cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 0x7fb6500cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 -- 172.21.15.3:0/1741118839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65015cc90 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64f400640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 0x7fb6500cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.780+0000 7fb64ea00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c4d0 tx=0x7fb64000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.781+0000 7fb646a00640 1 -- 172.21.15.3:0/1741118839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640018070 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.781+0000 7fb646a00640 1 -- 172.21.15.3:0/1741118839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640007500 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.781+0000 7fb646a00640 1 -- 172.21.15.3:0/1741118839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640004a20 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.781+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6500ce490 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.781+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65015d210 con 0x7fb6500cfe90 2024-09-10T07:47:56.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.783+0000 7fb646a00640 1 -- 172.21.15.3:0/1741118839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb64001b070 con 0x7fb6500cfe90 2024-09-10T07:47:56.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.784+0000 7fb646a00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.784+0000 7fb646a00640 1 -- 172.21.15.3:0/1741118839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb64008a110 con 0x7fb6500cfe90 2024-09-10T07:47:56.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.784+0000 7fb647e00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.785+0000 7fb658a6c740 1 Processor -- start 2024-09-10T07:47:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.785+0000 7fb658a6c740 1 -- start start 2024-09-10T07:47:56.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.785+0000 7fb647e00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 0x7fb62c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb638034000 tx=0x7fb63802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.785+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 0x55a4f545a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 0x55a4f54a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb658a6c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a4f53c3660 con 0x55a4f545b9e0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a4f536b680 con 0x55a4f54754f0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a4f53bf620 con 0x55a4f54a5ea0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb647e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 0x55a4f545a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:43342/0 (socket says 172.21.15.3:43342) 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 -- 172.21.15.3:0/2602783049 learned_addr learned my addr 172.21.15.3:0/2602783049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64f400640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 0x55a4f54a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 -- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 msgr2=0x55a4f54a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 0x55a4f54a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 -- 172.21.15.3:0/2602783049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 msgr2=0x55a4f545a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 0x55a4f545a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64ea00640 1 -- 172.21.15.3:0/2602783049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f51a07d0 con 0x55a4f54754f0 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb64f400640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 0x55a4f54a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.786+0000 7fb647e00640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 0x55a4f545a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb6400548c0 tx=0x7fb640058360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278d26ff29c26e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb63f400640 1 -- 172.21.15.3:0/2602783049 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640056760 con 0x55a4f54754f0 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb63f400640 1 -- 172.21.15.3:0/2602783049 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640054ad0 con 0x55a4f54754f0 2024-09-10T07:47:56.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb63f400640 1 -- 172.21.15.3:0/2602783049 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640070370 con 0x55a4f54754f0 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 -- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 msgr2=0x55a4f54758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb6400548c0 tx=0x7fb640058360 comp rx=0 tx=0).stop 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 -- 172.21.15.3:0/2602783049 shutdown_connections 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54a5ea0 0x55a4f54a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f545b9e0 0x55a4f545a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2602783049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54754f0 0x55a4f54758d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.787+0000 7fb658a6c740 1 -- 172.21.15.3:0/2602783049 >> 172.21.15.3:0/2602783049 conn(0x55a4f545c410 msgr2=0x55a4f54a4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.788+0000 7fb658a6c740 1 -- 172.21.15.3:0/2602783049 shutdown_connections 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.788+0000 7fb658a6c740 1 -- 172.21.15.3:0/2602783049 wait complete. 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.788+0000 7fb658a6c740 1 -- start start 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.788+0000 7fb658a6c740 1 Processor -- start 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f5462440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a4f545a940 con 0x55a4f54b7a40 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a4f534c4c0 con 0x55a4f54a5ea0 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.789+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a4f545ac20 con 0x55a4f54754f0 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb658a6c740 10 client.0 _ll_register_callbacks cb 0x55a4f53bbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38468/0 (socket says 172.21.15.3:38468) 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f5462440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64f400640 1 -- 172.21.15.3:0/2836814457 learned_addr learned my addr 172.21.15.3:0/2836814457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb647e00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 -- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 msgr2=0x55a4f54b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 -- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 msgr2=0x55a4f54b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 -- 172.21.15.3:0/2836814457 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f50d63f0 con 0x55a4f54754f0 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64f400640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb64ea00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f5462440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb640001120 tx=0x7fb64005b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2576bfae989ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.790+0000 7fb633e00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:47:56.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640004dc0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb6400578b0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb640054ad0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb640057ab0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 10 client.4832 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55a4f50960f0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640054ad0 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fb640088020 con 0x55a4f54754f0 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 10 client.4832 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 10 client.4832 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.c 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 20 client.4832 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f5464030 0x55a4f5464410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 4 client.4832 mds_command: new command op to 4601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.791+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 --> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f53d0d90 con 0x55a4f5464030 2024-09-10T07:47:56.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.792+0000 7fb647e00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f5464030 0x55a4f5464410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.792+0000 7fb647e00640 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f5464030 0x55a4f5464410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=930a0912238d3792 server_cookie=766f9a4cd839c2fd in_seq=0 out_seq=0 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.792+0000 7fb633e00640 10 client.4832 ms_handle_connect on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb633e00640 1 -- 172.21.15.3:0/2836814457 <== mds.2 v2:172.21.15.3:6835/708626559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4f53d0d90 con 0x55a4f5464030 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb633e00640 10 client.4832 handle_command_reply: tid=0 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 client.4832 shutdown 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 msgr2=0x55a4f5462440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f5462440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb640001120 tx=0x7fb64005b040 comp rx=0 tx=0).stop 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 shutdown_connections 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f5464030 0x55a4f5464410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb633e00640 0 client.4832 ms_handle_reset on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 --2- 172.21.15.3:0/2836814457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f5462440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 >> 172.21.15.3:0/2836814457 conn(0x55a4f545b9e0 msgr2=0x55a4f54a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 shutdown_connections 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 1 -- 172.21.15.3:0/2836814457 wait complete. 2024-09-10T07:47:56.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.793+0000 7fb658a6c740 20 client.4832 trim_cache size 0 max 16384 2024-09-10T07:47:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 Processor -- start 2024-09-10T07:47:56.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 -- start start 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb650200930 con 0x55a4f54b7a40 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ffbf0 con 0x55a4f54a5ea0 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.809+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb650200200 con 0x55a4f54754f0 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38476/0 (socket says 172.21.15.3:38476) 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 -- 172.21.15.3:0/3697428615 learned_addr learned my addr 172.21.15.3:0/3697428615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb647e00640 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64ea00640 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 -- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 msgr2=0x55a4f54c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 -- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 msgr2=0x55a4f54c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 -- 172.21.15.3:0/3697428615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f545c1a0 con 0x55a4f54b7a40 2024-09-10T07:47:56.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.810+0000 7fb64f400640 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb648004350 tx=0x7fb648004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4106ee6dd8043d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb63ea00640 1 -- 172.21.15.3:0/3697428615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480167a0 con 0x55a4f54b7a40 2024-09-10T07:47:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb63ea00640 1 -- 172.21.15.3:0/3697428615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb648016940 con 0x55a4f54b7a40 2024-09-10T07:47:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb63ea00640 1 -- 172.21.15.3:0/3697428615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648016d10 con 0x55a4f54b7a40 2024-09-10T07:47:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 -- 172.21.15.3:0/3697428615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 msgr2=0x55a4f5489970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb648004350 tx=0x7fb648004440 comp rx=0 tx=0).stop 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 -- 172.21.15.3:0/3697428615 shutdown_connections 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f5489970 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f54c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 --2- 172.21.15.3:0/3697428615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 -- 172.21.15.3:0/3697428615 >> 172.21.15.3:0/3697428615 conn(0x55a4f545b9e0 msgr2=0x55a4f549fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 -- 172.21.15.3:0/3697428615 shutdown_connections 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.811+0000 7fb658a6c740 1 -- 172.21.15.3:0/3697428615 wait complete. 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.812+0000 7fb658a6c740 1 -- start start 2024-09-10T07:47:56.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.812+0000 7fb658a6c740 1 Processor -- start 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f5521fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6500b8240 con 0x55a4f54b7a40 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a4f549f820 con 0x55a4f54a5ea0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a4f54beb00 con 0x55a4f54754f0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb658a6c740 10 client.0 _ll_register_callbacks cb 0x7fb64006f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.813+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb647e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f5521fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38480/0 (socket says 172.21.15.3:38480) 2024-09-10T07:47:56.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb647e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f5521fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:43386/0 (socket says 172.21.15.3:43386) 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 -- 172.21.15.3:0/27664666 learned_addr learned my addr 172.21.15.3:0/27664666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 -- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 msgr2=0x55a4f54a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 -- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 msgr2=0x55a4f5521fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f5521fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 -- 172.21.15.3:0/27664666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f5522510 con 0x55a4f54b7a40 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb64f400640 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a4f54c8650 tx=0x7fb648004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4f39b8cf727345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb633e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.814+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648016dc0 con 0x55a4f54b7a40 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb648025040 con 0x55a4f54b7a40 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb648016940 con 0x55a4f54b7a40 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb648034050 con 0x55a4f54b7a40 2024-09-10T07:47:56.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 10 client.5154 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55a4f549f620 con 0x55a4f54b7a40 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb648016940 con 0x55a4f54b7a40 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fb648037020 con 0x55a4f54b7a40 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 10 client.5154 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 10 client.5154 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.c 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 20 client.5154 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f549eb00 0x55a4f549ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 4 client.5154 mds_command: new command op to 4601 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 --> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4f53cb0e0 con 0x55a4f549eb00 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.815+0000 7fb64ea00640 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f549eb00 0x55a4f549ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.816+0000 7fb64ea00640 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f549eb00 0x55a4f549ef10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0577a3d1cd93e4b server_cookie=58995e9a97bfc9a3 in_seq=0 out_seq=0 2024-09-10T07:47:56.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.816+0000 7fb633e00640 10 client.5154 ms_handle_connect on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb633e00640 1 -- 172.21.15.3:0/27664666 <== mds.2 v2:172.21.15.3:6835/708626559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4f53cb0e0 con 0x55a4f549eb00 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb633e00640 10 client.5154 handle_command_reply: tid=0 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 client.5154 shutdown 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 msgr2=0x55a4f54a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a4f54c8650 tx=0x7fb648004ab0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 shutdown_connections 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:6835/708626559,v1:172.21.15.3:6837/708626559] conn(0x55a4f549eb00 0x55a4f549ef10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a4f54b7a40 0x55a4f54a46e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a4f54a5ea0 0x55a4f5521fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 --2- 172.21.15.3:0/27664666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a4f54754f0 0x55a4f54a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb633e00640 0 client.5154 ms_handle_reset on v2:172.21.15.3:6835/708626559 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 >> 172.21.15.3:0/27664666 conn(0x55a4f545b9e0 msgr2=0x55a4f5474180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.825+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 shutdown_connections 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb658a6c740 1 -- 172.21.15.3:0/27664666 wait complete. 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb658a6c740 20 client.5154 trim_cache size 0 max 16384 2024-09-10T07:47:56.826 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:47:56.827 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:47:56.827 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:56.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 msgr2=0x7fb62c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 0x7fb62c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb638034000 tx=0x7fb63802e000 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 msgr2=0x7fb650156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c4d0 tx=0x7fb64000c990 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 shutdown_connections 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb650161b70 0x7fb6500cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6500d45a0 0x7fb6500cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 --2- 172.21.15.3:0/1741118839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6500cfe90 0x7fb650156110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.826+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 >> 172.21.15.3:0/1741118839 conn(0x7fb650062630 msgr2=0x7fb650160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.827+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 shutdown_connections 2024-09-10T07:47:56.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:56.827+0000 7fb64fe00640 1 -- 172.21.15.3:0/1741118839 wait complete. 2024-09-10T07:47:56.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:56.841 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-10T07:47:56.886 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/ 2024-09-10T07:47:56.887 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4 2024-09-10T07:47:56.887 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5 2024-09-10T07:47:56.887 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6 2024-09-10T07:47:56.887 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-10T07:47:56.887 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-10T07:47:56.888 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:56.888 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:56.888 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:56.888 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:56.888 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-10T07:47:56.889 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.028 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627778 2024-09-10T07:47:57.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-10T07:47:57.039 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.179 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627779 2024-09-10T07:47:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.190 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:57.190 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:57.191 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:57.191 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:57.191 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-10T07:47:57.191 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.331 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627780 2024-09-10T07:47:57.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.342 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:57.343 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:57.343 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:57.343 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:57.343 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-10T07:47:57.343 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.482 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627781 2024-09-10T07:47:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.493 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:57.493 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:57.493 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:57.494 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:57.494 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-10T07:47:57.494 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.635 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627782 2024-09-10T07:47:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-10T07:47:57.646 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:47:57.787 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627783 2024-09-10T07:47:57.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:47:57.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2024-09-10T07:47:57.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57.998+0000 7f8db0c00640 1 Processor -- start 2024-09-10T07:47:57.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:57.999+0000 7f8db0c00640 1 -- start start 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8db0c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dac15c530 0x7f8dac0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8db0c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dac0cf1f0 con 0x7f8dac0cd740 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8db0c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dac0ce150 con 0x7f8dac0b26a0 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8db0c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac0cdbb0 con 0x7f8dac15c530 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8da9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dac15c530 0x7f8dac0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8da9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dac15c530 0x7f8dac0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38488/0 (socket says 172.21.15.3:38488) 2024-09-10T07:47:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.000+0000 7f8da9e00640 1 -- 172.21.15.3:0/315403356 learned_addr learned my addr 172.21.15.3:0/315403356 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.001+0000 7f8da9e00640 1 -- 172.21.15.3:0/315403356 >> v1:172.21.15.70:6789/0 conn(0x7f8dac0b26a0 legacy=0x7f8dac0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.001+0000 7f8da9e00640 1 -- 172.21.15.3:0/315403356 >> v1:172.21.15.3:6789/0 conn(0x7f8dac0cd740 legacy=0x7f8dac0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.001+0000 7f8da9e00640 1 -- 172.21.15.3:0/315403356 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac0ce430 con 0x7f8dac15c530 2024-09-10T07:47:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8da9e00640 1 --2- 172.21.15.3:0/315403356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac15c530 0x7f8dac0cd200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009870 tx=0x7f8d9c02a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f05c2eeea6c0e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8da9400640 1 -- 172.21.15.3:0/315403356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c037070 con 0x7f8dac15c530 2024-09-10T07:47:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8da9400640 1 -- 172.21.15.3:0/315403356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c032640 con 0x7f8dac15c530 2024-09-10T07:47:58.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8da9400640 1 -- 172.21.15.3:0/315403356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c032a10 con 0x7f8dac15c530 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8db0c00640 1 -- 172.21.15.3:0/315403356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac15c530 msgr2=0x7f8dac0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8db0c00640 1 --2- 172.21.15.3:0/315403356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac15c530 0x7f8dac0cd200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009870 tx=0x7f8d9c02a2f0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8db0c00640 1 -- 172.21.15.3:0/315403356 shutdown_connections 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8db0c00640 1 --2- 172.21.15.3:0/315403356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac15c530 0x7f8dac0cd200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.002+0000 7f8db0c00640 1 -- 172.21.15.3:0/315403356 >> 172.21.15.3:0/315403356 conn(0x7f8dac062cc0 msgr2=0x7f8dac06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.003+0000 7f8db0c00640 1 -- 172.21.15.3:0/315403356 shutdown_connections 2024-09-10T07:47:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.003+0000 7f8db0c00640 1 -- 172.21.15.3:0/315403356 wait complete. 2024-09-10T07:47:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.004+0000 7f8db0c00640 1 Processor -- start 2024-09-10T07:47:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.004+0000 7f8db0c00640 1 -- start start 2024-09-10T07:47:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8dac0cd740 0x7f8dac15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 0x7f8dac15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0ce9f0 con 0x7f8dac0b26a0 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0cecd0 con 0x7f8dac15c530 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8db0c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac15c910 con 0x7f8dac0cd740 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38500/0 (socket says 172.21.15.3:38500) 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.005+0000 7f8da9e00640 1 -- 172.21.15.3:0/1928747057 learned_addr learned my addr 172.21.15.3:0/1928747057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8daa800640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 0x7f8dac15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da3e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8dac0cd740 0x7f8dac15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 -- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8dac0cd740 msgr2=0x7f8dac15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8dac0cd740 0x7f8dac15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 -- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 msgr2=0x7f8dac15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 0x7f8dac15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 -- 172.21.15.3:0/1928747057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac15df30 con 0x7f8dac0b26a0 2024-09-10T07:47:58.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8daa800640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 0x7f8dac15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da9e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c000c00 tx=0x7f8d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da2000640 1 -- 172.21.15.3:0/1928747057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c037040 con 0x7f8dac0b26a0 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da2000640 1 -- 172.21.15.3:0/1928747057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c042070 con 0x7f8dac0b26a0 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8da2000640 1 -- 172.21.15.3:0/1928747057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c030bb0 con 0x7f8dac0b26a0 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.006+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac1e6ed0 con 0x7f8dac0b26a0 2024-09-10T07:47:58.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.007+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac15f860 con 0x7f8dac0b26a0 2024-09-10T07:47:58.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.010+0000 7f8da2000640 1 -- 172.21.15.3:0/1928747057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d9c032500 con 0x7f8dac0b26a0 2024-09-10T07:47:58.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.010+0000 7f8da2000640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 0x7f8d8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.011+0000 7f8da2000640 1 -- 172.21.15.3:0/1928747057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8d9c0ab940 con 0x7f8dac0b26a0 2024-09-10T07:47:58.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.011+0000 7f8da3e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 0x7f8d8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.011+0000 7f8db3a12740 1 Processor -- start 2024-09-10T07:47:58.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.011+0000 7f8db3a12740 1 -- start start 2024-09-10T07:47:58.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.011+0000 7f8da3e00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 0x7f8d8806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8dac159690 tx=0x7f8d94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 0x56215a486020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a0460 0x56215a4a9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56215a3bf970 con 0x56215a4a1ba0 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56215a3678d0 con 0x56215a485c40 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56215a3bb930 con 0x56215a4a0460 2024-09-10T07:47:58.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8da3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8da3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38510/0 (socket says 172.21.15.3:38510) 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8da3e00640 1 -- 172.21.15.3:0/2045679252 learned_addr learned my addr 172.21.15.3:0/2045679252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8daa800640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a0460 0x56215a4a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.012+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 0x56215a486020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 -- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a0460 msgr2=0x56215a4a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a0460 0x56215a4a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 -- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 msgr2=0x56215a486020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 0x56215a486020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 -- 172.21.15.3:0/2045679252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215a19ea20 con 0x56215a4a1ba0 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 0x56215a486020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8d94001db0 tx=0x7f8d94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2cb4433b2d10ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8d9a600640 1 -- 172.21.15.3:0/2045679252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d940035f0 con 0x56215a4a1ba0 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8d9a600640 1 -- 172.21.15.3:0/2045679252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d94003790 con 0x56215a4a1ba0 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.013+0000 7f8d9a600640 1 -- 172.21.15.3:0/2045679252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003b40 con 0x56215a4a1ba0 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 -- 172.21.15.3:0/2045679252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 msgr2=0x56215a4a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8d94001db0 tx=0x7f8d94011000 comp rx=0 tx=0).stop 2024-09-10T07:47:58.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 -- 172.21.15.3:0/2045679252 shutdown_connections 2024-09-10T07:47:58.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a0460 0x56215a4a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a4a9450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2045679252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a485c40 0x56215a486020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 -- 172.21.15.3:0/2045679252 >> 172.21.15.3:0/2045679252 conn(0x56215a458500 msgr2=0x56215a4a0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 -- 172.21.15.3:0/2045679252 shutdown_connections 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.014+0000 7f8db3a12740 1 -- 172.21.15.3:0/2045679252 wait complete. 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.015+0000 7f8db3a12740 1 -- start start 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.015+0000 7f8db3a12740 1 Processor -- start 2024-09-10T07:47:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a482260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 0x56215a49c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56215a45bbf0 con 0x56215a4a0460 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56215a37aaf0 con 0x56215a4827a0 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56215a45bed0 con 0x56215a4a1ba0 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8db3a12740 10 client.0 _ll_register_callbacks cb 0x56215a36c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38522/0 (socket says 172.21.15.3:38522) 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 -- 172.21.15.3:0/3873940093 learned_addr learned my addr 172.21.15.3:0/3873940093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da3e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a482260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8daa800640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 0x56215a49c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 -- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 msgr2=0x56215a482260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a482260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 -- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 msgr2=0x56215a49c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.016+0000 7f8da9e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 0x56215a49c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8da9e00640 1 -- 172.21.15.3:0/3873940093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215a092310 con 0x56215a4a0460 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8daa800640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 0x56215a49c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8da9e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c076040 tx=0x7f8d9c079600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9bab10218a5c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c091020 con 0x56215a4a0460 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8d9c0a4e70 con 0x56215a4a0460 2024-09-10T07:47:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d94003790 con 0x56215a4a0460 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8d9c0827c0 con 0x56215a4a0460 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8db3a12740 10 client.5169 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56215a1ff1b0 con 0x56215a4a0460 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.017+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d94003790 con 0x56215a4a0460 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f8d9c03a070 con 0x56215a4a0460 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 10 client.5169 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 10 client.5169 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 20 client.5169 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a4846b0 0x56215a4989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 4 client.5169 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56215a394100 con 0x56215a4846b0 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.018+0000 7f8da3e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a4846b0 0x56215a4989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8da3e00640 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a4846b0 0x56215a4989f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec156433b07915af server_cookie=906b7ffdc194591b in_seq=0 out_seq=0 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8d8ea00640 10 client.5169 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8d8ea00640 1 -- 172.21.15.3:0/3873940093 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56215a394100 con 0x56215a4846b0 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8d8ea00640 10 client.5169 handle_command_reply: tid=0 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8db3a12740 1 client.5169 shutdown 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 msgr2=0x56215a49ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.019+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c076040 tx=0x7f8d9c079600 comp rx=0 tx=0).stop 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 shutdown_connections 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a4846b0 0x56215a4989f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4827a0 0x56215a49c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a482260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 --2- 172.21.15.3:0/3873940093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a0460 0x56215a49ba60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8d8ea00640 0 client.5169 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 >> 172.21.15.3:0/3873940093 conn(0x56215a485c40 msgr2=0x56215a49edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 shutdown_connections 2024-09-10T07:47:58.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 1 -- 172.21.15.3:0/3873940093 wait complete. 2024-09-10T07:47:58.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.020+0000 7f8db3a12740 20 client.5169 trim_cache size 0 max 16384 2024-09-10T07:47:58.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.039+0000 7f8db3a12740 1 Processor -- start 2024-09-10T07:47:58.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.039+0000 7f8db3a12740 1 -- start start 2024-09-10T07:47:58.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a518350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a518890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac200d10 con 0x56215a4827a0 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1e8ba0 con 0x56215a4a0460 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac2005e0 con 0x56215a4a1ba0 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8da9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38536/0 (socket says 172.21.15.3:38536) 2024-09-10T07:47:58.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8da9e00640 1 -- 172.21.15.3:0/2294066850 learned_addr learned my addr 172.21.15.3:0/2294066850 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a518350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8da9e00640 1 -- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 msgr2=0x56215a518890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.040+0000 7f8daa800640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a518890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a518890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da9e00640 1 -- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 msgr2=0x56215a518350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a518350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da9e00640 1 -- 172.21.15.3:0/2294066850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215a06b3d0 con 0x56215a4827a0 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a518350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8daa800640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a518890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c02a330 tx=0x7f8d9c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f2613787d0e367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8d99c00640 1 -- 172.21.15.3:0/2294066850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c091030 con 0x56215a4827a0 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8d99c00640 1 -- 172.21.15.3:0/2294066850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c082450 con 0x56215a4827a0 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.041+0000 7f8d99c00640 1 -- 172.21.15.3:0/2294066850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c079230 con 0x56215a4827a0 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- 172.21.15.3:0/2294066850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 msgr2=0x56215a49b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c02a330 tx=0x7f8d9c03e000 comp rx=0 tx=0).stop 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- 172.21.15.3:0/2294066850 shutdown_connections 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4a1ba0 0x56215a518890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a518350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2294066850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4827a0 0x56215a49b050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- 172.21.15.3:0/2294066850 >> 172.21.15.3:0/2294066850 conn(0x56215a485c40 msgr2=0x56215a481d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- 172.21.15.3:0/2294066850 shutdown_connections 2024-09-10T07:47:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- 172.21.15.3:0/2294066850 wait complete. 2024-09-10T07:47:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 -- start start 2024-09-10T07:47:58.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.042+0000 7f8db3a12740 1 Processor -- start 2024-09-10T07:47:58.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4827a0 0x56215a4fc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a4c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56215a451be0 con 0x56215a4a1ba0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1f8960 con 0x56215a4a0460 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56215a499e60 con 0x56215a4827a0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.043+0000 7f8db3a12740 10 client.0 _ll_register_callbacks cb 0x7f8d9c079aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38538/0 (socket says 172.21.15.3:38538) 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 -- 172.21.15.3:0/2189437794 learned_addr learned my addr 172.21.15.3:0/2189437794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8da3e00640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a4c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4827a0 0x56215a4fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 -- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4827a0 msgr2=0x56215a4fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4827a0 0x56215a4fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 -- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 msgr2=0x56215a4c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a4c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 -- 172.21.15.3:0/2189437794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215a4fc5c0 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8daa800640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8da40012e0 tx=0x7f8da40019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73c915a75e72366 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.044+0000 7f8d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4003e20 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8da4010070 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d9c082450 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8da4006b50 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 10 client.5175 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56215a499590 con 0x56215a4a1ba0 2024-09-10T07:47:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c082450 con 0x56215a4a1ba0 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7f8da40155f0 con 0x56215a4a1ba0 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 10 client.5175 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 10 client.5175 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 20 client.5175 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a458080 0x56215a49efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 4 client.5175 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.045+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) -- 0x56215a3cd0a0 con 0x56215a458080 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.046+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a458080 0x56215a49efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.046+0000 7f8da9e00640 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a458080 0x56215a49efe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1131c249eece5aae server_cookie=6252c774c7fff246 in_seq=0 out_seq=0 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.046+0000 7f8d8ea00640 10 client.5175 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8d8ea00640 1 -- 172.21.15.3:0/2189437794 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56215a3cd0a0 con 0x56215a458080 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8d8ea00640 10 client.5175 handle_command_reply: tid=0 2024-09-10T07:47:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 client.5175 shutdown 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 msgr2=0x56215a49eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8da40012e0 tx=0x7f8da40019f0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 shutdown_connections 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x56215a458080 0x56215a49efe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56215a4a1ba0 0x56215a49eaa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56215a4a0460 0x56215a4c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 --2- 172.21.15.3:0/2189437794 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56215a4827a0 0x56215a4fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8d8ea00640 0 client.5175 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 >> 172.21.15.3:0/2189437794 conn(0x56215a485c40 msgr2=0x56215a4a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 shutdown_connections 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 1 -- 172.21.15.3:0/2189437794 wait complete. 2024-09-10T07:47:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.047+0000 7f8db3a12740 20 client.5175 trim_cache size 0 max 16384 2024-09-10T07:47:58.052 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-10T07:47:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "tag123", 2024-09-10T07:47:58.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-10T07:47:58.052 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.048+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 msgr2=0x7f8d8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.048+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 0x7f8d8806e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8dac159690 tx=0x7f8d94009040 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.048+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 msgr2=0x7f8dac1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.048+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c000c00 tx=0x7f8d9c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 shutdown_connections 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d8806be30 0x7f8d8806e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8dac15c530 0x7f8dac15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8dac0cd740 0x7f8dac15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 --2- 172.21.15.3:0/1928747057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dac0b26a0 0x7f8dac1558c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 >> 172.21.15.3:0/1928747057 conn(0x7f8dac062cc0 msgr2=0x7f8dac072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.049+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 shutdown_connections 2024-09-10T07:47:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.050+0000 7f8db0c00640 1 -- 172.21.15.3:0/1928747057 wait complete. 2024-09-10T07:47:58.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:47:58.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T07:47:58.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.223+0000 7fc9e8400640 1 Processor -- start 2024-09-10T07:47:58.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.224+0000 7fc9e8400640 1 -- start start 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e8400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e0145730 0x7fc9e006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e8400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e00cf270 0x7fc9e00cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e8400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00cd900 con 0x7fc9e01451d0 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e8400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e006d7c0 con 0x7fc9e00cf270 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e8400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00cdbe0 con 0x7fc9e0145730 2024-09-10T07:47:58.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e6400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e00cf270 0x7fc9e00cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e6400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e00cf270 0x7fc9e00cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38554/0 (socket says 172.21.15.3:38554) 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e6400640 1 -- 172.21.15.3:0/131979768 learned_addr learned my addr 172.21.15.3:0/131979768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e5a00640 1 --1- 172.21.15.3:0/131979768 >> v1:172.21.15.70:6789/0 conn(0x7fc9e01451d0 0x7fc9e00b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:52504/0 (socket says 172.21.15.3:52504) 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.225+0000 7fc9e5000640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e0145730 0x7fc9e006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.226+0000 7fc9e6400640 1 -- 172.21.15.3:0/131979768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e0145730 msgr2=0x7fc9e006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.226+0000 7fc9e6400640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e0145730 0x7fc9e006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.226+0000 7fc9e6400640 1 -- 172.21.15.3:0/131979768 >> v1:172.21.15.70:6789/0 conn(0x7fc9e01451d0 legacy=0x7fc9e00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.226+0000 7fc9e6400640 1 -- 172.21.15.3:0/131979768 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e006da10 con 0x7fc9e00cf270 2024-09-10T07:47:58.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.226+0000 7fc9dfe00640 1 -- 172.21.15.3:0/131979768 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140122751 0 0) 0x7fc9e00cd900 con 0x7fc9e01451d0 2024-09-10T07:47:58.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e6400640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00cf700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009e50 tx=0x7fc9d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f95f5323deb2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9dfe00640 1 -- 172.21.15.3:0/131979768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8036070 con 0x7fc9e00cf270 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9dfe00640 1 -- 172.21.15.3:0/131979768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8031600 con 0x7fc9e00cf270 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9dfe00640 1 -- 172.21.15.3:0/131979768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80319d0 con 0x7fc9e00cf270 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 -- 172.21.15.3:0/131979768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 msgr2=0x7fc9e00cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00cf700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009e50 tx=0x7fc9d802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 -- 172.21.15.3:0/131979768 shutdown_connections 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00cf700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 --2- 172.21.15.3:0/131979768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e0145730 0x7fc9e006cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.227+0000 7fc9e8400640 1 -- 172.21.15.3:0/131979768 >> 172.21.15.3:0/131979768 conn(0x7fc9e00b5cb0 msgr2=0x7fc9e006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.228+0000 7fc9e8400640 1 -- 172.21.15.3:0/131979768 shutdown_connections 2024-09-10T07:47:58.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.228+0000 7fc9e8400640 1 -- 172.21.15.3:0/131979768 wait complete. 2024-09-10T07:47:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.229+0000 7fc9e8400640 1 Processor -- start 2024-09-10T07:47:58.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.229+0000 7fc9e8400640 1 -- start start 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 0x7fc9e00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e0145730 0x7fc9e00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e006df30 con 0x7fc9e00cf270 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00cfc40 con 0x7fc9e0145730 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e8400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e006e0b0 con 0x7fc9e01451d0 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38566/0 (socket says 172.21.15.3:38566) 2024-09-10T07:47:58.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.230+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3316554908 learned_addr learned my addr 172.21.15.3:0/3316554908 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e6400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e0145730 0x7fc9e00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5000640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 0x7fc9e00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 msgr2=0x7fc9e00b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 0x7fc9e00b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e0145730 msgr2=0x7fc9e00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e0145730 0x7fc9e00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3316554908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e01f63e0 con 0x7fc9e00cf270 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5000640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 0x7fc9e00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c4a0 tx=0x7fc9d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9dd600640 1 -- 172.21.15.3:0/3316554908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4018070 con 0x7fc9e00cf270 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9dd600640 1 -- 172.21.15.3:0/3316554908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d40074e0 con 0x7fc9e00cf270 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00ae500 con 0x7fc9e00cf270 2024-09-10T07:47:58.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.231+0000 7fc9dd600640 1 -- 172.21.15.3:0/3316554908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4004a20 con 0x7fc9e00cf270 2024-09-10T07:47:58.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.232+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e00aeac0 con 0x7fc9e00cf270 2024-09-10T07:47:58.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.234+0000 7fc9dd600640 1 -- 172.21.15.3:0/3316554908 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9d401b070 con 0x7fc9e00cf270 2024-09-10T07:47:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.235+0000 7fc9dd600640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 0x7fc9c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.235+0000 7fc9dd600640 1 -- 172.21.15.3:0/3316554908 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc9d408a160 con 0x7fc9e00cf270 2024-09-10T07:47:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.235+0000 7fc9e5000640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 0x7fc9c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.235+0000 7fc9eb3d4740 1 Processor -- start 2024-09-10T07:47:58.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.235+0000 7fc9eb3d4740 1 -- start start 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9e5000640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 0x7fc9c006e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9e015c660 tx=0x7fc9cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 0x56527681af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56527673a660 con 0x56527681cc20 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652766e2680 con 0x5652767ec490 2024-09-10T07:47:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.236+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x565276736620 con 0x5652767ecbc0 2024-09-10T07:47:58.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38572/0 (socket says 172.21.15.3:38572) 2024-09-10T07:47:58.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 -- 172.21.15.3:0/571743757 learned_addr learned my addr 172.21.15.3:0/571743757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 0x56527681af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 -- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 msgr2=0x56527681d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 -- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 msgr2=0x56527681af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 0x56527681af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 -- 172.21.15.3:0/571743757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652765177d0 con 0x56527681cc20 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 0x56527681af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.237+0000 7fc9e6400640 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009f70 tx=0x7fc9d802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab8abee595e7428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9d2000640 1 -- 172.21.15.3:0/571743757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8036040 con 0x56527681cc20 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9d2000640 1 -- 172.21.15.3:0/571743757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8037040 con 0x56527681cc20 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9d2000640 1 -- 172.21.15.3:0/571743757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80318a0 con 0x56527681cc20 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/571743757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 msgr2=0x56527681d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009f70 tx=0x7fc9d802e000 comp rx=0 tx=0).stop 2024-09-10T07:47:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/571743757 shutdown_connections 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56527681cc20 0x56527681d740 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/571743757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767ec490 0x56527681af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/571743757 >> 172.21.15.3:0/571743757 conn(0x5652767d3220 msgr2=0x56527681b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/571743757 shutdown_connections 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.238+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/571743757 wait complete. 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.239+0000 7fc9eb3d4740 1 -- start start 2024-09-10T07:47:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.239+0000 7fc9eb3d4740 1 Processor -- start 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x5652767d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x565276803f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565276819ef0 con 0x5652767ecbc0 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652766c34c0 con 0x5652767d6420 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56527681a1d0 con 0x56527681cc20 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9eb3d4740 10 client.0 _ll_register_callbacks cb 0x565276732e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:58.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38574/0 (socket says 172.21.15.3:38574) 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.240+0000 7fc9e5a00640 1 -- 172.21.15.3:0/278398611 learned_addr learned my addr 172.21.15.3:0/278398611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5000640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x5652767d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e6400640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x565276803f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 -- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 msgr2=0x5652767d5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x5652767d5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 -- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 msgr2=0x565276803f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x565276803f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 -- 172.21.15.3:0/278398611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527644d3f0 con 0x5652767ecbc0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4054880 tx=0x7fc9d4058520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15c4a3e3eaed8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4004dc0 con 0x5652767ecbc0 2024-09-10T07:47:58.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc9d4057870 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.241+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d8037040 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc9d4057a90 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 10 client.5187 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56527640d0f0 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8037040 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fc9d40701b0 con 0x5652767ecbc0 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 10 client.5187 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 10 client.5187 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 20 client.5187 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:58.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652768131e0 0x5652768135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 4 client.5187 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565276747d90 con 0x5652768131e0 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.242+0000 7fc9e5000640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652768131e0 0x5652768135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.243+0000 7fc9e5000640 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652768131e0 0x5652768135f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5847f02254e7a24 server_cookie=368b34968026f240 in_seq=0 out_seq=0 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.243+0000 7fc9c6a00640 10 client.5187 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.243+0000 7fc9c6a00640 1 -- 172.21.15.3:0/278398611 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565276747d90 con 0x5652768131e0 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.243+0000 7fc9c6a00640 10 client.5187 handle_command_reply: tid=0 2024-09-10T07:47:58.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 client.5187 shutdown 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 msgr2=0x565276833ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4054880 tx=0x7fc9d4058520 comp rx=0 tx=0).stop 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 shutdown_connections 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652768131e0 0x5652768135f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x565276803f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x5652767d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/278398611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x565276833ee0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9c6a00640 0 client.5187 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 >> 172.21.15.3:0/278398611 conn(0x5652767ec490 msgr2=0x5652767cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 shutdown_connections 2024-09-10T07:47:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/278398611 wait complete. 2024-09-10T07:47:58.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.244+0000 7fc9eb3d4740 20 client.5187 trim_cache size 0 max 16384 2024-09-10T07:47:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 Processor -- start 2024-09-10T07:47:58.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 -- start start 2024-09-10T07:47:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56527681cc20 0x56527681e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01fc790 con 0x5652767d6420 2024-09-10T07:47:58.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00aa7a0 con 0x56527681cc20 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.263+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00aaea0 con 0x5652767ecbc0 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38586/0 (socket says 172.21.15.3:38586) 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3451546315 learned_addr learned my addr 172.21.15.3:0/3451546315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5000640 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e6400640 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56527681cc20 0x56527681e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 msgr2=0x56527681e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56527681cc20 msgr2=0x56527681e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56527681cc20 0x56527681e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 -- 172.21.15.3:0/3451546315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652768141c0 con 0x5652767d6420 2024-09-10T07:47:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.264+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4020030 tx=0x7fc9d40547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3e6217f6670cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9d1600640 1 -- 172.21.15.3:0/3451546315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4004bc0 con 0x5652767d6420 2024-09-10T07:47:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9d1600640 1 -- 172.21.15.3:0/3451546315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4004d60 con 0x5652767d6420 2024-09-10T07:47:58.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9d1600640 1 -- 172.21.15.3:0/3451546315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4054480 con 0x5652767d6420 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/3451546315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 msgr2=0x5652767d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4020030 tx=0x7fc9d40547f0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/3451546315 shutdown_connections 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56527681cc20 0x56527681e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652767ecbc0 0x56527681e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/3451546315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767d6420 0x5652767d6d30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/3451546315 >> 172.21.15.3:0/3451546315 conn(0x5652767ec490 msgr2=0x56527681d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/3451546315 shutdown_connections 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.265+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/3451546315 wait complete. 2024-09-10T07:47:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.266+0000 7fc9eb3d4740 1 -- start start 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.266+0000 7fc9eb3d4740 1 Processor -- start 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x5652768874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x565276817350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565276816980 con 0x5652767ecbc0 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00724e0 con 0x5652767d6420 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x565276819030 con 0x56527681cc20 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9eb3d4740 10 client.0 _ll_register_callbacks cb 0x7fc9d40076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9e5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9e5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38588/0 (socket says 172.21.15.3:38588) 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.267+0000 7fc9e5000640 1 -- 172.21.15.3:0/2469259768 learned_addr learned my addr 172.21.15.3:0/2469259768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x5652768874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e6400640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x565276817350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 -- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 msgr2=0x565276817350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x565276817350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 -- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 msgr2=0x5652768874e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x5652768874e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 -- 172.21.15.3:0/2469259768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652767ebaa0 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9e5000640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00ac1b0 tx=0x7fc9cc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621de72529ee6c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc01c040 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc9cc00f050 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.268+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d4004d60 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc037050 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 10 client.5193 fetch_fsmap learned FSMap version 24 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5652767d50f0 con 0x5652767ecbc0 2024-09-10T07:47:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4004d60 con 0x5652767ecbc0 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 24) ==== 3554+0+0 (secure 0 0 0) 0x7fc9cc03a020 con 0x5652767ecbc0 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 10 client.5193 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 10 client.5193 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 20 client.5193 populate_metadata read hostname 'smithi003' 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652767eb2b0 0x565276817890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 4 client.5193 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652767420e0 con 0x5652767eb2b0 2024-09-10T07:47:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.269+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652767eb2b0 0x565276817890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:47:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9e5a00640 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652767eb2b0 0x565276817890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b614188dea628b01 server_cookie=54d88a87557cbf4c in_seq=0 out_seq=0 2024-09-10T07:47:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9c6a00640 10 client.5193 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9c6a00640 1 -- 172.21.15.3:0/2469259768 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+207 (crc 0 0 0) 0x5652767420e0 con 0x5652767eb2b0 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9c6a00640 10 client.5193 handle_command_reply: tid=0 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 client.5193 shutdown 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 msgr2=0x5652767ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00ac1b0 tx=0x7fc9cc009190 comp rx=0 tx=0).stop 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 shutdown_connections 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x5652767eb2b0 0x565276817890 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56527681cc20 0x565276817350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652767ecbc0 0x5652767ead70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 --2- 172.21.15.3:0/2469259768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652767d6420 0x5652768874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 >> 172.21.15.3:0/2469259768 conn(0x5652767ec490 msgr2=0x5652767d6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9c6a00640 0 client.5193 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 shutdown_connections 2024-09-10T07:47:58.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.270+0000 7fc9eb3d4740 1 -- 172.21.15.3:0/2469259768 wait complete. 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9eb3d4740 20 client.5193 trim_cache size 0 max 16384 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tag123": { 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "tag123", 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:58.275 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:47:58.276 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:47:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 msgr2=0x7fc9c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 0x7fc9c006e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9e015c660 tx=0x7fc9cc008040 comp rx=0 tx=0).stop 2024-09-10T07:47:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 msgr2=0x7fc9e00af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c4a0 tx=0x7fc9d400c960 comp rx=0 tx=0).stop 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 shutdown_connections 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c006be30 0x7fc9c006e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e0145730 0x7fc9e00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e01451d0 0x7fc9e00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 --2- 172.21.15.3:0/3316554908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e00cf270 0x7fc9e00af2c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 >> 172.21.15.3:0/3316554908 conn(0x7fc9e00b5cb0 msgr2=0x7fc9e0143f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 shutdown_connections 2024-09-10T07:47:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:47:58.271+0000 7fc9e8400640 1 -- 172.21.15.3:0/3316554908 wait complete. 2024-09-10T07:47:58.286 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2024-09-10T07:48:21.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:48:21.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:48:28.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:48:28.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T07:48:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.459+0000 7f205b800640 1 Processor -- start 2024-09-10T07:48:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.459+0000 7f205b800640 1 -- start start 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f205b800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 0x7f205415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f205b800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2054072c50 0x7f20540b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f205b800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205415eff0 con 0x7f20540b4ac0 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f205b800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f20540b4f10 con 0x7f2054072c50 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f205b800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f205415fbe0 con 0x7f205415b230 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2053e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2054072c50 0x7f20540b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2059800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20540b4ac0 0x7f205406cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50322/0 (socket says 172.21.15.3:50322) 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2059800640 1 -- 172.21.15.3:0/3909357032 learned_addr learned my addr 172.21.15.3:0/3909357032 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2058e00640 1 --2- 172.21.15.3:0/3909357032 >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 0x7f205415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2053e00640 1 -- 172.21.15.3:0/3909357032 >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 msgr2=0x7f205415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2053e00640 1 --2- 172.21.15.3:0/3909357032 >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 0x7f205415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2053e00640 1 -- 172.21.15.3:0/3909357032 >> v1:172.21.15.3:6789/0 conn(0x7f20540b4ac0 legacy=0x7f205406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2053e00640 1 -- 172.21.15.3:0/3909357032 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205415f2b0 con 0x7f2054072c50 2024-09-10T07:48:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.461+0000 7f2058e00640 1 --2- 172.21.15.3:0/3909357032 >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 0x7f205415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.462+0000 7f2053e00640 1 --2- 172.21.15.3:0/3909357032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2054072c50 0x7f20540b4580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20540b65e0 tx=0x7f204002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7a580c8c6f4abb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f2053400640 1 -- 172.21.15.3:0/3909357032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040037070 con 0x7f2054072c50 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f2053400640 1 -- 172.21.15.3:0/3909357032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040032440 con 0x7f2054072c50 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f2053400640 1 -- 172.21.15.3:0/3909357032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040032810 con 0x7f2054072c50 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 -- 172.21.15.3:0/3909357032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2054072c50 msgr2=0x7f20540b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 --2- 172.21.15.3:0/3909357032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2054072c50 0x7f20540b4580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20540b65e0 tx=0x7f204002a1a0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 -- 172.21.15.3:0/3909357032 shutdown_connections 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 --2- 172.21.15.3:0/3909357032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2054072c50 0x7f20540b4580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 --2- 172.21.15.3:0/3909357032 >> v2:172.21.15.70:3300/0 conn(0x7f205415b230 0x7f205415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 -- 172.21.15.3:0/3909357032 >> 172.21.15.3:0/3909357032 conn(0x7f20540b5cb0 msgr2=0x7f205406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 -- 172.21.15.3:0/3909357032 shutdown_connections 2024-09-10T07:48:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.463+0000 7f205b800640 1 -- 172.21.15.3:0/3909357032 wait complete. 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.465+0000 7f205b800640 1 Processor -- start 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.465+0000 7f205b800640 1 -- start start 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2054072c50 0x7f205415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f205415b230 0x7f205414bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f205415f870 con 0x7f20540b4ac0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f205415c8d0 con 0x7f205415b230 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f205b800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f205415b610 con 0x7f2054072c50 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f2053e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f2053e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41244/0 (socket says 172.21.15.3:41244) 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.466+0000 7f2053e00640 1 -- 172.21.15.3:0/3647182682 learned_addr learned my addr 172.21.15.3:0/3647182682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2059800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f205415b230 0x7f205414bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2058e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2054072c50 0x7f205415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 -- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2054072c50 msgr2=0x7f205415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2054072c50 0x7f205415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 -- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f205415b230 msgr2=0x7f205414bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f205415b230 0x7f205414bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 -- 172.21.15.3:0/3647182682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205414d730 con 0x7f20540b4ac0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2053e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2040000c00 tx=0x7f20400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2050c00640 1 -- 172.21.15.3:0/3647182682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040037040 con 0x7f20540b4ac0 2024-09-10T07:48:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2050c00640 1 -- 172.21.15.3:0/3647182682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040042070 con 0x7f20540b4ac0 2024-09-10T07:48:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205414da10 con 0x7f20540b4ac0 2024-09-10T07:48:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.467+0000 7f2050c00640 1 -- 172.21.15.3:0/3647182682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040030bd0 con 0x7f20540b4ac0 2024-09-10T07:48:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.468+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2054135210 con 0x7f20540b4ac0 2024-09-10T07:48:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.469+0000 7f2050c00640 1 -- 172.21.15.3:0/3647182682 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2040047460 con 0x7f20540b4ac0 2024-09-10T07:48:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.470+0000 7f2050c00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 0x7f203406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.470+0000 7f2050c00640 1 -- 172.21.15.3:0/3647182682 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f20400abbd0 con 0x7f20540b4ac0 2024-09-10T07:48:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.471+0000 7f2058e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 0x7f203406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.471+0000 7f2058e00640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 0x7f203406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2054136480 tx=0x7f2048008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.471+0000 7f205e5e5740 1 Processor -- start 2024-09-10T07:48:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.471+0000 7f205e5e5740 1 -- start start 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c5a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 0x558a71c5a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558a71b77660 con 0x558a71c29420 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558a71b1f680 con 0x558a71c0f480 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.472+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558a71b73620 con 0x558a71c29800 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41260/0 (socket says 172.21.15.3:41260) 2024-09-10T07:48:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 -- 172.21.15.3:0/3529027220 learned_addr learned my addr 172.21.15.3:0/3529027220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2053e00640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c5a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2059800640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 0x558a71c5a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 -- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 msgr2=0x558a71c5a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c5a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 -- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 msgr2=0x558a71c5a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 0x558a71c5a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 -- 172.21.15.3:0/3529027220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a719547d0 con 0x558a71c29420 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2059800640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 0x558a71c5a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.473+0000 7f2058e00640 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20480108f0 tx=0x7f2048013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21e1f86c6daaa74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f2045600640 1 -- 172.21.15.3:0/3529027220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204801c050 con 0x558a71c29420 2024-09-10T07:48:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f2045600640 1 -- 172.21.15.3:0/3529027220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048026070 con 0x558a71c29420 2024-09-10T07:48:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f2045600640 1 -- 172.21.15.3:0/3529027220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800f040 con 0x558a71c29420 2024-09-10T07:48:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 -- 172.21.15.3:0/3529027220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 msgr2=0x558a71c59c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20480108f0 tx=0x7f2048013000 comp rx=0 tx=0).stop 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 -- 172.21.15.3:0/3529027220 shutdown_connections 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c0f480 0x558a71c5a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c5a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3529027220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c59c40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 -- 172.21.15.3:0/3529027220 >> 172.21.15.3:0/3529027220 conn(0x558a71c10220 msgr2=0x558a71c58840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 -- 172.21.15.3:0/3529027220 shutdown_connections 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.474+0000 7f205e5e5740 1 -- 172.21.15.3:0/3529027220 wait complete. 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.475+0000 7f205e5e5740 1 -- start start 2024-09-10T07:48:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.475+0000 7f205e5e5740 1 Processor -- start 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 0x558a71c55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558a71c0e3e0 con 0x558a71c29420 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558a71b004c0 con 0x558a71c37660 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558a71c0e6c0 con 0x558a71c29800 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f205e5e5740 10 client.0 _ll_register_callbacks cb 0x558a71b6fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.476+0000 7f2058e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41274/0 (socket says 172.21.15.3:41274) 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 -- 172.21.15.3:0/1247089121 learned_addr learned my addr 172.21.15.3:0/1247089121 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2053e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2059800640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 0x558a71c55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 -- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 msgr2=0x558a71c515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 -- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 msgr2=0x558a71c55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 0x558a71c55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 -- 172.21.15.3:0/1247089121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7188a3f0 con 0x558a71c29420 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2059800640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 0x558a71c55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f2058e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20480216d0 tx=0x7f2048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d30902aa9d0c18f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.477+0000 7f203a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204802f070 con 0x558a71c29420 2024-09-10T07:48:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f2048021cc0 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2048026070 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f204803c050 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 10 client.5208 fetch_fsmap learned FSMap version 25 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x558a7184a0f0 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048026070 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 3558+0+0 (secure 0 0 0) 0x7f204803f020 con 0x558a71c29420 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 10 client.5208 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 10 client.5208 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 20 client.5208 populate_metadata read hostname 'smithi003' 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c5a890 0x558a71c5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 4 client.5208 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.478+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a71b84d90 con 0x558a71c5a890 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.479+0000 7f2053e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c5a890 0x558a71c5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.479+0000 7f2053e00640 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c5a890 0x558a71c5aca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605132f1e48bf959 server_cookie=81087899a22bb8c5 in_seq=0 out_seq=0 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.479+0000 7f203a000640 10 client.5208 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f203a000640 1 -- 172.21.15.3:0/1247089121 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a71b84d90 con 0x558a71c5a890 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f203a000640 10 client.5208 handle_command_reply: tid=0 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 client.5208 shutdown 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 msgr2=0x558a71c26510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20480216d0 tx=0x7f2048004730 comp rx=0 tx=0).stop 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 shutdown_connections 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c5a890 0x558a71c5aca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c37660 0x558a71c55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 --2- 172.21.15.3:0/1247089121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c29420 0x558a71c26510 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 >> 172.21.15.3:0/1247089121 conn(0x558a71c0f480 msgr2=0x558a71c56ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f203a000640 0 client.5208 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:48:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 shutdown_connections 2024-09-10T07:48:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 1 -- 172.21.15.3:0/1247089121 wait complete. 2024-09-10T07:48:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.480+0000 7f205e5e5740 20 client.5208 trim_cache size 0 max 16384 2024-09-10T07:48:28.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.498+0000 7f205e5e5740 1 Processor -- start 2024-09-10T07:48:28.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 -- start start 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c56ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c36770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2054168990 con 0x558a71c37660 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2054167b50 con 0x558a71c29420 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.499+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2054168260 con 0x558a71c29800 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41278/0 (socket says 172.21.15.3:41278) 2024-09-10T07:48:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 -- 172.21.15.3:0/3747457641 learned_addr learned my addr 172.21.15.3:0/3747457641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2053e00640 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c36770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2058e00640 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c56ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 -- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 msgr2=0x558a71c36770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c36770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 -- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 msgr2=0x558a71c56ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c56ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 -- 172.21.15.3:0/3747457641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a71c0f990 con 0x558a71c37660 2024-09-10T07:48:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.500+0000 7f2059800640 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f204c002000 tx=0x7f204c002030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ba0f007b774b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f2044c00640 1 -- 172.21.15.3:0/3747457641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c0076f0 con 0x558a71c37660 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f2044c00640 1 -- 172.21.15.3:0/3747457641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c01b070 con 0x558a71c37660 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f2044c00640 1 -- 172.21.15.3:0/3747457641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c007a90 con 0x558a71c37660 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 -- 172.21.15.3:0/3747457641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 msgr2=0x558a71c57c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f204c002000 tx=0x7f204c002030 comp rx=0 tx=0).stop 2024-09-10T07:48:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 -- 172.21.15.3:0/3747457641 shutdown_connections 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c57c60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c36770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 --2- 172.21.15.3:0/3747457641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c56ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 -- 172.21.15.3:0/3747457641 >> 172.21.15.3:0/3747457641 conn(0x558a71c0f480 msgr2=0x558a71c5bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 -- 172.21.15.3:0/3747457641 shutdown_connections 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.501+0000 7f205e5e5740 1 -- 172.21.15.3:0/3747457641 wait complete. 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.502+0000 7f205e5e5740 1 -- start start 2024-09-10T07:48:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.502+0000 7f205e5e5740 1 Processor -- start 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c27a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20540b1740 con 0x558a71c37660 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558a71c26c30 con 0x558a71c29420 2024-09-10T07:48:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558a71cdc780 con 0x558a71c29800 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f205e5e5740 10 client.0 _ll_register_callbacks cb 0x7f204002a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.503+0000 7f2059800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2058e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41288/0 (socket says 172.21.15.3:41288) 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 -- 172.21.15.3:0/4141176586 learned_addr learned my addr 172.21.15.3:0/4141176586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2053e00640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c27a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 -- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 msgr2=0x558a71c27a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c27a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 -- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 msgr2=0x558a71c274e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c274e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 -- 172.21.15.3:0/4141176586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a71c585e0 con 0x558a71c37660 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f2059800640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47249757ca06d55d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f203a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.504+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c016c40 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f204c016de0 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f204c01b070 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f204c031050 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 10 client.5214 fetch_fsmap learned FSMap version 25 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x558a71c28e50 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c01b070 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 3558+0+0 (secure 0 0 0) 0x7f204c034020 con 0x558a71c37660 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 10 client.5214 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 10 client.5214 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 20 client.5214 populate_metadata read hostname 'smithi003' 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c0d4a0 0x558a71c137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 4 client.5214 mds_command: new command op to 4741 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 --> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a71b7f0e0 con 0x558a71c0d4a0 2024-09-10T07:48:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.505+0000 7f2058e00640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c0d4a0 0x558a71c137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f2058e00640 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c0d4a0 0x558a71c137d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705d3df831ab8d30 server_cookie=4af23be39eac0dbc in_seq=0 out_seq=0 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f203a000640 10 client.5214 ms_handle_connect on v2:172.21.15.3:6834/2294972972 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f203a000640 1 -- 172.21.15.3:0/4141176586 <== mds.0 v2:172.21.15.3:6834/2294972972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a71b7f0e0 con 0x558a71c0d4a0 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f203a000640 10 client.5214 handle_command_reply: tid=0 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 client.5214 shutdown 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 msgr2=0x558a71c0cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c001250 comp rx=0 tx=0).stop 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 shutdown_connections 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:6834/2294972972,v1:172.21.15.3:6836/2294972972] conn(0x558a71c0d4a0 0x558a71c137d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558a71c37660 0x558a71c0cf60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558a71c29800 0x558a71c27a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f203a000640 0 client.5214 ms_handle_reset on v2:172.21.15.3:6834/2294972972 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 --2- 172.21.15.3:0/4141176586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558a71c29420 0x558a71c274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.506+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 >> 172.21.15.3:0/4141176586 conn(0x558a71c0f480 msgr2=0x558a71c598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 shutdown_connections 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205e5e5740 1 -- 172.21.15.3:0/4141176586 wait complete. 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205e5e5740 20 client.5214 trim_cache size 0 max 16384 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-10T07:48:28.509 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 msgr2=0x7f203406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 0x7f203406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2054136480 tx=0x7f2048008040 comp rx=0 tx=0).stop 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 msgr2=0x7f205414ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2040000c00 tx=0x7f20400046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 shutdown_connections 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f203406be30 0x7f203406e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f205415b230 0x7f205414bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20540b4ac0 0x7f205414ba20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 --2- 172.21.15.3:0/3647182682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2054072c50 0x7f205415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.507+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 >> 172.21.15.3:0/3647182682 conn(0x7f20540b5cb0 msgr2=0x7f20541366b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.508+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 shutdown_connections 2024-09-10T07:48:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.508+0000 7f205b800640 1 -- 172.21.15.3:0/3647182682 wait complete. 2024-09-10T07:48:28.522 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:48:28.522 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2024-09-10T07:48:28.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.600+0000 7f00f851ef40 1 Processor -- start 2024-09-10T07:48:28.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.600+0000 7f00f851ef40 1 -- start start 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 0x55e807fffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e80807ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e807fe7150 con 0x55e807fe67a0 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e807f5cb30 con 0x55e807fe8220 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f851ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e80808d620 con 0x55e807fe7e40 2024-09-10T07:48:28.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41314/0 (socket says 172.21.15.3:41314) 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 -- 172.21.15.3:0/1490987856 learned_addr learned my addr 172.21.15.3:0/1490987856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00efe00640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 0x55e807fffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 -- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 msgr2=0x55e807fffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 0x55e807fffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 -- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 msgr2=0x55e80807ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f6200640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e80807ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e80807ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f5800640 1 -- 172.21.15.3:0/1490987856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80808e320 con 0x55e807fe67a0 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00efe00640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 0x55e807fffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.601+0000 7f00f6200640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e80807ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00f5800640 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b260 tx=0x7f00e800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2b741656f7d1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00ef400640 1 -- 172.21.15.3:0/1490987856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e800e5c0 con 0x55e807fe67a0 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00ef400640 1 -- 172.21.15.3:0/1490987856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e8006070 con 0x55e807fe67a0 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00ef400640 1 -- 172.21.15.3:0/1490987856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e800e9c0 con 0x55e807fe67a0 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00f851ef40 1 -- 172.21.15.3:0/1490987856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 msgr2=0x55e808082830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00f851ef40 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b260 tx=0x7f00e800b720 comp rx=0 tx=0).stop 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.602+0000 7f00f851ef40 1 -- 172.21.15.3:0/1490987856 shutdown_connections 2024-09-10T07:48:28.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e80807ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe7e40 0x55e807fffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 --2- 172.21.15.3:0/1490987856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe67a0 0x55e808082830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 -- 172.21.15.3:0/1490987856 >> 172.21.15.3:0/1490987856 conn(0x55e807fe4d40 msgr2=0x55e807fe5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 -- 172.21.15.3:0/1490987856 shutdown_connections 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 -- 172.21.15.3:0/1490987856 wait complete. 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 Processor -- start 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.603+0000 7f00f851ef40 1 -- start start 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe67a0 0x55e807fb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 0x55e80807a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e808087880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e80808e600 con 0x55e807fe7e40 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e808083060 con 0x55e807fe8220 2024-09-10T07:48:28.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00f851ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e80808e8e0 con 0x55e807fe67a0 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.604+0000 7f00efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 0x55e80807a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 0x55e80807a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41320/0 (socket says 172.21.15.3:41320) 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 -- 172.21.15.3:0/21696594 learned_addr learned my addr 172.21.15.3:0/21696594 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00f5800640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe67a0 0x55e807fb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00f6200640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e808087880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 -- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe67a0 msgr2=0x55e807fb44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e807fe67a0 0x55e807fb44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 -- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 msgr2=0x55e808087880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e807fe8220 0x55e808087880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 -- 172.21.15.3:0/21696594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e808087dc0 con 0x55e807fe7e40 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00efe00640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 0x55e80807a620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f00e0045b00 tx=0x7f00e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00ee000640 1 -- 172.21.15.3:0/21696594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0056040 con 0x55e807fe7e40 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00ee000640 1 -- 172.21.15.3:0/21696594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0052af0 con 0x55e807fe7e40 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.605+0000 7f00ee000640 1 -- 172.21.15.3:0/21696594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0066440 con 0x55e807fe7e40 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.606+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e808088080 con 0x55e807fe7e40 2024-09-10T07:48:28.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.606+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80807add0 con 0x55e807fe7e40 2024-09-10T07:48:28.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.607+0000 7f00ee000640 1 -- 172.21.15.3:0/21696594 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00e0059020 con 0x55e807fe7e40 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.608+0000 7f00ee000640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f00d406a810 0x7f00d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.608+0000 7f00ee000640 1 -- 172.21.15.3:0/21696594 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f00e00cdbc0 con 0x55e807fe7e40 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.608+0000 7f00f5800640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f00d406a810 0x7f00d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.609+0000 7f00f851ef40 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e80807bd60 0x55e80807cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.609+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 4.12 4:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55e808080b20 con 0x55e80807bd60 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.609+0000 7f00f5800640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f00d406a810 0x7f00d406ccd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b760 tx=0x7f00e8009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.609+0000 7f00f6200640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e80807bd60 0x55e80807cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f6200640 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e80807bd60 0x55e80807cb60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f6200640 1 -- 172.21.15.3:0/21696594 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f00f0011040 con 0x55e80807bd60 2024-09-10T07:48:28.609 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e80807bd60 msgr2=0x55e80807cb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e80807bd60 0x55e80807cb60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f00d406a810 msgr2=0x7f00d406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f00d406a810 0x7f00d406ccd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b760 tx=0x7f00e8009000 comp rx=0 tx=0).stop 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 msgr2=0x55e80807a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 --2- 172.21.15.3:0/21696594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e807fe7e40 0x55e80807a620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f00e0045b00 tx=0x7f00e0002fb0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f6200640 1 -- 172.21.15.3:0/21696594 reap_dead start 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 shutdown_connections 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.610+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 >> 172.21.15.3:0/21696594 conn(0x55e807fe4d40 msgr2=0x55e8080858d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.611+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 shutdown_connections 2024-09-10T07:48:28.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.611+0000 7f00f851ef40 1 -- 172.21.15.3:0/21696594 wait complete. 2024-09-10T07:48:28.614 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2024-09-10T07:48:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 Processor -- start 2024-09-10T07:48:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 -- start start 2024-09-10T07:48:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 0x55c2fdd1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 0x55c2fdd1a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c2fdbf7b30 con 0x55c2fdc83220 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c2fdbfbb70 con 0x55c2fdc82e40 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.692+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c2fdd28900 con 0x55c2fdc81eb0 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41338/0 (socket says 172.21.15.3:41338) 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 -- 172.21.15.3:0/3171241861 learned_addr learned my addr 172.21.15.3:0/3171241861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e2800640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 0x55c2fdd1a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 -- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 msgr2=0x55c2fdd1ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3200640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 0x55c2fdd1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 0x55c2fdd1ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 -- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 msgr2=0x55c2fdd1a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 0x55c2fdd1a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3c00640 1 -- 172.21.15.3:0/3171241861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fdc7e610 con 0x55c2fdc83220 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e2800640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 0x55c2fdd1a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.693+0000 7f24e3200640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 0x55c2fdd1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e3c00640 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24dc025b90 tx=0x7f24dc04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3dc77bc6f249 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e1400640 1 -- 172.21.15.3:0/3171241861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc05a050 con 0x55c2fdc83220 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e1400640 1 -- 172.21.15.3:0/3171241861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24dc066070 con 0x55c2fdc83220 2024-09-10T07:48:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e1400640 1 -- 172.21.15.3:0/3171241861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc055a30 con 0x55c2fdc83220 2024-09-10T07:48:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 -- 172.21.15.3:0/3171241861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 msgr2=0x55c2fdd1d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24dc025b90 tx=0x7f24dc04f000 comp rx=0 tx=0).stop 2024-09-10T07:48:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 -- 172.21.15.3:0/3171241861 shutdown_connections 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc83220 0x55c2fdd1d6e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc82e40 0x55c2fdd1a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/3171241861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc81eb0 0x55c2fdd1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 -- 172.21.15.3:0/3171241861 >> 172.21.15.3:0/3171241861 conn(0x55c2fdc7fd40 msgr2=0x55c2fdc80120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 -- 172.21.15.3:0/3171241861 shutdown_connections 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.694+0000 7f24e5f17f40 1 -- 172.21.15.3:0/3171241861 wait complete. 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.695+0000 7f24e5f17f40 1 Processor -- start 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.695+0000 7f24e5f17f40 1 -- start start 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 0x55c2fdd2aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc82e40 0x55c2fdc7df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc83220 0x55c2fdd1c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c2fdc9b210 con 0x55c2fdc81eb0 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c2fdc9a690 con 0x55c2fdc83220 2024-09-10T07:48:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e5f17f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c2fdd1a960 con 0x55c2fdc82e40 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 0x55c2fdd2aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 0x55c2fdd2aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41340/0 (socket says 172.21.15.3:41340) 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.696+0000 7f24e3200640 1 -- 172.21.15.3:0/1777307371 learned_addr learned my addr 172.21.15.3:0/1777307371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e2800640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc82e40 0x55c2fdc7df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3c00640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc83220 0x55c2fdd1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 -- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc82e40 msgr2=0x55c2fdc7df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c2fdc82e40 0x55c2fdc7df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 -- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc83220 msgr2=0x55c2fdd1c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc83220 0x55c2fdd1c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 -- 172.21.15.3:0/1777307371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fdd272b0 con 0x55c2fdc81eb0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3c00640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2fdc83220 0x55c2fdd1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e3200640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 0x55c2fdd2aa80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24d400cec0 tx=0x7f24d400a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24dbe00640 1 -- 172.21.15.3:0/1777307371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4010470 con 0x55c2fdc81eb0 2024-09-10T07:48:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24dbe00640 1 -- 172.21.15.3:0/1777307371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d4010610 con 0x55c2fdc81eb0 2024-09-10T07:48:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24dbe00640 1 -- 172.21.15.3:0/1777307371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40109c0 con 0x55c2fdc81eb0 2024-09-10T07:48:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.697+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c2fdd27590 con 0x55c2fdc81eb0 2024-09-10T07:48:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.698+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fdd1c060 con 0x55c2fdc81eb0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.699+0000 7f24dbe00640 1 -- 172.21.15.3:0/1777307371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24d400e040 con 0x55c2fdc81eb0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.700+0000 7f24dbe00640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24c406a810 0x7f24c406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.700+0000 7f24dbe00640 1 -- 172.21.15.3:0/1777307371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f24d4089b20 con 0x55c2fdc81eb0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.700+0000 7f24e2800640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24c406a810 0x7f24c406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.700+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c2fdd1cac0 0x55c2fdd23760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.700+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 4.12 4:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55c2fdd21170 con 0x55c2fdd1cac0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.701+0000 7f24e3c00640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c2fdd1cac0 0x55c2fdd23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.701+0000 7f24e2800640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24c406a810 0x7f24c406ccd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c2fdc9d1e0 tx=0x7f24d0001650 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e3c00640 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c2fdd1cac0 0x55c2fdd23760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e3c00640 1 -- 172.21.15.3:0/1777307371 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f24dc05d070 con 0x55c2fdd1cac0 2024-09-10T07:48:28.701 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c2fdd1cac0 msgr2=0x55c2fdd23760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c2fdd1cac0 0x55c2fdd23760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24c406a810 msgr2=0x7f24c406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24c406a810 0x7f24c406ccd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c2fdc9d1e0 tx=0x7f24d0001650 comp rx=0 tx=0).stop 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 msgr2=0x55c2fdd2aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 --2- 172.21.15.3:0/1777307371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c2fdc81eb0 0x55c2fdd2aa80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24d400cec0 tx=0x7f24d400a5a0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e3c00640 1 -- 172.21.15.3:0/1777307371 reap_dead start 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 shutdown_connections 2024-09-10T07:48:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 >> 172.21.15.3:0/1777307371 conn(0x55c2fdc7fd40 msgr2=0x55c2fdd1d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 shutdown_connections 2024-09-10T07:48:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.702+0000 7f24e5f17f40 1 -- 172.21.15.3:0/1777307371 wait complete. 2024-09-10T07:48:28.705 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2024-09-10T07:48:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.784+0000 7f9edfa29f40 1 Processor -- start 2024-09-10T07:48:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.784+0000 7f9edfa29f40 1 -- start start 2024-09-10T07:48:28.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98d7a0 0x557d4aa2cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4aa26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557d4a98e120 con 0x557d4a98ee40 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557d4a903b30 con 0x557d4a98f220 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557d4aa34620 con 0x557d4a98d7a0 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41350/0 (socket says 172.21.15.3:41350) 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 -- 172.21.15.3:0/661043929 learned_addr learned my addr 172.21.15.3:0/661043929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edcc00640 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98d7a0 0x557d4aa2cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9edd600640 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4aa26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 -- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98d7a0 msgr2=0x557d4aa2cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98d7a0 0x557d4aa2cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 -- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 msgr2=0x557d4aa26c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4aa26c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.785+0000 7f9ed7e00640 1 -- 172.21.15.3:0/661043929 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4aa35200 con 0x557d4a98ee40 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800a4f0 tx=0x7f9ec8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec6368d1373f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9ed6a00640 1 -- 172.21.15.3:0/661043929 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80537e0 con 0x557d4a98ee40 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9ed6a00640 1 -- 172.21.15.3:0/661043929 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec8063070 con 0x557d4a98ee40 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9ed6a00640 1 -- 172.21.15.3:0/661043929 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8053b80 con 0x557d4a98ee40 2024-09-10T07:48:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9edfa29f40 1 -- 172.21.15.3:0/661043929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 msgr2=0x557d4a95c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.786+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800a4f0 tx=0x7f9ec8051000 comp rx=0 tx=0).stop 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 -- 172.21.15.3:0/661043929 shutdown_connections 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4aa26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98ee40 0x557d4a95c9d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/661043929 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98d7a0 0x557d4aa2cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 -- 172.21.15.3:0/661043929 >> 172.21.15.3:0/661043929 conn(0x557d4a98bd40 msgr2=0x557d4a98c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 -- 172.21.15.3:0/661043929 shutdown_connections 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 -- 172.21.15.3:0/661043929 wait complete. 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.787+0000 7f9edfa29f40 1 Processor -- start 2024-09-10T07:48:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 -- start start 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 0x557d4aa29220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98ee40 0x557d4aa248e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4a98af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557d4aa354e0 con 0x557d4a98d7a0 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557d4aa2d590 con 0x557d4a98f220 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.788+0000 7f9edfa29f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557d4aa357c0 con 0x557d4a98ee40 2024-09-10T07:48:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 0x557d4aa29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 0x557d4aa29220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41352/0 (socket says 172.21.15.3:41352) 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 -- 172.21.15.3:0/3581741634 learned_addr learned my addr 172.21.15.3:0/3581741634 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edd600640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4a98af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98ee40 0x557d4aa248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 -- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98ee40 msgr2=0x557d4aa248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98ee40 0x557d4aa248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 -- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 msgr2=0x557d4a98af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4a98af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 -- 172.21.15.3:0/3581741634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4aa25180 con 0x557d4a98d7a0 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edd600640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d4a98f220 0x557d4a98af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d4a98ee40 0x557d4aa248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.789+0000 7f9edcc00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 0x557d4aa29220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0002d40 tx=0x7f9ed000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.790+0000 7f9ed5600640 1 -- 172.21.15.3:0/3581741634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0015020 con 0x557d4a98d7a0 2024-09-10T07:48:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.790+0000 7f9ed5600640 1 -- 172.21.15.3:0/3581741634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0011450 con 0x557d4a98d7a0 2024-09-10T07:48:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.790+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557d4aa298b0 con 0x557d4a98d7a0 2024-09-10T07:48:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.790+0000 7f9ed5600640 1 -- 172.21.15.3:0/3581741634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0011820 con 0x557d4a98d7a0 2024-09-10T07:48:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.790+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4aaca240 con 0x557d4a98d7a0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.792+0000 7f9ed5600640 1 -- 172.21.15.3:0/3581741634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ed0018020 con 0x557d4a98d7a0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.792+0000 7f9ed5600640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06a810 0x7f9ebc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9ed5600640 1 -- 172.21.15.3:0/3581741634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f9ed0089da0 con 0x557d4a98d7a0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06a810 0x7f9ebc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x557d4aacadd0 0x557d4aacb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- osd_op(unknown.0.0:1 4.5 4:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x557d4aacb770 con 0x557d4aacadd0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9edd600640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x557d4aacadd0 0x557d4aacb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.793+0000 7f9ed7e00640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06a810 0x7f9ebc06ccd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8053000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edd600640 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x557d4aacadd0 0x557d4aacb230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edd600640 1 -- 172.21.15.3:0/3581741634 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f9ed8002a60 con 0x557d4aacadd0 2024-09-10T07:48:28.793 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x557d4aacadd0 msgr2=0x557d4aacb230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x557d4aacadd0 0x557d4aacb230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06a810 msgr2=0x7f9ebc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06a810 0x7f9ebc06ccd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8053000 comp rx=0 tx=0).stop 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 msgr2=0x557d4aa29220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 --2- 172.21.15.3:0/3581741634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d4a98d7a0 0x557d4aa29220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0002d40 tx=0x7f9ed000c620 comp rx=0 tx=0).stop 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edd600640 1 -- 172.21.15.3:0/3581741634 reap_dead start 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 shutdown_connections 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.794+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 >> 172.21.15.3:0/3581741634 conn(0x557d4a98bd40 msgr2=0x557d4a98ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.795+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 shutdown_connections 2024-09-10T07:48:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.795+0000 7f9edfa29f40 1 -- 172.21.15.3:0/3581741634 wait complete. 2024-09-10T07:48:28.797 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2024-09-10T07:48:28.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.875+0000 7fd38a0ccf40 1 Processor -- start 2024-09-10T07:48:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.875+0000 7fd38a0ccf40 1 -- start start 2024-09-10T07:48:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 0x562bbee07bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 0x562bbed6cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bbece5b30 con 0x562bbed6feb0 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562bbece9b70 con 0x562bbed70e40 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562bbee16900 con 0x562bbed71220 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.876+0000 7fd387200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41364/0 (socket says 172.21.15.3:41364) 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 -- 172.21.15.3:0/313205247 learned_addr learned my addr 172.21.15.3:0/313205247 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd386800640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 0x562bbee07bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387c00640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 0x562bbed6cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 -- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 msgr2=0x562bbed6cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 0x562bbed6cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 -- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 msgr2=0x562bbee07bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 0x562bbee07bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 -- 172.21.15.3:0/313205247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbee0be40 con 0x562bbed6feb0 2024-09-10T07:48:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387c00640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 0x562bbed6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd386800640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 0x562bbee07bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd387200640 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd378001d80 tx=0x7fd378001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166efa4cc62a19ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd385400640 1 -- 172.21.15.3:0/313205247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800bd70 con 0x562bbed6feb0 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd385400640 1 -- 172.21.15.3:0/313205247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37800f400 con 0x562bbed6feb0 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.877+0000 7fd385400640 1 -- 172.21.15.3:0/313205247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800f7f0 con 0x562bbed6feb0 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/313205247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 msgr2=0x562bbee0d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd378001d80 tx=0x7fd378001e70 comp rx=0 tx=0).stop 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/313205247 shutdown_connections 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed71220 0x562bbed6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed70e40 0x562bbee07bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/313205247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee0d680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/313205247 >> 172.21.15.3:0/313205247 conn(0x562bbed6dd40 msgr2=0x562bbed3df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/313205247 shutdown_connections 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.878+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/313205247 wait complete. 2024-09-10T07:48:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.879+0000 7fd38a0ccf40 1 Processor -- start 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.879+0000 7fd38a0ccf40 1 -- start start 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee07020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed70e40 0x562bbed6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed71220 0x562bbee08d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562bbee0bb60 con 0x562bbed6feb0 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562bbee0b880 con 0x562bbed71220 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd38a0ccf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562bbed3ead0 con 0x562bbed70e40 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee07020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee07020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41370/0 (socket says 172.21.15.3:41370) 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 -- 172.21.15.3:0/1706354768 learned_addr learned my addr 172.21.15.3:0/1706354768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd386800640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed70e40 0x562bbed6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387c00640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed71220 0x562bbee08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 -- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed70e40 msgr2=0x562bbed6cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562bbed70e40 0x562bbed6cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 -- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed71220 msgr2=0x562bbee08d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562bbed71220 0x562bbee08d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.880+0000 7fd387200640 1 -- 172.21.15.3:0/1706354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbee10dc0 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd387200640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee07020 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd37800ba40 tx=0x7fd378000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd37fe00640 1 -- 172.21.15.3:0/1706354768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378020070 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd37fe00640 1 -- 172.21.15.3:0/1706354768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37801c440 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd37fe00640 1 -- 172.21.15.3:0/1706354768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801c7f0 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562bbee0a8a0 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.881+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbee11060 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.883+0000 7fd37fe00640 1 -- 172.21.15.3:0/1706354768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd378023070 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.884+0000 7fd37fe00640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd36806a810 0x7fd36806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.884+0000 7fd37fe00640 1 -- 172.21.15.3:0/1706354768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fd37808b440 con 0x562bbed6feb0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.884+0000 7fd386800640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd36806a810 0x7fd36806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.884+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x562bbee11860 0x562bbee12ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.884+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- osd_op(unknown.0.0:1 4.9 4:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x562bbed6d500 con 0x562bbee11860 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.885+0000 7fd387c00640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x562bbee11860 0x562bbee12ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.885+0000 7fd386800640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd36806a810 0x7fd36806ccd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x562bbed3e520 tx=0x7fd3740017f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.885+0000 7fd387c00640 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x562bbee11860 0x562bbee12ff0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.885+0000 7fd387c00640 1 -- 172.21.15.3:0/1706354768 <== osd.1 v2:172.21.15.3:6800/2429800591 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fd38004e040 con 0x562bbee11860 2024-09-10T07:48:28.919 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x562bbee11860 msgr2=0x562bbee12ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x562bbee11860 0x562bbee12ff0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd36806a810 msgr2=0x7fd36806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd36806a810 0x7fd36806ccd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x562bbed3e520 tx=0x7fd3740017f0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 msgr2=0x562bbee07020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 --2- 172.21.15.3:0/1706354768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bbed6feb0 0x562bbee07020 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd37800ba40 tx=0x7fd378000bc0 comp rx=0 tx=0).stop 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd387c00640 1 -- 172.21.15.3:0/1706354768 reap_dead start 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 shutdown_connections 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 >> 172.21.15.3:0/1706354768 conn(0x562bbed6dd40 msgr2=0x562bbee0e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 shutdown_connections 2024-09-10T07:48:28.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:28.886+0000 7fd38a0ccf40 1 -- 172.21.15.3:0/1706354768 wait complete. 2024-09-10T07:48:28.922 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2024-09-10T07:48:28.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.000+0000 7f27ef996f40 1 Processor -- start 2024-09-10T07:48:28.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.000+0000 7f27ef996f40 1 -- start start 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567b8ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567af0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562567af5150 con 0x562567af5e40 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562567a6ab30 con 0x562567af6220 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27ef996f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562567b9b620 con 0x562567af47a0 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41382/0 (socket says 172.21.15.3:41382) 2024-09-10T07:48:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.001+0000 7f27e7e00640 1 -- 172.21.15.3:0/3616942275 learned_addr learned my addr 172.21.15.3:0/3616942275 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567b8ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27ed600640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567af0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 -- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 msgr2=0x562567b8ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567b8ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 -- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 msgr2=0x562567af0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567af0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 -- 172.21.15.3:0/3616942275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562567b93d10 con 0x562567af5e40 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567b8ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:29.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27ed600640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567af0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001e50 tx=0x7f27d800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4318eddf296db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e6a00640 1 -- 172.21.15.3:0/3616942275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8012020 con 0x562567af5e40 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e6a00640 1 -- 172.21.15.3:0/3616942275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d800e670 con 0x562567af5e40 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.002+0000 7f27e6a00640 1 -- 172.21.15.3:0/3616942275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d800ea60 con 0x562567af5e40 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 -- 172.21.15.3:0/3616942275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 msgr2=0x562567b0da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001e50 tx=0x7f27d800b780 comp rx=0 tx=0).stop 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 -- 172.21.15.3:0/3616942275 shutdown_connections 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567af0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b0da50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3616942275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567b8ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 -- 172.21.15.3:0/3616942275 >> 172.21.15.3:0/3616942275 conn(0x562567af2d40 msgr2=0x562567af3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 -- 172.21.15.3:0/3616942275 shutdown_connections 2024-09-10T07:48:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.003+0000 7f27ef996f40 1 -- 172.21.15.3:0/3616942275 wait complete. 2024-09-10T07:48:29.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.004+0000 7f27ef996f40 1 Processor -- start 2024-09-10T07:48:29.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.004+0000 7f27ef996f40 1 -- start start 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567c30f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b89a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567b0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562567af2800 con 0x562567af5e40 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562567b947e0 con 0x562567af6220 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ef996f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562567b93a10 con 0x562567af47a0 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b89a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41398/0 (socket says 172.21.15.3:41398) 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 -- 172.21.15.3:0/3711134451 learned_addr learned my addr 172.21.15.3:0/3711134451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567c30f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 -- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 msgr2=0x562567c30f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567c30f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ed600640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567b0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 -- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 msgr2=0x562567b0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562567af6220 0x562567b0d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27e7e00640 1 -- 172.21.15.3:0/3711134451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562567af25b0 con 0x562567af5e40 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.005+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562567af47a0 0x562567c30f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27e7e00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b89a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27d800b7c0 tx=0x7f27d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27e5600640 1 -- 172.21.15.3:0/3711134451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8027070 con 0x562567af5e40 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27e5600640 1 -- 172.21.15.3:0/3711134451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d800e470 con 0x562567af5e40 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562567b99f80 con 0x562567af5e40 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27e5600640 1 -- 172.21.15.3:0/3711134451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8019a10 con 0x562567af5e40 2024-09-10T07:48:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.006+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562567c314b0 con 0x562567af5e40 2024-09-10T07:48:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.008+0000 7f27e5600640 1 -- 172.21.15.3:0/3711134451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27d802a070 con 0x562567af5e40 2024-09-10T07:48:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27e5600640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f27cc06a840 0x7f27cc06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27e5600640 1 -- 172.21.15.3:0/3711134451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f27d808b700 con 0x562567af5e40 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f27cc06a840 0x7f27cc06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x562567b8bcd0 0x562567c3d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- osd_op(unknown.0.0:1 4.5 4:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x562567b93450 con 0x562567b8bcd0 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.009+0000 7f27ed600640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x562567b8bcd0 0x562567c3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ecc00640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f27cc06a840 0x7f27cc06cd00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e0005a00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ed600640 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x562567b8bcd0 0x562567c3d550 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ed600640 1 -- 172.21.15.3:0/3711134451 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f27e8057030 con 0x562567b8bcd0 2024-09-10T07:48:29.010 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x562567b8bcd0 msgr2=0x562567c3d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x562567b8bcd0 0x562567c3d550 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f27cc06a840 msgr2=0x7f27cc06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.010+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f27cc06a840 0x7f27cc06cd00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e0005a00 comp rx=0 tx=0).stop 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 msgr2=0x562567b89a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 --2- 172.21.15.3:0/3711134451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562567af5e40 0x562567b89a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27d800b7c0 tx=0x7f27d8000bc0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ed600640 1 -- 172.21.15.3:0/3711134451 reap_dead start 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 shutdown_connections 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 >> 172.21.15.3:0/3711134451 conn(0x562567af2d40 msgr2=0x562567af3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 shutdown_connections 2024-09-10T07:48:29.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.011+0000 7f27ef996f40 1 -- 172.21.15.3:0/3711134451 wait complete. 2024-09-10T07:48:29.013 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2024-09-10T07:48:29.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.091+0000 7f6c7504af40 1 Processor -- start 2024-09-10T07:48:29.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.091+0000 7f6c7504af40 1 -- start start 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da39d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da39d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558da3a7d900 con 0x558da39f0130 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558da3a7d620 con 0x558da39d7e40 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c7504af40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558da3a7dbe0 con 0x558da39d8220 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c72c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c72c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41414/0 (socket says 172.21.15.3:41414) 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c72c00640 1 -- 172.21.15.3:0/2266549023 learned_addr learned my addr 172.21.15.3:0/2266549023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c71800640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da39d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.092+0000 7f6c72200640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da39d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 -- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 msgr2=0x558da39d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da39d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 -- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 msgr2=0x558da39d7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da39d7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 -- 172.21.15.3:0/2266549023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558da38d0b30 con 0x558da39f0130 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c71800640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da39d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c72c00640 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c440 tx=0x7f6c6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5768dc3a6d02f8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c6be00640 1 -- 172.21.15.3:0/2266549023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c010650 con 0x558da39f0130 2024-09-10T07:48:29.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c6be00640 1 -- 172.21.15.3:0/2266549023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c016070 con 0x558da39f0130 2024-09-10T07:48:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.093+0000 7f6c6be00640 1 -- 172.21.15.3:0/2266549023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c010a50 con 0x558da39f0130 2024-09-10T07:48:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 -- 172.21.15.3:0/2266549023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 msgr2=0x558da3a73420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c440 tx=0x7f6c6c00c900 comp rx=0 tx=0).stop 2024-09-10T07:48:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 -- 172.21.15.3:0/2266549023 shutdown_connections 2024-09-10T07:48:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a73420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da39d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 --2- 172.21.15.3:0/2266549023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da39d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 -- 172.21.15.3:0/2266549023 >> 172.21.15.3:0/2266549023 conn(0x558da39d4d40 msgr2=0x558da39a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 -- 172.21.15.3:0/2266549023 shutdown_connections 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.094+0000 7f6c7504af40 1 -- 172.21.15.3:0/2266549023 wait complete. 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.095+0000 7f6c7504af40 1 Processor -- start 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.095+0000 7f6c7504af40 1 -- start start 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.095+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da3a5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.095+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da3a71170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c7504af40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a7a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c7504af40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558da39f2530 con 0x558da39f0130 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c7504af40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558da3a606e0 con 0x558da39d7e40 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c7504af40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558da3a5ee10 con 0x558da39d8220 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a7a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a7a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41418/0 (socket says 172.21.15.3:41418) 2024-09-10T07:48:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 -- 172.21.15.3:0/1085586317 learned_addr learned my addr 172.21.15.3:0/1085586317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72200640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da3a5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c71800640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da3a71170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 -- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 msgr2=0x558da3a71170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558da39d8220 0x558da3a71170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 -- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 msgr2=0x558da3a5dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558da39d7e40 0x558da3a5dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 -- 172.21.15.3:0/1085586317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558da3a79720 con 0x558da39f0130 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.096+0000 7f6c72c00640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a7a7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x558da39f2980 tx=0x7f6c6c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.097+0000 7f6c6aa00640 1 -- 172.21.15.3:0/1085586317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c010cd0 con 0x558da39f0130 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.097+0000 7f6c6aa00640 1 -- 172.21.15.3:0/1085586317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c024430 con 0x558da39f0130 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.097+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558da3a79a00 con 0x558da39f0130 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.097+0000 7f6c6aa00640 1 -- 172.21.15.3:0/1085586317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c024820 con 0x558da39f0130 2024-09-10T07:48:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.097+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558da3a79fc0 con 0x558da39f0130 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.098+0000 7f6c6aa00640 1 -- 172.21.15.3:0/1085586317 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c6c02d430 con 0x558da39f0130 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.099+0000 7f6c6aa00640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c5406a810 0x7f6c5406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.099+0000 7f6c6aa00640 1 -- 172.21.15.3:0/1085586317 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f6c6c00e020 con 0x558da39f0130 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.100+0000 7f6c72200640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c5406a810 0x7f6c5406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.100+0000 7f6c7504af40 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x558da3a716b0 0x558da3b1e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.100+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 --> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] -- osd_op(unknown.0.0:1 4.1d 4:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x558da3b1ec90 con 0x558da3a716b0 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.100+0000 7f6c71800640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x558da3a716b0 0x558da3b1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.100+0000 7f6c72200640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c5406a810 0x7f6c5406ccd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c64057000 tx=0x7f6c64052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c71800640 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x558da3a716b0 0x558da3b1e750 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c71800640 1 -- 172.21.15.3:0/1085586317 <== osd.7 v2:172.21.15.70:6808/3218842598 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f6c60009a60 con 0x558da3a716b0 2024-09-10T07:48:29.101 INFO:teuthology.orchestra.run.smithi003.stdout:tag123 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x558da3a716b0 msgr2=0x558da3b1e750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x558da3a716b0 0x558da3b1e750 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c5406a810 msgr2=0x7f6c5406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c5406a810 0x7f6c5406ccd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c64057000 tx=0x7f6c64052000 comp rx=0 tx=0).stop 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 msgr2=0x558da3a7a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c7504af40 1 --2- 172.21.15.3:0/1085586317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558da39f0130 0x558da3a7a7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x558da39f2980 tx=0x7f6c6c0033a0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.101+0000 7f6c72c00640 1 -- 172.21.15.3:0/1085586317 reap_dead start 2024-09-10T07:48:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.102+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 shutdown_connections 2024-09-10T07:48:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.102+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 >> 172.21.15.3:0/1085586317 conn(0x558da39d4d40 msgr2=0x558da39d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.102+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 shutdown_connections 2024-09-10T07:48:29.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.102+0000 7f6c7504af40 1 -- 172.21.15.3:0/1085586317 wait complete. 2024-09-10T07:48:29.104 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:48:29.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:48:29.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:48:29.282 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:48:29.282 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:48:29.282 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:48:29.282 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:29.318 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:48:29.318 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:29.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:48:29.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.591+0000 7fb779c00640 1 Processor -- start 2024-09-10T07:48:29.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.592+0000 7fb779c00640 1 -- start start 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb779c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb77415b330 0x7fb7740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb779c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7740b2540 0x7fb7740cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb779c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77415bac0 con 0x7fb7740cfb00 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb779c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740b29a0 con 0x7fb77415b330 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb779c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb77415bd30 con 0x7fb7740b2540 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.593+0000 7fb773e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb77415b330 0x7fb7740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb77415b330 0x7fb7740b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41426/0 (socket says 172.21.15.3:41426) 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 -- 172.21.15.3:0/166285859 learned_addr learned my addr 172.21.15.3:0/166285859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb76be00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7740b2540 0x7fb7740cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 -- 172.21.15.3:0/166285859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7740b2540 msgr2=0x7fb7740cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7740b2540 0x7fb7740cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 -- 172.21.15.3:0/166285859 >> v1:172.21.15.70:6789/0 conn(0x7fb7740cfb00 legacy=0x7fb77415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.594+0000 7fb773e00640 1 -- 172.21.15.3:0/166285859 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7740b56a0 con 0x7fb77415b330 2024-09-10T07:48:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.595+0000 7fb773e00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77415b330 0x7fb7740b2000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb764009870 tx=0x7fb76402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853a7d13f0ebb7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.595+0000 7fb773400640 1 -- 172.21.15.3:0/166285859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764037070 con 0x7fb77415b330 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.595+0000 7fb773400640 1 -- 172.21.15.3:0/166285859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb764032440 con 0x7fb77415b330 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.595+0000 7fb773400640 1 -- 172.21.15.3:0/166285859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764032810 con 0x7fb77415b330 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 -- 172.21.15.3:0/166285859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77415b330 msgr2=0x7fb7740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77415b330 0x7fb7740b2000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb764009870 tx=0x7fb76402a210 comp rx=0 tx=0).stop 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 -- 172.21.15.3:0/166285859 shutdown_connections 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7740b2540 0x7fb7740cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 --2- 172.21.15.3:0/166285859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb77415b330 0x7fb7740b2000 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 -- 172.21.15.3:0/166285859 >> 172.21.15.3:0/166285859 conn(0x7fb77406bb20 msgr2=0x7fb77406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 -- 172.21.15.3:0/166285859 shutdown_connections 2024-09-10T07:48:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.596+0000 7fb779c00640 1 -- 172.21.15.3:0/166285859 wait complete. 2024-09-10T07:48:29.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.597+0000 7fb779c00640 1 Processor -- start 2024-09-10T07:48:29.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 -- start start 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7740b2540 0x7fb774150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 0x7fb77414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb77415c5d0 con 0x7fb7740cfb00 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb77415c2f0 con 0x7fb7740b2540 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.598+0000 7fb779c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7740ce1c0 con 0x7fb77415b330 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41442/0 (socket says 172.21.15.3:41442) 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 -- 172.21.15.3:0/3242957255 learned_addr learned my addr 172.21.15.3:0/3242957255 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb779200640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 0x7fb77414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb773e00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7740b2540 0x7fb774150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 -- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 msgr2=0x7fb77414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 0x7fb77414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 -- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7740b2540 msgr2=0x7fb774150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7740b2540 0x7fb774150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 -- 172.21.15.3:0/3242957255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77415ccf0 con 0x7fb7740cfb00 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb779200640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 0x7fb77414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.599+0000 7fb76be00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d830 tx=0x7fb75c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.600+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00b6c0 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.600+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c019070 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.600+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00ba80 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.600+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77415cfd0 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.600+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77415d590 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.601+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774063610 con 0x7fb7740cfb00 2024-09-10T07:48:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.602+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb75c01e460 con 0x7fb7740cfb00 2024-09-10T07:48:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.603+0000 7fb770c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 0x7fb75006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.603+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fb75c08b660 con 0x7fb7740cfb00 2024-09-10T07:48:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.603+0000 7fb773e00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 0x7fb75006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.604+0000 7fb773e00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 0x7fb75006e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb764034000 tx=0x7fb76402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.608+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb75c059ef0 con 0x7fb7740cfb00 2024-09-10T07:48:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.806+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb7740b3650 con 0x7fb7740cfb00 2024-09-10T07:48:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.806+0000 7fb770c00640 1 -- 172.21.15.3:0/3242957255 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7fb75c0600c0 con 0x7fb7740cfb00 2024-09-10T07:48:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 msgr2=0x7fb75006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 0x7fb75006e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb764034000 tx=0x7fb76402e000 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 msgr2=0x7fb77415e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d830 tx=0x7fb75c00dcf0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 shutdown_connections 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb75006be30 0x7fb75006e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb77415b330 0x7fb77414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7740cfb00 0x7fb77415e630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 --2- 172.21.15.3:0/3242957255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7740b2540 0x7fb774150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 >> 172.21.15.3:0/3242957255 conn(0x7fb77406bb20 msgr2=0x7fb774072330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 shutdown_connections 2024-09-10T07:48:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:29.809+0000 7fb779c00640 1 -- 172.21.15.3:0/3242957255 wait complete. 2024-09-10T07:48:29.824 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:48:29.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:48:29.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:48:29.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:48:29.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:48:29.889 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:48:29.925 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:48:29.925 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:48:29.925 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:29.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:48:29.966 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:48:29.966 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:48:29.966 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:30.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:48:30.004 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:48:30.004 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:48:30.004 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:30.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:48:30.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.200+0000 7f6b80a00640 1 Processor -- start 2024-09-10T07:48:30.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.200+0000 7f6b80a00640 1 -- start start 2024-09-10T07:48:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.201+0000 7f6b80a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b7c15b110 0x7f6b7c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.201+0000 7f6b80a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7c0b4250 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.201+0000 7f6b80a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7c0ccd80 con 0x7f6b7c0d08f0 2024-09-10T07:48:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.201+0000 7f6b80a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7c0b44f0 con 0x7f6b7c15b110 2024-09-10T07:48:30.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b73e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6b7c15b4f0 0x7f6b7c0d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50360/0 (socket says 172.21.15.3:50360) 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b73e00640 1 -- 172.21.15.3:0/4008686981 learned_addr learned my addr 172.21.15.3:0/4008686981 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79e00640 1 --2- 172.21.15.3:0/4008686981 >> v2:172.21.15.3:3300/0 conn(0x7f6b7c15b110 0x7f6b7c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79400640 1 -- 172.21.15.3:0/4008686981 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152816687 0 0) 0x7f6b7c0b4250 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79400640 1 -- 172.21.15.3:0/4008686981 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b60003520 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79e00640 1 -- 172.21.15.3:0/4008686981 >> v1:172.21.15.70:6789/0 conn(0x7f6b7c0d08f0 legacy=0x7f6b7c0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79e00640 1 -- 172.21.15.3:0/4008686981 >> v1:172.21.15.3:6789/0 conn(0x7f6b7c15b4f0 legacy=0x7f6b7c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.202+0000 7f6b79e00640 1 -- 172.21.15.3:0/4008686981 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c0cd060 con 0x7f6b7c15b110 2024-09-10T07:48:30.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.203+0000 7f6b79e00640 1 --2- 172.21.15.3:0/4008686981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b110 0x7f6b7c0b3d10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94d1d9be69011a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b79400640 1 -- 172.21.15.3:0/4008686981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c037070 con 0x7f6b7c15b110 2024-09-10T07:48:30.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b79400640 1 -- 172.21.15.3:0/4008686981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6c032440 con 0x7f6b7c15b110 2024-09-10T07:48:30.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b79400640 1 -- 172.21.15.3:0/4008686981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c032810 con 0x7f6b7c15b110 2024-09-10T07:48:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 -- 172.21.15.3:0/4008686981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b110 msgr2=0x7f6b7c0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4008686981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b110 0x7f6b7c0b3d10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02a230 comp rx=0 tx=0).stop 2024-09-10T07:48:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 -- 172.21.15.3:0/4008686981 shutdown_connections 2024-09-10T07:48:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4008686981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b110 0x7f6b7c0b3d10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 -- 172.21.15.3:0/4008686981 >> 172.21.15.3:0/4008686981 conn(0x7f6b7c06d000 msgr2=0x7f6b7c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 -- 172.21.15.3:0/4008686981 shutdown_connections 2024-09-10T07:48:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.204+0000 7f6b80a00640 1 -- 172.21.15.3:0/4008686981 wait complete. 2024-09-10T07:48:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.206+0000 7f6b80a00640 1 Processor -- start 2024-09-10T07:48:30.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.206+0000 7f6b80a00640 1 -- start start 2024-09-10T07:48:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 0x7f6b7c1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b7c15b110 0x7f6b7c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c073d10 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c0cd620 con 0x7f6b7c0d08f0 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b80a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c073e90 con 0x7f6b7c15b110 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b7a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b79e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 0x7f6b7c1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b79e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 0x7f6b7c1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:57596/0 (socket says 172.21.15.3:57596) 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b79e00640 1 -- 172.21.15.3:0/4121444561 learned_addr learned my addr 172.21.15.3:0/4121444561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b7a800640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41450/0 (socket says 172.21.15.3:41450) 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.207+0000 7f6b73e00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b7c15b110 0x7f6b7c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 -- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b7c15b110 msgr2=0x7f6b7c15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b7c15b110 0x7f6b7c15dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 -- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 msgr2=0x7f6b7c1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 0x7f6b7c1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 -- 172.21.15.3:0/4121444561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c1f3c60 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b7a800640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c690 tx=0x7f6b7400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74003b70 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b74003d10 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74005310 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.208+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1f3f40 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.209+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1f4500 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.210+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c0b3d10 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.210+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b740054b0 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.211+0000 7f6b72000640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 0x7f6b5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.211+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f6b7408ac90 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.212+0000 7f6b79e00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 0x7f6b5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.212+0000 7f6b79e00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 0x7f6b5806e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c02a270 tx=0x7f6b6c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.216+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b74058ed0 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.439+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7c0d0560 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.439+0000 7f6b72000640 1 -- 172.21.15.3:0/4121444561 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5258 (secure 0 0 0) 0x7f6b7405d1d0 con 0x7f6b7c15b4f0 2024-09-10T07:48:30.438 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:30.438 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-09-10T07:48:00:021413+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:59.965937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741,4607,4601]},"id":2}]} 2024-09-10T07:48:30.439 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 msgr2=0x7f6b5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 0x7f6b5806e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c02a270 tx=0x7f6b6c02e000 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 msgr2=0x7f6b7c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c690 tx=0x7f6b7400cb50 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 shutdown_connections 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6b5806be30 0x7f6b5806e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b7c15b4f0 0x7f6b7c15e270 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b7c15b110 0x7f6b7c15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 --2- 172.21.15.3:0/4121444561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0d08f0 0x7f6b7c1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 >> 172.21.15.3:0/4121444561 conn(0x7f6b7c06d000 msgr2=0x7f6b7c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 shutdown_connections 2024-09-10T07:48:30.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.442+0000 7f6b80a00640 1 -- 172.21.15.3:0/4121444561 wait complete. 2024-09-10T07:48:30.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.613+0000 7f21e0200640 1 Processor -- start 2024-09-10T07:48:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.614+0000 7f21e0200640 1 -- start start 2024-09-10T07:48:30.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21e0200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 0x7f21d80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21e0200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21d80d05f0 0x7f21d8072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21e0200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d80cfdd0 con 0x7f21d8160710 2024-09-10T07:48:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21e0200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f21d815cc90 con 0x7f21d80d05f0 2024-09-10T07:48:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21e0200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21d815cf70 con 0x7f21d8160af0 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21dce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 0x7f21d80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21dd800640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f21d8160710 0x7f21d80cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:53996/0 (socket says 172.21.15.3:53996) 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21dd800640 1 -- 172.21.15.3:0/2047525075 learned_addr learned my addr 172.21.15.3:0/2047525075 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.615+0000 7f21de200640 1 --2- 172.21.15.3:0/2047525075 >> v2:172.21.15.70:3300/0 conn(0x7f21d80d05f0 0x7f21d8072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.616+0000 7f21dce00640 1 -- 172.21.15.3:0/2047525075 >> v2:172.21.15.70:3300/0 conn(0x7f21d80d05f0 msgr2=0x7f21d8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.616+0000 7f21dce00640 1 --2- 172.21.15.3:0/2047525075 >> v2:172.21.15.70:3300/0 conn(0x7f21d80d05f0 0x7f21d8072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.616+0000 7f21dce00640 1 -- 172.21.15.3:0/2047525075 >> v1:172.21.15.70:6789/0 conn(0x7f21d8160710 legacy=0x7f21d80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.616+0000 7f21dce00640 1 -- 172.21.15.3:0/2047525075 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d815d250 con 0x7f21d8160af0 2024-09-10T07:48:30.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.616+0000 7f21d7e00640 1 -- 172.21.15.3:0/2047525075 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209627466 0 0) 0x7f21d80cfdd0 con 0x7f21d8160710 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.617+0000 7f21dce00640 1 --2- 172.21.15.3:0/2047525075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 0x7f21d80d00b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21c40097a0 tx=0x7f21c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65f54c11ff09ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.617+0000 7f21d7e00640 1 -- 172.21.15.3:0/2047525075 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4037070 con 0x7f21d8160af0 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.617+0000 7f21d7e00640 1 -- 172.21.15.3:0/2047525075 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c4032440 con 0x7f21d8160af0 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.617+0000 7f21d7e00640 1 -- 172.21.15.3:0/2047525075 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4032810 con 0x7f21d8160af0 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 -- 172.21.15.3:0/2047525075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 msgr2=0x7f21d80d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 --2- 172.21.15.3:0/2047525075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 0x7f21d80d00b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21c40097a0 tx=0x7f21c402a180 comp rx=0 tx=0).stop 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 -- 172.21.15.3:0/2047525075 shutdown_connections 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 --2- 172.21.15.3:0/2047525075 >> v2:172.21.15.70:3300/0 conn(0x7f21d80d05f0 0x7f21d8072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 --2- 172.21.15.3:0/2047525075 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160af0 0x7f21d80d00b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 -- 172.21.15.3:0/2047525075 >> 172.21.15.3:0/2047525075 conn(0x7f21d806d000 msgr2=0x7f21d806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 -- 172.21.15.3:0/2047525075 shutdown_connections 2024-09-10T07:48:30.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.618+0000 7f21e0200640 1 -- 172.21.15.3:0/2047525075 wait complete. 2024-09-10T07:48:30.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.620+0000 7f21e0200640 1 Processor -- start 2024-09-10T07:48:30.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.620+0000 7f21e0200640 1 -- start start 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21d80d05f0 0x7f21d814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160710 0x7f21d814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d815d770 con 0x7f21d8160af0 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f21d80d0a10 con 0x7f21d80d05f0 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21e0200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21d814ce60 con 0x7f21d8160710 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21de200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21de200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41462/0 (socket says 172.21.15.3:41462) 2024-09-10T07:48:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.621+0000 7f21de200640 1 -- 172.21.15.3:0/22599967 learned_addr learned my addr 172.21.15.3:0/22599967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21dce00640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160710 0x7f21d814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21dd800640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21d80d05f0 0x7f21d814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 -- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160710 msgr2=0x7f21d814df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160710 0x7f21d814df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 -- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21d80d05f0 msgr2=0x7f21d814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21d80d05f0 0x7f21d814c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 -- 172.21.15.3:0/22599967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d814d140 con 0x7f21d8160af0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21de200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f21d000efc0 tx=0x7f21d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0018020 con 0x7f21d8160af0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d0014480 con 0x7f21d8160af0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0014870 con 0x7f21d8160af0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d814d420 con 0x7f21d8160af0 2024-09-10T07:48:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.622+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d814cb70 con 0x7f21d8160af0 2024-09-10T07:48:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.623+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d80cfaf0 con 0x7f21d8160af0 2024-09-10T07:48:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.624+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21d001b020 con 0x7f21d8160af0 2024-09-10T07:48:30.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.626+0000 7f21d5600640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 0x7f21b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:30.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.626+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f21d008baa0 con 0x7f21d8160af0 2024-09-10T07:48:30.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.626+0000 7f21dd800640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 0x7f21b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:30.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.626+0000 7f21dd800640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 0x7f21b806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f60 tx=0x7f21cc007b90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:30.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.630+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f21d0022030 con 0x7f21d8160af0 2024-09-10T07:48:30.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.853+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21d814ba00 con 0x7f21d8160af0 2024-09-10T07:48:30.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.853+0000 7f21d5600640 1 -- 172.21.15.3:0/22599967 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5258 (secure 0 0 0) 0x7f21d0010040 con 0x7f21d8160af0 2024-09-10T07:48:30.853 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:30.853 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-09-10T07:48:00:021413+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:59.965937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741,4607,4601]},"id":2}]} 2024-09-10T07:48:30.853 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-09-10T07:48:30.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 msgr2=0x7f21b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 0x7f21b806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f60 tx=0x7f21cc007b90 comp rx=0 tx=0).stop 2024-09-10T07:48:30.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 msgr2=0x7f21d81527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f21d000efc0 tx=0x7f21d000c490 comp rx=0 tx=0).stop 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 shutdown_connections 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21b806be30 0x7f21b806e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8160af0 0x7f21d81527f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21d8160710 0x7f21d814df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 --2- 172.21.15.3:0/22599967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21d80d05f0 0x7f21d814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.855+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 >> 172.21.15.3:0/22599967 conn(0x7f21d806d000 msgr2=0x7f21d8151120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.856+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 shutdown_connections 2024-09-10T07:48:30.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:30.856+0000 7f21e0200640 1 -- 172.21.15.3:0/22599967 wait complete. 2024-09-10T07:48:30.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.027+0000 7f0eb7400640 1 Processor -- start 2024-09-10T07:48:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.027+0000 7f0eb7400640 1 -- start start 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.028+0000 7f0eb7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0eb015b110 0x7f0eb00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb00cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb7400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb00b4250 con 0x7f0eb015b4f0 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb7400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb00b44f0 con 0x7f0eb015b110 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb7400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb00ccd80 con 0x7f0eb00d08f0 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0eb015b110 0x7f0eb00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eafe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0eb015b4f0 0x7f0eb00d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50376/0 (socket says 172.21.15.3:50376) 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eafe00640 1 -- 172.21.15.3:0/3831073645 learned_addr learned my addr 172.21.15.3:0/3831073645 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb5400640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb00cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb4a00640 1 -- 172.21.15.3:0/3831073645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 msgr2=0x7f0eb00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb4a00640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb00cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb4a00640 1 -- 172.21.15.3:0/3831073645 >> v1:172.21.15.3:6789/0 conn(0x7f0eb015b4f0 legacy=0x7f0eb00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.029+0000 7f0eb4a00640 1 -- 172.21.15.3:0/3831073645 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb00cd060 con 0x7f0eb015b110 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.030+0000 7f0eb4a00640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b110 0x7f0eb00b3d10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00b4670 tx=0x7f0ea402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b2eba4964fff21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eaf400640 1 -- 172.21.15.3:0/3831073645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4037070 con 0x7f0eb015b110 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eaf400640 1 -- 172.21.15.3:0/3831073645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea4032440 con 0x7f0eb015b110 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eaf400640 1 -- 172.21.15.3:0/3831073645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4032810 con 0x7f0eb015b110 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 -- 172.21.15.3:0/3831073645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b110 msgr2=0x7f0eb00b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b110 0x7f0eb00b3d10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00b4670 tx=0x7f0ea402a180 comp rx=0 tx=0).stop 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 -- 172.21.15.3:0/3831073645 shutdown_connections 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb00cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 --2- 172.21.15.3:0/3831073645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b110 0x7f0eb00b3d10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 -- 172.21.15.3:0/3831073645 >> 172.21.15.3:0/3831073645 conn(0x7f0eb006d000 msgr2=0x7f0eb006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 -- 172.21.15.3:0/3831073645 shutdown_connections 2024-09-10T07:48:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.031+0000 7f0eb7400640 1 -- 172.21.15.3:0/3831073645 wait complete. 2024-09-10T07:48:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.033+0000 7f0eb7400640 1 Processor -- start 2024-09-10T07:48:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.033+0000 7f0eb7400640 1 -- start start 2024-09-10T07:48:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb01465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 0x7f0eb00aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0073d10 con 0x7f0eb015b4f0 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb00cd620 con 0x7f0eb015b110 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb7400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb0073e90 con 0x7f0eb00d08f0 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41472/0 (socket says 172.21.15.3:41472) 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb5400640 1 -- 172.21.15.3:0/1720062866 learned_addr learned my addr 172.21.15.3:0/1720062866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eb4a00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb01465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.034+0000 7f0eafe00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 0x7f0eb00aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 -- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 msgr2=0x7f0eb01465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb01465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 -- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 msgr2=0x7f0eb00aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 0x7f0eb00aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 -- 172.21.15.3:0/1720062866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb00aaee0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eafe00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 0x7f0eb00aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb5400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8007e40 tx=0x7f0ea800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8014620 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea8019070 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8014a20 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb00ab1c0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.035+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb00ab780 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.036+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb00b3d10 con 0x7f0eb015b4f0 2024-09-10T07:48:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.037+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ea8014bc0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.038+0000 7f0eacc00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 0x7f0e9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.039+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f0ea808b840 con 0x7f0eb015b4f0 2024-09-10T07:48:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.039+0000 7f0eb4a00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 0x7f0e9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.039+0000 7f0eb4a00640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 0x7f0e9006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4034000 tx=0x7f0ea402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.043+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ea805a0d0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.230+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eb015dda0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.231+0000 7f0eacc00640 1 -- 172.21.15.3:0/1720062866 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16846 (secure 0 0 0) 0x7f0ea80599d0 con 0x7f0eb015b4f0 2024-09-10T07:48:31.231 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:31.231 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:13.588540+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:47:25.477493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T07:47:26.113999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.5","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 msgr2=0x7f0e9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 0x7f0e9006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4034000 tx=0x7f0ea402e000 comp rx=0 tx=0).stop 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 msgr2=0x7f0eb00cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8007e40 tx=0x7f0ea800e4b0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 shutdown_connections 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e9006be30 0x7f0e9006e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0eb015b4f0 0x7f0eb00cda40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eb015b110 0x7f0eb00aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 --2- 172.21.15.3:0/1720062866 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eb00d08f0 0x7f0eb01465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 >> 172.21.15.3:0/1720062866 conn(0x7f0eb006d000 msgr2=0x7f0eb015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:31.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 shutdown_connections 2024-09-10T07:48:31.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.233+0000 7f0eb7400640 1 -- 172.21.15.3:0/1720062866 wait complete. 2024-09-10T07:48:31.250 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:48:31.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:31.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.406+0000 7f8f73800640 1 Processor -- start 2024-09-10T07:48:31.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.406+0000 7f8f73800640 1 -- start start 2024-09-10T07:48:31.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f73800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f73800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c073d10 0x7f8f6c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f73800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c0ccd80 con 0x7f8f6c15b110 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f73800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c0b44f0 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f73800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c0cd060 con 0x7f8f6c073d10 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f6be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f6be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41484/0 (socket says 172.21.15.3:41484) 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f6be00640 1 -- 172.21.15.3:0/1085088372 learned_addr learned my addr 172.21.15.3:0/1085088372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:31.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f71800640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c073d10 0x7f8f6c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f6be00640 1 -- 172.21.15.3:0/1085088372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c073d10 msgr2=0x7f8f6c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:31.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.408+0000 7f8f6be00640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c073d10 0x7f8f6c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.409+0000 7f8f6be00640 1 -- 172.21.15.3:0/1085088372 >> v1:172.21.15.70:6789/0 conn(0x7f8f6c15b110 legacy=0x7f8f6c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.409+0000 7f8f6be00640 1 -- 172.21.15.3:0/1085088372 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c0cd340 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.409+0000 7f8f6be00640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009870 tx=0x7f8f5802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2db34167f3b75c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f6b400640 1 -- 172.21.15.3:0/1085088372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58037070 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f6b400640 1 -- 172.21.15.3:0/1085088372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f58032440 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f6b400640 1 -- 172.21.15.3:0/1085088372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58032810 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 -- 172.21.15.3:0/1085088372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c15b4f0 msgr2=0x7f8f6c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009870 tx=0x7f8f5802a1c0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 -- 172.21.15.3:0/1085088372 shutdown_connections 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c073d10 0x7f8f6c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 --2- 172.21.15.3:0/1085088372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c15b4f0 0x7f8f6c0d01e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.410+0000 7f8f73800640 1 -- 172.21.15.3:0/1085088372 >> 172.21.15.3:0/1085088372 conn(0x7f8f6c06d000 msgr2=0x7f8f6c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.411+0000 7f8f73800640 1 -- 172.21.15.3:0/1085088372 shutdown_connections 2024-09-10T07:48:31.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.411+0000 7f8f73800640 1 -- 172.21.15.3:0/1085088372 wait complete. 2024-09-10T07:48:31.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.412+0000 7f8f73800640 1 Processor -- start 2024-09-10T07:48:31.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.412+0000 7f8f73800640 1 -- start start 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15b110 0x7f8f6c147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c15b4f0 0x7f8f6c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0cdcc0 con 0x7f8f6c073d10 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0cdb40 con 0x7f8f6c15b110 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f73800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c0cde40 con 0x7f8f6c15b4f0 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f70e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f70e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41488/0 (socket says 172.21.15.3:41488) 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f70e00640 1 -- 172.21.15.3:0/1290775632 learned_addr learned my addr 172.21.15.3:0/1290775632 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:31.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f6be00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15b110 0x7f8f6c147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.413+0000 7f8f71800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c15b4f0 0x7f8f6c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 -- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c15b4f0 msgr2=0x7f8f6c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c15b4f0 0x7f8f6c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 -- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15b110 msgr2=0x7f8f6c147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15b110 0x7f8f6c147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 -- 172.21.15.3:0/1290775632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c1f3f70 con 0x7f8f6c073d10 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f70e00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c400 tx=0x7f8f6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600052d0 con 0x7f8f6c073d10 2024-09-10T07:48:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60005490 con 0x7f8f6c073d10 2024-09-10T07:48:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60007260 con 0x7f8f6c073d10 2024-09-10T07:48:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.414+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1f4250 con 0x7f8f6c073d10 2024-09-10T07:48:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.415+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c148420 con 0x7f8f6c073d10 2024-09-10T07:48:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.416+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c0b3d10 con 0x7f8f6c073d10 2024-09-10T07:48:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.416+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f60007400 con 0x7f8f6c073d10 2024-09-10T07:48:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.417+0000 7f8f68c00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 0x7f8f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.417+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f8f6008a380 con 0x7f8f6c073d10 2024-09-10T07:48:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.417+0000 7f8f6be00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 0x7f8f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.418+0000 7f8f6be00640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 0x7f8f4c06e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f58034000 tx=0x7f8f5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.422+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f60058650 con 0x7f8f6c073d10 2024-09-10T07:48:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.647+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f6c06c5d0 con 0x7f8f6c073d10 2024-09-10T07:48:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.648+0000 7f8f68c00640 1 -- 172.21.15.3:0/1290775632 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5258 (secure 0 0 0) 0x7f8f6005c950 con 0x7f8f6c073d10 2024-09-10T07:48:31.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:31.648 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-09-10T07:48:00:021413+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:59.965937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741,4607,4601]},"id":2}]} 2024-09-10T07:48:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-09-10T07:48:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 msgr2=0x7f8f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 0x7f8f4c06e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f58034000 tx=0x7f8f5802e000 comp rx=0 tx=0).stop 2024-09-10T07:48:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 msgr2=0x7f8f6c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000c400 tx=0x7f8f6000c8c0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 shutdown_connections 2024-09-10T07:48:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f4c06be30 0x7f8f4c06e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f6c15b4f0 0x7f8f6c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15b110 0x7f8f6c147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 --2- 172.21.15.3:0/1290775632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f6c073d10 0x7f8f6c154c80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.650+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 >> 172.21.15.3:0/1290775632 conn(0x7f8f6c06d000 msgr2=0x7f8f6c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:31.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.651+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 shutdown_connections 2024-09-10T07:48:31.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.651+0000 7f8f73800640 1 -- 172.21.15.3:0/1290775632 wait complete. 2024-09-10T07:48:31.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.826+0000 7f51f5400640 1 Processor -- start 2024-09-10T07:48:31.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.826+0000 7f51f5400640 1 -- start start 2024-09-10T07:48:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51f5400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51f0160710 0x7f51f00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51f5400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 0x7f51f0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51f5400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f00cfdd0 con 0x7f51f0160af0 2024-09-10T07:48:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51f5400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f51f015cc90 con 0x7f51f0160710 2024-09-10T07:48:31.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51f5400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51f015cf70 con 0x7f51f00d07c0 2024-09-10T07:48:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51e7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f51f0160af0 0x7f51f00d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50388/0 (socket says 172.21.15.3:50388) 2024-09-10T07:48:31.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51e7e00640 1 -- 172.21.15.3:0/3746156305 learned_addr learned my addr 172.21.15.3:0/3746156305 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ee800640 1 --2- 172.21.15.3:0/3746156305 >> v2:172.21.15.3:3300/0 conn(0x7f51f0160710 0x7f51f00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ef200640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 0x7f51f0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ee800640 1 -- 172.21.15.3:0/3746156305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 msgr2=0x7f51f0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ee800640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 0x7f51f0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ee800640 1 -- 172.21.15.3:0/3746156305 >> v1:172.21.15.3:6789/0 conn(0x7f51f0160af0 legacy=0x7f51f00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ee800640 1 -- 172.21.15.3:0/3746156305 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f015d250 con 0x7f51f0160710 2024-09-10T07:48:31.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.828+0000 7f51ef200640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 0x7f51f0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.829+0000 7f51ee800640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160710 0x7f51f00cf2c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51e00097a0 tx=0x7f51e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2916d92591b9a2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51ede00640 1 -- 172.21.15.3:0/3746156305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0037070 con 0x7f51f0160710 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51ede00640 1 -- 172.21.15.3:0/3746156305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e0032440 con 0x7f51f0160710 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51ede00640 1 -- 172.21.15.3:0/3746156305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0032810 con 0x7f51f0160710 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 -- 172.21.15.3:0/3746156305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160710 msgr2=0x7f51f00cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160710 0x7f51f00cf2c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51e00097a0 tx=0x7f51e002a180 comp rx=0 tx=0).stop 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 -- 172.21.15.3:0/3746156305 shutdown_connections 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f00d07c0 0x7f51f0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 --2- 172.21.15.3:0/3746156305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160710 0x7f51f00cf2c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 -- 172.21.15.3:0/3746156305 >> 172.21.15.3:0/3746156305 conn(0x7f51f006d000 msgr2=0x7f51f006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 -- 172.21.15.3:0/3746156305 shutdown_connections 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.830+0000 7f51f5400640 1 -- 172.21.15.3:0/3746156305 wait complete. 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.832+0000 7f51f5400640 1 Processor -- start 2024-09-10T07:48:31.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.832+0000 7f51f5400640 1 -- start start 2024-09-10T07:48:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f00d07c0 0x7f51f014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f0160710 0x7f51f0152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51f015d770 con 0x7f51f0160af0 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00cf800 con 0x7f51f00d07c0 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51f5400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51f01f5650 con 0x7f51f0160710 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51ef200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51ef200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41506/0 (socket says 172.21.15.3:41506) 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51ef200640 1 -- 172.21.15.3:0/1739588194 learned_addr learned my addr 172.21.15.3:0/1739588194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51ee800640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f00d07c0 0x7f51f014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.833+0000 7f51e7e00640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f0160710 0x7f51f0152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 -- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f0160710 msgr2=0x7f51f0152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f0160710 0x7f51f0152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 -- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f00d07c0 msgr2=0x7f51f014ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f00d07c0 0x7f51f014ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 -- 172.21.15.3:0/1739588194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f01f4a40 con 0x7f51f0160af0 2024-09-10T07:48:31.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51ef200640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51e800cf20 tx=0x7f51e800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e801a070 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e800d840 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e800dc30 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f01f4d20 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.834+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f01f52e0 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.835+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f01e7480 con 0x7f51f0160af0 2024-09-10T07:48:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.836+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51e801d070 con 0x7f51f0160af0 2024-09-10T07:48:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.837+0000 7f51e6a00640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 0x7f51cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.837+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f51e808b7a0 con 0x7f51f0160af0 2024-09-10T07:48:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.837+0000 7f51ee800640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 0x7f51cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.838+0000 7f51ee800640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 0x7f51cc06e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f51e0034000 tx=0x7f51e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:31.842+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51e80599e0 con 0x7f51f0160af0 2024-09-10T07:48:32.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.065+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51f01e71a0 con 0x7f51f0160af0 2024-09-10T07:48:32.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.066+0000 7f51e6a00640 1 -- 172.21.15.3:0/1739588194 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5258 (secure 0 0 0) 0x7f51e805dce0 con 0x7f51f0160af0 2024-09-10T07:48:32.066 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:32.066 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-09-10T07:48:00:021413+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:47:27.121193+0000","modified":"2024-09-10T07:47:59.965937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4741,"mds_1":4607,"mds_2":4601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4601":{"gid":4601,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/708626559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":708626559},{"type":"v1","addr":"172.21.15.3:6837","nonce":708626559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4607":{"gid":4607,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.70:6835/2716956690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2716956690},{"type":"v1","addr":"172.21.15.70:6835","nonce":2716956690}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6836/2294972972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2294972972},{"type":"v1","addr":"172.21.15.3:6836","nonce":2294972972}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4741,"qdb_cluster":[4741,4607,4601]},"id":2}]} 2024-09-10T07:48:32.066 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-09-10T07:48:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 msgr2=0x7f51cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 0x7f51cc06e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f51e0034000 tx=0x7f51e002e000 comp rx=0 tx=0).stop 2024-09-10T07:48:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 msgr2=0x7f51f0152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51e800cf20 tx=0x7f51e800a7d0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 shutdown_connections 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51cc06be30 0x7f51cc06e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f0160af0 0x7f51f0152e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51f0160710 0x7f51f0152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 --2- 172.21.15.3:0/1739588194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51f00d07c0 0x7f51f014ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 >> 172.21.15.3:0/1739588194 conn(0x7f51f006d000 msgr2=0x7f51f01e7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 shutdown_connections 2024-09-10T07:48:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.068+0000 7f51f5400640 1 -- 172.21.15.3:0/1739588194 wait complete. 2024-09-10T07:48:32.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:32.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.240+0000 7f2691800640 1 Processor -- start 2024-09-10T07:48:32.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.240+0000 7f2691800640 1 -- start start 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2691800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 0x7f268c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2691800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f268c0d07c0 0x7f268c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2691800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268c0cfdd0 con 0x7f268c160af0 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2691800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f268c15cc90 con 0x7f268c0d07c0 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2691800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f268c15cf70 con 0x7f268c160710 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f268aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 0x7f268c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2683e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f268c160af0 0x7f268c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:54014/0 (socket says 172.21.15.3:54014) 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f268b400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f268c0d07c0 0x7f268c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.242+0000 7f2683e00640 1 -- 172.21.15.3:0/1638868715 learned_addr learned my addr 172.21.15.3:0/1638868715 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268a000640 1 -- 172.21.15.3:0/1638868715 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737462444 0 0) 0x7f268c0cfdd0 con 0x7f268c160af0 2024-09-10T07:48:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268a000640 1 -- 172.21.15.3:0/1638868715 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2670003520 con 0x7f268c160af0 2024-09-10T07:48:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268aa00640 1 -- 172.21.15.3:0/1638868715 >> v2:172.21.15.70:3300/0 conn(0x7f268c0d07c0 msgr2=0x7f268c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268aa00640 1 --2- 172.21.15.3:0/1638868715 >> v2:172.21.15.70:3300/0 conn(0x7f268c0d07c0 0x7f268c072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268aa00640 1 -- 172.21.15.3:0/1638868715 >> v1:172.21.15.70:6789/0 conn(0x7f268c160af0 legacy=0x7f268c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.243+0000 7f268aa00640 1 -- 172.21.15.3:0/1638868715 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c15d250 con 0x7f268c160710 2024-09-10T07:48:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.244+0000 7f268aa00640 1 --2- 172.21.15.3:0/1638868715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 0x7f268c0cf2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f267c0097a0 tx=0x7f267c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177d340fdbd5965f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f268a000640 1 -- 172.21.15.3:0/1638868715 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c037070 con 0x7f268c160710 2024-09-10T07:48:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f268a000640 1 -- 172.21.15.3:0/1638868715 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267c032440 con 0x7f268c160710 2024-09-10T07:48:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f268a000640 1 -- 172.21.15.3:0/1638868715 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0327f0 con 0x7f268c160710 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 -- 172.21.15.3:0/1638868715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 msgr2=0x7f268c0cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 --2- 172.21.15.3:0/1638868715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 0x7f268c0cf2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f267c0097a0 tx=0x7f267c02a180 comp rx=0 tx=0).stop 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 -- 172.21.15.3:0/1638868715 shutdown_connections 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 --2- 172.21.15.3:0/1638868715 >> v2:172.21.15.70:3300/0 conn(0x7f268c0d07c0 0x7f268c072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 --2- 172.21.15.3:0/1638868715 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160710 0x7f268c0cf2c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 -- 172.21.15.3:0/1638868715 >> 172.21.15.3:0/1638868715 conn(0x7f268c06d000 msgr2=0x7f268c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 -- 172.21.15.3:0/1638868715 shutdown_connections 2024-09-10T07:48:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.245+0000 7f2691800640 1 -- 172.21.15.3:0/1638868715 wait complete. 2024-09-10T07:48:32.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.247+0000 7f2691800640 1 Processor -- start 2024-09-10T07:48:32.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.247+0000 7f2691800640 1 -- start start 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f268c0d07c0 0x7f268c146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160af0 0x7f268c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f268c15d770 con 0x7f268c160710 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f268c0cf800 con 0x7f268c0d07c0 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2691800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f268c1f8e60 con 0x7f268c160af0 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2683e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2683e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41510/0 (socket says 172.21.15.3:41510) 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f2683e00640 1 -- 172.21.15.3:0/260439402 learned_addr learned my addr 172.21.15.3:0/260439402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f268aa00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f268c0d07c0 0x7f268c146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.248+0000 7f268b400640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160af0 0x7f268c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 -- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160af0 msgr2=0x7f268c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160af0 0x7f268c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 -- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f268c0d07c0 msgr2=0x7f268c146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f268c0d07c0 0x7f268c146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 -- 172.21.15.3:0/260439402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1f8250 con 0x7f268c160710 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2683e00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f267400c390 tx=0x7f267400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f268c160710 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674005ce0 con 0x7f268c160710 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740072f0 con 0x7f268c160710 2024-09-10T07:48:32.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.249+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1f8530 con 0x7f268c160710 2024-09-10T07:48:32.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.250+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1f8af0 con 0x7f268c160710 2024-09-10T07:48:32.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.250+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c0afd50 con 0x7f268c160710 2024-09-10T07:48:32.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.251+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f267401a070 con 0x7f268c160710 2024-09-10T07:48:32.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.252+0000 7f2682a00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 0x7f266806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.252+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f267408a2d0 con 0x7f268c160710 2024-09-10T07:48:32.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.253+0000 7f268aa00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 0x7f266806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.253+0000 7f268aa00640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 0x7f266806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.256+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2674058510 con 0x7f268c160710 2024-09-10T07:48:32.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.441+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f268c0725f0 con 0x7f268c160710 2024-09-10T07:48:32.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.443+0000 7f2682a00640 1 -- 172.21.15.3:0/260439402 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16846 (secure 0 0 0) 0x7f267405c810 con 0x7f268c160710 2024-09-10T07:48:32.443 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:32.443 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:13.588540+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:47:25.477493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T07:47:26.113999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.5","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 msgr2=0x7f266806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 0x7f266806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 msgr2=0x7f268c1e5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f267400c390 tx=0x7f267400c850 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 shutdown_connections 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f266806be30 0x7f266806e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f268c160af0 0x7f268c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f268c160710 0x7f268c1e5d70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 --2- 172.21.15.3:0/260439402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f268c0d07c0 0x7f268c146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 >> 172.21.15.3:0/260439402 conn(0x7f268c06d000 msgr2=0x7f268c06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:32.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 shutdown_connections 2024-09-10T07:48:32.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.445+0000 7f2691800640 1 -- 172.21.15.3:0/260439402 wait complete. 2024-09-10T07:48:32.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:48:32.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.621+0000 7fedffe00640 1 Processor -- start 2024-09-10T07:48:32.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.621+0000 7fedffe00640 1 -- start start 2024-09-10T07:48:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedffe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee000d3c10 0x7fee00073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedffe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee000d4050 0x7fee00073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedffe00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee000cfdc0 con 0x7fee0015e860 2024-09-10T07:48:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedffe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fee000d3780 con 0x7fee000d3c10 2024-09-10T07:48:32.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedffe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fee000cee40 con 0x7fee000d4050 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedfe000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee000d3c10 0x7fee00073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.622+0000 7fedfe000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee000d3c10 0x7fee00073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41516/0 (socket says 172.21.15.3:41516) 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedfe000640 1 -- 172.21.15.3:0/1432644665 learned_addr learned my addr 172.21.15.3:0/1432644665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedff400640 1 --2- 172.21.15.3:0/1432644665 >> v2:172.21.15.70:3300/0 conn(0x7fee000d4050 0x7fee00073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedfe000640 1 -- 172.21.15.3:0/1432644665 >> v2:172.21.15.70:3300/0 conn(0x7fee000d4050 msgr2=0x7fee00073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedfe000640 1 --2- 172.21.15.3:0/1432644665 >> v2:172.21.15.70:3300/0 conn(0x7fee000d4050 0x7fee00073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedfe000640 1 -- 172.21.15.3:0/1432644665 >> v1:172.21.15.70:6789/0 conn(0x7fee0015e860 legacy=0x7fee000cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.623+0000 7fedfe000640 1 -- 172.21.15.3:0/1432644665 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000cf120 con 0x7fee000d3c10 2024-09-10T07:48:32.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.624+0000 7fedfe000640 1 --2- 172.21.15.3:0/1432644665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee000d3c10 0x7fee00073150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fee000b4060 tx=0x7fedec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad91fb06dfa9e441 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.624+0000 7fedfd600640 1 -- 172.21.15.3:0/1432644665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec037070 con 0x7fee000d3c10 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.624+0000 7fedfd600640 1 -- 172.21.15.3:0/1432644665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedec032440 con 0x7fee000d3c10 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.624+0000 7fedfd600640 1 -- 172.21.15.3:0/1432644665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec032810 con 0x7fee000d3c10 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 -- 172.21.15.3:0/1432644665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee000d3c10 msgr2=0x7fee00073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 --2- 172.21.15.3:0/1432644665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee000d3c10 0x7fee00073150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fee000b4060 tx=0x7fedec02a270 comp rx=0 tx=0).stop 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 -- 172.21.15.3:0/1432644665 shutdown_connections 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 --2- 172.21.15.3:0/1432644665 >> v2:172.21.15.70:3300/0 conn(0x7fee000d4050 0x7fee00073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 --2- 172.21.15.3:0/1432644665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee000d3c10 0x7fee00073150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 -- 172.21.15.3:0/1432644665 >> 172.21.15.3:0/1432644665 conn(0x7fee000d4650 msgr2=0x7fee000d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 -- 172.21.15.3:0/1432644665 shutdown_connections 2024-09-10T07:48:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.625+0000 7fedffe00640 1 -- 172.21.15.3:0/1432644665 wait complete. 2024-09-10T07:48:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.627+0000 7fedffe00640 1 Processor -- start 2024-09-10T07:48:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.627+0000 7fedffe00640 1 -- start start 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee000d3c10 0x7fee000aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee000d4050 0x7fee0015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee000d2460 con 0x7fee0015e860 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fee000cf400 con 0x7fee000d4050 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedffe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fee000b7880 con 0x7fee000d3c10 2024-09-10T07:48:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41520/0 (socket says 172.21.15.3:41520) 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 -- 172.21.15.3:0/2228358454 learned_addr learned my addr 172.21.15.3:0/2228358454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 -- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee000d3c10 msgr2=0x7fee000aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee000d3c10 0x7fee000aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 -- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee000d4050 msgr2=0x7fee0015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee000d4050 0x7fee0015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.628+0000 7fedff400640 1 -- 172.21.15.3:0/2228358454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee000cd780 con 0x7fee0015e860 2024-09-10T07:48:32.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedff400640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fedf8007e70 tx=0x7fedf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8014620 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8019070 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8014a20 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee000cda60 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.629+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee000ab500 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.630+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee000693c0 con 0x7fee0015e860 2024-09-10T07:48:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.631+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fedf8014bc0 con 0x7fee0015e860 2024-09-10T07:48:32.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.632+0000 7fedf6a00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 0x7fede006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:32.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.632+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fedf808b8d0 con 0x7fee0015e860 2024-09-10T07:48:32.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.632+0000 7fedfea00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 0x7fede006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.633+0000 7fedfea00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 0x7fede006e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedf0000f60 tx=0x7fedf00079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.636+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee000693c0 con 0x7fee0015e860 2024-09-10T07:48:32.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.858+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fee000ab700 con 0x7fee0015e860 2024-09-10T07:48:32.860 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:48:32.858+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:48:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.860+0000 7fedf6a00640 1 -- 172.21.15.3:0/2228358454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fee000ab700 con 0x7fee0015e860 2024-09-10T07:48:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:48:32.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 msgr2=0x7fede006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 0x7fede006e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedf0000f60 tx=0x7fedf00079d0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 msgr2=0x7fee000ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fedf8007e70 tx=0x7fedf800e4b0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 shutdown_connections 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fede006be30 0x7fede006e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee0015e860 0x7fee000ce310 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee000d4050 0x7fee0015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 --2- 172.21.15.3:0/2228358454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee000d3c10 0x7fee000aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 >> 172.21.15.3:0/2228358454 conn(0x7fee000d4650 msgr2=0x7fee000b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 shutdown_connections 2024-09-10T07:48:32.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:32.864+0000 7fedffe00640 1 -- 172.21.15.3:0/2228358454 wait complete. 2024-09-10T07:48:32.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:48:32.922 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:48:32.926 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:48:32.939 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:48:33.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.051+0000 7fe79fe00640 1 Processor -- start 2024-09-10T07:48:33.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.052+0000 7fe79fe00640 1 -- start start 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe79806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 0x7fe7980b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7980cd900 con 0x7fe7980cf270 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79fe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe79806d7c0 con 0x7fe798145730 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe7980cdbe0 con 0x7fe7981451d0 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79de00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe7980cf270 0x7fe7980cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50408/0 (socket says 172.21.15.3:50408) 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79de00640 1 -- 172.21.15.3:0/4028963617 learned_addr learned my addr 172.21.15.3:0/4028963617 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79d400640 1 --2- 172.21.15.3:0/4028963617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe79806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.053+0000 7fe79ca00640 1 --2- 172.21.15.3:0/4028963617 >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 0x7fe7980b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe79d400640 1 -- 172.21.15.3:0/4028963617 >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 msgr2=0x7fe7980b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe79d400640 1 --2- 172.21.15.3:0/4028963617 >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 0x7fe7980b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe79d400640 1 -- 172.21.15.3:0/4028963617 >> v1:172.21.15.3:6789/0 conn(0x7fe7980cf270 legacy=0x7fe7980cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe797e00640 1 -- 172.21.15.3:0/4028963617 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731167820 0 0) 0x7fe7980cd900 con 0x7fe7980cf270 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe79d400640 1 -- 172.21.15.3:0/4028963617 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79806da10 con 0x7fe7981451d0 2024-09-10T07:48:33.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.054+0000 7fe79ca00640 1 --2- 172.21.15.3:0/4028963617 >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 0x7fe7980b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.055+0000 7fe79d400640 1 --2- 172.21.15.3:0/4028963617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe79806cfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0097a0 tx=0x7fe78c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605b2f1f037a270b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.055+0000 7fe797e00640 1 -- 172.21.15.3:0/4028963617 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c037070 con 0x7fe7981451d0 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.055+0000 7fe797e00640 1 -- 172.21.15.3:0/4028963617 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c02ae50 con 0x7fe7981451d0 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.055+0000 7fe797e00640 1 -- 172.21.15.3:0/4028963617 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c032660 con 0x7fe7981451d0 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 -- 172.21.15.3:0/4028963617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 msgr2=0x7fe79806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 --2- 172.21.15.3:0/4028963617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe79806cfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0097a0 tx=0x7fe78c02a100 comp rx=0 tx=0).stop 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 -- 172.21.15.3:0/4028963617 shutdown_connections 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 --2- 172.21.15.3:0/4028963617 >> v2:172.21.15.70:3300/0 conn(0x7fe798145730 0x7fe7980b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 --2- 172.21.15.3:0/4028963617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe79806cfc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 -- 172.21.15.3:0/4028963617 >> 172.21.15.3:0/4028963617 conn(0x7fe7980b5cb0 msgr2=0x7fe79806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 -- 172.21.15.3:0/4028963617 shutdown_connections 2024-09-10T07:48:33.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.056+0000 7fe79fe00640 1 -- 172.21.15.3:0/4028963617 wait complete. 2024-09-10T07:48:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.058+0000 7fe79fe00640 1 Processor -- start 2024-09-10T07:48:33.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.058+0000 7fe79fe00640 1 -- start start 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 0x7fe79813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe7980ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe79806df30 con 0x7fe798145730 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe7980b38b0 con 0x7fe7980cf270 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe79806e0b0 con 0x7fe7981451d0 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41546/0 (socket says 172.21.15.3:41546) 2024-09-10T07:48:33.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 -- 172.21.15.3:0/2153478167 learned_addr learned my addr 172.21.15.3:0/2153478167 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79d400640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 0x7fe79813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79ca00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe7980ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 -- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 msgr2=0x7fe7980ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe7980ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 -- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 msgr2=0x7fe79813f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 0x7fe79813f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79de00640 1 -- 172.21.15.3:0/2153478167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79813e560 con 0x7fe798145730 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.059+0000 7fe79d400640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 0x7fe79813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe79de00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c7d0 tx=0x7fe79000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790018070 con 0x7fe798145730 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79000a640 con 0x7fe798145730 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000aa70 con 0x7fe798145730 2024-09-10T07:48:33.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79813e800 con 0x7fe798145730 2024-09-10T07:48:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.060+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79813f2e0 con 0x7fe798145730 2024-09-10T07:48:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.061+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79806cfc0 con 0x7fe798145730 2024-09-10T07:48:33.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.062+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe79001b070 con 0x7fe798145730 2024-09-10T07:48:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.063+0000 7fe795600640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 0x7fe77806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.063+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6083+0+0 (secure 0 0 0) 0x7fe79008a600 con 0x7fe798145730 2024-09-10T07:48:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.064+0000 7fe79d400640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 0x7fe77806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:33.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.064+0000 7fe79d400640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 0x7fe77806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe78c034000 tx=0x7fe78c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:33.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.067+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe790058e20 con 0x7fe798145730 2024-09-10T07:48:33.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.287+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe7980b0f40 con 0x7fe798145730 2024-09-10T07:48:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.866+0000 7fe795600640 1 -- 172.21.15.3:0/2153478167 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fe790058720 con 0x7fe798145730 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 msgr2=0x7fe77806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 0x7fe77806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe78c034000 tx=0x7fe78c02e000 comp rx=0 tx=0).stop 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 msgr2=0x7fe7980aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c7d0 tx=0x7fe79000cc90 comp rx=0 tx=0).stop 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 shutdown_connections 2024-09-10T07:48:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe77806be30 0x7fe77806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe798145730 0x7fe7980aede0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7981451d0 0x7fe7980ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 --2- 172.21.15.3:0/2153478167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980cf270 0x7fe79813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 >> 172.21.15.3:0/2153478167 conn(0x7fe7980b5cb0 msgr2=0x7fe7980b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.869+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 shutdown_connections 2024-09-10T07:48:33.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:33.870+0000 7fe79fe00640 1 -- 172.21.15.3:0/2153478167 wait complete. 2024-09-10T07:48:33.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:48:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.099+0000 7f12df000640 1 Processor -- start 2024-09-10T07:48:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.099+0000 7f12df000640 1 -- start start 2024-09-10T07:48:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.100+0000 7f12df000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f12d815c530 0x7f12d80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.100+0000 7f12df000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.100+0000 7f12df000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d80cf1f0 con 0x7f12d80b24d0 2024-09-10T07:48:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.100+0000 7f12df000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f12d80d00c0 con 0x7f12d815c530 2024-09-10T07:48:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.100+0000 7f12df000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f12d80cdb80 con 0x7f12d80cd740 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.101+0000 7f12cf400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.101+0000 7f12dd000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f12d80b24d0 0x7f12d806bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:52776/0 (socket says 172.21.15.3:52776) 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.101+0000 7f12cf400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:54258/0 (socket says 172.21.15.3:54258) 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.101+0000 7f12dd000640 1 -- 172.21.15.3:0/343683999 learned_addr learned my addr 172.21.15.3:0/343683999 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.101+0000 7f12d7e00640 1 --2- 172.21.15.3:0/343683999 >> v2:172.21.15.70:3300/0 conn(0x7f12d815c530 0x7f12d80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7400640 1 -- 172.21.15.3:0/343683999 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070501205 0 0) 0x7f12d80cf1f0 con 0x7f12d80b24d0 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7400640 1 -- 172.21.15.3:0/343683999 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12bc003520 con 0x7f12d80b24d0 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7e00640 1 -- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 msgr2=0x7f12d80b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7e00640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7e00640 1 -- 172.21.15.3:0/343683999 >> v1:172.21.15.70:6789/0 conn(0x7f12d80b24d0 legacy=0x7f12d806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12d7e00640 1 -- 172.21.15.3:0/343683999 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d80b2910 con 0x7f12d815c530 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.102+0000 7f12cf400640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12d7e00640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d80cd200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097a0 tx=0x7f12c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b1e73ce2c16897 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12d7400640 1 -- 172.21.15.3:0/343683999 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8037070 con 0x7f12d815c530 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12d7400640 1 -- 172.21.15.3:0/343683999 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c8032440 con 0x7f12d815c530 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12d7400640 1 -- 172.21.15.3:0/343683999 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80327f0 con 0x7f12d815c530 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12df000640 1 -- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 msgr2=0x7f12d80cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.103+0000 7f12df000640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d80cd200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097a0 tx=0x7f12c802a180 comp rx=0 tx=0).stop 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 -- 172.21.15.3:0/343683999 shutdown_connections 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d80b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 --2- 172.21.15.3:0/343683999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d80cd200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 -- 172.21.15.3:0/343683999 >> 172.21.15.3:0/343683999 conn(0x7f12d8062cc0 msgr2=0x7f12d806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 -- 172.21.15.3:0/343683999 shutdown_connections 2024-09-10T07:48:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.104+0000 7f12df000640 1 -- 172.21.15.3:0/343683999 wait complete. 2024-09-10T07:48:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.105+0000 7f12df000640 1 Processor -- start 2024-09-10T07:48:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.105+0000 7f12df000640 1 -- start start 2024-09-10T07:48:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d8144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80ce710 con 0x7f12d80b24d0 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80ce430 con 0x7f12d815c530 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.106+0000 7f12df000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f12d80ce9f0 con 0x7f12d80cd740 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52406/0 (socket says 172.21.15.3:52406) 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 -- 172.21.15.3:0/995557624 learned_addr learned my addr 172.21.15.3:0/995557624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12cf400640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d8144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12dd000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 -- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 msgr2=0x7f12d8144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d8144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 -- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 msgr2=0x7f12d815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 -- 172.21.15.3:0/995557624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d8151ff0 con 0x7f12d80b24d0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12cf400640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d8144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12dd000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.107+0000 7f12d7e00640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f12c802a1c0 tx=0x7f12c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.108+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8037040 con 0x7f12d80b24d0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.108+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c8042070 con 0x7f12d80b24d0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.108+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c803b930 con 0x7f12d80b24d0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.108+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d81522d0 con 0x7f12d80b24d0 2024-09-10T07:48:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.108+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d8152890 con 0x7f12d80b24d0 2024-09-10T07:48:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.109+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d80cfba0 con 0x7f12d80b24d0 2024-09-10T07:48:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.110+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12c803bad0 con 0x7f12d80b24d0 2024-09-10T07:48:34.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.111+0000 7f12d4c00640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 0x7f12b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:34.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.111+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6083+0+0 (secure 0 0 0) 0x7f12c80ab890 con 0x7f12d80b24d0 2024-09-10T07:48:34.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.111+0000 7f12cf400640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 0x7f12b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:34.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.112+0000 7f12cf400640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 0x7f12b406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12d8151e00 tx=0x7f12c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.115+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12c807a1d0 con 0x7f12d80b24d0 2024-09-10T07:48:34.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.301+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f12d80cd2a0 con 0x7f12d80b24d0 2024-09-10T07:48:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.868+0000 7f12d4c00640 1 -- 172.21.15.3:0/995557624 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7f12c8079ad0 con 0x7f12d80b24d0 2024-09-10T07:48:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:48:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 msgr2=0x7f12b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 0x7f12b406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12d8151e00 tx=0x7f12c0008040 comp rx=0 tx=0).stop 2024-09-10T07:48:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 msgr2=0x7f12d814e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:34.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f12c802a1c0 tx=0x7f12c80046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 shutdown_connections 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12b406be30 0x7f12b406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12d815c530 0x7f12d815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f12d80cd740 0x7f12d8144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 --2- 172.21.15.3:0/995557624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d80b24d0 0x7f12d814e980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:34.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.872+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 >> 172.21.15.3:0/995557624 conn(0x7f12d8062cc0 msgr2=0x7f12d80d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.873+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 shutdown_connections 2024-09-10T07:48:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:34.873+0000 7f12df000640 1 -- 172.21.15.3:0/995557624 wait complete. 2024-09-10T07:48:34.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:48:35.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.062+0000 7f66c4a00640 1 Processor -- start 2024-09-10T07:48:35.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.062+0000 7f66c4a00640 1 -- start start 2024-09-10T07:48:35.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.063+0000 7f66c4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 0x7f66c0141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.063+0000 7f66c4a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 0x7f66c00d48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.063+0000 7f66c4a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c0141fb0 con 0x7f66c015e9c0 2024-09-10T07:48:35.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.063+0000 7f66c4a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f66c00d3d50 con 0x7f66c00d4490 2024-09-10T07:48:35.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.063+0000 7f66c4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66c015f960 con 0x7f66c0141310 2024-09-10T07:48:35.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bf200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66c015e9c0 0x7f66c015ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46046/0 (socket says 172.21.15.3:46046) 2024-09-10T07:48:35.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bf200640 1 -- 172.21.15.3:0/1978309361 learned_addr learned my addr 172.21.15.3:0/1978309361 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:35.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bde00640 1 --2- 172.21.15.3:0/1978309361 >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 0x7f66c00d48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66be800640 1 --2- 172.21.15.3:0/1978309361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 0x7f66c0141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bd400640 1 -- 172.21.15.3:0/1978309361 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042482343 0 0) 0x7f66c0141fb0 con 0x7f66c015e9c0 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bd400640 1 -- 172.21.15.3:0/1978309361 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a4003520 con 0x7f66c015e9c0 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66be800640 1 -- 172.21.15.3:0/1978309361 >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 msgr2=0x7f66c00d48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66be800640 1 --2- 172.21.15.3:0/1978309361 >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 0x7f66c00d48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66be800640 1 -- 172.21.15.3:0/1978309361 >> v1:172.21.15.3:6789/0 conn(0x7f66c015e9c0 legacy=0x7f66c015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66be800640 1 -- 172.21.15.3:0/1978309361 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c015f680 con 0x7f66c0141310 2024-09-10T07:48:35.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.064+0000 7f66bde00640 1 --2- 172.21.15.3:0/1978309361 >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 0x7f66c00d48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:35.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66be800640 1 --2- 172.21.15.3:0/1978309361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 0x7f66c0141a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097a0 tx=0x7f66b002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc4a2a84e14a70a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66bd400640 1 -- 172.21.15.3:0/1978309361 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0037070 con 0x7f66c0141310 2024-09-10T07:48:35.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66bd400640 1 -- 172.21.15.3:0/1978309361 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b002ae50 con 0x7f66c0141310 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66bd400640 1 -- 172.21.15.3:0/1978309361 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0032640 con 0x7f66c0141310 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 -- 172.21.15.3:0/1978309361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 msgr2=0x7f66c0141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1978309361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 0x7f66c0141a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66b00097a0 tx=0x7f66b002a100 comp rx=0 tx=0).stop 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 -- 172.21.15.3:0/1978309361 shutdown_connections 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1978309361 >> v2:172.21.15.70:3300/0 conn(0x7f66c00d4490 0x7f66c00d48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1978309361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c0141310 0x7f66c0141a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.066+0000 7f66c4a00640 1 -- 172.21.15.3:0/1978309361 >> 172.21.15.3:0/1978309361 conn(0x7f66c0062a50 msgr2=0x7f66c0072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.067+0000 7f66c4a00640 1 -- 172.21.15.3:0/1978309361 shutdown_connections 2024-09-10T07:48:35.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.067+0000 7f66c4a00640 1 -- 172.21.15.3:0/1978309361 wait complete. 2024-09-10T07:48:35.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.068+0000 7f66c4a00640 1 Processor -- start 2024-09-10T07:48:35.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.068+0000 7f66c4a00640 1 -- start start 2024-09-10T07:48:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c00d4490 0x7f66c00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c0141310 0x7f66c01f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00b5c50 con 0x7f66c015e9c0 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00d4e10 con 0x7f66c0141310 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.069+0000 7f66c4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00b5f30 con 0x7f66c00d4490 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bde00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c0141310 0x7f66c01f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52408/0 (socket says 172.21.15.3:52408) 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 -- 172.21.15.3:0/1162606405 learned_addr learned my addr 172.21.15.3:0/1162606405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66be800640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c00d4490 0x7f66c00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 -- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c00d4490 msgr2=0x7f66c00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c00d4490 0x7f66c00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 -- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c0141310 msgr2=0x7f66c01f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c0141310 0x7f66c01f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 -- 172.21.15.3:0/1162606405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c01592b0 con 0x7f66c015e9c0 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.070+0000 7f66bf200640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f66b800caa0 tx=0x7f66b800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.071+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8017070 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.071+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b800ac00 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.071+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c01f5600 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.071+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8012400 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.071+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c0159b10 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.072+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c0159660 con 0x7f66c015e9c0 2024-09-10T07:48:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.073+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66b801a070 con 0x7f66c015e9c0 2024-09-10T07:48:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.074+0000 7f66b6a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 0x7f66a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.074+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5645+0+0 (secure 0 0 0) 0x7f66b808ab60 con 0x7f66c015e9c0 2024-09-10T07:48:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.074+0000 7f66be800640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 0x7f66a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.075+0000 7f66be800640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 0x7f66a006e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66b0034000 tx=0x7f66b002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.078+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66b8058ef0 con 0x7f66c015e9c0 2024-09-10T07:48:35.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.262+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66c01416f0 con 0x7f66c015e9c0 2024-09-10T07:48:35.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.873+0000 7f66b6a00640 1 -- 172.21.15.3:0/1162606405 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7f66b805d1f0 con 0x7f66c015e9c0 2024-09-10T07:48:35.873 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 msgr2=0x7f66a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 0x7f66a006e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66b0034000 tx=0x7f66b002e000 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 msgr2=0x7f66c015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f66b800caa0 tx=0x7f66b800cf60 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 shutdown_connections 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66a006be30 0x7f66a006e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66c015e9c0 0x7f66c015a310 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c0141310 0x7f66c01f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 --2- 172.21.15.3:0/1162606405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66c00d4490 0x7f66c00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:35.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 >> 172.21.15.3:0/1162606405 conn(0x7f66c0062a50 msgr2=0x7f66c00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:35.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 shutdown_connections 2024-09-10T07:48:35.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:35.876+0000 7f66c4a00640 1 -- 172.21.15.3:0/1162606405 wait complete. 2024-09-10T07:48:35.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:48:36.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.062+0000 7f6cb4000640 1 Processor -- start 2024-09-10T07:48:36.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.063+0000 7f6cb4000640 1 -- start start 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb4000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 0x7f6cac06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb4000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cac0cd900 con 0x7f6cac1451d0 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb4000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cac0cdbe0 con 0x7f6cac145730 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb4000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac06d7c0 con 0x7f6cac0cf980 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb1600640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6cac1451d0 0x7f6cac0b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46048/0 (socket says 172.21.15.3:46048) 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb1600640 1 -- 172.21.15.3:0/2964610626 learned_addr learned my addr 172.21.15.3:0/2964610626 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:36.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cb2000640 1 --2- 172.21.15.3:0/2964610626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 0x7f6cac06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.064+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892491077 0 0) 0x7f6cac0cd900 con 0x7f6cac1451d0 2024-09-10T07:48:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c94003520 con 0x7f6cac1451d0 2024-09-10T07:48:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414109961 0 0) 0x7f6cac0cdbe0 con 0x7f6cac145730 2024-09-10T07:48:36.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cac0cd900 con 0x7f6cac145730 2024-09-10T07:48:36.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cb2000640 1 -- 172.21.15.3:0/2964610626 >> v1:172.21.15.70:6789/0 conn(0x7f6cac145730 legacy=0x7f6cac0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3763812907 0 0) 0x7f6c94003520 con 0x7f6cac1451d0 2024-09-10T07:48:36.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cb2000640 1 -- 172.21.15.3:0/2964610626 >> v1:172.21.15.3:6789/0 conn(0x7f6cac1451d0 legacy=0x7f6cac0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.065+0000 7f6cb2000640 1 -- 172.21.15.3:0/2964610626 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac06da10 con 0x7f6cac0cf980 2024-09-10T07:48:36.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.066+0000 7f6cb2000640 1 --2- 172.21.15.3:0/2964610626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 0x7f6cac06cfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009dc0 tx=0x7f6ca402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9258915c1ffe7a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.066+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0001690 con 0x7f6cac0cf980 2024-09-10T07:48:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.066+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca4007540 con 0x7f6cac0cf980 2024-09-10T07:48:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.066+0000 7f6cabe00640 1 -- 172.21.15.3:0/2964610626 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40078f0 con 0x7f6cac0cf980 2024-09-10T07:48:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 -- 172.21.15.3:0/2964610626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 msgr2=0x7f6cac06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 --2- 172.21.15.3:0/2964610626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 0x7f6cac06cfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009dc0 tx=0x7f6ca402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 -- 172.21.15.3:0/2964610626 shutdown_connections 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 --2- 172.21.15.3:0/2964610626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac0cf980 0x7f6cac06cfc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 -- 172.21.15.3:0/2964610626 >> 172.21.15.3:0/2964610626 conn(0x7f6cac0b5cb0 msgr2=0x7f6cac06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 -- 172.21.15.3:0/2964610626 shutdown_connections 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.067+0000 7f6cb4000640 1 -- 172.21.15.3:0/2964610626 wait complete. 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.068+0000 7f6cb4000640 1 Processor -- start 2024-09-10T07:48:36.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 -- start start 2024-09-10T07:48:36.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 0x7f6cac15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac145730 0x7f6cac0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac06e0b0 con 0x7f6cac1451d0 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac06df30 con 0x7f6cac0cf980 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.069+0000 7f6cb4000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac06e230 con 0x7f6cac145730 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52410/0 (socket says 172.21.15.3:52410) 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 -- 172.21.15.3:0/1046707057 learned_addr learned my addr 172.21.15.3:0/1046707057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb1600640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 0x7f6cac15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb2000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac145730 0x7f6cac0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 -- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac145730 msgr2=0x7f6cac0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac145730 0x7f6cac0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 -- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 msgr2=0x7f6cac15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 0x7f6cac15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 -- 172.21.15.3:0/1046707057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac072c50 con 0x7f6cac1451d0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb1600640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 0x7f6cac15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.070+0000 7f6cb0c00640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800c430 tx=0x7f6c9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.071+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98017070 con 0x7f6cac1451d0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.071+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac072970 con 0x7f6cac1451d0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.071+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c980052f0 con 0x7f6cac1451d0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.071+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98003830 con 0x7f6cac1451d0 2024-09-10T07:48:36.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.071+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac15cc20 con 0x7f6cac1451d0 2024-09-10T07:48:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.072+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac06cfc0 con 0x7f6cac1451d0 2024-09-10T07:48:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.073+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c9801a070 con 0x7f6cac1451d0 2024-09-10T07:48:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.073+0000 7f6ca9600640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 0x7f6c8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.074+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f6c9808a100 con 0x7f6cac1451d0 2024-09-10T07:48:36.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.074+0000 7f6cb1600640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 0x7f6c8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.074+0000 7f6cb1600640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 0x7f6c8c06e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0001eb0 tx=0x7f6ca00015b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.078+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c98058c70 con 0x7f6cac1451d0 2024-09-10T07:48:36.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.260+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6cac15fc00 con 0x7f6cac1451d0 2024-09-10T07:48:36.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.264+0000 7f6ca9600640 1 -- 172.21.15.3:0/1046707057 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f6c98058570 con 0x7f6cac1451d0 2024-09-10T07:48:36.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 msgr2=0x7f6c8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 0x7f6c8c06e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0001eb0 tx=0x7f6ca00015b0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 msgr2=0x7f6cac071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800c430 tx=0x7f6c9800c8f0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 shutdown_connections 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c8c06be60 0x7f6c8c06e320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6cac145730 0x7f6cac0b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cac1451d0 0x7f6cac071ef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 --2- 172.21.15.3:0/1046707057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cac0cf980 0x7f6cac15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.266+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 >> 172.21.15.3:0/1046707057 conn(0x7f6cac0b5cb0 msgr2=0x7f6cac0cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.267+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 shutdown_connections 2024-09-10T07:48:36.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.267+0000 7f6cb4000640 1 -- 172.21.15.3:0/1046707057 wait complete. 2024-09-10T07:48:36.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-10T07:48:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.439+0000 7fc359000640 1 Processor -- start 2024-09-10T07:48:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.440+0000 7fc359000640 1 -- start start 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc359000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 0x7fc3540cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc359000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc354160af0 0x7fc3540d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc359000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3540cfdd0 con 0x7fc3540cf800 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc359000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc35415cc90 con 0x7fc354160af0 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc359000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc35415cf70 con 0x7fc354160710 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc34be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc354160af0 0x7fc3540d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc352800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc3540cf800 0x7fc35406bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46052/0 (socket says 172.21.15.3:46052) 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc352800640 1 -- 172.21.15.3:0/1251148878 learned_addr learned my addr 172.21.15.3:0/1251148878 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:36.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.441+0000 7fc351e00640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 0x7fc3540cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.442+0000 7fc34be00640 1 -- 172.21.15.3:0/1251148878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 msgr2=0x7fc3540cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.442+0000 7fc34be00640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 0x7fc3540cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.442+0000 7fc34be00640 1 -- 172.21.15.3:0/1251148878 >> v1:172.21.15.3:6789/0 conn(0x7fc3540cf800 legacy=0x7fc35406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.442+0000 7fc34be00640 1 -- 172.21.15.3:0/1251148878 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35415d250 con 0x7fc354160af0 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.442+0000 7fc351e00640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 0x7fc3540cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc34be00640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc354160af0 0x7fc3540d00b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0097a0 tx=0x7fc33c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2dc83f7eb4bd5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc351400640 1 -- 172.21.15.3:0/1251148878 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c037070 con 0x7fc354160af0 2024-09-10T07:48:36.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc351400640 1 -- 172.21.15.3:0/1251148878 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c032440 con 0x7fc354160af0 2024-09-10T07:48:36.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc351400640 1 -- 172.21.15.3:0/1251148878 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c032810 con 0x7fc354160af0 2024-09-10T07:48:36.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 -- 172.21.15.3:0/1251148878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc354160af0 msgr2=0x7fc3540d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc354160af0 0x7fc3540d00b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0097a0 tx=0x7fc33c02a180 comp rx=0 tx=0).stop 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 -- 172.21.15.3:0/1251148878 shutdown_connections 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc354160af0 0x7fc3540d00b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 --2- 172.21.15.3:0/1251148878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160710 0x7fc3540cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.443+0000 7fc359000640 1 -- 172.21.15.3:0/1251148878 >> 172.21.15.3:0/1251148878 conn(0x7fc35406d000 msgr2=0x7fc35406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.444+0000 7fc359000640 1 -- 172.21.15.3:0/1251148878 shutdown_connections 2024-09-10T07:48:36.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.444+0000 7fc359000640 1 -- 172.21.15.3:0/1251148878 wait complete. 2024-09-10T07:48:36.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.445+0000 7fc359000640 1 Processor -- start 2024-09-10T07:48:36.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.445+0000 7fc359000640 1 -- start start 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 0x7fc354151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160af0 0x7fc3540aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc35415d770 con 0x7fc3540cf800 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc354073080 con 0x7fc354160710 2024-09-10T07:48:36.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc359000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc35414fcd0 con 0x7fc354160af0 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.446+0000 7fc351e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52430/0 (socket says 172.21.15.3:52430) 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 -- 172.21.15.3:0/3358522004 learned_addr learned my addr 172.21.15.3:0/3358522004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc34be00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 0x7fc354151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc352800640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160af0 0x7fc3540aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 -- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160af0 msgr2=0x7fc3540aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160af0 0x7fc3540aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 -- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 msgr2=0x7fc354151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 0x7fc354151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 -- 172.21.15.3:0/3358522004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35414e710 con 0x7fc3540cf800 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc34be00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 0x7fc354151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc351e00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e760 tx=0x7fc34400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34400f040 con 0x7fc3540cf800 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc344004230 con 0x7fc3540cf800 2024-09-10T07:48:36.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34400ee70 con 0x7fc3540cf800 2024-09-10T07:48:36.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.447+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35414ff00 con 0x7fc3540cf800 2024-09-10T07:48:36.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.448+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3541504c0 con 0x7fc3540cf800 2024-09-10T07:48:36.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.449+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3540afd50 con 0x7fc3540cf800 2024-09-10T07:48:36.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.450+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc34400c6d0 con 0x7fc3540cf800 2024-09-10T07:48:36.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.450+0000 7fc34a000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:36.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.450+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7fc34408a990 con 0x7fc3540cf800 2024-09-10T07:48:36.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.451+0000 7fc34be00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:36.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.451+0000 7fc34be00640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc33c034000 tx=0x7fc33c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:36.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.455+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc35414e710 con 0x7fc3540cf800 2024-09-10T07:48:36.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.708+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7fc35406c6c0 con 0x7fc3540cf800 2024-09-10T07:48:36.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.876+0000 7fc34a000640 1 -- 172.21.15.3:0/3358522004 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v126) ==== 136+0+0 (secure 0 0 0) 0x7fc344017020 con 0x7fc3540cf800 2024-09-10T07:48:36.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.878+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 msgr2=0x7fc33006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.878+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc33c034000 tx=0x7fc33c02e000 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.878+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 msgr2=0x7fc35414e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.878+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e760 tx=0x7fc34400ec20 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 shutdown_connections 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc354160af0 0x7fc3540aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc354160710 0x7fc354151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 --2- 172.21.15.3:0/3358522004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3540cf800 0x7fc35414e1d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 >> 172.21.15.3:0/3358522004 conn(0x7fc35406d000 msgr2=0x7fc3540b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 shutdown_connections 2024-09-10T07:48:36.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:36.879+0000 7fc359000640 1 -- 172.21.15.3:0/3358522004 wait complete. 2024-09-10T07:48:36.896 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:48:36.896 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:48:36.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:37.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.052+0000 7f6976600640 1 Processor -- start 2024-09-10T07:48:37.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.053+0000 7f6976600640 1 -- start start 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6976600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f697015b4f0 0x7f69700d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6976600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6970073d10 0x7f69700d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6976600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69700ccd80 con 0x7f697015b110 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6976600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f69700b44f0 con 0x7f697015b4f0 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6976600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f69700cd060 con 0x7f6970073d10 2024-09-10T07:48:37.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f697015b4f0 0x7f69700d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f697015b4f0 0x7f69700d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52438/0 (socket says 172.21.15.3:52438) 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 -- 172.21.15.3:0/712861676 learned_addr learned my addr 172.21.15.3:0/712861676 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f696fe00640 1 --2- 172.21.15.3:0/712861676 >> v2:172.21.15.70:3300/0 conn(0x7f6970073d10 0x7f69700d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 -- 172.21.15.3:0/712861676 >> v2:172.21.15.70:3300/0 conn(0x7f6970073d10 msgr2=0x7f69700d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 --2- 172.21.15.3:0/712861676 >> v2:172.21.15.70:3300/0 conn(0x7f6970073d10 0x7f69700d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.054+0000 7f6967e00640 1 -- 172.21.15.3:0/712861676 >> v1:172.21.15.70:6789/0 conn(0x7f697015b110 legacy=0x7f69700b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.055+0000 7f6967e00640 1 -- 172.21.15.3:0/712861676 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69700cd340 con 0x7f697015b4f0 2024-09-10T07:48:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6967e00640 1 --2- 172.21.15.3:0/712861676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f697015b4f0 0x7f69700d01e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f69700aff40 tx=0x7f695802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323ac94d9a018fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f696ea00640 1 -- 172.21.15.3:0/712861676 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958037070 con 0x7f697015b4f0 2024-09-10T07:48:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f696ea00640 1 -- 172.21.15.3:0/712861676 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958032440 con 0x7f697015b4f0 2024-09-10T07:48:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f696ea00640 1 -- 172.21.15.3:0/712861676 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958032810 con 0x7f697015b4f0 2024-09-10T07:48:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 -- 172.21.15.3:0/712861676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f697015b4f0 msgr2=0x7f69700d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 --2- 172.21.15.3:0/712861676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f697015b4f0 0x7f69700d01e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f69700aff40 tx=0x7f695802a270 comp rx=0 tx=0).stop 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 -- 172.21.15.3:0/712861676 shutdown_connections 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 --2- 172.21.15.3:0/712861676 >> v2:172.21.15.70:3300/0 conn(0x7f6970073d10 0x7f69700d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 --2- 172.21.15.3:0/712861676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f697015b4f0 0x7f69700d01e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 -- 172.21.15.3:0/712861676 >> 172.21.15.3:0/712861676 conn(0x7f697006d000 msgr2=0x7f697006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.056+0000 7f6976600640 1 -- 172.21.15.3:0/712861676 shutdown_connections 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.057+0000 7f6976600640 1 -- 172.21.15.3:0/712861676 wait complete. 2024-09-10T07:48:37.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.058+0000 7f6976600640 1 Processor -- start 2024-09-10T07:48:37.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.058+0000 7f6976600640 1 -- start start 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 0x7f697015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f697015b4f0 0x7f6970152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f697006bca0 con 0x7f6970073d10 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f697006bb20 con 0x7f697015b110 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6976600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f697006be20 con 0x7f697015b4f0 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f696f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f696f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52444/0 (socket says 172.21.15.3:52444) 2024-09-10T07:48:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f696f400640 1 -- 172.21.15.3:0/688885988 learned_addr learned my addr 172.21.15.3:0/688885988 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f6967e00640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 0x7f697015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.059+0000 7f696fe00640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f697015b4f0 0x7f6970152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 -- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f697015b4f0 msgr2=0x7f6970152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f697015b4f0 0x7f6970152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 -- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 msgr2=0x7f697015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 0x7f697015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 -- 172.21.15.3:0/688885988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697014c3d0 con 0x7f6970073d10 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f6967e00640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 0x7f697015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f696f400640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f696000c430 tx=0x7f696000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69600052d0 con 0x7f6970073d10 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960005490 con 0x7f6970073d10 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960007260 con 0x7f6970073d10 2024-09-10T07:48:37.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.060+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697015c530 con 0x7f6970073d10 2024-09-10T07:48:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.061+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697015caf0 con 0x7f6970073d10 2024-09-10T07:48:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.061+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69700b3d10 con 0x7f6970073d10 2024-09-10T07:48:37.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.062+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6960007400 con 0x7f6970073d10 2024-09-10T07:48:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.064+0000 7f6967400640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.064+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f696008a120 con 0x7f6970073d10 2024-09-10T07:48:37.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.064+0000 7f6967e00640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.065+0000 7f6967e00640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 0x7f694c06e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6958034000 tx=0x7f695802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.068+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f697014c3d0 con 0x7f6970073d10 2024-09-10T07:48:37.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.257+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69701563a0 con 0x7f6970073d10 2024-09-10T07:48:37.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.257+0000 7f6967400640 1 -- 172.21.15.3:0/688885988 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13214 (secure 0 0 0) 0x7f6960058630 con 0x7f6970073d10 2024-09-10T07:48:37.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:37.257 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:35.869397+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6837/708626559":"2024-09-11T07:48:32.859530+0000","172.21.15.3:6835/708626559":"2024-09-11T07:48:32.859530+0000","172.21.15.3:6836/2294972972":"2024-09-11T07:48:32.859546+0000","172.21.15.3:6834/2294972972":"2024-09-11T07:48:32.859546+0000","172.21.15.70:6835/2716956690":"2024-09-11T07:48:32.859541+0000","172.21.15.70:6834/2716956690":"2024-09-11T07:48:32.859541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:37.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 msgr2=0x7f694c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 0x7f694c06e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6958034000 tx=0x7f695802e000 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 msgr2=0x7f697014be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f696000c430 tx=0x7f696000c8f0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 shutdown_connections 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f694c06be60 0x7f694c06e320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f697015b4f0 0x7f6970152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f697015b110 0x7f697015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 --2- 172.21.15.3:0/688885988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6970073d10 0x7f697014be90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 >> 172.21.15.3:0/688885988 conn(0x7f697006d000 msgr2=0x7f69700b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 shutdown_connections 2024-09-10T07:48:37.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.260+0000 7f6976600640 1 -- 172.21.15.3:0/688885988 wait complete. 2024-09-10T07:48:37.276 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:48:37.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:48:37.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.436+0000 7f618e400640 1 Processor -- start 2024-09-10T07:48:37.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.436+0000 7f618e400640 1 -- start start 2024-09-10T07:48:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618e400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61880d45a0 0x7f61880d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618e400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880cfe90 0x7f61880d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618e400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61880d4f00 con 0x7f6188161b70 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618e400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6188161f50 con 0x7f61880d45a0 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618e400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f61880d3de0 con 0x7f61880cfe90 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618d000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6188161b70 0x7f61880d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46062/0 (socket says 172.21.15.3:46062) 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f618d000640 1 -- 172.21.15.3:0/1918782144 learned_addr learned my addr 172.21.15.3:0/1918782144 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.437+0000 7f6187e00640 1 --2- 172.21.15.3:0/1918782144 >> v2:172.21.15.3:3300/0 conn(0x7f61880d45a0 0x7f61880d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.438+0000 7f618da00640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880cfe90 0x7f61880d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.438+0000 7f6187e00640 1 -- 172.21.15.3:0/1918782144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880cfe90 msgr2=0x7f61880d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.438+0000 7f6187e00640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880cfe90 0x7f61880d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.438+0000 7f6187e00640 1 -- 172.21.15.3:0/1918782144 >> v1:172.21.15.3:6789/0 conn(0x7f6188161b70 legacy=0x7f61880d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.438+0000 7f6187e00640 1 -- 172.21.15.3:0/1918782144 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880d40c0 con 0x7f61880d45a0 2024-09-10T07:48:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.439+0000 7f6187e00640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d45a0 0x7f61880d49c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f617402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6314e79c569174f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.439+0000 7f6187400640 1 -- 172.21.15.3:0/1918782144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174037070 con 0x7f61880d45a0 2024-09-10T07:48:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.439+0000 7f6187400640 1 -- 172.21.15.3:0/1918782144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174032440 con 0x7f61880d45a0 2024-09-10T07:48:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.439+0000 7f6187400640 1 -- 172.21.15.3:0/1918782144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174032810 con 0x7f61880d45a0 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 -- 172.21.15.3:0/1918782144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d45a0 msgr2=0x7f61880d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d45a0 0x7f61880d49c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f61740097a0 tx=0x7f617402a180 comp rx=0 tx=0).stop 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 -- 172.21.15.3:0/1918782144 shutdown_connections 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880cfe90 0x7f61880d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 --2- 172.21.15.3:0/1918782144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61880d45a0 0x7f61880d49c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 -- 172.21.15.3:0/1918782144 >> 172.21.15.3:0/1918782144 conn(0x7f6188062630 msgr2=0x7f6188062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 -- 172.21.15.3:0/1918782144 shutdown_connections 2024-09-10T07:48:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.440+0000 7f618e400640 1 -- 172.21.15.3:0/1918782144 wait complete. 2024-09-10T07:48:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.442+0000 7f618e400640 1 Processor -- start 2024-09-10T07:48:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.442+0000 7f618e400640 1 -- start start 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61880cfe90 0x7f61880b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880d45a0 0x7f61880b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f61880c9fc0 con 0x7f6188161b70 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f61880cf970 con 0x7f61880cfe90 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618e400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f61880d3a30 con 0x7f61880d45a0 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52474/0 (socket says 172.21.15.3:52474) 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 -- 172.21.15.3:0/3876458839 learned_addr learned my addr 172.21.15.3:0/3876458839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:37.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f6187e00640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880d45a0 0x7f61880b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618d000640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61880cfe90 0x7f61880b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 -- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880d45a0 msgr2=0x7f61880b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880d45a0 0x7f61880b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 -- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61880cfe90 msgr2=0x7f61880b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61880cfe90 0x7f61880b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.443+0000 7f618da00640 1 -- 172.21.15.3:0/3876458839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6188146550 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f618da00640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6180007e40 tx=0x7f618000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180014620 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6180019070 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180014a20 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6188146830 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.444+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61880cbaf0 con 0x7f6188161b70 2024-09-10T07:48:37.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.445+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61880cf2d0 con 0x7f6188161b70 2024-09-10T07:48:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.446+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6180014bc0 con 0x7f6188161b70 2024-09-10T07:48:37.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.447+0000 7f6184c00640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.447+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f618008b530 con 0x7f6188161b70 2024-09-10T07:48:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.447+0000 7f618d000640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.448+0000 7f618d000640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 0x7f616806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f617c000f60 tx=0x7f617c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.451+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f618005a120 con 0x7f6188161b70 2024-09-10T07:48:37.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.640+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f618815e670 con 0x7f6188161b70 2024-09-10T07:48:37.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.640+0000 7f6184c00640 1 -- 172.21.15.3:0/3876458839 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f6180059a20 con 0x7f6188161b70 2024-09-10T07:48:37.640 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:48:37.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 msgr2=0x7f616806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 0x7f616806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f617c000f60 tx=0x7f617c009040 comp rx=0 tx=0).stop 2024-09-10T07:48:37.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 msgr2=0x7f6188145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6180007e40 tx=0x7f618000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 shutdown_connections 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f616806be30 0x7f616806e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6188161b70 0x7f6188145970 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61880d45a0 0x7f61880b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 --2- 172.21.15.3:0/3876458839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61880cfe90 0x7f61880b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.642+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 >> 172.21.15.3:0/3876458839 conn(0x7f6188062630 msgr2=0x7f6188062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.643+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 shutdown_connections 2024-09-10T07:48:37.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.643+0000 7f618e400640 1 -- 172.21.15.3:0/3876458839 wait complete. 2024-09-10T07:48:37.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-10T07:48:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.815+0000 7f9c51800640 1 Processor -- start 2024-09-10T07:48:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.815+0000 7f9c51800640 1 -- start start 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c51800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c4c15e860 0x7f9c4c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c51800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c4c073150 0x7f9c4c073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c51800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4c0cee40 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c51800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c0d3780 con 0x7f9c4c15e860 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c51800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c0cf120 con 0x7f9c4c073150 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.816+0000 7f9c4be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c4c15e860 0x7f9c4c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c4c0cfbf0 0x7f9c4c0d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46066/0 (socket says 172.21.15.3:46066) 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4b400640 1 -- 172.21.15.3:0/3968364306 learned_addr learned my addr 172.21.15.3:0/3968364306 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c50e00640 1 --2- 172.21.15.3:0/3968364306 >> v2:172.21.15.70:3300/0 conn(0x7f9c4c073150 0x7f9c4c073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4be00640 1 -- 172.21.15.3:0/3968364306 >> v2:172.21.15.70:3300/0 conn(0x7f9c4c073150 msgr2=0x7f9c4c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4be00640 1 --2- 172.21.15.3:0/3968364306 >> v2:172.21.15.70:3300/0 conn(0x7f9c4c073150 0x7f9c4c073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4be00640 1 -- 172.21.15.3:0/3968364306 >> v1:172.21.15.3:6789/0 conn(0x7f9c4c0cfbf0 legacy=0x7f9c4c0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4be00640 1 -- 172.21.15.3:0/3968364306 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c0d3410 con 0x7f9c4c15e860 2024-09-10T07:48:37.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.817+0000 7f9c4aa00640 1 -- 172.21.15.3:0/3968364306 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749154053 0 0) 0x7f9c4c0cee40 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.818+0000 7f9c4be00640 1 --2- 172.21.15.3:0/3968364306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c15e860 0x7f9c4c0cf6b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0b4060 tx=0x7f9c3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e73d3f4731fdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.818+0000 7f9c4aa00640 1 -- 172.21.15.3:0/3968364306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c037070 con 0x7f9c4c15e860 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.818+0000 7f9c4aa00640 1 -- 172.21.15.3:0/3968364306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c032440 con 0x7f9c4c15e860 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.818+0000 7f9c4aa00640 1 -- 172.21.15.3:0/3968364306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c032810 con 0x7f9c4c15e860 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 -- 172.21.15.3:0/3968364306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c15e860 msgr2=0x7f9c4c0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 --2- 172.21.15.3:0/3968364306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c15e860 0x7f9c4c0cf6b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0b4060 tx=0x7f9c3c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 -- 172.21.15.3:0/3968364306 shutdown_connections 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 --2- 172.21.15.3:0/3968364306 >> v2:172.21.15.70:3300/0 conn(0x7f9c4c073150 0x7f9c4c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 --2- 172.21.15.3:0/3968364306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c15e860 0x7f9c4c0cf6b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 -- 172.21.15.3:0/3968364306 >> 172.21.15.3:0/3968364306 conn(0x7f9c4c0d4650 msgr2=0x7f9c4c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 -- 172.21.15.3:0/3968364306 shutdown_connections 2024-09-10T07:48:37.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.819+0000 7f9c51800640 1 -- 172.21.15.3:0/3968364306 wait complete. 2024-09-10T07:48:37.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.821+0000 7f9c51800640 1 Processor -- start 2024-09-10T07:48:37.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.821+0000 7f9c51800640 1 -- start start 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c073150 0x7f9c4c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 0x7f9c4c1612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0d2460 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0cf400 con 0x7f9c4c073150 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c51800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4c0b7880 con 0x7f9c4c15e860 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52486/0 (socket says 172.21.15.3:52486) 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 -- 172.21.15.3:0/3625638732 learned_addr learned my addr 172.21.15.3:0/3625638732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4be00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c073150 0x7f9c4c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c50e00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 0x7f9c4c1612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 -- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 msgr2=0x7f9c4c1612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 0x7f9c4c1612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 -- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c073150 msgr2=0x7f9c4c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c073150 0x7f9c4c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c4b400640 1 -- 172.21.15.3:0/3625638732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c15c480 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.822+0000 7f9c50e00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 0x7f9c4c1612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c4b400640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c4d0 tx=0x7f9c3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38018070 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c38007500 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38004a20 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c15f820 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.823+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c15fde0 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.824+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c0cf6b0 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.825+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c3801b070 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.826+0000 7f9c43e00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.826+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f9c38089f40 con 0x7f9c4c0cfbf0 2024-09-10T07:48:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.826+0000 7f9c4be00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.827+0000 7f9c4be00640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 0x7f9c2c06e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c034000 tx=0x7f9c3c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:37.831+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c38058b30 con 0x7f9c4c0cfbf0 2024-09-10T07:48:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.082+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7f9c4c160540 con 0x7f9c4c0cfbf0 2024-09-10T07:48:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.885+0000 7f9c43e00640 1 -- 172.21.15.3:0/3625638732 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v128) ==== 146+0+0 (secure 0 0 0) 0x7f9c38058430 con 0x7f9c4c0cfbf0 2024-09-10T07:48:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.888+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 msgr2=0x7f9c2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.888+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 0x7f9c2c06e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c034000 tx=0x7f9c3c02e000 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 msgr2=0x7f9c4c160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c4d0 tx=0x7f9c3800c990 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 shutdown_connections 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c4c15e860 0x7f9c4c1612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c4c0cfbf0 0x7f9c4c160d70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 --2- 172.21.15.3:0/3625638732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c073150 0x7f9c4c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 >> 172.21.15.3:0/3625638732 conn(0x7f9c4c0d4650 msgr2=0x7f9c4c072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 shutdown_connections 2024-09-10T07:48:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:38.889+0000 7f9c51800640 1 -- 172.21.15.3:0/3625638732 wait complete. 2024-09-10T07:48:38.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:48:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.073+0000 7f6942c00640 1 Processor -- start 2024-09-10T07:48:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.073+0000 7f6942c00640 1 -- start start 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0d3de0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0d40c0 con 0x7f693c0d3860 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f693c161f50 con 0x7f693c161b70 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f693c0d4cb0 0x7f693c0cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46074/0 (socket says 172.21.15.3:46074) 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6942200640 1 -- 172.21.15.3:0/600961664 learned_addr learned my addr 172.21.15.3:0/600961664 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.074+0000 7f6941800640 1 --2- 172.21.15.3:0/600961664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146967755 0 0) 0x7f693c0d3de0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6924003520 con 0x7f693c0d4cb0 2024-09-10T07:48:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181000795 0 0) 0x7f693c0d40c0 con 0x7f693c0d3860 2024-09-10T07:48:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f693c0d3de0 con 0x7f693c0d3860 2024-09-10T07:48:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f6941800640 1 -- 172.21.15.3:0/600961664 >> v1:172.21.15.70:6789/0 conn(0x7f693c0d3860 legacy=0x7f693c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f6941800640 1 -- 172.21.15.3:0/600961664 >> v1:172.21.15.3:6789/0 conn(0x7f693c0d4cb0 legacy=0x7f693c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.075+0000 7f6941800640 1 -- 172.21.15.3:0/600961664 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c0cf690 con 0x7f693c161b70 2024-09-10T07:48:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.076+0000 7f6941800640 1 --2- 172.21.15.3:0/600961664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c0d3320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69300097d0 tx=0x7f693002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd8a268b1bc1330 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.076+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930037070 con 0x7f693c161b70 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.076+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6930032440 con 0x7f693c161b70 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.076+0000 7f693be00640 1 -- 172.21.15.3:0/600961664 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930032810 con 0x7f693c161b70 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 -- 172.21.15.3:0/600961664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 msgr2=0x7f693c0d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 --2- 172.21.15.3:0/600961664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c0d3320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69300097d0 tx=0x7f693002a210 comp rx=0 tx=0).stop 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 -- 172.21.15.3:0/600961664 shutdown_connections 2024-09-10T07:48:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 --2- 172.21.15.3:0/600961664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c0d3320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 -- 172.21.15.3:0/600961664 >> 172.21.15.3:0/600961664 conn(0x7f693c062630 msgr2=0x7f693c062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 -- 172.21.15.3:0/600961664 shutdown_connections 2024-09-10T07:48:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.077+0000 7f6942c00640 1 -- 172.21.15.3:0/600961664 wait complete. 2024-09-10T07:48:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.079+0000 7f6942c00640 1 Processor -- start 2024-09-10T07:48:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.079+0000 7f6942c00640 1 -- start start 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c0d3860 0x7f693c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0cf2d0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0c9fc0 con 0x7f693c0d3860 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f693c0cf450 con 0x7f693c161b70 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52488/0 (socket says 172.21.15.3:52488) 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 -- 172.21.15.3:0/2619825615 learned_addr learned my addr 172.21.15.3:0/2619825615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6941800640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c0d3860 0x7f693c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942200640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 -- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 msgr2=0x7f693c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 -- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c0d3860 msgr2=0x7f693c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c0d3860 0x7f693c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6940e00640 1 -- 172.21.15.3:0/2619825615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1596d0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.080+0000 7f6942200640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6940e00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f692800c400 tx=0x7f692800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928017070 con 0x7f693c0d4cb0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69280052d0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928003810 con 0x7f693c0d4cb0 2024-09-10T07:48:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1f8450 con 0x7f693c0d4cb0 2024-09-10T07:48:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.081+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c159970 con 0x7f693c0d4cb0 2024-09-10T07:48:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.082+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c0b75d0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.083+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f692801a070 con 0x7f693c0d4cb0 2024-09-10T07:48:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.084+0000 7f6939600640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.084+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f6928089fc0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.084+0000 7f6941800640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.085+0000 7f6941800640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 0x7f691c06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6930034000 tx=0x7f693002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.088+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6928058bb0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.272+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f693c0cfe30 con 0x7f693c0d4cb0 2024-09-10T07:48:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.277+0000 7f6939600640 1 -- 172.21.15.3:0/2619825615 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f69280584b0 con 0x7f693c0d4cb0 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 msgr2=0x7f691c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 0x7f691c06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6930034000 tx=0x7f693002e000 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 msgr2=0x7f693c15a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f692800c400 tx=0x7f692800c8c0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 shutdown_connections 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f691c06be30 0x7f691c06e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f693c161b70 0x7f693c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f693c0d4cb0 0x7f693c15a650 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 --2- 172.21.15.3:0/2619825615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f693c0d3860 0x7f693c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 >> 172.21.15.3:0/2619825615 conn(0x7f693c062630 msgr2=0x7f693c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 shutdown_connections 2024-09-10T07:48:39.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.280+0000 7f6942c00640 1 -- 172.21.15.3:0/2619825615 wait complete. 2024-09-10T07:48:39.296 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:48:39.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:48:39.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:48:39.496 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:48:39.496 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:48:39.496 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:39.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:48:39.532 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:48:39.532 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:39.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:48:39.569 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:48:39.569 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:39.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:48:39.607 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:48:39.607 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:48:39.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:48:39.646 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:48:39.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:39.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.805+0000 7f30a3200640 1 Processor -- start 2024-09-10T07:48:39.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.805+0000 7f30a3200640 1 -- start start 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a3200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f30a4073860 0x7f30a40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a3200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a40cfdc0 con 0x7f30a415e860 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a3200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a40cee40 con 0x7f30a40d3c10 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a3200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f30a40d3780 con 0x7f30a4073860 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a1e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f30a415e860 0x7f30a40cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46082/0 (socket says 172.21.15.3:46082) 2024-09-10T07:48:39.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.806+0000 7f30a1e00640 1 -- 172.21.15.3:0/2631675299 learned_addr learned my addr 172.21.15.3:0/2631675299 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a2800640 1 --2- 172.21.15.3:0/2631675299 >> v2:172.21.15.70:3300/0 conn(0x7f30a4073860 0x7f30a40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235747310 0 0) 0x7f30a40cfdc0 con 0x7f30a415e860 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3088003520 con 0x7f30a415e860 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 904266756 0 0) 0x7f3088003520 con 0x7f30a415e860 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 >> v2:172.21.15.70:3300/0 conn(0x7f30a4073860 msgr2=0x7f30a40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 --2- 172.21.15.3:0/2631675299 >> v2:172.21.15.70:3300/0 conn(0x7f30a4073860 0x7f30a40b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 >> v1:172.21.15.70:6789/0 conn(0x7f30a40d3c10 legacy=0x7f30a4073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a40cf120 con 0x7f30a415e860 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f3094001670 con 0x7f30a415e860 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 >> v1:172.21.15.3:6789/0 conn(0x7f30a415e860 legacy=0x7f30a40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3088003a70 0x7f3088003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f3088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3088005210 0x7f308800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40d3780 con 0x7f3088004800 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f308800c140 con 0x7f3088003a70 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f308800c340 con 0x7f3088005210 2024-09-10T07:48:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.807+0000 7f30a1400640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f3088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1e00640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3088003a70 0x7f3088003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a2800640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3088005210 0x7f308800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1400640 1 -- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3088005210 msgr2=0x7f308800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1400640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3088005210 0x7f308800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1400640 1 -- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3088003a70 msgr2=0x7f3088003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1400640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3088003a70 0x7f3088003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a1400640 1 -- 172.21.15.3:0/2631675299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3088003520 con 0x7f3088004800 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a2800640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3088005210 0x7f308800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.808+0000 7f30a2800640 1 -- 172.21.15.3:0/2631675299 reap_dead start 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.809+0000 7f30a1400640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f3088004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3090001750 tx=0x7f3088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a883a87895471660 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.809+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090001370 con 0x7f3088004800 2024-09-10T07:48:39.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.809+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30900052d0 con 0x7f3088004800 2024-09-10T07:48:39.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.809+0000 7f30a0a00640 1 -- 172.21.15.3:0/2631675299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309002abc0 con 0x7f3088004800 2024-09-10T07:48:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 -- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 msgr2=0x7f3088004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f3088004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3090001750 tx=0x7f3088005830 comp rx=0 tx=0).stop 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 -- 172.21.15.3:0/2631675299 shutdown_connections 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 --2- 172.21.15.3:0/2631675299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f3088004c80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 -- 172.21.15.3:0/2631675299 >> 172.21.15.3:0/2631675299 conn(0x7f30a40d4650 msgr2=0x7f30a40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 -- 172.21.15.3:0/2631675299 shutdown_connections 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.810+0000 7f30a3200640 1 -- 172.21.15.3:0/2631675299 wait complete. 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.811+0000 7f30a3200640 1 Processor -- start 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 -- start start 2024-09-10T07:48:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a4161170 0x7f30a41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 0x7f30a40ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40d2460 con 0x7f3088004800 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40cf400 con 0x7f30a4161170 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.812+0000 7f30a3200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f30a40d4050 con 0x7f30a40ab2e0 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52504/0 (socket says 172.21.15.3:52504) 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 -- 172.21.15.3:0/1508381276 learned_addr learned my addr 172.21.15.3:0/1508381276 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1400640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a4161170 0x7f30a41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a2800640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 0x7f30a40ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 -- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 msgr2=0x7f30a40ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 0x7f30a40ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 -- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a4161170 msgr2=0x7f30a41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a4161170 0x7f30a41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 -- 172.21.15.3:0/1508381276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a40ac030 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a2800640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 0x7f30a40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.813+0000 7f30a1e00640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.814+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309400f6d0 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.814+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3094018070 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.814+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309400fad0 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.814+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a40ac310 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.814+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a40abca0 con 0x7f3088004800 2024-09-10T07:48:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.815+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a40ac830 con 0x7f3088004800 2024-09-10T07:48:39.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.816+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f309400fc70 con 0x7f3088004800 2024-09-10T07:48:39.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.817+0000 7f309a000640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 0x7f308406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:39.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.817+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5115+0+0 (secure 0 0 0) 0x7f309408abb0 con 0x7f3088004800 2024-09-10T07:48:39.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.817+0000 7f30a1400640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 0x7f308406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:39.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.818+0000 7f30a1400640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 0x7f308406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3090032000 tx=0x7f309002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:39.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:39.821+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3094059710 con 0x7f3088004800 2024-09-10T07:48:40.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.039+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30a40cf6b0 con 0x7f3088004800 2024-09-10T07:48:40.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.040+0000 7f309a000640 1 -- 172.21.15.3:0/1508381276 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3896 (secure 0 0 0) 0x7f3094059010 con 0x7f3088004800 2024-09-10T07:48:40.040 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:40.040 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-10T07:48:33:862588+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3862835705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3862835705},{"type":"v1","addr":"172.21.15.70:6833","nonce":3862835705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/757765702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":757765702},{"type":"v1","addr":"172.21.15.70:6835","nonce":757765702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/667928240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":667928240},{"type":"v1","addr":"172.21.15.3:6835","nonce":667928240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/823957339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":823957339},{"type":"v1","addr":"172.21.15.3:6837","nonce":823957339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-09-10T07:48:40.040 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-10T07:48:40.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 msgr2=0x7f308406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:40.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 0x7f308406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3090032000 tx=0x7f309002e000 comp rx=0 tx=0).stop 2024-09-10T07:48:40.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 msgr2=0x7f30a414fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:40.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094007430 comp rx=0 tx=0).stop 2024-09-10T07:48:40.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 shutdown_connections 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f308406be60 0x7f308406e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f30a40ab2e0 0x7f30a40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a4161170 0x7f30a41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 --2- 172.21.15.3:0/1508381276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3088004800 0x7f30a414fb10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 >> 172.21.15.3:0/1508381276 conn(0x7f30a40d4650 msgr2=0x7f30a40cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 shutdown_connections 2024-09-10T07:48:40.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:40.042+0000 7f30a3200640 1 -- 172.21.15.3:0/1508381276 wait complete. 2024-09-10T07:48:40.058 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:48:40.059 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:48:40.059 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:48:40.059 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:48:40.059 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:48:40.059 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:48:40.060 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:48:40.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:48:40.060 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:48:40.060 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:48:40.060 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:48:40.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:48:40.061 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:48:40.061 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:48:40.061 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:48:40.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:48:40.073 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:48:40.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:48:40.074 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:48:40.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:48:40.078 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:48:40.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:48:40.118 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:48:40.118 DEBUG:teuthology.parallel:result is None 2024-09-10T07:48:40.121 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:48:40.121 DEBUG:teuthology.parallel:result is None 2024-09-10T07:48:40.123 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:48:40.124 DEBUG:teuthology.parallel:result is None 2024-09-10T07:48:40.248 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:48:40.248 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:48:40.253 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:48:44.088 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-10T07:48:46.078 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:48:46.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:48:46.084 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:48:46.084 DEBUG:teuthology.parallel:result is None 2024-09-10T07:48:46.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:48:46.220 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:48:46.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.253+0000 7fdb73e00640 1 Processor -- start 2024-09-10T07:48:46.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.253+0000 7fdb73e00640 1 -- start start 2024-09-10T07:48:46.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 0x7fdb6c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb6c160af0 0x7fdb6c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 0x7fdb6c0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdb6c0cfdd0 con 0x7fdb6c160af0 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdb6c15cc90 con 0x7fdb6c160710 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb73e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6c15cf70 con 0x7fdb6c0cf800 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb6c160af0 0x7fdb6c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb6c160af0 0x7fdb6c0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50832/0 (socket says 172.21.15.3:50832) 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 -- 172.21.15.3:0/2789921130 learned_addr learned my addr 172.21.15.3:0/2789921130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb71400640 1 --2- 172.21.15.3:0/2789921130 >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 0x7fdb6c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb71e00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 0x7fdb6c0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 -- 172.21.15.3:0/2789921130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 msgr2=0x7fdb6c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 0x7fdb6c0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 -- 172.21.15.3:0/2789921130 >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 msgr2=0x7fdb6c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.255+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2789921130 >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 0x7fdb6c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.256+0000 7fdb70a00640 1 -- 172.21.15.3:0/2789921130 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c15d250 con 0x7fdb6c160af0 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.256+0000 7fdb71e00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 0x7fdb6c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:46.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.256+0000 7fdb71400640 1 --2- 172.21.15.3:0/2789921130 >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 0x7fdb6c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:46.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.256+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c160af0 0x7fdb6c0d00b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c06ccc0 tx=0x7fdb5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d52542d7ae1fa28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb6be00640 1 -- 172.21.15.3:0/2789921130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58037070 con 0x7fdb6c160af0 2024-09-10T07:48:46.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb6be00640 1 -- 172.21.15.3:0/2789921130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb58032440 con 0x7fdb6c160af0 2024-09-10T07:48:46.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb6be00640 1 -- 172.21.15.3:0/2789921130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58032810 con 0x7fdb6c160af0 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 -- 172.21.15.3:0/2789921130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c160af0 msgr2=0x7fdb6c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c160af0 0x7fdb6c0d00b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c06ccc0 tx=0x7fdb5802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 -- 172.21.15.3:0/2789921130 shutdown_connections 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c0cf800 0x7fdb6c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2789921130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c160af0 0x7fdb6c0d00b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2789921130 >> v2:172.21.15.70:3300/0 conn(0x7fdb6c160710 0x7fdb6c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.257+0000 7fdb73e00640 1 -- 172.21.15.3:0/2789921130 >> 172.21.15.3:0/2789921130 conn(0x7fdb6c06d000 msgr2=0x7fdb6c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.258+0000 7fdb73e00640 1 -- 172.21.15.3:0/2789921130 shutdown_connections 2024-09-10T07:48:46.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.258+0000 7fdb73e00640 1 -- 172.21.15.3:0/2789921130 wait complete. 2024-09-10T07:48:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.259+0000 7fdb73e00640 1 Processor -- start 2024-09-10T07:48:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.259+0000 7fdb73e00640 1 -- start start 2024-09-10T07:48:46.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 0x7fdb6c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c160af0 0x7fdb6c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c15d770 con 0x7fdb6c0cf800 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c06bb20 con 0x7fdb6c160710 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb73e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6c0b37c0 con 0x7fdb6c160af0 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb71400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb71400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50840/0 (socket says 172.21.15.3:50840) 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.260+0000 7fdb71400640 1 -- 172.21.15.3:0/2288283888 learned_addr learned my addr 172.21.15.3:0/2288283888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 0x7fdb6c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c160af0 0x7fdb6c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 -- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c160af0 msgr2=0x7fdb6c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c160af0 0x7fdb6c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 -- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 msgr2=0x7fdb6c152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 0x7fdb6c152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 -- 172.21.15.3:0/2288283888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c0b3aa0 con 0x7fdb6c0cf800 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 0x7fdb6c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb71400640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000ebe0 tx=0x7fdb6000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.261+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60017070 con 0x7fdb6c0cf800 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.262+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6000cc30 con 0x7fdb6c0cf800 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.262+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c0b3d80 con 0x7fdb6c0cf800 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.262+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60013470 con 0x7fdb6c0cf800 2024-09-10T07:48:46.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.262+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c0b4340 con 0x7fdb6c0cf800 2024-09-10T07:48:46.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.263+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6c0afd50 con 0x7fdb6c0cf800 2024-09-10T07:48:46.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.264+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb6001a070 con 0x7fdb6c0cf800 2024-09-10T07:48:46.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.264+0000 7fdb69600640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 0x7fdb4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:46.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.265+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5115+0+0 (secure 0 0 0) 0x7fdb6008aaa0 con 0x7fdb6c0cf800 2024-09-10T07:48:46.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.265+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 0x7fdb4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:46.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.266+0000 7fdb70a00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 0x7fdb4c06e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb58034000 tx=0x7fdb5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:46.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.269+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb60058fb0 con 0x7fdb6c0cf800 2024-09-10T07:48:46.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:46.465+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdb6c0b2bf0 con 0x7fdb6c0cf800 2024-09-10T07:48:47.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.576+0000 7fdb69600640 1 -- 172.21.15.3:0/2288283888 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7fdb6005d2b0 con 0x7fdb6c0cf800 2024-09-10T07:48:47.576 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:48:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.580+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 msgr2=0x7fdb4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.580+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 0x7fdb4c06e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb58034000 tx=0x7fdb5802e000 comp rx=0 tx=0).stop 2024-09-10T07:48:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.580+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 msgr2=0x7fdb6c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.580+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000ebe0 tx=0x7fdb6000c520 comp rx=0 tx=0).stop 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.580+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 shutdown_connections 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb4c06be60 0x7fdb4c06e320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb6c160af0 0x7fdb6c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb6c160710 0x7fdb6c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 --2- 172.21.15.3:0/2288283888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb6c0cf800 0x7fdb6c148ff0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 >> 172.21.15.3:0/2288283888 conn(0x7fdb6c06d000 msgr2=0x7fdb6c06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 shutdown_connections 2024-09-10T07:48:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.581+0000 7fdb73e00640 1 -- 172.21.15.3:0/2288283888 wait complete. 2024-09-10T07:48:47.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:48:47.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.762+0000 7fb30ac00640 1 Processor -- start 2024-09-10T07:48:47.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.762+0000 7fb30ac00640 1 -- start start 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30ac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30ac00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3040d4050 0x7fb304073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30ac00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3040cfdc0 con 0x7fb30415e860 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30ac00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb3040d3780 con 0x7fb3040d4050 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30ac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb3040cee40 con 0x7fb3040d3c10 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb309800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb30415e860 0x7fb3040cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44520/0 (socket says 172.21.15.3:44520) 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb309800640 1 -- 172.21.15.3:0/2990684352 learned_addr learned my addr 172.21.15.3:0/2990684352 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:47.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb30a200640 1 --2- 172.21.15.3:0/2990684352 >> v2:172.21.15.3:3300/0 conn(0x7fb3040d4050 0x7fb304073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.763+0000 7fb308e00640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.764+0000 7fb30a200640 1 -- 172.21.15.3:0/2990684352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 msgr2=0x7fb304073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.764+0000 7fb30a200640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.764+0000 7fb30a200640 1 -- 172.21.15.3:0/2990684352 >> v1:172.21.15.3:6789/0 conn(0x7fb30415e860 legacy=0x7fb3040cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.764+0000 7fb30a200640 1 -- 172.21.15.3:0/2990684352 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3040cf120 con 0x7fb3040d4050 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.764+0000 7fb303e00640 1 -- 172.21.15.3:0/2990684352 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1885438254 0 0) 0x7fb3040cfdc0 con 0x7fb30415e860 2024-09-10T07:48:47.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30a200640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3040d4050 0x7fb304073690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009d90 tx=0x7fb2fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172788b86c64483c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:47.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb303e00640 1 -- 172.21.15.3:0/2990684352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc007740 con 0x7fb3040d4050 2024-09-10T07:48:47.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb303e00640 1 -- 172.21.15.3:0/2990684352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc0078e0 con 0x7fb3040d4050 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb303e00640 1 -- 172.21.15.3:0/2990684352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc007cb0 con 0x7fb3040d4050 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 -- 172.21.15.3:0/2990684352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3040d4050 msgr2=0x7fb304073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3040d4050 0x7fb304073690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009d90 tx=0x7fb2fc02a660 comp rx=0 tx=0).stop 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 -- 172.21.15.3:0/2990684352 shutdown_connections 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3040d4050 0x7fb304073690 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 --2- 172.21.15.3:0/2990684352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.765+0000 7fb30ac00640 1 -- 172.21.15.3:0/2990684352 >> 172.21.15.3:0/2990684352 conn(0x7fb3040d4650 msgr2=0x7fb3040d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.766+0000 7fb30ac00640 1 -- 172.21.15.3:0/2990684352 shutdown_connections 2024-09-10T07:48:47.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.766+0000 7fb30ac00640 1 -- 172.21.15.3:0/2990684352 wait complete. 2024-09-10T07:48:47.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.767+0000 7fb30ac00640 1 Processor -- start 2024-09-10T07:48:47.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.767+0000 7fb30ac00640 1 -- start start 2024-09-10T07:48:47.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3040d4050 0x7fb3040b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040d2460 con 0x7fb30415e860 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040cf400 con 0x7fb3040d4050 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30ac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb3040b7880 con 0x7fb3040d3c10 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50848/0 (socket says 172.21.15.3:50848) 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.768+0000 7fb30a200640 1 -- 172.21.15.3:0/1858418448 learned_addr learned my addr 172.21.15.3:0/1858418448 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb309800640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb308e00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3040d4050 0x7fb3040b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 -- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 msgr2=0x7fb304158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 -- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3040d4050 msgr2=0x7fb3040b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3040d4050 0x7fb3040b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 -- 172.21.15.3:0/1858418448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30415ff30 con 0x7fb30415e860 2024-09-10T07:48:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb309800640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:47.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30a200640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009eb0 tx=0x7fb2fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:47.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc040070 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc031bc0 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc045460 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.769+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30415ee90 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.770+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30415f450 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.771+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3040b7c40 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.771+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2fc039040 con 0x7fb30415e860 2024-09-10T07:48:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.772+0000 7fb301600640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 0x7fb2e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.772+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7fb2fc0ac400 con 0x7fb30415e860 2024-09-10T07:48:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.773+0000 7fb309800640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 0x7fb2e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.773+0000 7fb309800640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 0x7fb2e406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000f60 tx=0x7fb2f8007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:47.777+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2fc07b0f0 con 0x7fb30415e860 2024-09-10T07:48:48.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.031+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb3040d2ce0 con 0x7fb30415e860 2024-09-10T07:48:48.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.032+0000 7fb301600640 1 -- 172.21.15.3:0/1858418448 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7fb2fc07a9f0 con 0x7fb30415e860 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:48:48.032 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.033 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.034 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.035 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.037 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:48:48.038 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.040 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.041 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:48:48.042 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:48:48.044 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:48:48.045 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 msgr2=0x7fb2e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 0x7fb2e406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000f60 tx=0x7fb2f8007ca0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 msgr2=0x7fb3040cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009eb0 tx=0x7fb2fc004a40 comp rx=0 tx=0).stop 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 shutdown_connections 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb2e406be30 0x7fb2e406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb30415e860 0x7fb3040cdcc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3040d4050 0x7fb3040b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 --2- 172.21.15.3:0/1858418448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3040d3c10 0x7fb304158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 >> 172.21.15.3:0/1858418448 conn(0x7fb3040d4650 msgr2=0x7fb3040b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:48.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 shutdown_connections 2024-09-10T07:48:48.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.034+0000 7fb30ac00640 1 -- 172.21.15.3:0/1858418448 wait complete. 2024-09-10T07:48:48.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:48:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.205+0000 7f9db3c00640 1 Processor -- start 2024-09-10T07:48:48.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.205+0000 7f9db3c00640 1 -- start start 2024-09-10T07:48:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.206+0000 7f9db3c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9dac15c720 0x7f9dac071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.206+0000 7f9db3c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dac070f30 0x7f9dac073610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.206+0000 7f9db3c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dac0cd7f0 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.206+0000 7f9db3c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9dac072210 con 0x7f9dac070f30 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.206+0000 7f9db3c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9dac0cf880 con 0x7f9dac15c720 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dac070f30 0x7f9dac073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9db1c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9dac0cfdc0 0x7f9dac0ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44522/0 (socket says 172.21.15.3:44522) 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dac070f30 0x7f9dac073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50860/0 (socket says 172.21.15.3:50860) 2024-09-10T07:48:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9db1c00640 1 -- 172.21.15.3:0/3885531934 learned_addr learned my addr 172.21.15.3:0/3885531934 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9db1200640 1 --2- 172.21.15.3:0/3885531934 >> v2:172.21.15.70:3300/0 conn(0x7f9dac15c720 0x7f9dac071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 -- 172.21.15.3:0/3885531934 >> v2:172.21.15.70:3300/0 conn(0x7f9dac15c720 msgr2=0x7f9dac071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 --2- 172.21.15.3:0/3885531934 >> v2:172.21.15.70:3300/0 conn(0x7f9dac15c720 0x7f9dac071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 -- 172.21.15.3:0/3885531934 >> v1:172.21.15.3:6789/0 conn(0x7f9dac0cfdc0 legacy=0x7f9dac0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dabe00640 1 -- 172.21.15.3:0/3885531934 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac0cda90 con 0x7f9dac070f30 2024-09-10T07:48:48.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.207+0000 7f9dab400640 1 -- 172.21.15.3:0/3885531934 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326687414 0 0) 0x7f9dac0cd7f0 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.208+0000 7f9dabe00640 1 --2- 172.21.15.3:0/3885531934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac070f30 0x7f9dac073610 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dac075bd0 tx=0x7f9d9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e774084747ca7912 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9dab400640 1 -- 172.21.15.3:0/3885531934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98037070 con 0x7f9dac070f30 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9dab400640 1 -- 172.21.15.3:0/3885531934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98032440 con 0x7f9dac070f30 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9dab400640 1 -- 172.21.15.3:0/3885531934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9802ae60 con 0x7f9dac070f30 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 -- 172.21.15.3:0/3885531934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac070f30 msgr2=0x7f9dac073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 --2- 172.21.15.3:0/3885531934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac070f30 0x7f9dac073610 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dac075bd0 tx=0x7f9d9802a110 comp rx=0 tx=0).stop 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 -- 172.21.15.3:0/3885531934 shutdown_connections 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 --2- 172.21.15.3:0/3885531934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac070f30 0x7f9dac073610 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 --2- 172.21.15.3:0/3885531934 >> v2:172.21.15.70:3300/0 conn(0x7f9dac15c720 0x7f9dac071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 -- 172.21.15.3:0/3885531934 >> 172.21.15.3:0/3885531934 conn(0x7f9dac06cfc0 msgr2=0x7f9dac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 -- 172.21.15.3:0/3885531934 shutdown_connections 2024-09-10T07:48:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.209+0000 7f9db3c00640 1 -- 172.21.15.3:0/3885531934 wait complete. 2024-09-10T07:48:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.211+0000 7f9db3c00640 1 Processor -- start 2024-09-10T07:48:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.211+0000 7f9db3c00640 1 -- start start 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9dac070f30 0x7f9dac150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dac15c720 0x7f9dac15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac073b50 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac06ed10 con 0x7f9dac15c720 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9db3c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac073cd0 con 0x7f9dac070f30 2024-09-10T07:48:48.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9dabe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9dabe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50870/0 (socket says 172.21.15.3:50870) 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.212+0000 7f9dabe00640 1 -- 172.21.15.3:0/1861308458 learned_addr learned my addr 172.21.15.3:0/1861308458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9db1200640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9dac070f30 0x7f9dac150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9db1c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dac15c720 0x7f9dac15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 -- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9dac070f30 msgr2=0x7f9dac150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9dac070f30 0x7f9dac150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 -- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dac15c720 msgr2=0x7f9dac15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dac15c720 0x7f9dac15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 -- 172.21.15.3:0/1861308458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1f4f40 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9dabe00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009890 tx=0x7f9d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98037040 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98042070 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1f5220 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98030bd0 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.213+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac150890 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.214+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac073610 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.215+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d98047460 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.216+0000 7f9da8c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.216+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f9d980ab780 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.216+0000 7f9db1200640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.217+0000 7f9db1200640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9da0006fd0 tx=0x7f9da0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.221+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d98032e20 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.407+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9dac0cf2b0 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.408+0000 7f9da8c00640 1 -- 172.21.15.3:0/1861308458 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) ==== 84+0+65 (secure 0 0 0) 0x7f9d9807a3e0 con 0x7f9dac0cfdc0 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:48:48.408 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 msgr2=0x7f9d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9da0006fd0 tx=0x7f9da0008040 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 msgr2=0x7f9dac1f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009890 tx=0x7f9d980046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 shutdown_connections 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dac15c720 0x7f9dac15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dac0cfdc0 0x7f9dac1f4580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 --2- 172.21.15.3:0/1861308458 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9dac070f30 0x7f9dac150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 >> 172.21.15.3:0/1861308458 conn(0x7f9dac06cfc0 msgr2=0x7f9dac0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 shutdown_connections 2024-09-10T07:48:48.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.410+0000 7f9db3c00640 1 -- 172.21.15.3:0/1861308458 wait complete. 2024-09-10T07:48:48.425 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:48:48.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:48:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.586+0000 7f79b9c00640 1 Processor -- start 2024-09-10T07:48:48.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.587+0000 7f79b9c00640 1 -- start start 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b9c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79b40b61f0 0x7f79b40b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b9c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40b6b50 con 0x7f79b40d3320 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b9c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40cfe30 con 0x7f79b415e860 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b9c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f79b40b3f10 con 0x7f79b40b61f0 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b2000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79b40d3320 0x7f79b40d4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44526/0 (socket says 172.21.15.3:44526) 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.588+0000 7f79b2000640 1 -- 172.21.15.3:0/3711594419 learned_addr learned my addr 172.21.15.3:0/3711594419 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:48.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b3400640 1 --2- 172.21.15.3:0/3711594419 >> v2:172.21.15.70:3300/0 conn(0x7f79b40b61f0 0x7f79b40b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523883963 0 0) 0x7f79b40b6b50 con 0x7f79b40d3320 2024-09-10T07:48:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7998003520 con 0x7f79b40d3320 2024-09-10T07:48:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014184090 0 0) 0x7f79b40cfe30 con 0x7f79b415e860 2024-09-10T07:48:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79b40b6b50 con 0x7f79b415e860 2024-09-10T07:48:48.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1572682406 0 0) 0x7f7998003520 con 0x7f79b40d3320 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 >> v2:172.21.15.70:3300/0 conn(0x7f79b40b61f0 msgr2=0x7f79b40b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 --2- 172.21.15.3:0/3711594419 >> v2:172.21.15.70:3300/0 conn(0x7f79b40b61f0 0x7f79b40b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 >> v1:172.21.15.70:6789/0 conn(0x7f79b415e860 legacy=0x7f79b40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b40d5070 con 0x7f79b40d3320 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f79a0001670 con 0x7f79b40d3320 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 >> v1:172.21.15.3:6789/0 conn(0x7f79b40d3320 legacy=0x7f79b40d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f7998003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40b3f10 con 0x7f7998003b90 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f799800c090 con 0x7f7998004800 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f799800c290 con 0x7f79980051c0 2024-09-10T07:48:48.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b3400640 1 --2- 172.21.15.3:0/3711594419 >> v2:172.21.15.70:3300/0 conn(0x7f79b40b61f0 0x7f79b40b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.589+0000 7f79b2a00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f7998003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2000640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b3400640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2a00640 1 -- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79980051c0 msgr2=0x7f799800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2a00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2a00640 1 -- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7998004800 msgr2=0x7f7998004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2a00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b2a00640 1 -- 172.21.15.3:0/3711594419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7998003520 con 0x7f7998003b90 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.590+0000 7f79b3400640 1 -- 172.21.15.3:0/3711594419 reap_dead start 2024-09-10T07:48:48.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.591+0000 7f79b2a00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f7998003f90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f79a40018c0 tx=0x7f7998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b02252647ad062 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.591+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a40014e0 con 0x7f7998003b90 2024-09-10T07:48:48.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.591+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a40052f0 con 0x7f7998003b90 2024-09-10T07:48:48.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.591+0000 7f79b1600640 1 -- 172.21.15.3:0/3711594419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a402ac50 con 0x7f7998003b90 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 -- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 msgr2=0x7f7998003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f7998003f90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f79a40018c0 tx=0x7f7998005830 comp rx=0 tx=0).stop 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 -- 172.21.15.3:0/3711594419 shutdown_connections 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 --2- 172.21.15.3:0/3711594419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f7998003f90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 -- 172.21.15.3:0/3711594419 >> 172.21.15.3:0/3711594419 conn(0x7f79b4072560 msgr2=0x7f79b4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 -- 172.21.15.3:0/3711594419 shutdown_connections 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.592+0000 7f79b9c00640 1 -- 172.21.15.3:0/3711594419 wait complete. 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.593+0000 7f79b9c00640 1 Processor -- start 2024-09-10T07:48:48.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 -- start start 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b40cb3a0 0x7f79b41f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 0x7f79b40c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40d3e80 con 0x7f7998003b90 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40d3ba0 con 0x7f79b40cb3a0 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.594+0000 7f79b9c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f79b40d4160 con 0x7f79b41f4f40 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50880/0 (socket says 172.21.15.3:50880) 2024-09-10T07:48:48.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 -- 172.21.15.3:0/1677054988 learned_addr learned my addr 172.21.15.3:0/1677054988 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2000640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b40cb3a0 0x7f79b41f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b3400640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 0x7f79b40c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 -- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 msgr2=0x7f79b40c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 0x7f79b40c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 -- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b40cb3a0 msgr2=0x7f79b41f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b40cb3a0 0x7f79b41f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 -- 172.21.15.3:0/1677054988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b40ca300 con 0x7f7998003b90 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b3400640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 0x7f79b40c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.595+0000 7f79b2a00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f79a4000c00 tx=0x7f79a40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.596+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4007a50 con 0x7f7998003b90 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.596+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a40054a0 con 0x7f7998003b90 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.596+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a40069d0 con 0x7f7998003b90 2024-09-10T07:48:48.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.596+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b40ca5e0 con 0x7f7998003b90 2024-09-10T07:48:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.596+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b40caba0 con 0x7f7998003b90 2024-09-10T07:48:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.597+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b40d4680 con 0x7f7998003b90 2024-09-10T07:48:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.598+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79a4006b70 con 0x7f7998003b90 2024-09-10T07:48:48.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.599+0000 7f79aaa00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 0x7f799406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.599+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f79a40aad90 con 0x7f7998003b90 2024-09-10T07:48:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.599+0000 7f79b2000640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 0x7f799406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:48.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.600+0000 7f79b2000640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 0x7f799406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f79b40cb7a0 tx=0x7f79a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:48.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.603+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f79b40d4680 con 0x7f7998003b90 2024-09-10T07:48:48.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:48.786+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f79b40d4440 con 0x7f7998003b90 2024-09-10T07:48:49.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.591+0000 7f79aaa00640 1 -- 172.21.15.3:0/1677054988 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) ==== 138+0+0 (secure 0 0 0) 0x7f79a4031390 con 0x7f7998003b90 2024-09-10T07:48:49.591 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:48:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.595+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 msgr2=0x7f799406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.595+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 0x7f799406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f79b40cb7a0 tx=0x7f79a0009040 comp rx=0 tx=0).stop 2024-09-10T07:48:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.595+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 msgr2=0x7f79b41f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.595+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f79a4000c00 tx=0x7f79a40012d0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 shutdown_connections 2024-09-10T07:48:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f799406be30 0x7f799406e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79b41f4f40 0x7f79b40c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b40cb3a0 0x7f79b41f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 --2- 172.21.15.3:0/1677054988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7998003b90 0x7f79b41f1610 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 >> 172.21.15.3:0/1677054988 conn(0x7f79b4072560 msgr2=0x7f79b415ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.596+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 shutdown_connections 2024-09-10T07:48:49.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.597+0000 7f79b9c00640 1 -- 172.21.15.3:0/1677054988 wait complete. 2024-09-10T07:48:49.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:48:49.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.787+0000 7f419a600640 1 Processor -- start 2024-09-10T07:48:49.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.787+0000 7f419a600640 1 -- start start 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f419a600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 0x7f419406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f419a600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41940cf270 0x7f41940cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f419a600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41940cd900 con 0x7f4194145730 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f419a600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f419406d7c0 con 0x7f41940cf270 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f419a600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f41940cdbe0 con 0x7f41941451d0 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f4193e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41940cf270 0x7f41940cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f4193e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41940cf270 0x7f41940cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50886/0 (socket says 172.21.15.3:50886) 2024-09-10T07:48:49.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.788+0000 7f4193e00640 1 -- 172.21.15.3:0/2706517525 learned_addr learned my addr 172.21.15.3:0/2706517525 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193400640 1 --2- 172.21.15.3:0/2706517525 >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 0x7f419406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193e00640 1 -- 172.21.15.3:0/2706517525 >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 msgr2=0x7f419406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:49.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193e00640 1 --2- 172.21.15.3:0/2706517525 >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 0x7f419406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193e00640 1 -- 172.21.15.3:0/2706517525 >> v1:172.21.15.3:6789/0 conn(0x7f4194145730 legacy=0x7f41940b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193e00640 1 -- 172.21.15.3:0/2706517525 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419406da10 con 0x7f41940cf270 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.789+0000 7f4193400640 1 --2- 172.21.15.3:0/2706517525 >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 0x7f419406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f4193e00640 1 --2- 172.21.15.3:0/2706517525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41940cf270 0x7f41940cf6d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41940732d0 tx=0x7f418c02a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b039b9907ab98c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f4192a00640 1 -- 172.21.15.3:0/2706517525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c036070 con 0x7f41940cf270 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f4192a00640 1 -- 172.21.15.3:0/2706517525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c031600 con 0x7f41940cf270 2024-09-10T07:48:49.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f4192a00640 1 -- 172.21.15.3:0/2706517525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c0319d0 con 0x7f41940cf270 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f419a600640 1 -- 172.21.15.3:0/2706517525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41940cf270 msgr2=0x7f41940cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.790+0000 7f419a600640 1 --2- 172.21.15.3:0/2706517525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41940cf270 0x7f41940cf6d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41940732d0 tx=0x7f418c02a6b0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 -- 172.21.15.3:0/2706517525 shutdown_connections 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 --2- 172.21.15.3:0/2706517525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41940cf270 0x7f41940cf6d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 --2- 172.21.15.3:0/2706517525 >> v2:172.21.15.70:3300/0 conn(0x7f41941451d0 0x7f419406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 -- 172.21.15.3:0/2706517525 >> 172.21.15.3:0/2706517525 conn(0x7f41940b5cb0 msgr2=0x7f419406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 -- 172.21.15.3:0/2706517525 shutdown_connections 2024-09-10T07:48:49.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.791+0000 7f419a600640 1 -- 172.21.15.3:0/2706517525 wait complete. 2024-09-10T07:48:49.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.792+0000 7f419a600640 1 Processor -- start 2024-09-10T07:48:49.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.792+0000 7f419a600640 1 -- start start 2024-09-10T07:48:49.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 0x7f4194072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4194145730 0x7f4194134530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f419406df30 con 0x7f41941451d0 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f41940cfc10 con 0x7f41940cf270 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f419a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f419406e0b0 con 0x7f4194145730 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f418be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f418be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50898/0 (socket says 172.21.15.3:50898) 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.793+0000 7f418be00640 1 -- 172.21.15.3:0/413326950 learned_addr learned my addr 172.21.15.3:0/413326950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f4193e00640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4194145730 0x7f4194134530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f4193400640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 0x7f4194072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 -- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4194145730 msgr2=0x7f4194134530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4194145730 0x7f4194134530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 -- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 msgr2=0x7f4194072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 0x7f4194072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 -- 172.21.15.3:0/413326950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419415caf0 con 0x7f41941451d0 2024-09-10T07:48:49.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f4193400640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 0x7f4194072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:49.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418be00640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c3c0 tx=0x7f417c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:49.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.794+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0052d0 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.795+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c005490 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.795+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c007260 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.795+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419415cdd0 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.795+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419415d390 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.796+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41940cf6d0 con 0x7f41941451d0 2024-09-10T07:48:49.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.797+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f417c007400 con 0x7f41941451d0 2024-09-10T07:48:49.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.798+0000 7f418b400640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 0x7f417006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:49.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.798+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5227+0+0 (secure 0 0 0) 0x7f417c08a280 con 0x7f41941451d0 2024-09-10T07:48:49.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.798+0000 7f4193400640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 0x7f417006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:49.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.798+0000 7f4193400640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 0x7f417006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4184006fd0 tx=0x7f4184008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:49.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.802+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f417c058cf0 con 0x7f41941451d0 2024-09-10T07:48:49.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:49.987+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f41941429d0 con 0x7f41941451d0 2024-09-10T07:48:50.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.598+0000 7f418b400640 1 -- 172.21.15.3:0/413326950 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) ==== 170+0+0 (secure 0 0 0) 0x7f417c0585f0 con 0x7f41941451d0 2024-09-10T07:48:50.598 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:48:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.603+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 msgr2=0x7f417006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.603+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 0x7f417006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4184006fd0 tx=0x7f4184008040 comp rx=0 tx=0).stop 2024-09-10T07:48:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.603+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 msgr2=0x7f41940aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.603+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c3c0 tx=0x7f417c00c880 comp rx=0 tx=0).stop 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 shutdown_connections 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f417006be30 0x7f417006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4194145730 0x7f4194134530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41941451d0 0x7f41940aeb10 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c3c0 tx=0x7f417c00c880 comp rx=0 tx=0).stop 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 --2- 172.21.15.3:0/413326950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41940cf270 0x7f4194072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.604+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 >> 172.21.15.3:0/413326950 conn(0x7f41940b5cb0 msgr2=0x7f41941441d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.605+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 shutdown_connections 2024-09-10T07:48:50.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.605+0000 7f419a600640 1 -- 172.21.15.3:0/413326950 wait complete. 2024-09-10T07:48:50.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:48:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.794+0000 7f5bd9600640 1 Processor -- start 2024-09-10T07:48:50.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.794+0000 7f5bd9600640 1 -- start start 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd9600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5bd41521d0 0x7f5bd40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd9600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd9600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd40b3b20 con 0x7f5bd40d0130 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd9600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd40d0c90 con 0x7f5bd41521d0 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd9600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd40b3dc0 con 0x7f5bd40b48a0 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5bd41521d0 0x7f5bd40cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd8c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5bd40d0130 0x7f5bd40d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44530/0 (socket says 172.21.15.3:44530) 2024-09-10T07:48:50.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.795+0000 7f5bd8c00640 1 -- 172.21.15.3:0/321308135 learned_addr learned my addr 172.21.15.3:0/321308135 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.796+0000 7f5bcb400640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.796+0000 7f5bd3e00640 1 -- 172.21.15.3:0/321308135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 msgr2=0x7f5bd40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.796+0000 7f5bd3e00640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.796+0000 7f5bd3e00640 1 -- 172.21.15.3:0/321308135 >> v1:172.21.15.3:6789/0 conn(0x7f5bd40d0130 legacy=0x7f5bd40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.796+0000 7f5bd3e00640 1 -- 172.21.15.3:0/321308135 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd40b40a0 con 0x7f5bd41521d0 2024-09-10T07:48:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.797+0000 7f5bd3e00640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd40cf150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097a0 tx=0x7f5bc402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6c8b3c36a14a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.797+0000 7f5bd3400640 1 -- 172.21.15.3:0/321308135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4037070 con 0x7f5bd41521d0 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.797+0000 7f5bd3400640 1 -- 172.21.15.3:0/321308135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc402ae40 con 0x7f5bd41521d0 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.797+0000 7f5bd3400640 1 -- 172.21.15.3:0/321308135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4032660 con 0x7f5bd41521d0 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 -- 172.21.15.3:0/321308135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 msgr2=0x7f5bd40cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd40cf150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097a0 tx=0x7f5bc402a0f0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 -- 172.21.15.3:0/321308135 shutdown_connections 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 --2- 172.21.15.3:0/321308135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd40cf150 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 -- 172.21.15.3:0/321308135 >> 172.21.15.3:0/321308135 conn(0x7f5bd40b1e90 msgr2=0x7f5bd40b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 -- 172.21.15.3:0/321308135 shutdown_connections 2024-09-10T07:48:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.798+0000 7f5bd9600640 1 -- 172.21.15.3:0/321308135 wait complete. 2024-09-10T07:48:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.800+0000 7f5bd9600640 1 Processor -- start 2024-09-10T07:48:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.800+0000 7f5bd9600640 1 -- start start 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd41f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd40d0130 0x7f5bd4154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41525b0 con 0x7f5bd41521d0 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40d0ed0 con 0x7f5bd40d0130 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd9600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd4152730 con 0x7f5bd40b48a0 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd41f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bcb400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd40d0130 0x7f5bd4154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50914/0 (socket says 172.21.15.3:50914) 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.801+0000 7f5bd8c00640 1 -- 172.21.15.3:0/2822882319 learned_addr learned my addr 172.21.15.3:0/2822882319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 -- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 msgr2=0x7f5bd41f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd41f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 -- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd40d0130 msgr2=0x7f5bd4154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd40d0130 0x7f5bd4154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 -- 172.21.15.3:0/2822882319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd4155260 con 0x7f5bd41521d0 2024-09-10T07:48:50.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-10T07:48:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd8c00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00cbf0 tx=0x7f5bcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc017070 con 0x7f5bd41521d0 2024-09-10T07:48:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc00ad40 con 0x7f5bd41521d0 2024-09-10T07:48:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc012600 con 0x7f5bd41521d0 2024-09-10T07:48:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd4071360 con 0x7f5bd41521d0 2024-09-10T07:48:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.802+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd4071920 con 0x7f5bd41521d0 2024-09-10T07:48:50.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.803+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd40d0580 con 0x7f5bd41521d0 2024-09-10T07:48:50.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.805+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bcc01a070 con 0x7f5bd41521d0 2024-09-10T07:48:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.806+0000 7f5bd0c00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 0x7f5bb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.806+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5609+0+0 (secure 0 0 0) 0x7f5bcc08af30 con 0x7f5bd41521d0 2024-09-10T07:48:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.807+0000 7f5bd3e00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 0x7f5bb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.807+0000 7f5bd3e00640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 0x7f5bb006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4034000 tx=0x7f5bc402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:50.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:50.810+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5bcc0b9d50 con 0x7f5bd41521d0 2024-09-10T07:48:51.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.030+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5bd415b1d0 con 0x7f5bd41521d0 2024-09-10T07:48:51.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:48:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.611+0000 7f5bd0c00640 1 -- 172.21.15.3:0/2822882319 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v30) ==== 172+0+0 (secure 0 0 0) 0x7f5bcc059360 con 0x7f5bd41521d0 2024-09-10T07:48:51.612 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T07:48:51.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:48:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.622+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 msgr2=0x7f5bb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.622+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 0x7f5bb006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4034000 tx=0x7f5bc402e000 comp rx=0 tx=0).stop 2024-09-10T07:48:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.622+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 msgr2=0x7f5bd41462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.622+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00cbf0 tx=0x7f5bcc00a430 comp rx=0 tx=0).stop 2024-09-10T07:48:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 shutdown_connections 2024-09-10T07:48:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5bb006be30 0x7f5bb006e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bd41521d0 0x7f5bd41462e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd40d0130 0x7f5bd4154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 --2- 172.21.15.3:0/2822882319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5bd40b48a0 0x7f5bd41f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.623+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 >> 172.21.15.3:0/2822882319 conn(0x7f5bd40b1e90 msgr2=0x7f5bd415f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.626+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 shutdown_connections 2024-09-10T07:48:51.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.626+0000 7f5bd9600640 1 -- 172.21.15.3:0/2822882319 wait complete. 2024-09-10T07:48:51.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.853+0000 7fb837a00640 1 Processor -- start 2024-09-10T07:48:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.853+0000 7fb837a00640 1 -- start start 2024-09-10T07:48:51.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.854+0000 7fb837a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 0x7fb8300cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:51.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.854+0000 7fb837a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300cff10 con 0x7fb8300d16e0 2024-09-10T07:48:51.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.854+0000 7fb837a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300d1d90 con 0x7fb8300b6a70 2024-09-10T07:48:51.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.854+0000 7fb837a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb8300d1b70 con 0x7fb8300b6690 2024-09-10T07:48:51.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb8300d16e0 0x7fb8300d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44546/0 (socket says 172.21.15.3:44546) 2024-09-10T07:48:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835a00640 1 -- 172.21.15.3:0/2856829649 learned_addr learned my addr 172.21.15.3:0/2856829649 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835000640 1 --2- 172.21.15.3:0/2856829649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 0x7fb8300cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777461505 0 0) 0x7fb8300cff10 con 0x7fb8300d16e0 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb818003520 con 0x7fb8300d16e0 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63080345 0 0) 0x7fb8300d1d90 con 0x7fb8300b6a70 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8300cff10 con 0x7fb8300b6a70 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835000640 1 -- 172.21.15.3:0/2856829649 >> v1:172.21.15.70:6789/0 conn(0x7fb8300b6a70 legacy=0x7fb8300d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835000640 1 -- 172.21.15.3:0/2856829649 >> v1:172.21.15.3:6789/0 conn(0x7fb8300d16e0 legacy=0x7fb8300d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.855+0000 7fb835000640 1 -- 172.21.15.3:0/2856829649 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8300ceef0 con 0x7fb8300b6690 2024-09-10T07:48:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.856+0000 7fb835000640 1 --2- 172.21.15.3:0/2856829649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 0x7fb8300cf9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097d0 tx=0x7fb82402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f2528278454b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824037070 con 0x7fb8300b6690 2024-09-10T07:48:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824032440 con 0x7fb8300b6690 2024-09-10T07:48:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb82f400640 1 -- 172.21.15.3:0/2856829649 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824032810 con 0x7fb8300b6690 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 -- 172.21.15.3:0/2856829649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 msgr2=0x7fb8300cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 --2- 172.21.15.3:0/2856829649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 0x7fb8300cf9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097d0 tx=0x7fb82402a210 comp rx=0 tx=0).stop 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 -- 172.21.15.3:0/2856829649 shutdown_connections 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 --2- 172.21.15.3:0/2856829649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300b6690 0x7fb8300cf9d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 -- 172.21.15.3:0/2856829649 >> 172.21.15.3:0/2856829649 conn(0x7fb830072560 msgr2=0x7fb830072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 -- 172.21.15.3:0/2856829649 shutdown_connections 2024-09-10T07:48:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.857+0000 7fb837a00640 1 -- 172.21.15.3:0/2856829649 wait complete. 2024-09-10T07:48:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.859+0000 7fb837a00640 1 Processor -- start 2024-09-10T07:48:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.859+0000 7fb837a00640 1 -- start start 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 0x7fb8300b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 0x7fb8300ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300b6ed0 con 0x7fb8300b6690 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300cf540 con 0x7fb8300b6a70 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb837a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb83015c530 con 0x7fb8300d16e0 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb835000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb835000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50916/0 (socket says 172.21.15.3:50916) 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb835000640 1 -- 172.21.15.3:0/1953950988 learned_addr learned my addr 172.21.15.3:0/1953950988 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb82fe00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 0x7fb8300b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.860+0000 7fb835a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 0x7fb8300ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 -- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 msgr2=0x7fb8300ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 0x7fb8300ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 -- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 msgr2=0x7fb8300b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 0x7fb8300b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 -- 172.21.15.3:0/1953950988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8300cb900 con 0x7fb8300b6690 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 0x7fb8300ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb82fe00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 0x7fb8300b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb835000640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb824035040 tx=0x7fb8240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824041020 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82403d720 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824037040 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8300cbbe0 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.861+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8300cc1a0 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.862+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830076190 con 0x7fb8300b6690 2024-09-10T07:48:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.863+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb824044020 con 0x7fb8300b6690 2024-09-10T07:48:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.865+0000 7fb82cc00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 0x7fb81006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.865+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fb8240ab500 con 0x7fb8300b6690 2024-09-10T07:48:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.865+0000 7fb82fe00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 0x7fb81006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.865+0000 7fb82fe00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 0x7fb81006e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8301f7ac0 tx=0x7fb81c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:51.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:51.869+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb830076190 con 0x7fb8300b6690 2024-09-10T07:48:52.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.056+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8300cf9d0 con 0x7fb8300b6690 2024-09-10T07:48:52.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.057+0000 7fb82cc00640 1 -- 172.21.15.3:0/1953950988 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16610 (secure 0 0 0) 0x7fb82404b030 con 0x7fb8300b6690 2024-09-10T07:48:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:52.057 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":49,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:51.031661+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:48:48.829038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T07:48:50.029907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 msgr2=0x7fb81006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 0x7fb81006e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8301f7ac0 tx=0x7fb81c009040 comp rx=0 tx=0).stop 2024-09-10T07:48:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 msgr2=0x7fb8300b9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb824035040 tx=0x7fb8240046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 shutdown_connections 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb81006be30 0x7fb81006e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8300d16e0 0x7fb8300ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8300b6a70 0x7fb8300b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 --2- 172.21.15.3:0/1953950988 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8300b6690 0x7fb8300b9470 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.059+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 >> 172.21.15.3:0/1953950988 conn(0x7fb830072560 msgr2=0x7fb8300d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.060+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 shutdown_connections 2024-09-10T07:48:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.060+0000 7fb837a00640 1 -- 172.21.15.3:0/1953950988 wait complete. 2024-09-10T07:48:52.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.231+0000 7f93da000640 1 Processor -- start 2024-09-10T07:48:52.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.231+0000 7f93da000640 1 -- start start 2024-09-10T07:48:52.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93da000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93d406c230 0x7f93d40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93da000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40b36a0 con 0x7f93d4161b30 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93da000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40b3980 con 0x7f93d415fdb0 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93da000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40b3c60 con 0x7f93d406c230 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d3400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f93d4161b30 0x7f93d415f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44556/0 (socket says 172.21.15.3:44556) 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d3400640 1 -- 172.21.15.3:0/4138296475 learned_addr learned my addr 172.21.15.3:0/4138296475 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d3e00640 1 --2- 172.21.15.3:0/4138296475 >> v2:172.21.15.70:3300/0 conn(0x7f93d406c230 0x7f93d40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573981190 0 0) 0x7f93d40b36a0 con 0x7f93d4161b30 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93b8003520 con 0x7f93d4161b30 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265454504 0 0) 0x7f93d40b3980 con 0x7f93d415fdb0 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.233+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d40b36a0 con 0x7f93d415fdb0 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.234+0000 7f93d3e00640 1 -- 172.21.15.3:0/4138296475 >> v1:172.21.15.70:6789/0 conn(0x7f93d415fdb0 legacy=0x7f93d406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.234+0000 7f93d3e00640 1 -- 172.21.15.3:0/4138296475 >> v1:172.21.15.3:6789/0 conn(0x7f93d4161b30 legacy=0x7f93d415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.234+0000 7f93d3e00640 1 -- 172.21.15.3:0/4138296475 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40b3f40 con 0x7f93d406c230 2024-09-10T07:48:52.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93d3e00640 1 --2- 172.21.15.3:0/4138296475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40b4740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa099bb931a7619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc007740 con 0x7f93d406c230 2024-09-10T07:48:52.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc0078e0 con 0x7f93d406c230 2024-09-10T07:48:52.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93d2a00640 1 -- 172.21.15.3:0/4138296475 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc007cb0 con 0x7f93d406c230 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93da000640 1 -- 172.21.15.3:0/4138296475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 msgr2=0x7f93d40b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.235+0000 7f93da000640 1 --2- 172.21.15.3:0/4138296475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40b4740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc02a6f0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.236+0000 7f93da000640 1 -- 172.21.15.3:0/4138296475 shutdown_connections 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.236+0000 7f93da000640 1 --2- 172.21.15.3:0/4138296475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40b4740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.236+0000 7f93da000640 1 -- 172.21.15.3:0/4138296475 >> 172.21.15.3:0/4138296475 conn(0x7f93d406d000 msgr2=0x7f93d406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.236+0000 7f93da000640 1 -- 172.21.15.3:0/4138296475 shutdown_connections 2024-09-10T07:48:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.236+0000 7f93da000640 1 -- 172.21.15.3:0/4138296475 wait complete. 2024-09-10T07:48:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.237+0000 7f93da000640 1 Processor -- start 2024-09-10T07:48:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.237+0000 7f93da000640 1 -- start start 2024-09-10T07:48:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 0x7f93d40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40aefa0 con 0x7f93d4161b30 2024-09-10T07:48:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4161f10 con 0x7f93d406c230 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93da000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40b3300 con 0x7f93d415fdb0 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.238+0000 7f93d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50932/0 (socket says 172.21.15.3:50932) 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 -- 172.21.15.3:0/662550204 learned_addr learned my addr 172.21.15.3:0/662550204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93cbe00640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 0x7f93d40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3400640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 -- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 msgr2=0x7f93d40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 0x7f93d40cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 -- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 msgr2=0x7f93d40738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 -- 172.21.15.3:0/662550204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40cfe00 con 0x7f93d4161b30 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93cbe00640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 0x7f93d40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.239+0000 7f93d3e00640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009eb0 tx=0x7f93cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.240+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03f070 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.240+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc031bc0 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.240+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc009450 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.240+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d40d00a0 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.240+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d4071100 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.241+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d40af460 con 0x7f93d4161b30 2024-09-10T07:48:52.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.242+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93cc03b410 con 0x7f93d4161b30 2024-09-10T07:48:52.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.242+0000 7f93cb400640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 0x7f93b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.243+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f93cc0abf20 con 0x7f93d4161b30 2024-09-10T07:48:52.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.243+0000 7f93d3400640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 0x7f93b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.243+0000 7f93d3400640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 0x7f93b006e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93c4000f60 tx=0x7f93c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.247+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f93d40af460 con 0x7f93d4161b30 2024-09-10T07:48:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.434+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93d406c940 con 0x7f93d4161b30 2024-09-10T07:48:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.435+0000 7f93cb400640 1 -- 172.21.15.3:0/662550204 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16610 (secure 0 0 0) 0x7f93cc07a830 con 0x7f93d4161b30 2024-09-10T07:48:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:52.435 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":49,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:51.031661+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:48:48.829038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T07:48:50.029907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 msgr2=0x7f93b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 0x7f93b006e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93c4000f60 tx=0x7f93c4009040 comp rx=0 tx=0).stop 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 msgr2=0x7f93d414c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009eb0 tx=0x7f93cc004a40 comp rx=0 tx=0).stop 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 shutdown_connections 2024-09-10T07:48:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93b006be60 0x7f93b006e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93d4161b30 0x7f93d414c340 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93d415fdb0 0x7f93d40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 --2- 172.21.15.3:0/662550204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93d406c230 0x7f93d40738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 >> 172.21.15.3:0/662550204 conn(0x7f93d406d000 msgr2=0x7f93d406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.437+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 shutdown_connections 2024-09-10T07:48:52.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.438+0000 7f93da000640 1 -- 172.21.15.3:0/662550204 wait complete. 2024-09-10T07:48:52.453 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:48:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.616+0000 7f8cde800640 1 Processor -- start 2024-09-10T07:48:52.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.616+0000 7f8cde800640 1 -- start start 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cde800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 0x7f8cd80b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cde800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd81620f0 con 0x7f8cd8161d10 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cde800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd80d3f40 con 0x7f8cd80cf6a0 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cde800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd8072140 con 0x7f8cd80d3b20 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cd7400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8cd8161d10 0x7f8cd80cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44570/0 (socket says 172.21.15.3:44570) 2024-09-10T07:48:52.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cd7400640 1 -- 172.21.15.3:0/3247141641 learned_addr learned my addr 172.21.15.3:0/3247141641 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.617+0000 7f8cd7e00640 1 --2- 172.21.15.3:0/3247141641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 0x7f8cd80b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.618+0000 7f8cd6000640 1 -- 172.21.15.3:0/3247141641 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7990908 0 0) 0x7f8cd81620f0 con 0x7f8cd8161d10 2024-09-10T07:48:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.618+0000 7f8cd7e00640 1 -- 172.21.15.3:0/3247141641 >> v1:172.21.15.70:6789/0 conn(0x7f8cd80cf6a0 legacy=0x7f8cd8072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.618+0000 7f8cd7e00640 1 -- 172.21.15.3:0/3247141641 >> v1:172.21.15.3:6789/0 conn(0x7f8cd8161d10 legacy=0x7f8cd80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.618+0000 7f8cd7e00640 1 -- 172.21.15.3:0/3247141641 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd80d4200 con 0x7f8cd80d3b20 2024-09-10T07:48:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.619+0000 7f8cd7e00640 1 --2- 172.21.15.3:0/3247141641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 0x7f8cd80b76c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009dc0 tx=0x7f8cd002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc693116ca89c41a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.619+0000 7f8cd6000640 1 -- 172.21.15.3:0/3247141641 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd002ae60 con 0x7f8cd80d3b20 2024-09-10T07:48:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.619+0000 7f8cd6000640 1 -- 172.21.15.3:0/3247141641 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd0007740 con 0x7f8cd80d3b20 2024-09-10T07:48:52.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.619+0000 7f8cd6000640 1 -- 172.21.15.3:0/3247141641 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0007b10 con 0x7f8cd80d3b20 2024-09-10T07:48:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 -- 172.21.15.3:0/3247141641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 msgr2=0x7f8cd80b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 --2- 172.21.15.3:0/3247141641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 0x7f8cd80b76c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009dc0 tx=0x7f8cd002a640 comp rx=0 tx=0).stop 2024-09-10T07:48:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 -- 172.21.15.3:0/3247141641 shutdown_connections 2024-09-10T07:48:52.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 --2- 172.21.15.3:0/3247141641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd80d3b20 0x7f8cd80b76c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 -- 172.21.15.3:0/3247141641 >> 172.21.15.3:0/3247141641 conn(0x7f8cd80d4730 msgr2=0x7f8cd80d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:52.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 -- 172.21.15.3:0/3247141641 shutdown_connections 2024-09-10T07:48:52.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.620+0000 7f8cde800640 1 -- 172.21.15.3:0/3247141641 wait complete. 2024-09-10T07:48:52.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.622+0000 7f8cde800640 1 Processor -- start 2024-09-10T07:48:52.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.622+0000 7f8cde800640 1 -- start start 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd80cf6a0 0x7f8cd80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd8161d10 0x7f8cd80ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8072950 con 0x7f8cd80d3b20 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8072670 con 0x7f8cd80cf6a0 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cde800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd80d3280 con 0x7f8cd8161d10 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cd6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cd6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50946/0 (socket says 172.21.15.3:50946) 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cd6a00640 1 -- 172.21.15.3:0/654253868 learned_addr learned my addr 172.21.15.3:0/654253868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cd7400640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd80cf6a0 0x7f8cd80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.623+0000 7f8cd7e00640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd8161d10 0x7f8cd80ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 -- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd8161d10 msgr2=0x7f8cd80ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd8161d10 0x7f8cd80ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 -- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd80cf6a0 msgr2=0x7f8cd80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd80cf6a0 0x7f8cd80b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 -- 172.21.15.3:0/654253868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd8151b40 con 0x7f8cd80d3b20 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cd6a00640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c450 tx=0x7f8cc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40052d0 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc4005470 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4012870 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd8151e20 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd81523e0 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.625+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd80d36a0 con 0x7f8cd80d3b20 2024-09-10T07:48:52.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.626+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cc4012a10 con 0x7f8cd80d3b20 2024-09-10T07:48:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.627+0000 7f8ccf400640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.628+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f8cc408aa40 con 0x7f8cd80d3b20 2024-09-10T07:48:52.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.628+0000 7f8cd7400640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.628+0000 7f8cd7400640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 0x7f8cb806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8007fd0 tx=0x7f8cc8007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.632+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8cc4058c70 con 0x7f8cd80d3b20 2024-09-10T07:48:52.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:52.852+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8cd80c8000 con 0x7f8cd80d3b20 2024-09-10T07:48:53.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.621+0000 7f8ccf400640 1 -- 172.21.15.3:0/654253868 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f8cc405cf70 con 0x7f8cd80d3b20 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 msgr2=0x7f8cb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 0x7f8cb806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8007fd0 tx=0x7f8cc8007de0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 msgr2=0x7f8cd80ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c450 tx=0x7f8cc400c910 comp rx=0 tx=0).stop 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 shutdown_connections 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cb806be30 0x7f8cb806e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8cd8161d10 0x7f8cd80ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd80d3b20 0x7f8cd80ae170 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 --2- 172.21.15.3:0/654253868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd80cf6a0 0x7f8cd80b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 >> 172.21.15.3:0/654253868 conn(0x7f8cd80d4730 msgr2=0x7f8cd80b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 shutdown_connections 2024-09-10T07:48:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.624+0000 7f8cde800640 1 -- 172.21.15.3:0/654253868 wait complete. 2024-09-10T07:48:53.643 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:48:53.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:48:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.807+0000 7f144be00640 1 Processor -- start 2024-09-10T07:48:53.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.807+0000 7f144be00640 1 -- start start 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f144c0d45a0 0x7f144c0d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 0x7f144c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144be00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f144c0d4f00 con 0x7f144c161b70 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144be00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f144c161f50 con 0x7f144c0d45a0 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f144c0d3de0 con 0x7f144c0cfe90 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.808+0000 7f144aa00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f144c161b70 0x7f144c0d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37058/0 (socket says 172.21.15.3:37058) 2024-09-10T07:48:53.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144aa00640 1 -- 172.21.15.3:0/3047138247 learned_addr learned my addr 172.21.15.3:0/3047138247 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:53.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144a000640 1 --2- 172.21.15.3:0/3047138247 >> v2:172.21.15.70:3300/0 conn(0x7f144c0d45a0 0x7f144c0d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144b400640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 0x7f144c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f1449600640 1 -- 172.21.15.3:0/3047138247 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401383066 0 0) 0x7f144c0d4f00 con 0x7f144c161b70 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f1449600640 1 -- 172.21.15.3:0/3047138247 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1430003520 con 0x7f144c161b70 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144a000640 1 -- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 msgr2=0x7f144c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144a000640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 0x7f144c0d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144a000640 1 -- 172.21.15.3:0/3047138247 >> v1:172.21.15.3:6789/0 conn(0x7f144c161b70 legacy=0x7f144c0d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144a000640 1 -- 172.21.15.3:0/3047138247 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c0d40c0 con 0x7f144c0d45a0 2024-09-10T07:48:53.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.809+0000 7f144b400640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 0x7f144c0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:53.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.810+0000 7f144a000640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c0d45a0 0x7f144c0d49c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14380097a0 tx=0x7f143802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f11531f588c8db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.811+0000 7f1449600640 1 -- 172.21.15.3:0/3047138247 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438037070 con 0x7f144c0d45a0 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.811+0000 7f1449600640 1 -- 172.21.15.3:0/3047138247 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1438032440 con 0x7f144c0d45a0 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.811+0000 7f1449600640 1 -- 172.21.15.3:0/3047138247 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380327f0 con 0x7f144c0d45a0 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.811+0000 7f144be00640 1 -- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c0d45a0 msgr2=0x7f144c0d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.811+0000 7f144be00640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c0d45a0 0x7f144c0d49c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14380097a0 tx=0x7f143802a180 comp rx=0 tx=0).stop 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 -- 172.21.15.3:0/3047138247 shutdown_connections 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0cfe90 0x7f144c0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 --2- 172.21.15.3:0/3047138247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c0d45a0 0x7f144c0d49c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 -- 172.21.15.3:0/3047138247 >> 172.21.15.3:0/3047138247 conn(0x7f144c062630 msgr2=0x7f144c062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 -- 172.21.15.3:0/3047138247 shutdown_connections 2024-09-10T07:48:53.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.812+0000 7f144be00640 1 -- 172.21.15.3:0/3047138247 wait complete. 2024-09-10T07:48:53.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.813+0000 7f144be00640 1 Processor -- start 2024-09-10T07:48:53.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.814+0000 7f144be00640 1 -- start start 2024-09-10T07:48:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.814+0000 7f144be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.814+0000 7f144be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0d45a0 0x7f144c1f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.814+0000 7f144be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c161b70 0x7f144c1f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0c9fc0 con 0x7f144c0cfe90 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0cf970 con 0x7f144c161b70 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f144c0d3a30 con 0x7f144c0d45a0 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54582/0 (socket says 172.21.15.3:54582) 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 -- 172.21.15.3:0/2832035362 learned_addr learned my addr 172.21.15.3:0/2832035362 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144b400640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c161b70 0x7f144c1f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144a000640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0d45a0 0x7f144c1f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 -- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0d45a0 msgr2=0x7f144c1f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0d45a0 0x7f144c1f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 -- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c161b70 msgr2=0x7f144c1f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c161b70 0x7f144c1f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 -- 172.21.15.3:0/2832035362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c1f4fd0 con 0x7f144c0cfe90 2024-09-10T07:48:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.815+0000 7f144aa00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f143c00c390 tx=0x7f143c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.816+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c017070 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.816+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f143c005ce0 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.816+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1f7380 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.816+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0072f0 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.816+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c0b5890 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.817+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c0cf2d0 con 0x7f144c0cfe90 2024-09-10T07:48:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.818+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f143c01a070 con 0x7f144c0cfe90 2024-09-10T07:48:53.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.819+0000 7f1442a00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 0x7f142c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:53.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.819+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f143c08a2b0 con 0x7f144c0cfe90 2024-09-10T07:48:53.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.819+0000 7f144a000640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 0x7f142c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.819+0000 7f144a000640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 0x7f142c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1438034000 tx=0x7f143802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:53.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:53.823+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f143c058c50 con 0x7f144c0cfe90 2024-09-10T07:48:54.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.049+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f144c1610c0 con 0x7f144c0cfe90 2024-09-10T07:48:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.623+0000 7f1442a00640 1 -- 172.21.15.3:0/2832035362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f143c058550 con 0x7f144c0cfe90 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 msgr2=0x7f142c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 0x7f142c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1438034000 tx=0x7f143802e000 comp rx=0 tx=0).stop 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 msgr2=0x7f144c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f143c00c390 tx=0x7f143c00c850 comp rx=0 tx=0).stop 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 shutdown_connections 2024-09-10T07:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f142c06be30 0x7f142c06e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f144c161b70 0x7f144c1f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f144c0d45a0 0x7f144c1f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 --2- 172.21.15.3:0/2832035362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0cfe90 0x7f144c0b4ea0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 >> 172.21.15.3:0/2832035362 conn(0x7f144c062630 msgr2=0x7f144c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 shutdown_connections 2024-09-10T07:48:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.626+0000 7f144be00640 1 -- 172.21.15.3:0/2832035362 wait complete. 2024-09-10T07:48:54.643 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:48:54.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:48:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.801+0000 7f9e55a00640 1 Processor -- start 2024-09-10T07:48:54.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.802+0000 7f9e55a00640 1 -- start start 2024-09-10T07:48:54.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e55a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e5015b4f0 0x7f9e500d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e500d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e55a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e500ccd80 con 0x7f9e5015b110 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e55a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9e500b44f0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e55a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e500cd060 con 0x7f9e50073d10 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e47e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e5015b4f0 0x7f9e500d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e4ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9e5015b110 0x7f9e500b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37064/0 (socket says 172.21.15.3:37064) 2024-09-10T07:48:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e4ea00640 1 -- 172.21.15.3:0/3529389768 learned_addr learned my addr 172.21.15.3:0/3529389768 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e4f400640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e500d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e47e00640 1 -- 172.21.15.3:0/3529389768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 msgr2=0x7f9e500d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e47e00640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e500d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e47e00640 1 -- 172.21.15.3:0/3529389768 >> v1:172.21.15.3:6789/0 conn(0x7f9e5015b110 legacy=0x7f9e500b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.803+0000 7f9e47e00640 1 -- 172.21.15.3:0/3529389768 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e500cd340 con 0x7f9e5015b4f0 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e47e00640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e500d01e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097a0 tx=0x7f9e3802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c5ab3a05cc03d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e4e000640 1 -- 172.21.15.3:0/3529389768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38037070 con 0x7f9e5015b4f0 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e4e000640 1 -- 172.21.15.3:0/3529389768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3802ae40 con 0x7f9e5015b4f0 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e4e000640 1 -- 172.21.15.3:0/3529389768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38032660 con 0x7f9e5015b4f0 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e55a00640 1 -- 172.21.15.3:0/3529389768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 msgr2=0x7f9e500d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e55a00640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e500d01e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097a0 tx=0x7f9e3802a0f0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.805+0000 7f9e55a00640 1 -- 172.21.15.3:0/3529389768 shutdown_connections 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.806+0000 7f9e55a00640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e500d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.806+0000 7f9e55a00640 1 --2- 172.21.15.3:0/3529389768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e500d01e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.806+0000 7f9e55a00640 1 -- 172.21.15.3:0/3529389768 >> 172.21.15.3:0/3529389768 conn(0x7f9e5006d000 msgr2=0x7f9e5006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.806+0000 7f9e55a00640 1 -- 172.21.15.3:0/3529389768 shutdown_connections 2024-09-10T07:48:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.806+0000 7f9e55a00640 1 -- 172.21.15.3:0/3529389768 wait complete. 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.807+0000 7f9e55a00640 1 Processor -- start 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.807+0000 7f9e55a00640 1 -- start start 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e50157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e5015b110 0x7f9e5014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e500cdcc0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9e500cdb40 con 0x7f9e5015b110 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.808+0000 7f9e55a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e500cde40 con 0x7f9e50073d10 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54606/0 (socket says 172.21.15.3:54606) 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 -- 172.21.15.3:0/1576746289 learned_addr learned my addr 172.21.15.3:0/1576746289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4ea00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e50157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e47e00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e5015b110 0x7f9e5014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 -- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 msgr2=0x7f9e50157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e50157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 -- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e5015b110 msgr2=0x7f9e5014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e5015b110 0x7f9e5014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 -- 172.21.15.3:0/1576746289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e50071e80 con 0x7f9e5015b4f0 2024-09-10T07:48:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.809+0000 7f9e4f400640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9e48007e70 tx=0x7f9e4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.810+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48014620 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.810+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48019070 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.810+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48014a20 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.810+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e500aebb0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.810+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e50072100 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.811+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e500b3cf0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.812+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e48014bc0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.812+0000 7f9e46a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 0x7f9e2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.813+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f9e4808b5c0 con 0x7f9e5015b4f0 2024-09-10T07:48:54.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.813+0000 7f9e4ea00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 0x7f9e2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:54.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.813+0000 7f9e4ea00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 0x7f9e2c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e40007fd0 tx=0x7f9e40007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:54.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:54.817+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9e50071e80 con 0x7f9e5015b4f0 2024-09-10T07:48:55.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.038+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e5006c3c0 con 0x7f9e5015b4f0 2024-09-10T07:48:55.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.039+0000 7f9e46a00640 1 -- 172.21.15.3:0/1576746289 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5211 (secure 0 0 0) 0x7f9e480599b0 con 0x7f9e5015b4f0 2024-09-10T07:48:55.039 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:55.039 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":34,"btime":"2024-09-10T07:48:54:622365+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:48:54.051392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-10T07:48:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 34 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 msgr2=0x7f9e2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 0x7f9e2c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e40007fd0 tx=0x7f9e40007e80 comp rx=0 tx=0).stop 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 msgr2=0x7f9e50159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9e48007e70 tx=0x7f9e4800e4b0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 shutdown_connections 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9e2c06be30 0x7f9e2c06e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e5015b4f0 0x7f9e50159590 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e5015b110 0x7f9e5014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 --2- 172.21.15.3:0/1576746289 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e50073d10 0x7f9e50157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 >> 172.21.15.3:0/1576746289 conn(0x7f9e5006d000 msgr2=0x7f9e5015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 shutdown_connections 2024-09-10T07:48:55.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.041+0000 7f9e55a00640 1 -- 172.21.15.3:0/1576746289 wait complete. 2024-09-10T07:48:55.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:48:55.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.212+0000 7fa798800640 1 Processor -- start 2024-09-10T07:48:55.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.212+0000 7fa798800640 1 -- start start 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.213+0000 7fa798800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa798800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7900ccd80 con 0x7fa79015b110 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa798800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7900cd060 con 0x7fa790073d10 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa798800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7900b44f0 con 0x7fa79015b4f0 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa79015b110 0x7fa7900b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37068/0 (socket says 172.21.15.3:37068) 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795e00640 1 -- 172.21.15.3:0/1996215219 learned_addr learned my addr 172.21.15.3:0/1996215219 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795400640 1 --2- 172.21.15.3:0/1996215219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303559203 0 0) 0x7fa7900ccd80 con 0x7fa79015b110 2024-09-10T07:48:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa778003520 con 0x7fa79015b110 2024-09-10T07:48:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93361330 0 0) 0x7fa7900cd060 con 0x7fa790073d10 2024-09-10T07:48:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795400640 1 -- 172.21.15.3:0/1996215219 >> v1:172.21.15.70:6789/0 conn(0x7fa790073d10 legacy=0x7fa7900cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795400640 1 -- 172.21.15.3:0/1996215219 >> v1:172.21.15.3:6789/0 conn(0x7fa79015b110 legacy=0x7fa7900b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.214+0000 7fa795400640 1 -- 172.21.15.3:0/1996215219 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7900cd340 con 0x7fa79015b4f0 2024-09-10T07:48:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa795400640 1 --2- 172.21.15.3:0/1996215219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900d01e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa7900b4670 tx=0x7fa78002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677e71169e6ce668 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780037070 con 0x7fa79015b4f0 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa780032440 con 0x7fa79015b4f0 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa794a00640 1 -- 172.21.15.3:0/1996215219 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780032810 con 0x7fa79015b4f0 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa798800640 1 -- 172.21.15.3:0/1996215219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 msgr2=0x7fa7900d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa798800640 1 --2- 172.21.15.3:0/1996215219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900d01e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa7900b4670 tx=0x7fa78002a200 comp rx=0 tx=0).stop 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa798800640 1 -- 172.21.15.3:0/1996215219 shutdown_connections 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa798800640 1 --2- 172.21.15.3:0/1996215219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900d01e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.216+0000 7fa798800640 1 -- 172.21.15.3:0/1996215219 >> 172.21.15.3:0/1996215219 conn(0x7fa79006d000 msgr2=0x7fa79006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.217+0000 7fa798800640 1 -- 172.21.15.3:0/1996215219 shutdown_connections 2024-09-10T07:48:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.217+0000 7fa798800640 1 -- 172.21.15.3:0/1996215219 wait complete. 2024-09-10T07:48:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.218+0000 7fa798800640 1 Processor -- start 2024-09-10T07:48:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.218+0000 7fa798800640 1 -- start start 2024-09-10T07:48:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa79015b110 0x7fa7900aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900d08a0 con 0x7fa790073d10 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900d0720 con 0x7fa79015b110 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa798800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7900d0a20 con 0x7fa79015b4f0 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa795e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa795e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54614/0 (socket says 172.21.15.3:54614) 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.219+0000 7fa795e00640 1 -- 172.21.15.3:0/3079247943 learned_addr learned my addr 172.21.15.3:0/3079247943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa796800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795400640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa79015b110 0x7fa7900aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 -- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 msgr2=0x7fa7900acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 -- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa79015b110 msgr2=0x7fa7900aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa79015b110 0x7fa7900aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 -- 172.21.15.3:0/3079247943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7901e6670 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.220+0000 7fa795e00640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa78400c430 tx=0x7fa78400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.221+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784017070 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.221+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa784005ce0 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.221+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840072f0 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.221+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7901e6950 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.221+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901e6f10 con 0x7fa790073d10 2024-09-10T07:48:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.222+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7901e7340 con 0x7fa790073d10 2024-09-10T07:48:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.223+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa78401a070 con 0x7fa790073d10 2024-09-10T07:48:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.224+0000 7fa78a000640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 0x7fa77406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.224+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa78408a2e0 con 0x7fa790073d10 2024-09-10T07:48:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.224+0000 7fa795400640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 0x7fa77406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.224+0000 7fa795400640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 0x7fa77406e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa780034000 tx=0x7fa78002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.228+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7840585a0 con 0x7fa790073d10 2024-09-10T07:48:55.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.415+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7900b2bc0 con 0x7fa790073d10 2024-09-10T07:48:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.416+0000 7fa78a000640 1 -- 172.21.15.3:0/3079247943 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16597 (secure 0 0 0) 0x7fa78405c8a0 con 0x7fa790073d10 2024-09-10T07:48:55.416 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:55.416 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":50,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:48:52.601276+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:48:48.829038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T07:48:50.029907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 msgr2=0x7fa77406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 0x7fa77406e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa780034000 tx=0x7fa78002e000 comp rx=0 tx=0).stop 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 msgr2=0x7fa7901565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa78400c430 tx=0x7fa78400c8f0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 shutdown_connections 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa77406be60 0x7fa77406e320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa79015b4f0 0x7fa7900acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa79015b110 0x7fa7900aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 --2- 172.21.15.3:0/3079247943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa790073d10 0x7fa7901565b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 >> 172.21.15.3:0/3079247943 conn(0x7fa79006d000 msgr2=0x7fa7900b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 shutdown_connections 2024-09-10T07:48:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.418+0000 7fa798800640 1 -- 172.21.15.3:0/3079247943 wait complete. 2024-09-10T07:48:55.434 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:48:55.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:48:55.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.591+0000 7f4f7a000640 1 Processor -- start 2024-09-10T07:48:55.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.591+0000 7f4f7a000640 1 -- start start 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.592+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 0x7f4f740b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.592+0000 7f4f7a000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f7415b4f0 0x7f4f740d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f7a000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f740b4250 con 0x7f4f74073d10 2024-09-10T07:48:55.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f7a000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4f740b44f0 con 0x7f4f7415b4f0 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f7a000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f740ccd80 con 0x7f4f7415b110 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4f74073d10 0x7f4f7415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37076/0 (socket says 172.21.15.3:37076) 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73e00640 1 -- 172.21.15.3:0/1969788672 learned_addr learned my addr 172.21.15.3:0/1969788672 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73400640 1 --2- 172.21.15.3:0/1969788672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 0x7f4f740b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f6be00640 1 --2- 172.21.15.3:0/1969788672 >> v2:172.21.15.70:3300/0 conn(0x7f4f7415b4f0 0x7f4f740d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73400640 1 -- 172.21.15.3:0/1969788672 >> v2:172.21.15.70:3300/0 conn(0x7f4f7415b4f0 msgr2=0x7f4f740d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73400640 1 --2- 172.21.15.3:0/1969788672 >> v2:172.21.15.70:3300/0 conn(0x7f4f7415b4f0 0x7f4f740d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73400640 1 -- 172.21.15.3:0/1969788672 >> v1:172.21.15.3:6789/0 conn(0x7f4f74073d10 legacy=0x7f4f7415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f73400640 1 -- 172.21.15.3:0/1969788672 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f740cd060 con 0x7f4f7415b110 2024-09-10T07:48:55.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.593+0000 7f4f72a00640 1 -- 172.21.15.3:0/1969788672 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784504325 0 0) 0x7f4f740b4250 con 0x7f4f74073d10 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f73400640 1 --2- 172.21.15.3:0/1969788672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 0x7f4f740b3d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f640097a0 tx=0x7f4f6402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24fcc280089bf96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f72a00640 1 -- 172.21.15.3:0/1969788672 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64037070 con 0x7f4f7415b110 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f72a00640 1 -- 172.21.15.3:0/1969788672 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f64032440 con 0x7f4f7415b110 2024-09-10T07:48:55.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f72a00640 1 -- 172.21.15.3:0/1969788672 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64032810 con 0x7f4f7415b110 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f7a000640 1 -- 172.21.15.3:0/1969788672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 msgr2=0x7f4f740b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.595+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1969788672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 0x7f4f740b3d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f640097a0 tx=0x7f4f6402a180 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 -- 172.21.15.3:0/1969788672 shutdown_connections 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1969788672 >> v2:172.21.15.70:3300/0 conn(0x7f4f7415b4f0 0x7f4f740d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1969788672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b110 0x7f4f740b3d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 -- 172.21.15.3:0/1969788672 >> 172.21.15.3:0/1969788672 conn(0x7f4f7406d000 msgr2=0x7f4f7406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 -- 172.21.15.3:0/1969788672 shutdown_connections 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.596+0000 7f4f7a000640 1 -- 172.21.15.3:0/1969788672 wait complete. 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.597+0000 7f4f7a000640 1 Processor -- start 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.597+0000 7f4f7a000640 1 -- start start 2024-09-10T07:48:55.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f7415b110 0x7f4f74144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b4f0 0x7f4f74155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f740ceb50 con 0x7f4f74073d10 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f740cd620 con 0x7f4f7415b110 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.598+0000 7f4f7a000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f740cecd0 con 0x7f4f7415b4f0 2024-09-10T07:48:55.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54624/0 (socket says 172.21.15.3:54624) 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 -- 172.21.15.3:0/1779419546 learned_addr learned my addr 172.21.15.3:0/1779419546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f6be00640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f7415b110 0x7f4f74144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73e00640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b4f0 0x7f4f74155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 -- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b4f0 msgr2=0x7f4f74155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b4f0 0x7f4f74155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 -- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f7415b110 msgr2=0x7f4f74144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f7415b110 0x7f4f74144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 -- 172.21.15.3:0/1779419546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f741453f0 con 0x7f4f74073d10 2024-09-10T07:48:55.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.599+0000 7f4f73400640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f64035040 tx=0x7f4f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.600+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64037040 con 0x7f4f74073d10 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.600+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f64030500 con 0x7f4f74073d10 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.600+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f74156dd0 con 0x7f4f74073d10 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.600+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64030910 con 0x7f4f74073d10 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.600+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74157390 con 0x7f4f74073d10 2024-09-10T07:48:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.601+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f740b3d10 con 0x7f4f74073d10 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.602+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f64044020 con 0x7f4f74073d10 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.602+0000 7f4f6b400640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 0x7f4f5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.603+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f4f640aaff0 con 0x7f4f74073d10 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.603+0000 7f4f6be00640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 0x7f4f5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.604+0000 7f4f6be00640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 0x7f4f5006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b17c0 tx=0x7f4f5c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.607+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f741453f0 con 0x7f4f74073d10 2024-09-10T07:48:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.791+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f74069f70 con 0x7f4f5006be30 2024-09-10T07:48:55.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.799+0000 7f4f6b400640 1 -- 172.21.15.3:0/1779419546 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249857 (secure 0 0 0) 0x7f4f74069f70 con 0x7f4f5006be30 2024-09-10T07:48:55.802 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 msgr2=0x7f4f5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 0x7f4f5006e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b17c0 tx=0x7f4f5c009190 comp rx=0 tx=0).stop 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 msgr2=0x7f4f74157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f64035040 tx=0x7f4f640046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 shutdown_connections 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f5006be30 0x7f4f5006e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f7415b4f0 0x7f4f74155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f7415b110 0x7f4f74144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 --2- 172.21.15.3:0/1779419546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f74073d10 0x7f4f74157e70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.802+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 >> 172.21.15.3:0/1779419546 conn(0x7f4f7406d000 msgr2=0x7f4f740aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.803+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 shutdown_connections 2024-09-10T07:48:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:55.803+0000 7f4f7a000640 1 -- 172.21.15.3:0/1779419546 wait complete. 2024-09-10T07:48:55.819 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T07:48:55.520865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238360,"kb_used_data":5476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736360,"statfs":{"total":773094113280,"available":772850032640,"internally_reserved":0,"allocated":5607424,"data_stored":2963918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927936"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621887+0000","last_change":"2024-09-10T07:48:51.617751+0000","last_active":"2024-09-10T07:48:51.621887+0000","last_peered":"2024-09-10T07:48:51.621887+0000","last_clean":"2024-09-10T07:48:51.621887+0000","last_became_active":"2024-09-10T07:48:51.617450+0000","last_became_peered":"2024-09-10T07:48:51.617450+0000","last_unstale":"2024-09-10T07:48:51.621887+0000","last_undegraded":"2024-09-10T07:48:51.621887+0000","last_fullsized":"2024-09-10T07:48:51.621887+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632949+0000","last_change":"2024-09-10T07:48:51.620056+0000","last_active":"2024-09-10T07:48:51.632949+0000","last_peered":"2024-09-10T07:48:51.632949+0000","last_clean":"2024-09-10T07:48:51.632949+0000","last_became_active":"2024-09-10T07:48:51.619801+0000","last_became_peered":"2024-09-10T07:48:51.619801+0000","last_unstale":"2024-09-10T07:48:51.632949+0000","last_undegraded":"2024-09-10T07:48:51.632949+0000","last_fullsized":"2024-09-10T07:48:51.632949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:12.498291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617258+0000","last_change":"2024-09-10T07:48:51.612764+0000","last_active":"2024-09-10T07:48:51.617258+0000","last_peered":"2024-09-10T07:48:51.617258+0000","last_clean":"2024-09-10T07:48:51.617258+0000","last_became_active":"2024-09-10T07:48:51.611550+0000","last_became_peered":"2024-09-10T07:48:51.611550+0000","last_unstale":"2024-09-10T07:48:51.617258+0000","last_undegraded":"2024-09-10T07:48:51.617258+0000","last_fullsized":"2024-09-10T07:48:51.617258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:07.243280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611003+0000","last_change":"2024-09-10T07:48:50.611003+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611003+0000","last_undegraded":"2024-09-10T07:48:50.611003+0000","last_fullsized":"2024-09-10T07:48:50.611003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631377+0000","last_change":"2024-09-10T07:48:51.624149+0000","last_active":"2024-09-10T07:48:51.631377+0000","last_peered":"2024-09-10T07:48:51.631377+0000","last_clean":"2024-09-10T07:48:51.631377+0000","last_became_active":"2024-09-10T07:48:51.623849+0000","last_became_peered":"2024-09-10T07:48:51.623849+0000","last_unstale":"2024-09-10T07:48:51.631377+0000","last_undegraded":"2024-09-10T07:48:51.631377+0000","last_fullsized":"2024-09-10T07:48:51.631377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.308850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608067+0000","last_change":"2024-09-10T07:48:50.608067+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608067+0000","last_undegraded":"2024-09-10T07:48:50.608067+0000","last_fullsized":"2024-09-10T07:48:50.608067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639319+0000","last_change":"2024-09-10T07:48:51.628760+0000","last_active":"2024-09-10T07:48:51.639319+0000","last_peered":"2024-09-10T07:48:51.639319+0000","last_clean":"2024-09-10T07:48:51.639319+0000","last_became_active":"2024-09-10T07:48:51.628462+0000","last_became_peered":"2024-09-10T07:48:51.628462+0000","last_unstale":"2024-09-10T07:48:51.639319+0000","last_undegraded":"2024-09-10T07:48:51.639319+0000","last_fullsized":"2024-09-10T07:48:51.639319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:19.292466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614856+0000","last_change":"2024-09-10T07:48:50.614856+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614856+0000","last_undegraded":"2024-09-10T07:48:50.614856+0000","last_fullsized":"2024-09-10T07:48:50.614856+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609844+0000","last_change":"2024-09-10T07:48:50.609844+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609844+0000","last_undegraded":"2024-09-10T07:48:50.609844+0000","last_fullsized":"2024-09-10T07:48:50.609844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607402+0000","last_change":"2024-09-10T07:48:51.617674+0000","last_active":"2024-09-10T07:48:52.607402+0000","last_peered":"2024-09-10T07:48:52.607402+0000","last_clean":"2024-09-10T07:48:52.607402+0000","last_became_active":"2024-09-10T07:48:51.617378+0000","last_became_peered":"2024-09-10T07:48:51.617378+0000","last_unstale":"2024-09-10T07:48:52.607402+0000","last_undegraded":"2024-09-10T07:48:52.607402+0000","last_fullsized":"2024-09-10T07:48:52.607402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:02.393228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611643+0000","last_change":"2024-09-10T07:48:50.611643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611643+0000","last_undegraded":"2024-09-10T07:48:50.611643+0000","last_fullsized":"2024-09-10T07:48:50.611643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633458+0000","last_change":"2024-09-10T07:48:51.620489+0000","last_active":"2024-09-10T07:48:51.633458+0000","last_peered":"2024-09-10T07:48:51.633458+0000","last_clean":"2024-09-10T07:48:51.633458+0000","last_became_active":"2024-09-10T07:48:51.620237+0000","last_became_peered":"2024-09-10T07:48:51.620237+0000","last_unstale":"2024-09-10T07:48:51.633458+0000","last_undegraded":"2024-09-10T07:48:51.633458+0000","last_fullsized":"2024-09-10T07:48:51.633458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:25.714474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609643+0000","last_change":"2024-09-10T07:48:50.609643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609643+0000","last_undegraded":"2024-09-10T07:48:50.609643+0000","last_fullsized":"2024-09-10T07:48:50.609643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608771+0000","last_change":"2024-09-10T07:48:50.608771+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608771+0000","last_undegraded":"2024-09-10T07:48:50.608771+0000","last_fullsized":"2024-09-10T07:48:50.608771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609579+0000","last_change":"2024-09-10T07:48:50.609579+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609579+0000","last_undegraded":"2024-09-10T07:48:50.609579+0000","last_fullsized":"2024-09-10T07:48:50.609579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637116+0000","last_change":"2024-09-10T07:48:51.631668+0000","last_active":"2024-09-10T07:48:51.637116+0000","last_peered":"2024-09-10T07:48:51.637116+0000","last_clean":"2024-09-10T07:48:51.637116+0000","last_became_active":"2024-09-10T07:48:51.631459+0000","last_became_peered":"2024-09-10T07:48:51.631459+0000","last_unstale":"2024-09-10T07:48:51.637116+0000","last_undegraded":"2024-09-10T07:48:51.637116+0000","last_fullsized":"2024-09-10T07:48:51.637116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:12.993962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632892+0000","last_change":"2024-09-10T07:48:51.622041+0000","last_active":"2024-09-10T07:48:51.632892+0000","last_peered":"2024-09-10T07:48:51.632892+0000","last_clean":"2024-09-10T07:48:51.632892+0000","last_became_active":"2024-09-10T07:48:51.621622+0000","last_became_peered":"2024-09-10T07:48:51.621622+0000","last_unstale":"2024-09-10T07:48:51.632892+0000","last_undegraded":"2024-09-10T07:48:51.632892+0000","last_fullsized":"2024-09-10T07:48:51.632892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:50.994249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638066+0000","last_change":"2024-09-10T07:48:51.632824+0000","last_active":"2024-09-10T07:48:51.638066+0000","last_peered":"2024-09-10T07:48:51.638066+0000","last_clean":"2024-09-10T07:48:51.638066+0000","last_became_active":"2024-09-10T07:48:51.629313+0000","last_became_peered":"2024-09-10T07:48:51.629313+0000","last_unstale":"2024-09-10T07:48:51.638066+0000","last_undegraded":"2024-09-10T07:48:51.638066+0000","last_fullsized":"2024-09-10T07:48:51.638066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:25.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630089+0000","last_change":"2024-09-10T07:48:51.623911+0000","last_active":"2024-09-10T07:48:51.630089+0000","last_peered":"2024-09-10T07:48:51.630089+0000","last_clean":"2024-09-10T07:48:51.630089+0000","last_became_active":"2024-09-10T07:48:51.623645+0000","last_became_peered":"2024-09-10T07:48:51.623645+0000","last_unstale":"2024-09-10T07:48:51.630089+0000","last_undegraded":"2024-09-10T07:48:51.630089+0000","last_fullsized":"2024-09-10T07:48:51.630089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:54.995968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606836+0000","last_change":"2024-09-10T07:48:51.624623+0000","last_active":"2024-09-10T07:48:52.606836+0000","last_peered":"2024-09-10T07:48:52.606836+0000","last_clean":"2024-09-10T07:48:52.606836+0000","last_became_active":"2024-09-10T07:48:51.621627+0000","last_became_peered":"2024-09-10T07:48:51.621627+0000","last_unstale":"2024-09-10T07:48:52.606836+0000","last_undegraded":"2024-09-10T07:48:52.606836+0000","last_fullsized":"2024-09-10T07:48:52.606836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:19.844611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608827+0000","last_change":"2024-09-10T07:48:50.608827+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608827+0000","last_undegraded":"2024-09-10T07:48:50.608827+0000","last_fullsized":"2024-09-10T07:48:50.608827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605350+0000","last_change":"2024-09-10T07:48:50.605350+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605350+0000","last_undegraded":"2024-09-10T07:48:50.605350+0000","last_fullsized":"2024-09-10T07:48:50.605350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.624565+0000","last_change":"2024-09-10T07:48:51.616950+0000","last_active":"2024-09-10T07:48:51.624565+0000","last_peered":"2024-09-10T07:48:51.624565+0000","last_clean":"2024-09-10T07:48:51.624565+0000","last_became_active":"2024-09-10T07:48:51.614672+0000","last_became_peered":"2024-09-10T07:48:51.614672+0000","last_unstale":"2024-09-10T07:48:51.624565+0000","last_undegraded":"2024-09-10T07:48:51.624565+0000","last_fullsized":"2024-09-10T07:48:51.624565+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:48.008128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635731+0000","last_change":"2024-09-10T07:48:51.630986+0000","last_active":"2024-09-10T07:48:51.635731+0000","last_peered":"2024-09-10T07:48:51.635731+0000","last_clean":"2024-09-10T07:48:51.635731+0000","last_became_active":"2024-09-10T07:48:51.630754+0000","last_became_peered":"2024-09-10T07:48:51.630754+0000","last_unstale":"2024-09-10T07:48:51.635731+0000","last_undegraded":"2024-09-10T07:48:51.635731+0000","last_fullsized":"2024-09-10T07:48:51.635731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621864+0000","last_change":"2024-09-10T07:48:51.617701+0000","last_active":"2024-09-10T07:48:51.621864+0000","last_peered":"2024-09-10T07:48:51.621864+0000","last_clean":"2024-09-10T07:48:51.621864+0000","last_became_active":"2024-09-10T07:48:51.617418+0000","last_became_peered":"2024-09-10T07:48:51.617418+0000","last_unstale":"2024-09-10T07:48:51.621864+0000","last_undegraded":"2024-09-10T07:48:51.621864+0000","last_fullsized":"2024-09-10T07:48:51.621864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612838+0000","last_change":"2024-09-10T07:48:50.612838+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612838+0000","last_undegraded":"2024-09-10T07:48:50.612838+0000","last_fullsized":"2024-09-10T07:48:50.612838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630570+0000","last_change":"2024-09-10T07:48:51.620624+0000","last_active":"2024-09-10T07:48:51.630570+0000","last_peered":"2024-09-10T07:48:51.630570+0000","last_clean":"2024-09-10T07:48:51.630570+0000","last_became_active":"2024-09-10T07:48:51.619782+0000","last_became_peered":"2024-09-10T07:48:51.619782+0000","last_unstale":"2024-09-10T07:48:51.630570+0000","last_undegraded":"2024-09-10T07:48:51.630570+0000","last_fullsized":"2024-09-10T07:48:51.630570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.304141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623981+0000","last_change":"2024-09-10T07:48:51.616307+0000","last_active":"2024-09-10T07:48:51.623981+0000","last_peered":"2024-09-10T07:48:51.623981+0000","last_clean":"2024-09-10T07:48:51.623981+0000","last_became_active":"2024-09-10T07:48:51.615851+0000","last_became_peered":"2024-09-10T07:48:51.615851+0000","last_unstale":"2024-09-10T07:48:51.623981+0000","last_undegraded":"2024-09-10T07:48:51.623981+0000","last_fullsized":"2024-09-10T07:48:51.623981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:05.714041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623649+0000","last_change":"2024-09-10T07:48:51.616551+0000","last_active":"2024-09-10T07:48:51.623649+0000","last_peered":"2024-09-10T07:48:51.623649+0000","last_clean":"2024-09-10T07:48:51.623649+0000","last_became_active":"2024-09-10T07:48:51.616241+0000","last_became_peered":"2024-09-10T07:48:51.616241+0000","last_unstale":"2024-09-10T07:48:51.623649+0000","last_undegraded":"2024-09-10T07:48:51.623649+0000","last_fullsized":"2024-09-10T07:48:51.623649+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633988+0000","last_change":"2024-09-10T07:48:51.628969+0000","last_active":"2024-09-10T07:48:51.633988+0000","last_peered":"2024-09-10T07:48:51.633988+0000","last_clean":"2024-09-10T07:48:51.633988+0000","last_became_active":"2024-09-10T07:48:51.628200+0000","last_became_peered":"2024-09-10T07:48:51.628200+0000","last_unstale":"2024-09-10T07:48:51.633988+0000","last_undegraded":"2024-09-10T07:48:51.633988+0000","last_fullsized":"2024-09-10T07:48:51.633988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:12.551381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.613199+0000","last_change":"2024-09-10T07:48:50.613199+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.613199+0000","last_undegraded":"2024-09-10T07:48:50.613199+0000","last_fullsized":"2024-09-10T07:48:50.613199+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614337+0000","last_change":"2024-09-10T07:48:50.614337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614337+0000","last_undegraded":"2024-09-10T07:48:50.614337+0000","last_fullsized":"2024-09-10T07:48:50.614337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607331+0000","last_change":"2024-09-10T07:48:50.607331+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607331+0000","last_undegraded":"2024-09-10T07:48:50.607331+0000","last_fullsized":"2024-09-10T07:48:50.607331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.628793+0000","last_change":"2024-09-10T07:48:50.627953+0000","last_active":"2024-09-10T07:48:50.628793+0000","last_peered":"2024-09-10T07:48:50.628793+0000","last_clean":"2024-09-10T07:48:50.628793+0000","last_became_active":"2024-09-10T07:48:50.627666+0000","last_became_peered":"2024-09-10T07:48:50.627666+0000","last_unstale":"2024-09-10T07:48:50.628793+0000","last_undegraded":"2024-09-10T07:48:50.628793+0000","last_fullsized":"2024-09-10T07:48:50.628793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:57.026249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630003+0000","last_change":"2024-09-10T07:48:51.621847+0000","last_active":"2024-09-10T07:48:51.630003+0000","last_peered":"2024-09-10T07:48:51.630003+0000","last_clean":"2024-09-10T07:48:51.630003+0000","last_became_active":"2024-09-10T07:48:51.621554+0000","last_became_peered":"2024-09-10T07:48:51.621554+0000","last_unstale":"2024-09-10T07:48:51.630003+0000","last_undegraded":"2024-09-10T07:48:51.630003+0000","last_fullsized":"2024-09-10T07:48:51.630003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634481+0000","last_change":"2024-09-10T07:48:50.619796+0000","last_active":"2024-09-10T07:48:51.634481+0000","last_peered":"2024-09-10T07:48:51.634481+0000","last_clean":"2024-09-10T07:48:51.634481+0000","last_became_active":"2024-09-10T07:48:50.619535+0000","last_became_peered":"2024-09-10T07:48:50.619535+0000","last_unstale":"2024-09-10T07:48:51.634481+0000","last_undegraded":"2024-09-10T07:48:51.634481+0000","last_fullsized":"2024-09-10T07:48:51.634481+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:44.514806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633602+0000","last_change":"2024-09-10T07:48:51.629435+0000","last_active":"2024-09-10T07:48:51.633602+0000","last_peered":"2024-09-10T07:48:51.633602+0000","last_clean":"2024-09-10T07:48:51.633602+0000","last_became_active":"2024-09-10T07:48:51.628928+0000","last_became_peered":"2024-09-10T07:48:51.628928+0000","last_unstale":"2024-09-10T07:48:51.633602+0000","last_undegraded":"2024-09-10T07:48:51.633602+0000","last_fullsized":"2024-09-10T07:48:51.633602+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622265+0000","last_change":"2024-09-10T07:48:50.619441+0000","last_active":"2024-09-10T07:48:50.622265+0000","last_peered":"2024-09-10T07:48:50.622265+0000","last_clean":"2024-09-10T07:48:50.622265+0000","last_became_active":"2024-09-10T07:48:50.619148+0000","last_became_peered":"2024-09-10T07:48:50.619148+0000","last_unstale":"2024-09-10T07:48:50.622265+0000","last_undegraded":"2024-09-10T07:48:50.622265+0000","last_fullsized":"2024-09-10T07:48:50.622265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.560270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610493+0000","last_change":"2024-09-10T07:48:50.610493+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610493+0000","last_undegraded":"2024-09-10T07:48:50.610493+0000","last_fullsized":"2024-09-10T07:48:50.610493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639518+0000","last_change":"2024-09-10T07:48:50.617904+0000","last_active":"2024-09-10T07:48:51.639518+0000","last_peered":"2024-09-10T07:48:51.639518+0000","last_clean":"2024-09-10T07:48:51.639518+0000","last_became_active":"2024-09-10T07:48:50.617656+0000","last_became_peered":"2024-09-10T07:48:50.617656+0000","last_unstale":"2024-09-10T07:48:51.639518+0000","last_undegraded":"2024-09-10T07:48:51.639518+0000","last_fullsized":"2024-09-10T07:48:51.639518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.305959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.627946+0000","last_change":"2024-09-10T07:48:51.620554+0000","last_active":"2024-09-10T07:48:51.627946+0000","last_peered":"2024-09-10T07:48:51.627946+0000","last_clean":"2024-09-10T07:48:51.627946+0000","last_became_active":"2024-09-10T07:48:51.619526+0000","last_became_peered":"2024-09-10T07:48:51.619526+0000","last_unstale":"2024-09-10T07:48:51.627946+0000","last_undegraded":"2024-09-10T07:48:51.627946+0000","last_fullsized":"2024-09-10T07:48:51.627946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:33.856786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.625037+0000","last_change":"2024-09-10T07:48:50.623162+0000","last_active":"2024-09-10T07:48:50.625037+0000","last_peered":"2024-09-10T07:48:50.625037+0000","last_clean":"2024-09-10T07:48:50.625037+0000","last_became_active":"2024-09-10T07:48:50.622765+0000","last_became_peered":"2024-09-10T07:48:50.622765+0000","last_unstale":"2024-09-10T07:48:50.625037+0000","last_undegraded":"2024-09-10T07:48:50.625037+0000","last_fullsized":"2024-09-10T07:48:50.625037+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:24.341239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606939+0000","last_change":"2024-09-10T07:48:51.628666+0000","last_active":"2024-09-10T07:48:52.606939+0000","last_peered":"2024-09-10T07:48:52.606939+0000","last_clean":"2024-09-10T07:48:52.606939+0000","last_became_active":"2024-09-10T07:48:51.628411+0000","last_became_peered":"2024-09-10T07:48:51.628411+0000","last_unstale":"2024-09-10T07:48:52.606939+0000","last_undegraded":"2024-09-10T07:48:52.606939+0000","last_fullsized":"2024-09-10T07:48:52.606939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:13.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635593+0000","last_change":"2024-09-10T07:48:50.616830+0000","last_active":"2024-09-10T07:48:51.635593+0000","last_peered":"2024-09-10T07:48:51.635593+0000","last_clean":"2024-09-10T07:48:51.635593+0000","last_became_active":"2024-09-10T07:48:50.616586+0000","last_became_peered":"2024-09-10T07:48:50.616586+0000","last_unstale":"2024-09-10T07:48:51.635593+0000","last_undegraded":"2024-09-10T07:48:51.635593+0000","last_fullsized":"2024-09-10T07:48:51.635593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:25.042914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606676+0000","last_change":"2024-09-10T07:48:51.629828+0000","last_active":"2024-09-10T07:48:52.606676+0000","last_peered":"2024-09-10T07:48:52.606676+0000","last_clean":"2024-09-10T07:48:52.606676+0000","last_became_active":"2024-09-10T07:48:51.628172+0000","last_became_peered":"2024-09-10T07:48:51.628172+0000","last_unstale":"2024-09-10T07:48:52.606676+0000","last_undegraded":"2024-09-10T07:48:52.606676+0000","last_fullsized":"2024-09-10T07:48:52.606676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609352+0000","last_change":"2024-09-10T07:48:50.627131+0000","last_active":"2024-09-10T07:48:51.609352+0000","last_peered":"2024-09-10T07:48:51.609352+0000","last_clean":"2024-09-10T07:48:51.609352+0000","last_became_active":"2024-09-10T07:48:50.626968+0000","last_became_peered":"2024-09-10T07:48:50.626968+0000","last_unstale":"2024-09-10T07:48:51.609352+0000","last_undegraded":"2024-09-10T07:48:51.609352+0000","last_fullsized":"2024-09-10T07:48:51.609352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:11.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634368+0000","last_change":"2024-09-10T07:48:51.624704+0000","last_active":"2024-09-10T07:48:51.634368+0000","last_peered":"2024-09-10T07:48:51.634368+0000","last_clean":"2024-09-10T07:48:51.634368+0000","last_became_active":"2024-09-10T07:48:51.624391+0000","last_became_peered":"2024-09-10T07:48:51.624391+0000","last_unstale":"2024-09-10T07:48:51.634368+0000","last_undegraded":"2024-09-10T07:48:51.634368+0000","last_fullsized":"2024-09-10T07:48:51.634368+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:39.576381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606654+0000","last_change":"2024-09-10T07:48:50.618315+0000","last_active":"2024-09-10T07:48:52.606654+0000","last_peered":"2024-09-10T07:48:52.606654+0000","last_clean":"2024-09-10T07:48:52.606654+0000","last_became_active":"2024-09-10T07:48:50.618022+0000","last_became_peered":"2024-09-10T07:48:50.618022+0000","last_unstale":"2024-09-10T07:48:52.606654+0000","last_undegraded":"2024-09-10T07:48:52.606654+0000","last_fullsized":"2024-09-10T07:48:52.606654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:24.272006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.622150+0000","last_change":"2024-09-10T07:48:51.616784+0000","last_active":"2024-09-10T07:48:51.622150+0000","last_peered":"2024-09-10T07:48:51.622150+0000","last_clean":"2024-09-10T07:48:51.622150+0000","last_became_active":"2024-09-10T07:48:51.616520+0000","last_became_peered":"2024-09-10T07:48:51.616520+0000","last_unstale":"2024-09-10T07:48:51.622150+0000","last_undegraded":"2024-09-10T07:48:51.622150+0000","last_fullsized":"2024-09-10T07:48:51.622150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:47.124772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.604652+0000","last_change":"2024-09-10T07:48:50.615394+0000","last_active":"2024-09-10T07:48:51.604652+0000","last_peered":"2024-09-10T07:48:51.604652+0000","last_clean":"2024-09-10T07:48:51.604652+0000","last_became_active":"2024-09-10T07:48:50.615139+0000","last_became_peered":"2024-09-10T07:48:50.615139+0000","last_unstale":"2024-09-10T07:48:51.604652+0000","last_undegraded":"2024-09-10T07:48:51.604652+0000","last_fullsized":"2024-09-10T07:48:51.604652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623257+0000","last_change":"2024-09-10T07:48:51.619010+0000","last_active":"2024-09-10T07:48:51.623257+0000","last_peered":"2024-09-10T07:48:51.623257+0000","last_clean":"2024-09-10T07:48:51.623257+0000","last_became_active":"2024-09-10T07:48:51.618676+0000","last_became_peered":"2024-09-10T07:48:51.618676+0000","last_unstale":"2024-09-10T07:48:51.623257+0000","last_undegraded":"2024-09-10T07:48:51.623257+0000","last_fullsized":"2024-09-10T07:48:51.623257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:47.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607323+0000","last_change":"2024-09-10T07:48:50.620699+0000","last_active":"2024-09-10T07:48:52.607323+0000","last_peered":"2024-09-10T07:48:52.607323+0000","last_clean":"2024-09-10T07:48:52.607323+0000","last_became_active":"2024-09-10T07:48:50.620417+0000","last_became_peered":"2024-09-10T07:48:50.620417+0000","last_unstale":"2024-09-10T07:48:52.607323+0000","last_undegraded":"2024-09-10T07:48:52.607323+0000","last_fullsized":"2024-09-10T07:48:52.607323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:31.155316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610526+0000","last_change":"2024-09-10T07:48:50.610526+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610526+0000","last_undegraded":"2024-09-10T07:48:50.610526+0000","last_fullsized":"2024-09-10T07:48:50.610526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644403+0000","last_change":"2024-09-10T07:48:50.620839+0000","last_active":"2024-09-10T07:48:51.644403+0000","last_peered":"2024-09-10T07:48:51.644403+0000","last_clean":"2024-09-10T07:48:51.644403+0000","last_became_active":"2024-09-10T07:48:50.620581+0000","last_became_peered":"2024-09-10T07:48:50.620581+0000","last_unstale":"2024-09-10T07:48:51.644403+0000","last_undegraded":"2024-09-10T07:48:51.644403+0000","last_fullsized":"2024-09-10T07:48:51.644403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639187+0000","last_change":"2024-09-10T07:48:51.630681+0000","last_active":"2024-09-10T07:48:51.639187+0000","last_peered":"2024-09-10T07:48:51.639187+0000","last_clean":"2024-09-10T07:48:51.639187+0000","last_became_active":"2024-09-10T07:48:51.630321+0000","last_became_peered":"2024-09-10T07:48:51.630321+0000","last_unstale":"2024-09-10T07:48:51.639187+0000","last_undegraded":"2024-09-10T07:48:51.639187+0000","last_fullsized":"2024-09-10T07:48:51.639187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:11.469475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606961+0000","last_change":"2024-09-10T07:48:50.620804+0000","last_active":"2024-09-10T07:48:52.606961+0000","last_peered":"2024-09-10T07:48:52.606961+0000","last_clean":"2024-09-10T07:48:52.606961+0000","last_became_active":"2024-09-10T07:48:50.620518+0000","last_became_peered":"2024-09-10T07:48:50.620518+0000","last_unstale":"2024-09-10T07:48:52.606961+0000","last_undegraded":"2024-09-10T07:48:52.606961+0000","last_fullsized":"2024-09-10T07:48:52.606961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:26.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637082+0000","last_change":"2024-09-10T07:48:51.632456+0000","last_active":"2024-09-10T07:48:51.637082+0000","last_peered":"2024-09-10T07:48:51.637082+0000","last_clean":"2024-09-10T07:48:51.637082+0000","last_became_active":"2024-09-10T07:48:51.632175+0000","last_became_peered":"2024-09-10T07:48:51.632175+0000","last_unstale":"2024-09-10T07:48:51.637082+0000","last_undegraded":"2024-09-10T07:48:51.637082+0000","last_fullsized":"2024-09-10T07:48:51.637082+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:59.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624685+0000","last_change":"2024-09-10T07:48:50.623794+0000","last_active":"2024-09-10T07:48:50.624685+0000","last_peered":"2024-09-10T07:48:50.624685+0000","last_clean":"2024-09-10T07:48:50.624685+0000","last_became_active":"2024-09-10T07:48:50.623196+0000","last_became_peered":"2024-09-10T07:48:50.623196+0000","last_unstale":"2024-09-10T07:48:50.624685+0000","last_undegraded":"2024-09-10T07:48:50.624685+0000","last_fullsized":"2024-09-10T07:48:50.624685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605758+0000","last_change":"2024-09-10T07:48:50.605758+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605758+0000","last_undegraded":"2024-09-10T07:48:50.605758+0000","last_fullsized":"2024-09-10T07:48:50.605758+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.608400+0000","last_change":"2024-09-10T07:48:50.617395+0000","last_active":"2024-09-10T07:48:51.608400+0000","last_peered":"2024-09-10T07:48:51.608400+0000","last_clean":"2024-09-10T07:48:51.608400+0000","last_became_active":"2024-09-10T07:48:50.615852+0000","last_became_peered":"2024-09-10T07:48:50.615852+0000","last_unstale":"2024-09-10T07:48:51.608400+0000","last_undegraded":"2024-09-10T07:48:51.608400+0000","last_fullsized":"2024-09-10T07:48:51.608400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605337+0000","last_change":"2024-09-10T07:48:50.605337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605337+0000","last_undegraded":"2024-09-10T07:48:50.605337+0000","last_fullsized":"2024-09-10T07:48:50.605337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624312+0000","last_change":"2024-09-10T07:48:50.620280+0000","last_active":"2024-09-10T07:48:50.624312+0000","last_peered":"2024-09-10T07:48:50.624312+0000","last_clean":"2024-09-10T07:48:50.624312+0000","last_became_active":"2024-09-10T07:48:50.619988+0000","last_became_peered":"2024-09-10T07:48:50.619988+0000","last_unstale":"2024-09-10T07:48:50.624312+0000","last_undegraded":"2024-09-10T07:48:50.624312+0000","last_fullsized":"2024-09-10T07:48:50.624312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:03.156373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632347+0000","last_change":"2024-09-10T07:48:51.616228+0000","last_active":"2024-09-10T07:48:51.632347+0000","last_peered":"2024-09-10T07:48:51.632347+0000","last_clean":"2024-09-10T07:48:51.632347+0000","last_became_active":"2024-09-10T07:48:51.615946+0000","last_became_peered":"2024-09-10T07:48:51.615946+0000","last_unstale":"2024-09-10T07:48:51.632347+0000","last_undegraded":"2024-09-10T07:48:51.632347+0000","last_fullsized":"2024-09-10T07:48:51.632347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606882+0000","last_change":"2024-09-10T07:48:50.615137+0000","last_active":"2024-09-10T07:48:52.606882+0000","last_peered":"2024-09-10T07:48:52.606882+0000","last_clean":"2024-09-10T07:48:52.606882+0000","last_became_active":"2024-09-10T07:48:50.614618+0000","last_became_peered":"2024-09-10T07:48:50.614618+0000","last_unstale":"2024-09-10T07:48:52.606882+0000","last_undegraded":"2024-09-10T07:48:52.606882+0000","last_fullsized":"2024-09-10T07:48:52.606882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:16.791861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.606876+0000","last_change":"2024-09-10T07:48:50.606876+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.606876+0000","last_undegraded":"2024-09-10T07:48:50.606876+0000","last_fullsized":"2024-09-10T07:48:50.606876+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624888+0000","last_change":"2024-09-10T07:48:50.622636+0000","last_active":"2024-09-10T07:48:50.624888+0000","last_peered":"2024-09-10T07:48:50.624888+0000","last_clean":"2024-09-10T07:48:50.624888+0000","last_became_active":"2024-09-10T07:48:50.621702+0000","last_became_peered":"2024-09-10T07:48:50.621702+0000","last_unstale":"2024-09-10T07:48:50.624888+0000","last_undegraded":"2024-09-10T07:48:50.624888+0000","last_fullsized":"2024-09-10T07:48:50.624888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:21.875313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607227+0000","last_change":"2024-09-10T07:48:50.607227+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607227+0000","last_undegraded":"2024-09-10T07:48:50.607227+0000","last_fullsized":"2024-09-10T07:48:50.607227+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644901+0000","last_change":"2024-09-10T07:48:50.621912+0000","last_active":"2024-09-10T07:48:51.644901+0000","last_peered":"2024-09-10T07:48:51.644901+0000","last_clean":"2024-09-10T07:48:51.644901+0000","last_became_active":"2024-09-10T07:48:50.620580+0000","last_became_peered":"2024-09-10T07:48:50.620580+0000","last_unstale":"2024-09-10T07:48:51.644901+0000","last_undegraded":"2024-09-10T07:48:51.644901+0000","last_fullsized":"2024-09-10T07:48:51.644901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637323+0000","last_change":"2024-09-10T07:48:51.634091+0000","last_active":"2024-09-10T07:48:51.637323+0000","last_peered":"2024-09-10T07:48:51.637323+0000","last_clean":"2024-09-10T07:48:51.637323+0000","last_became_active":"2024-09-10T07:48:51.633805+0000","last_became_peered":"2024-09-10T07:48:51.633805+0000","last_unstale":"2024-09-10T07:48:51.637323+0000","last_undegraded":"2024-09-10T07:48:51.637323+0000","last_fullsized":"2024-09-10T07:48:51.637323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.641630+0000","last_change":"2024-09-10T07:48:50.616978+0000","last_active":"2024-09-10T07:48:51.641630+0000","last_peered":"2024-09-10T07:48:51.641630+0000","last_clean":"2024-09-10T07:48:51.641630+0000","last_became_active":"2024-09-10T07:48:50.616723+0000","last_became_peered":"2024-09-10T07:48:50.616723+0000","last_unstale":"2024-09-10T07:48:51.641630+0000","last_undegraded":"2024-09-10T07:48:51.641630+0000","last_fullsized":"2024-09-10T07:48:51.641630+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617278+0000","last_change":"2024-09-10T07:48:51.612987+0000","last_active":"2024-09-10T07:48:51.617278+0000","last_peered":"2024-09-10T07:48:51.617278+0000","last_clean":"2024-09-10T07:48:51.617278+0000","last_became_active":"2024-09-10T07:48:51.611732+0000","last_became_peered":"2024-09-10T07:48:51.611732+0000","last_unstale":"2024-09-10T07:48:51.617278+0000","last_undegraded":"2024-09-10T07:48:51.617278+0000","last_fullsized":"2024-09-10T07:48:51.617278+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.626733+0000","last_change":"2024-09-10T07:48:50.625853+0000","last_active":"2024-09-10T07:48:50.626733+0000","last_peered":"2024-09-10T07:48:50.626733+0000","last_clean":"2024-09-10T07:48:50.626733+0000","last_became_active":"2024-09-10T07:48:50.625576+0000","last_became_peered":"2024-09-10T07:48:50.625576+0000","last_unstale":"2024-09-10T07:48:50.626733+0000","last_undegraded":"2024-09-10T07:48:50.626733+0000","last_fullsized":"2024-09-10T07:48:50.626733+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:54.365655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606693+0000","last_change":"2024-09-10T07:48:51.622658+0000","last_active":"2024-09-10T07:48:52.606693+0000","last_peered":"2024-09-10T07:48:52.606693+0000","last_clean":"2024-09-10T07:48:52.606693+0000","last_became_active":"2024-09-10T07:48:51.622387+0000","last_became_peered":"2024-09-10T07:48:51.622387+0000","last_unstale":"2024-09-10T07:48:52.606693+0000","last_undegraded":"2024-09-10T07:48:52.606693+0000","last_fullsized":"2024-09-10T07:48:52.606693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:20.188379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607033+0000","last_change":"2024-09-10T07:48:50.617463+0000","last_active":"2024-09-10T07:48:52.607033+0000","last_peered":"2024-09-10T07:48:52.607033+0000","last_clean":"2024-09-10T07:48:52.607033+0000","last_became_active":"2024-09-10T07:48:50.617117+0000","last_became_peered":"2024-09-10T07:48:50.617117+0000","last_unstale":"2024-09-10T07:48:52.607033+0000","last_undegraded":"2024-09-10T07:48:52.607033+0000","last_fullsized":"2024-09-10T07:48:52.607033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:12.913436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609194+0000","last_change":"2024-09-10T07:48:50.609194+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609194+0000","last_undegraded":"2024-09-10T07:48:50.609194+0000","last_fullsized":"2024-09-10T07:48:50.609194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609587+0000","last_change":"2024-09-10T07:48:50.624873+0000","last_active":"2024-09-10T07:48:51.609587+0000","last_peered":"2024-09-10T07:48:51.609587+0000","last_clean":"2024-09-10T07:48:51.609587+0000","last_became_active":"2024-09-10T07:48:50.622063+0000","last_became_peered":"2024-09-10T07:48:50.622063+0000","last_unstale":"2024-09-10T07:48:51.609587+0000","last_undegraded":"2024-09-10T07:48:51.609587+0000","last_fullsized":"2024-09-10T07:48:51.609587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:49.987651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608768+0000","last_change":"2024-09-10T07:48:50.608768+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608768+0000","last_undegraded":"2024-09-10T07:48:50.608768+0000","last_fullsized":"2024-09-10T07:48:50.608768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606490+0000","last_change":"2024-09-10T07:48:50.619456+0000","last_active":"2024-09-10T07:48:52.606490+0000","last_peered":"2024-09-10T07:48:52.606490+0000","last_clean":"2024-09-10T07:48:52.606490+0000","last_became_active":"2024-09-10T07:48:50.618866+0000","last_became_peered":"2024-09-10T07:48:50.618866+0000","last_unstale":"2024-09-10T07:48:52.606490+0000","last_undegraded":"2024-09-10T07:48:52.606490+0000","last_fullsized":"2024-09-10T07:48:52.606490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:58.806978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609027+0000","last_change":"2024-09-10T07:48:50.609027+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609027+0000","last_undegraded":"2024-09-10T07:48:50.609027+0000","last_fullsized":"2024-09-10T07:48:50.609027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606169+0000","last_change":"2024-09-10T07:48:50.617552+0000","last_active":"2024-09-10T07:48:51.606169+0000","last_peered":"2024-09-10T07:48:51.606169+0000","last_clean":"2024-09-10T07:48:51.606169+0000","last_became_active":"2024-09-10T07:48:50.617260+0000","last_became_peered":"2024-09-10T07:48:50.617260+0000","last_unstale":"2024-09-10T07:48:51.606169+0000","last_undegraded":"2024-09-10T07:48:51.606169+0000","last_fullsized":"2024-09-10T07:48:51.606169+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:03.137368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634402+0000","last_change":"2024-09-10T07:48:51.625611+0000","last_active":"2024-09-10T07:48:51.634402+0000","last_peered":"2024-09-10T07:48:51.634402+0000","last_clean":"2024-09-10T07:48:51.634402+0000","last_became_active":"2024-09-10T07:48:51.625352+0000","last_became_peered":"2024-09-10T07:48:51.625352+0000","last_unstale":"2024-09-10T07:48:51.634402+0000","last_undegraded":"2024-09-10T07:48:51.634402+0000","last_fullsized":"2024-09-10T07:48:51.634402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:49.068278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622838+0000","last_change":"2024-09-10T07:48:50.621422+0000","last_active":"2024-09-10T07:48:50.622838+0000","last_peered":"2024-09-10T07:48:50.622838+0000","last_clean":"2024-09-10T07:48:50.622838+0000","last_became_active":"2024-09-10T07:48:50.621142+0000","last_became_peered":"2024-09-10T07:48:50.621142+0000","last_unstale":"2024-09-10T07:48:50.622838+0000","last_undegraded":"2024-09-10T07:48:50.622838+0000","last_fullsized":"2024-09-10T07:48:50.622838+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:25.494380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606946+0000","last_change":"2024-09-10T07:48:51.632002+0000","last_active":"2024-09-10T07:48:52.606946+0000","last_peered":"2024-09-10T07:48:52.606946+0000","last_clean":"2024-09-10T07:48:52.606946+0000","last_became_active":"2024-09-10T07:48:51.629180+0000","last_became_peered":"2024-09-10T07:48:51.629180+0000","last_unstale":"2024-09-10T07:48:52.606946+0000","last_undegraded":"2024-09-10T07:48:52.606946+0000","last_fullsized":"2024-09-10T07:48:52.606946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.610075+0000","last_change":"2024-09-10T07:48:50.622779+0000","last_active":"2024-09-10T07:48:51.610075+0000","last_peered":"2024-09-10T07:48:51.610075+0000","last_clean":"2024-09-10T07:48:51.610075+0000","last_became_active":"2024-09-10T07:48:50.622556+0000","last_became_peered":"2024-09-10T07:48:50.622556+0000","last_unstale":"2024-09-10T07:48:51.610075+0000","last_undegraded":"2024-09-10T07:48:51.610075+0000","last_fullsized":"2024-09-10T07:48:51.610075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:11.544783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638933+0000","last_change":"2024-09-10T07:48:51.628390+0000","last_active":"2024-09-10T07:48:51.638933+0000","last_peered":"2024-09-10T07:48:51.638933+0000","last_clean":"2024-09-10T07:48:51.638933+0000","last_became_active":"2024-09-10T07:48:51.628095+0000","last_became_peered":"2024-09-10T07:48:51.628095+0000","last_unstale":"2024-09-10T07:48:51.638933+0000","last_undegraded":"2024-09-10T07:48:51.638933+0000","last_fullsized":"2024-09-10T07:48:51.638933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:48.707363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622111+0000","last_change":"2024-09-10T07:48:50.619655+0000","last_active":"2024-09-10T07:48:50.622111+0000","last_peered":"2024-09-10T07:48:50.622111+0000","last_clean":"2024-09-10T07:48:50.622111+0000","last_became_active":"2024-09-10T07:48:50.619360+0000","last_became_peered":"2024-09-10T07:48:50.619360+0000","last_unstale":"2024-09-10T07:48:50.622111+0000","last_undegraded":"2024-09-10T07:48:50.622111+0000","last_fullsized":"2024-09-10T07:48:50.622111+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606864+0000","last_change":"2024-09-10T07:48:51.623695+0000","last_active":"2024-09-10T07:48:52.606864+0000","last_peered":"2024-09-10T07:48:52.606864+0000","last_clean":"2024-09-10T07:48:52.606864+0000","last_became_active":"2024-09-10T07:48:51.623049+0000","last_became_peered":"2024-09-10T07:48:51.623049+0000","last_unstale":"2024-09-10T07:48:52.606864+0000","last_undegraded":"2024-09-10T07:48:52.606864+0000","last_fullsized":"2024-09-10T07:48:52.606864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:42.211425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609403+0000","last_change":"2024-09-10T07:48:50.619068+0000","last_active":"2024-09-10T07:48:51.609403+0000","last_peered":"2024-09-10T07:48:51.609403+0000","last_clean":"2024-09-10T07:48:51.609403+0000","last_became_active":"2024-09-10T07:48:50.618824+0000","last_became_peered":"2024-09-10T07:48:50.618824+0000","last_unstale":"2024-09-10T07:48:51.609403+0000","last_undegraded":"2024-09-10T07:48:51.609403+0000","last_fullsized":"2024-09-10T07:48:51.609403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:24.707876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611851+0000","last_change":"2024-09-10T07:48:50.611851+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611851+0000","last_undegraded":"2024-09-10T07:48:50.611851+0000","last_fullsized":"2024-09-10T07:48:50.611851+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606363+0000","last_change":"2024-09-10T07:48:50.614453+0000","last_active":"2024-09-10T07:48:51.606363+0000","last_peered":"2024-09-10T07:48:51.606363+0000","last_clean":"2024-09-10T07:48:51.606363+0000","last_became_active":"2024-09-10T07:48:50.614159+0000","last_became_peered":"2024-09-10T07:48:50.614159+0000","last_unstale":"2024-09-10T07:48:51.606363+0000","last_undegraded":"2024-09-10T07:48:51.606363+0000","last_fullsized":"2024-09-10T07:48:51.606363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.792276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631226+0000","last_change":"2024-09-10T07:48:51.628493+0000","last_active":"2024-09-10T07:48:51.631226+0000","last_peered":"2024-09-10T07:48:51.631226+0000","last_clean":"2024-09-10T07:48:51.631226+0000","last_became_active":"2024-09-10T07:48:51.628229+0000","last_became_peered":"2024-09-10T07:48:51.628229+0000","last_unstale":"2024-09-10T07:48:51.631226+0000","last_undegraded":"2024-09-10T07:48:51.631226+0000","last_fullsized":"2024-09-10T07:48:51.631226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:04.162971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624400+0000","last_change":"2024-09-10T07:48:50.620900+0000","last_active":"2024-09-10T07:48:50.624400+0000","last_peered":"2024-09-10T07:48:50.624400+0000","last_clean":"2024-09-10T07:48:50.624400+0000","last_became_active":"2024-09-10T07:48:50.620412+0000","last_became_peered":"2024-09-10T07:48:50.620412+0000","last_unstale":"2024-09-10T07:48:50.624400+0000","last_undegraded":"2024-09-10T07:48:50.624400+0000","last_fullsized":"2024-09-10T07:48:50.624400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:58.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635716+0000","last_change":"2024-09-10T07:48:51.632214+0000","last_active":"2024-09-10T07:48:51.635716+0000","last_peered":"2024-09-10T07:48:51.635716+0000","last_clean":"2024-09-10T07:48:51.635716+0000","last_became_active":"2024-09-10T07:48:51.631932+0000","last_became_peered":"2024-09-10T07:48:51.631932+0000","last_unstale":"2024-09-10T07:48:51.635716+0000","last_undegraded":"2024-09-10T07:48:51.635716+0000","last_fullsized":"2024-09-10T07:48:51.635716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.618234+0000","last_change":"2024-09-10T07:48:50.615382+0000","last_active":"2024-09-10T07:48:50.618234+0000","last_peered":"2024-09-10T07:48:50.618234+0000","last_clean":"2024-09-10T07:48:50.618234+0000","last_became_active":"2024-09-10T07:48:50.614269+0000","last_became_peered":"2024-09-10T07:48:50.614269+0000","last_unstale":"2024-09-10T07:48:50.618234+0000","last_undegraded":"2024-09-10T07:48:50.618234+0000","last_fullsized":"2024-09-10T07:48:50.618234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612430+0000","last_change":"2024-09-10T07:48:50.612430+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612430+0000","last_undegraded":"2024-09-10T07:48:50.612430+0000","last_fullsized":"2024-09-10T07:48:50.612430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.645507+0000","last_change":"2024-09-10T07:48:50.624209+0000","last_active":"2024-09-10T07:48:51.645507+0000","last_peered":"2024-09-10T07:48:51.645507+0000","last_clean":"2024-09-10T07:48:51.645507+0000","last_became_active":"2024-09-10T07:48:50.623406+0000","last_became_peered":"2024-09-10T07:48:50.623406+0000","last_unstale":"2024-09-10T07:48:51.645507+0000","last_undegraded":"2024-09-10T07:48:51.645507+0000","last_fullsized":"2024-09-10T07:48:51.645507+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:29.913985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":217,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.603655+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:48:51.603655+0000","last_peered":"2024-09-10T07:48:51.603655+0000","last_clean":"2024-09-10T07:48:51.603655+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:48:51.603655+0000","last_undegraded":"2024-09-10T07:48:51.603655+0000","last_fullsized":"2024-09-10T07:48:51.603655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343688,"statfs":{"total":96636764160,"available":96607936512,"internally_reserved":0,"allocated":1146880,"data_stored":814559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":808284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:48:55.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:48:55.896 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:48:55.897+0000 7f6862c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:48:56.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.016+0000 7f7dfec00640 1 Processor -- start 2024-09-10T07:48:56.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.017+0000 7f7dfec00640 1 -- start start 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df8160710 0x7f7df80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfec00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df80d07c0 0x7f7df8072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df80cfdd0 con 0x7f7df8160af0 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfec00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7df815cc90 con 0x7f7df80d07c0 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7df815cf70 con 0x7f7df8160710 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfcc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df80d07c0 0x7f7df8072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfcc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7df80d07c0 0x7f7df8072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54626/0 (socket says 172.21.15.3:54626) 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7dfcc00640 1 -- 172.21.15.3:0/546410252 learned_addr learned my addr 172.21.15.3:0/546410252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:56.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.018+0000 7f7df7e00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df8160710 0x7f7df80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.019+0000 7f7dfcc00640 1 -- 172.21.15.3:0/546410252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df8160710 msgr2=0x7f7df80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:56.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.019+0000 7f7dfcc00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df8160710 0x7f7df80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.019+0000 7f7dfcc00640 1 -- 172.21.15.3:0/546410252 >> v1:172.21.15.70:6789/0 conn(0x7f7df8160af0 legacy=0x7f7df80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.019+0000 7f7dfcc00640 1 -- 172.21.15.3:0/546410252 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df815d250 con 0x7f7df80d07c0 2024-09-10T07:48:56.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.020+0000 7f7dfcc00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df80d07c0 0x7f7df8072070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009e50 tx=0x7f7df002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfda3878c01f7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.020+0000 7f7df7400640 1 -- 172.21.15.3:0/546410252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0036070 con 0x7f7df80d07c0 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.020+0000 7f7df7400640 1 -- 172.21.15.3:0/546410252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df0031600 con 0x7f7df80d07c0 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.020+0000 7f7df7400640 1 -- 172.21.15.3:0/546410252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00319d0 con 0x7f7df80d07c0 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 -- 172.21.15.3:0/546410252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df80d07c0 msgr2=0x7f7df8072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df80d07c0 0x7f7df8072070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009e50 tx=0x7f7df002a710 comp rx=0 tx=0).stop 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 -- 172.21.15.3:0/546410252 shutdown_connections 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df80d07c0 0x7f7df8072070 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 --2- 172.21.15.3:0/546410252 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df8160710 0x7f7df80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 -- 172.21.15.3:0/546410252 >> 172.21.15.3:0/546410252 conn(0x7f7df806d000 msgr2=0x7f7df806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 -- 172.21.15.3:0/546410252 shutdown_connections 2024-09-10T07:48:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.021+0000 7f7dfec00640 1 -- 172.21.15.3:0/546410252 wait complete. 2024-09-10T07:48:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.022+0000 7f7dfec00640 1 Processor -- start 2024-09-10T07:48:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.022+0000 7f7dfec00640 1 -- start start 2024-09-10T07:48:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 0x7f7df8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 0x7f7df8154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7df815d770 con 0x7f7df8160af0 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7df80cf800 con 0x7f7df8160710 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.023+0000 7f7dfec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7df814e490 con 0x7f7df80d07c0 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54632/0 (socket says 172.21.15.3:54632) 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 -- 172.21.15.3:0/2151603932 learned_addr learned my addr 172.21.15.3:0/2151603932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7df7e00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 0x7f7df8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7def400640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 0x7f7df8154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 -- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 msgr2=0x7f7df8151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 0x7f7df8151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 -- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 msgr2=0x7f7df8154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 0x7f7df8154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 -- 172.21.15.3:0/2151603932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81505c0 con 0x7f7df8160af0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7df7e00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 0x7f7df8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7def400640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 0x7f7df8154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7dfcc00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7df0034040 tx=0x7f7df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.024+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0036050 con 0x7f7df8160af0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.025+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df0042070 con 0x7f7df8160af0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.025+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0009510 con 0x7f7df8160af0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.025+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81508a0 con 0x7f7df8160af0 2024-09-10T07:48:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.025+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df8150e60 con 0x7f7df8160af0 2024-09-10T07:48:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.026+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df8151030 con 0x7f7df8160af0 2024-09-10T07:48:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.027+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7df0007bf0 con 0x7f7df8160af0 2024-09-10T07:48:56.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.028+0000 7f7df4c00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.028+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f7df00abd40 con 0x7f7df8160af0 2024-09-10T07:48:56.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.028+0000 7f7df7e00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.029+0000 7f7df7e00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 0x7f7dd406e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009ac0 tx=0x7f7de8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.032+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7df007a650 con 0x7f7df8160af0 2024-09-10T07:48:56.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.211+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7df80afd50 con 0x7f7dd406be30 2024-09-10T07:48:56.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.217+0000 7f7df4c00640 1 -- 172.21.15.3:0/2151603932 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249857 (secure 0 0 0) 0x7f7df80afd50 con 0x7f7dd406be30 2024-09-10T07:48:56.220 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:56.221 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.221+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 msgr2=0x7f7dd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.221+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 0x7f7dd406e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009ac0 tx=0x7f7de8008040 comp rx=0 tx=0).stop 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.221+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 msgr2=0x7f7df815db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.221+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7df0034040 tx=0x7f7df0004a40 comp rx=0 tx=0).stop 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.221+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 shutdown_connections 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7dd406be30 0x7f7dd406e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7df8160af0 0x7f7df815db20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df8160710 0x7f7df8154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 --2- 172.21.15.3:0/2151603932 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7df80d07c0 0x7f7df8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 >> 172.21.15.3:0/2151603932 conn(0x7f7df806d000 msgr2=0x7f7df8074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:56.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 shutdown_connections 2024-09-10T07:48:56.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.222+0000 7f7dfec00640 1 -- 172.21.15.3:0/2151603932 wait complete. 2024-09-10T07:48:56.237 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T07:48:55.520865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238360,"kb_used_data":5476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736360,"statfs":{"total":773094113280,"available":772850032640,"internally_reserved":0,"allocated":5607424,"data_stored":2963918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927936"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621887+0000","last_change":"2024-09-10T07:48:51.617751+0000","last_active":"2024-09-10T07:48:51.621887+0000","last_peered":"2024-09-10T07:48:51.621887+0000","last_clean":"2024-09-10T07:48:51.621887+0000","last_became_active":"2024-09-10T07:48:51.617450+0000","last_became_peered":"2024-09-10T07:48:51.617450+0000","last_unstale":"2024-09-10T07:48:51.621887+0000","last_undegraded":"2024-09-10T07:48:51.621887+0000","last_fullsized":"2024-09-10T07:48:51.621887+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632949+0000","last_change":"2024-09-10T07:48:51.620056+0000","last_active":"2024-09-10T07:48:51.632949+0000","last_peered":"2024-09-10T07:48:51.632949+0000","last_clean":"2024-09-10T07:48:51.632949+0000","last_became_active":"2024-09-10T07:48:51.619801+0000","last_became_peered":"2024-09-10T07:48:51.619801+0000","last_unstale":"2024-09-10T07:48:51.632949+0000","last_undegraded":"2024-09-10T07:48:51.632949+0000","last_fullsized":"2024-09-10T07:48:51.632949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:12.498291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617258+0000","last_change":"2024-09-10T07:48:51.612764+0000","last_active":"2024-09-10T07:48:51.617258+0000","last_peered":"2024-09-10T07:48:51.617258+0000","last_clean":"2024-09-10T07:48:51.617258+0000","last_became_active":"2024-09-10T07:48:51.611550+0000","last_became_peered":"2024-09-10T07:48:51.611550+0000","last_unstale":"2024-09-10T07:48:51.617258+0000","last_undegraded":"2024-09-10T07:48:51.617258+0000","last_fullsized":"2024-09-10T07:48:51.617258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:07.243280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611003+0000","last_change":"2024-09-10T07:48:50.611003+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611003+0000","last_undegraded":"2024-09-10T07:48:50.611003+0000","last_fullsized":"2024-09-10T07:48:50.611003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631377+0000","last_change":"2024-09-10T07:48:51.624149+0000","last_active":"2024-09-10T07:48:51.631377+0000","last_peered":"2024-09-10T07:48:51.631377+0000","last_clean":"2024-09-10T07:48:51.631377+0000","last_became_active":"2024-09-10T07:48:51.623849+0000","last_became_peered":"2024-09-10T07:48:51.623849+0000","last_unstale":"2024-09-10T07:48:51.631377+0000","last_undegraded":"2024-09-10T07:48:51.631377+0000","last_fullsized":"2024-09-10T07:48:51.631377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.308850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608067+0000","last_change":"2024-09-10T07:48:50.608067+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608067+0000","last_undegraded":"2024-09-10T07:48:50.608067+0000","last_fullsized":"2024-09-10T07:48:50.608067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639319+0000","last_change":"2024-09-10T07:48:51.628760+0000","last_active":"2024-09-10T07:48:51.639319+0000","last_peered":"2024-09-10T07:48:51.639319+0000","last_clean":"2024-09-10T07:48:51.639319+0000","last_became_active":"2024-09-10T07:48:51.628462+0000","last_became_peered":"2024-09-10T07:48:51.628462+0000","last_unstale":"2024-09-10T07:48:51.639319+0000","last_undegraded":"2024-09-10T07:48:51.639319+0000","last_fullsized":"2024-09-10T07:48:51.639319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:19.292466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614856+0000","last_change":"2024-09-10T07:48:50.614856+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614856+0000","last_undegraded":"2024-09-10T07:48:50.614856+0000","last_fullsized":"2024-09-10T07:48:50.614856+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609844+0000","last_change":"2024-09-10T07:48:50.609844+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609844+0000","last_undegraded":"2024-09-10T07:48:50.609844+0000","last_fullsized":"2024-09-10T07:48:50.609844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607402+0000","last_change":"2024-09-10T07:48:51.617674+0000","last_active":"2024-09-10T07:48:52.607402+0000","last_peered":"2024-09-10T07:48:52.607402+0000","last_clean":"2024-09-10T07:48:52.607402+0000","last_became_active":"2024-09-10T07:48:51.617378+0000","last_became_peered":"2024-09-10T07:48:51.617378+0000","last_unstale":"2024-09-10T07:48:52.607402+0000","last_undegraded":"2024-09-10T07:48:52.607402+0000","last_fullsized":"2024-09-10T07:48:52.607402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:02.393228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611643+0000","last_change":"2024-09-10T07:48:50.611643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611643+0000","last_undegraded":"2024-09-10T07:48:50.611643+0000","last_fullsized":"2024-09-10T07:48:50.611643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633458+0000","last_change":"2024-09-10T07:48:51.620489+0000","last_active":"2024-09-10T07:48:51.633458+0000","last_peered":"2024-09-10T07:48:51.633458+0000","last_clean":"2024-09-10T07:48:51.633458+0000","last_became_active":"2024-09-10T07:48:51.620237+0000","last_became_peered":"2024-09-10T07:48:51.620237+0000","last_unstale":"2024-09-10T07:48:51.633458+0000","last_undegraded":"2024-09-10T07:48:51.633458+0000","last_fullsized":"2024-09-10T07:48:51.633458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:25.714474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609643+0000","last_change":"2024-09-10T07:48:50.609643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609643+0000","last_undegraded":"2024-09-10T07:48:50.609643+0000","last_fullsized":"2024-09-10T07:48:50.609643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608771+0000","last_change":"2024-09-10T07:48:50.608771+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608771+0000","last_undegraded":"2024-09-10T07:48:50.608771+0000","last_fullsized":"2024-09-10T07:48:50.608771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609579+0000","last_change":"2024-09-10T07:48:50.609579+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609579+0000","last_undegraded":"2024-09-10T07:48:50.609579+0000","last_fullsized":"2024-09-10T07:48:50.609579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637116+0000","last_change":"2024-09-10T07:48:51.631668+0000","last_active":"2024-09-10T07:48:51.637116+0000","last_peered":"2024-09-10T07:48:51.637116+0000","last_clean":"2024-09-10T07:48:51.637116+0000","last_became_active":"2024-09-10T07:48:51.631459+0000","last_became_peered":"2024-09-10T07:48:51.631459+0000","last_unstale":"2024-09-10T07:48:51.637116+0000","last_undegraded":"2024-09-10T07:48:51.637116+0000","last_fullsized":"2024-09-10T07:48:51.637116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:12.993962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632892+0000","last_change":"2024-09-10T07:48:51.622041+0000","last_active":"2024-09-10T07:48:51.632892+0000","last_peered":"2024-09-10T07:48:51.632892+0000","last_clean":"2024-09-10T07:48:51.632892+0000","last_became_active":"2024-09-10T07:48:51.621622+0000","last_became_peered":"2024-09-10T07:48:51.621622+0000","last_unstale":"2024-09-10T07:48:51.632892+0000","last_undegraded":"2024-09-10T07:48:51.632892+0000","last_fullsized":"2024-09-10T07:48:51.632892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:50.994249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638066+0000","last_change":"2024-09-10T07:48:51.632824+0000","last_active":"2024-09-10T07:48:51.638066+0000","last_peered":"2024-09-10T07:48:51.638066+0000","last_clean":"2024-09-10T07:48:51.638066+0000","last_became_active":"2024-09-10T07:48:51.629313+0000","last_became_peered":"2024-09-10T07:48:51.629313+0000","last_unstale":"2024-09-10T07:48:51.638066+0000","last_undegraded":"2024-09-10T07:48:51.638066+0000","last_fullsized":"2024-09-10T07:48:51.638066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:25.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630089+0000","last_change":"2024-09-10T07:48:51.623911+0000","last_active":"2024-09-10T07:48:51.630089+0000","last_peered":"2024-09-10T07:48:51.630089+0000","last_clean":"2024-09-10T07:48:51.630089+0000","last_became_active":"2024-09-10T07:48:51.623645+0000","last_became_peered":"2024-09-10T07:48:51.623645+0000","last_unstale":"2024-09-10T07:48:51.630089+0000","last_undegraded":"2024-09-10T07:48:51.630089+0000","last_fullsized":"2024-09-10T07:48:51.630089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:54.995968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606836+0000","last_change":"2024-09-10T07:48:51.624623+0000","last_active":"2024-09-10T07:48:52.606836+0000","last_peered":"2024-09-10T07:48:52.606836+0000","last_clean":"2024-09-10T07:48:52.606836+0000","last_became_active":"2024-09-10T07:48:51.621627+0000","last_became_peered":"2024-09-10T07:48:51.621627+0000","last_unstale":"2024-09-10T07:48:52.606836+0000","last_undegraded":"2024-09-10T07:48:52.606836+0000","last_fullsized":"2024-09-10T07:48:52.606836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:19.844611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608827+0000","last_change":"2024-09-10T07:48:50.608827+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608827+0000","last_undegraded":"2024-09-10T07:48:50.608827+0000","last_fullsized":"2024-09-10T07:48:50.608827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605350+0000","last_change":"2024-09-10T07:48:50.605350+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605350+0000","last_undegraded":"2024-09-10T07:48:50.605350+0000","last_fullsized":"2024-09-10T07:48:50.605350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.624565+0000","last_change":"2024-09-10T07:48:51.616950+0000","last_active":"2024-09-10T07:48:51.624565+0000","last_peered":"2024-09-10T07:48:51.624565+0000","last_clean":"2024-09-10T07:48:51.624565+0000","last_became_active":"2024-09-10T07:48:51.614672+0000","last_became_peered":"2024-09-10T07:48:51.614672+0000","last_unstale":"2024-09-10T07:48:51.624565+0000","last_undegraded":"2024-09-10T07:48:51.624565+0000","last_fullsized":"2024-09-10T07:48:51.624565+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:48.008128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635731+0000","last_change":"2024-09-10T07:48:51.630986+0000","last_active":"2024-09-10T07:48:51.635731+0000","last_peered":"2024-09-10T07:48:51.635731+0000","last_clean":"2024-09-10T07:48:51.635731+0000","last_became_active":"2024-09-10T07:48:51.630754+0000","last_became_peered":"2024-09-10T07:48:51.630754+0000","last_unstale":"2024-09-10T07:48:51.635731+0000","last_undegraded":"2024-09-10T07:48:51.635731+0000","last_fullsized":"2024-09-10T07:48:51.635731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621864+0000","last_change":"2024-09-10T07:48:51.617701+0000","last_active":"2024-09-10T07:48:51.621864+0000","last_peered":"2024-09-10T07:48:51.621864+0000","last_clean":"2024-09-10T07:48:51.621864+0000","last_became_active":"2024-09-10T07:48:51.617418+0000","last_became_peered":"2024-09-10T07:48:51.617418+0000","last_unstale":"2024-09-10T07:48:51.621864+0000","last_undegraded":"2024-09-10T07:48:51.621864+0000","last_fullsized":"2024-09-10T07:48:51.621864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612838+0000","last_change":"2024-09-10T07:48:50.612838+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612838+0000","last_undegraded":"2024-09-10T07:48:50.612838+0000","last_fullsized":"2024-09-10T07:48:50.612838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630570+0000","last_change":"2024-09-10T07:48:51.620624+0000","last_active":"2024-09-10T07:48:51.630570+0000","last_peered":"2024-09-10T07:48:51.630570+0000","last_clean":"2024-09-10T07:48:51.630570+0000","last_became_active":"2024-09-10T07:48:51.619782+0000","last_became_peered":"2024-09-10T07:48:51.619782+0000","last_unstale":"2024-09-10T07:48:51.630570+0000","last_undegraded":"2024-09-10T07:48:51.630570+0000","last_fullsized":"2024-09-10T07:48:51.630570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.304141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623981+0000","last_change":"2024-09-10T07:48:51.616307+0000","last_active":"2024-09-10T07:48:51.623981+0000","last_peered":"2024-09-10T07:48:51.623981+0000","last_clean":"2024-09-10T07:48:51.623981+0000","last_became_active":"2024-09-10T07:48:51.615851+0000","last_became_peered":"2024-09-10T07:48:51.615851+0000","last_unstale":"2024-09-10T07:48:51.623981+0000","last_undegraded":"2024-09-10T07:48:51.623981+0000","last_fullsized":"2024-09-10T07:48:51.623981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:05.714041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623649+0000","last_change":"2024-09-10T07:48:51.616551+0000","last_active":"2024-09-10T07:48:51.623649+0000","last_peered":"2024-09-10T07:48:51.623649+0000","last_clean":"2024-09-10T07:48:51.623649+0000","last_became_active":"2024-09-10T07:48:51.616241+0000","last_became_peered":"2024-09-10T07:48:51.616241+0000","last_unstale":"2024-09-10T07:48:51.623649+0000","last_undegraded":"2024-09-10T07:48:51.623649+0000","last_fullsized":"2024-09-10T07:48:51.623649+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633988+0000","last_change":"2024-09-10T07:48:51.628969+0000","last_active":"2024-09-10T07:48:51.633988+0000","last_peered":"2024-09-10T07:48:51.633988+0000","last_clean":"2024-09-10T07:48:51.633988+0000","last_became_active":"2024-09-10T07:48:51.628200+0000","last_became_peered":"2024-09-10T07:48:51.628200+0000","last_unstale":"2024-09-10T07:48:51.633988+0000","last_undegraded":"2024-09-10T07:48:51.633988+0000","last_fullsized":"2024-09-10T07:48:51.633988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:12.551381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.613199+0000","last_change":"2024-09-10T07:48:50.613199+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.613199+0000","last_undegraded":"2024-09-10T07:48:50.613199+0000","last_fullsized":"2024-09-10T07:48:50.613199+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614337+0000","last_change":"2024-09-10T07:48:50.614337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614337+0000","last_undegraded":"2024-09-10T07:48:50.614337+0000","last_fullsized":"2024-09-10T07:48:50.614337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607331+0000","last_change":"2024-09-10T07:48:50.607331+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607331+0000","last_undegraded":"2024-09-10T07:48:50.607331+0000","last_fullsized":"2024-09-10T07:48:50.607331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.628793+0000","last_change":"2024-09-10T07:48:50.627953+0000","last_active":"2024-09-10T07:48:50.628793+0000","last_peered":"2024-09-10T07:48:50.628793+0000","last_clean":"2024-09-10T07:48:50.628793+0000","last_became_active":"2024-09-10T07:48:50.627666+0000","last_became_peered":"2024-09-10T07:48:50.627666+0000","last_unstale":"2024-09-10T07:48:50.628793+0000","last_undegraded":"2024-09-10T07:48:50.628793+0000","last_fullsized":"2024-09-10T07:48:50.628793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:57.026249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630003+0000","last_change":"2024-09-10T07:48:51.621847+0000","last_active":"2024-09-10T07:48:51.630003+0000","last_peered":"2024-09-10T07:48:51.630003+0000","last_clean":"2024-09-10T07:48:51.630003+0000","last_became_active":"2024-09-10T07:48:51.621554+0000","last_became_peered":"2024-09-10T07:48:51.621554+0000","last_unstale":"2024-09-10T07:48:51.630003+0000","last_undegraded":"2024-09-10T07:48:51.630003+0000","last_fullsized":"2024-09-10T07:48:51.630003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634481+0000","last_change":"2024-09-10T07:48:50.619796+0000","last_active":"2024-09-10T07:48:51.634481+0000","last_peered":"2024-09-10T07:48:51.634481+0000","last_clean":"2024-09-10T07:48:51.634481+0000","last_became_active":"2024-09-10T07:48:50.619535+0000","last_became_peered":"2024-09-10T07:48:50.619535+0000","last_unstale":"2024-09-10T07:48:51.634481+0000","last_undegraded":"2024-09-10T07:48:51.634481+0000","last_fullsized":"2024-09-10T07:48:51.634481+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:44.514806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633602+0000","last_change":"2024-09-10T07:48:51.629435+0000","last_active":"2024-09-10T07:48:51.633602+0000","last_peered":"2024-09-10T07:48:51.633602+0000","last_clean":"2024-09-10T07:48:51.633602+0000","last_became_active":"2024-09-10T07:48:51.628928+0000","last_became_peered":"2024-09-10T07:48:51.628928+0000","last_unstale":"2024-09-10T07:48:51.633602+0000","last_undegraded":"2024-09-10T07:48:51.633602+0000","last_fullsized":"2024-09-10T07:48:51.633602+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622265+0000","last_change":"2024-09-10T07:48:50.619441+0000","last_active":"2024-09-10T07:48:50.622265+0000","last_peered":"2024-09-10T07:48:50.622265+0000","last_clean":"2024-09-10T07:48:50.622265+0000","last_became_active":"2024-09-10T07:48:50.619148+0000","last_became_peered":"2024-09-10T07:48:50.619148+0000","last_unstale":"2024-09-10T07:48:50.622265+0000","last_undegraded":"2024-09-10T07:48:50.622265+0000","last_fullsized":"2024-09-10T07:48:50.622265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.560270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610493+0000","last_change":"2024-09-10T07:48:50.610493+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610493+0000","last_undegraded":"2024-09-10T07:48:50.610493+0000","last_fullsized":"2024-09-10T07:48:50.610493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639518+0000","last_change":"2024-09-10T07:48:50.617904+0000","last_active":"2024-09-10T07:48:51.639518+0000","last_peered":"2024-09-10T07:48:51.639518+0000","last_clean":"2024-09-10T07:48:51.639518+0000","last_became_active":"2024-09-10T07:48:50.617656+0000","last_became_peered":"2024-09-10T07:48:50.617656+0000","last_unstale":"2024-09-10T07:48:51.639518+0000","last_undegraded":"2024-09-10T07:48:51.639518+0000","last_fullsized":"2024-09-10T07:48:51.639518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.305959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.627946+0000","last_change":"2024-09-10T07:48:51.620554+0000","last_active":"2024-09-10T07:48:51.627946+0000","last_peered":"2024-09-10T07:48:51.627946+0000","last_clean":"2024-09-10T07:48:51.627946+0000","last_became_active":"2024-09-10T07:48:51.619526+0000","last_became_peered":"2024-09-10T07:48:51.619526+0000","last_unstale":"2024-09-10T07:48:51.627946+0000","last_undegraded":"2024-09-10T07:48:51.627946+0000","last_fullsized":"2024-09-10T07:48:51.627946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:33.856786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.625037+0000","last_change":"2024-09-10T07:48:50.623162+0000","last_active":"2024-09-10T07:48:50.625037+0000","last_peered":"2024-09-10T07:48:50.625037+0000","last_clean":"2024-09-10T07:48:50.625037+0000","last_became_active":"2024-09-10T07:48:50.622765+0000","last_became_peered":"2024-09-10T07:48:50.622765+0000","last_unstale":"2024-09-10T07:48:50.625037+0000","last_undegraded":"2024-09-10T07:48:50.625037+0000","last_fullsized":"2024-09-10T07:48:50.625037+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:24.341239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606939+0000","last_change":"2024-09-10T07:48:51.628666+0000","last_active":"2024-09-10T07:48:52.606939+0000","last_peered":"2024-09-10T07:48:52.606939+0000","last_clean":"2024-09-10T07:48:52.606939+0000","last_became_active":"2024-09-10T07:48:51.628411+0000","last_became_peered":"2024-09-10T07:48:51.628411+0000","last_unstale":"2024-09-10T07:48:52.606939+0000","last_undegraded":"2024-09-10T07:48:52.606939+0000","last_fullsized":"2024-09-10T07:48:52.606939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:13.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635593+0000","last_change":"2024-09-10T07:48:50.616830+0000","last_active":"2024-09-10T07:48:51.635593+0000","last_peered":"2024-09-10T07:48:51.635593+0000","last_clean":"2024-09-10T07:48:51.635593+0000","last_became_active":"2024-09-10T07:48:50.616586+0000","last_became_peered":"2024-09-10T07:48:50.616586+0000","last_unstale":"2024-09-10T07:48:51.635593+0000","last_undegraded":"2024-09-10T07:48:51.635593+0000","last_fullsized":"2024-09-10T07:48:51.635593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:25.042914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606676+0000","last_change":"2024-09-10T07:48:51.629828+0000","last_active":"2024-09-10T07:48:52.606676+0000","last_peered":"2024-09-10T07:48:52.606676+0000","last_clean":"2024-09-10T07:48:52.606676+0000","last_became_active":"2024-09-10T07:48:51.628172+0000","last_became_peered":"2024-09-10T07:48:51.628172+0000","last_unstale":"2024-09-10T07:48:52.606676+0000","last_undegraded":"2024-09-10T07:48:52.606676+0000","last_fullsized":"2024-09-10T07:48:52.606676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609352+0000","last_change":"2024-09-10T07:48:50.627131+0000","last_active":"2024-09-10T07:48:51.609352+0000","last_peered":"2024-09-10T07:48:51.609352+0000","last_clean":"2024-09-10T07:48:51.609352+0000","last_became_active":"2024-09-10T07:48:50.626968+0000","last_became_peered":"2024-09-10T07:48:50.626968+0000","last_unstale":"2024-09-10T07:48:51.609352+0000","last_undegraded":"2024-09-10T07:48:51.609352+0000","last_fullsized":"2024-09-10T07:48:51.609352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:11.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634368+0000","last_change":"2024-09-10T07:48:51.624704+0000","last_active":"2024-09-10T07:48:51.634368+0000","last_peered":"2024-09-10T07:48:51.634368+0000","last_clean":"2024-09-10T07:48:51.634368+0000","last_became_active":"2024-09-10T07:48:51.624391+0000","last_became_peered":"2024-09-10T07:48:51.624391+0000","last_unstale":"2024-09-10T07:48:51.634368+0000","last_undegraded":"2024-09-10T07:48:51.634368+0000","last_fullsized":"2024-09-10T07:48:51.634368+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:39.576381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606654+0000","last_change":"2024-09-10T07:48:50.618315+0000","last_active":"2024-09-10T07:48:52.606654+0000","last_peered":"2024-09-10T07:48:52.606654+0000","last_clean":"2024-09-10T07:48:52.606654+0000","last_became_active":"2024-09-10T07:48:50.618022+0000","last_became_peered":"2024-09-10T07:48:50.618022+0000","last_unstale":"2024-09-10T07:48:52.606654+0000","last_undegraded":"2024-09-10T07:48:52.606654+0000","last_fullsized":"2024-09-10T07:48:52.606654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:24.272006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.622150+0000","last_change":"2024-09-10T07:48:51.616784+0000","last_active":"2024-09-10T07:48:51.622150+0000","last_peered":"2024-09-10T07:48:51.622150+0000","last_clean":"2024-09-10T07:48:51.622150+0000","last_became_active":"2024-09-10T07:48:51.616520+0000","last_became_peered":"2024-09-10T07:48:51.616520+0000","last_unstale":"2024-09-10T07:48:51.622150+0000","last_undegraded":"2024-09-10T07:48:51.622150+0000","last_fullsized":"2024-09-10T07:48:51.622150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:47.124772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.604652+0000","last_change":"2024-09-10T07:48:50.615394+0000","last_active":"2024-09-10T07:48:51.604652+0000","last_peered":"2024-09-10T07:48:51.604652+0000","last_clean":"2024-09-10T07:48:51.604652+0000","last_became_active":"2024-09-10T07:48:50.615139+0000","last_became_peered":"2024-09-10T07:48:50.615139+0000","last_unstale":"2024-09-10T07:48:51.604652+0000","last_undegraded":"2024-09-10T07:48:51.604652+0000","last_fullsized":"2024-09-10T07:48:51.604652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623257+0000","last_change":"2024-09-10T07:48:51.619010+0000","last_active":"2024-09-10T07:48:51.623257+0000","last_peered":"2024-09-10T07:48:51.623257+0000","last_clean":"2024-09-10T07:48:51.623257+0000","last_became_active":"2024-09-10T07:48:51.618676+0000","last_became_peered":"2024-09-10T07:48:51.618676+0000","last_unstale":"2024-09-10T07:48:51.623257+0000","last_undegraded":"2024-09-10T07:48:51.623257+0000","last_fullsized":"2024-09-10T07:48:51.623257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:47.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607323+0000","last_change":"2024-09-10T07:48:50.620699+0000","last_active":"2024-09-10T07:48:52.607323+0000","last_peered":"2024-09-10T07:48:52.607323+0000","last_clean":"2024-09-10T07:48:52.607323+0000","last_became_active":"2024-09-10T07:48:50.620417+0000","last_became_peered":"2024-09-10T07:48:50.620417+0000","last_unstale":"2024-09-10T07:48:52.607323+0000","last_undegraded":"2024-09-10T07:48:52.607323+0000","last_fullsized":"2024-09-10T07:48:52.607323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:31.155316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610526+0000","last_change":"2024-09-10T07:48:50.610526+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610526+0000","last_undegraded":"2024-09-10T07:48:50.610526+0000","last_fullsized":"2024-09-10T07:48:50.610526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644403+0000","last_change":"2024-09-10T07:48:50.620839+0000","last_active":"2024-09-10T07:48:51.644403+0000","last_peered":"2024-09-10T07:48:51.644403+0000","last_clean":"2024-09-10T07:48:51.644403+0000","last_became_active":"2024-09-10T07:48:50.620581+0000","last_became_peered":"2024-09-10T07:48:50.620581+0000","last_unstale":"2024-09-10T07:48:51.644403+0000","last_undegraded":"2024-09-10T07:48:51.644403+0000","last_fullsized":"2024-09-10T07:48:51.644403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639187+0000","last_change":"2024-09-10T07:48:51.630681+0000","last_active":"2024-09-10T07:48:51.639187+0000","last_peered":"2024-09-10T07:48:51.639187+0000","last_clean":"2024-09-10T07:48:51.639187+0000","last_became_active":"2024-09-10T07:48:51.630321+0000","last_became_peered":"2024-09-10T07:48:51.630321+0000","last_unstale":"2024-09-10T07:48:51.639187+0000","last_undegraded":"2024-09-10T07:48:51.639187+0000","last_fullsized":"2024-09-10T07:48:51.639187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:11.469475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606961+0000","last_change":"2024-09-10T07:48:50.620804+0000","last_active":"2024-09-10T07:48:52.606961+0000","last_peered":"2024-09-10T07:48:52.606961+0000","last_clean":"2024-09-10T07:48:52.606961+0000","last_became_active":"2024-09-10T07:48:50.620518+0000","last_became_peered":"2024-09-10T07:48:50.620518+0000","last_unstale":"2024-09-10T07:48:52.606961+0000","last_undegraded":"2024-09-10T07:48:52.606961+0000","last_fullsized":"2024-09-10T07:48:52.606961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:26.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637082+0000","last_change":"2024-09-10T07:48:51.632456+0000","last_active":"2024-09-10T07:48:51.637082+0000","last_peered":"2024-09-10T07:48:51.637082+0000","last_clean":"2024-09-10T07:48:51.637082+0000","last_became_active":"2024-09-10T07:48:51.632175+0000","last_became_peered":"2024-09-10T07:48:51.632175+0000","last_unstale":"2024-09-10T07:48:51.637082+0000","last_undegraded":"2024-09-10T07:48:51.637082+0000","last_fullsized":"2024-09-10T07:48:51.637082+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:59.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624685+0000","last_change":"2024-09-10T07:48:50.623794+0000","last_active":"2024-09-10T07:48:50.624685+0000","last_peered":"2024-09-10T07:48:50.624685+0000","last_clean":"2024-09-10T07:48:50.624685+0000","last_became_active":"2024-09-10T07:48:50.623196+0000","last_became_peered":"2024-09-10T07:48:50.623196+0000","last_unstale":"2024-09-10T07:48:50.624685+0000","last_undegraded":"2024-09-10T07:48:50.624685+0000","last_fullsized":"2024-09-10T07:48:50.624685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605758+0000","last_change":"2024-09-10T07:48:50.605758+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605758+0000","last_undegraded":"2024-09-10T07:48:50.605758+0000","last_fullsized":"2024-09-10T07:48:50.605758+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.608400+0000","last_change":"2024-09-10T07:48:50.617395+0000","last_active":"2024-09-10T07:48:51.608400+0000","last_peered":"2024-09-10T07:48:51.608400+0000","last_clean":"2024-09-10T07:48:51.608400+0000","last_became_active":"2024-09-10T07:48:50.615852+0000","last_became_peered":"2024-09-10T07:48:50.615852+0000","last_unstale":"2024-09-10T07:48:51.608400+0000","last_undegraded":"2024-09-10T07:48:51.608400+0000","last_fullsized":"2024-09-10T07:48:51.608400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605337+0000","last_change":"2024-09-10T07:48:50.605337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605337+0000","last_undegraded":"2024-09-10T07:48:50.605337+0000","last_fullsized":"2024-09-10T07:48:50.605337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624312+0000","last_change":"2024-09-10T07:48:50.620280+0000","last_active":"2024-09-10T07:48:50.624312+0000","last_peered":"2024-09-10T07:48:50.624312+0000","last_clean":"2024-09-10T07:48:50.624312+0000","last_became_active":"2024-09-10T07:48:50.619988+0000","last_became_peered":"2024-09-10T07:48:50.619988+0000","last_unstale":"2024-09-10T07:48:50.624312+0000","last_undegraded":"2024-09-10T07:48:50.624312+0000","last_fullsized":"2024-09-10T07:48:50.624312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:03.156373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632347+0000","last_change":"2024-09-10T07:48:51.616228+0000","last_active":"2024-09-10T07:48:51.632347+0000","last_peered":"2024-09-10T07:48:51.632347+0000","last_clean":"2024-09-10T07:48:51.632347+0000","last_became_active":"2024-09-10T07:48:51.615946+0000","last_became_peered":"2024-09-10T07:48:51.615946+0000","last_unstale":"2024-09-10T07:48:51.632347+0000","last_undegraded":"2024-09-10T07:48:51.632347+0000","last_fullsized":"2024-09-10T07:48:51.632347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606882+0000","last_change":"2024-09-10T07:48:50.615137+0000","last_active":"2024-09-10T07:48:52.606882+0000","last_peered":"2024-09-10T07:48:52.606882+0000","last_clean":"2024-09-10T07:48:52.606882+0000","last_became_active":"2024-09-10T07:48:50.614618+0000","last_became_peered":"2024-09-10T07:48:50.614618+0000","last_unstale":"2024-09-10T07:48:52.606882+0000","last_undegraded":"2024-09-10T07:48:52.606882+0000","last_fullsized":"2024-09-10T07:48:52.606882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:16.791861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.606876+0000","last_change":"2024-09-10T07:48:50.606876+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.606876+0000","last_undegraded":"2024-09-10T07:48:50.606876+0000","last_fullsized":"2024-09-10T07:48:50.606876+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624888+0000","last_change":"2024-09-10T07:48:50.622636+0000","last_active":"2024-09-10T07:48:50.624888+0000","last_peered":"2024-09-10T07:48:50.624888+0000","last_clean":"2024-09-10T07:48:50.624888+0000","last_became_active":"2024-09-10T07:48:50.621702+0000","last_became_peered":"2024-09-10T07:48:50.621702+0000","last_unstale":"2024-09-10T07:48:50.624888+0000","last_undegraded":"2024-09-10T07:48:50.624888+0000","last_fullsized":"2024-09-10T07:48:50.624888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:21.875313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607227+0000","last_change":"2024-09-10T07:48:50.607227+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607227+0000","last_undegraded":"2024-09-10T07:48:50.607227+0000","last_fullsized":"2024-09-10T07:48:50.607227+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644901+0000","last_change":"2024-09-10T07:48:50.621912+0000","last_active":"2024-09-10T07:48:51.644901+0000","last_peered":"2024-09-10T07:48:51.644901+0000","last_clean":"2024-09-10T07:48:51.644901+0000","last_became_active":"2024-09-10T07:48:50.620580+0000","last_became_peered":"2024-09-10T07:48:50.620580+0000","last_unstale":"2024-09-10T07:48:51.644901+0000","last_undegraded":"2024-09-10T07:48:51.644901+0000","last_fullsized":"2024-09-10T07:48:51.644901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637323+0000","last_change":"2024-09-10T07:48:51.634091+0000","last_active":"2024-09-10T07:48:51.637323+0000","last_peered":"2024-09-10T07:48:51.637323+0000","last_clean":"2024-09-10T07:48:51.637323+0000","last_became_active":"2024-09-10T07:48:51.633805+0000","last_became_peered":"2024-09-10T07:48:51.633805+0000","last_unstale":"2024-09-10T07:48:51.637323+0000","last_undegraded":"2024-09-10T07:48:51.637323+0000","last_fullsized":"2024-09-10T07:48:51.637323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.641630+0000","last_change":"2024-09-10T07:48:50.616978+0000","last_active":"2024-09-10T07:48:51.641630+0000","last_peered":"2024-09-10T07:48:51.641630+0000","last_clean":"2024-09-10T07:48:51.641630+0000","last_became_active":"2024-09-10T07:48:50.616723+0000","last_became_peered":"2024-09-10T07:48:50.616723+0000","last_unstale":"2024-09-10T07:48:51.641630+0000","last_undegraded":"2024-09-10T07:48:51.641630+0000","last_fullsized":"2024-09-10T07:48:51.641630+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617278+0000","last_change":"2024-09-10T07:48:51.612987+0000","last_active":"2024-09-10T07:48:51.617278+0000","last_peered":"2024-09-10T07:48:51.617278+0000","last_clean":"2024-09-10T07:48:51.617278+0000","last_became_active":"2024-09-10T07:48:51.611732+0000","last_became_peered":"2024-09-10T07:48:51.611732+0000","last_unstale":"2024-09-10T07:48:51.617278+0000","last_undegraded":"2024-09-10T07:48:51.617278+0000","last_fullsized":"2024-09-10T07:48:51.617278+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.626733+0000","last_change":"2024-09-10T07:48:50.625853+0000","last_active":"2024-09-10T07:48:50.626733+0000","last_peered":"2024-09-10T07:48:50.626733+0000","last_clean":"2024-09-10T07:48:50.626733+0000","last_became_active":"2024-09-10T07:48:50.625576+0000","last_became_peered":"2024-09-10T07:48:50.625576+0000","last_unstale":"2024-09-10T07:48:50.626733+0000","last_undegraded":"2024-09-10T07:48:50.626733+0000","last_fullsized":"2024-09-10T07:48:50.626733+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:54.365655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606693+0000","last_change":"2024-09-10T07:48:51.622658+0000","last_active":"2024-09-10T07:48:52.606693+0000","last_peered":"2024-09-10T07:48:52.606693+0000","last_clean":"2024-09-10T07:48:52.606693+0000","last_became_active":"2024-09-10T07:48:51.622387+0000","last_became_peered":"2024-09-10T07:48:51.622387+0000","last_unstale":"2024-09-10T07:48:52.606693+0000","last_undegraded":"2024-09-10T07:48:52.606693+0000","last_fullsized":"2024-09-10T07:48:52.606693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:20.188379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607033+0000","last_change":"2024-09-10T07:48:50.617463+0000","last_active":"2024-09-10T07:48:52.607033+0000","last_peered":"2024-09-10T07:48:52.607033+0000","last_clean":"2024-09-10T07:48:52.607033+0000","last_became_active":"2024-09-10T07:48:50.617117+0000","last_became_peered":"2024-09-10T07:48:50.617117+0000","last_unstale":"2024-09-10T07:48:52.607033+0000","last_undegraded":"2024-09-10T07:48:52.607033+0000","last_fullsized":"2024-09-10T07:48:52.607033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:12.913436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609194+0000","last_change":"2024-09-10T07:48:50.609194+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609194+0000","last_undegraded":"2024-09-10T07:48:50.609194+0000","last_fullsized":"2024-09-10T07:48:50.609194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609587+0000","last_change":"2024-09-10T07:48:50.624873+0000","last_active":"2024-09-10T07:48:51.609587+0000","last_peered":"2024-09-10T07:48:51.609587+0000","last_clean":"2024-09-10T07:48:51.609587+0000","last_became_active":"2024-09-10T07:48:50.622063+0000","last_became_peered":"2024-09-10T07:48:50.622063+0000","last_unstale":"2024-09-10T07:48:51.609587+0000","last_undegraded":"2024-09-10T07:48:51.609587+0000","last_fullsized":"2024-09-10T07:48:51.609587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:49.987651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608768+0000","last_change":"2024-09-10T07:48:50.608768+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608768+0000","last_undegraded":"2024-09-10T07:48:50.608768+0000","last_fullsized":"2024-09-10T07:48:50.608768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606490+0000","last_change":"2024-09-10T07:48:50.619456+0000","last_active":"2024-09-10T07:48:52.606490+0000","last_peered":"2024-09-10T07:48:52.606490+0000","last_clean":"2024-09-10T07:48:52.606490+0000","last_became_active":"2024-09-10T07:48:50.618866+0000","last_became_peered":"2024-09-10T07:48:50.618866+0000","last_unstale":"2024-09-10T07:48:52.606490+0000","last_undegraded":"2024-09-10T07:48:52.606490+0000","last_fullsized":"2024-09-10T07:48:52.606490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:58.806978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609027+0000","last_change":"2024-09-10T07:48:50.609027+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609027+0000","last_undegraded":"2024-09-10T07:48:50.609027+0000","last_fullsized":"2024-09-10T07:48:50.609027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606169+0000","last_change":"2024-09-10T07:48:50.617552+0000","last_active":"2024-09-10T07:48:51.606169+0000","last_peered":"2024-09-10T07:48:51.606169+0000","last_clean":"2024-09-10T07:48:51.606169+0000","last_became_active":"2024-09-10T07:48:50.617260+0000","last_became_peered":"2024-09-10T07:48:50.617260+0000","last_unstale":"2024-09-10T07:48:51.606169+0000","last_undegraded":"2024-09-10T07:48:51.606169+0000","last_fullsized":"2024-09-10T07:48:51.606169+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:03.137368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634402+0000","last_change":"2024-09-10T07:48:51.625611+0000","last_active":"2024-09-10T07:48:51.634402+0000","last_peered":"2024-09-10T07:48:51.634402+0000","last_clean":"2024-09-10T07:48:51.634402+0000","last_became_active":"2024-09-10T07:48:51.625352+0000","last_became_peered":"2024-09-10T07:48:51.625352+0000","last_unstale":"2024-09-10T07:48:51.634402+0000","last_undegraded":"2024-09-10T07:48:51.634402+0000","last_fullsized":"2024-09-10T07:48:51.634402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:49.068278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622838+0000","last_change":"2024-09-10T07:48:50.621422+0000","last_active":"2024-09-10T07:48:50.622838+0000","last_peered":"2024-09-10T07:48:50.622838+0000","last_clean":"2024-09-10T07:48:50.622838+0000","last_became_active":"2024-09-10T07:48:50.621142+0000","last_became_peered":"2024-09-10T07:48:50.621142+0000","last_unstale":"2024-09-10T07:48:50.622838+0000","last_undegraded":"2024-09-10T07:48:50.622838+0000","last_fullsized":"2024-09-10T07:48:50.622838+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:25.494380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606946+0000","last_change":"2024-09-10T07:48:51.632002+0000","last_active":"2024-09-10T07:48:52.606946+0000","last_peered":"2024-09-10T07:48:52.606946+0000","last_clean":"2024-09-10T07:48:52.606946+0000","last_became_active":"2024-09-10T07:48:51.629180+0000","last_became_peered":"2024-09-10T07:48:51.629180+0000","last_unstale":"2024-09-10T07:48:52.606946+0000","last_undegraded":"2024-09-10T07:48:52.606946+0000","last_fullsized":"2024-09-10T07:48:52.606946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.610075+0000","last_change":"2024-09-10T07:48:50.622779+0000","last_active":"2024-09-10T07:48:51.610075+0000","last_peered":"2024-09-10T07:48:51.610075+0000","last_clean":"2024-09-10T07:48:51.610075+0000","last_became_active":"2024-09-10T07:48:50.622556+0000","last_became_peered":"2024-09-10T07:48:50.622556+0000","last_unstale":"2024-09-10T07:48:51.610075+0000","last_undegraded":"2024-09-10T07:48:51.610075+0000","last_fullsized":"2024-09-10T07:48:51.610075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:11.544783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638933+0000","last_change":"2024-09-10T07:48:51.628390+0000","last_active":"2024-09-10T07:48:51.638933+0000","last_peered":"2024-09-10T07:48:51.638933+0000","last_clean":"2024-09-10T07:48:51.638933+0000","last_became_active":"2024-09-10T07:48:51.628095+0000","last_became_peered":"2024-09-10T07:48:51.628095+0000","last_unstale":"2024-09-10T07:48:51.638933+0000","last_undegraded":"2024-09-10T07:48:51.638933+0000","last_fullsized":"2024-09-10T07:48:51.638933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:48.707363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622111+0000","last_change":"2024-09-10T07:48:50.619655+0000","last_active":"2024-09-10T07:48:50.622111+0000","last_peered":"2024-09-10T07:48:50.622111+0000","last_clean":"2024-09-10T07:48:50.622111+0000","last_became_active":"2024-09-10T07:48:50.619360+0000","last_became_peered":"2024-09-10T07:48:50.619360+0000","last_unstale":"2024-09-10T07:48:50.622111+0000","last_undegraded":"2024-09-10T07:48:50.622111+0000","last_fullsized":"2024-09-10T07:48:50.622111+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606864+0000","last_change":"2024-09-10T07:48:51.623695+0000","last_active":"2024-09-10T07:48:52.606864+0000","last_peered":"2024-09-10T07:48:52.606864+0000","last_clean":"2024-09-10T07:48:52.606864+0000","last_became_active":"2024-09-10T07:48:51.623049+0000","last_became_peered":"2024-09-10T07:48:51.623049+0000","last_unstale":"2024-09-10T07:48:52.606864+0000","last_undegraded":"2024-09-10T07:48:52.606864+0000","last_fullsized":"2024-09-10T07:48:52.606864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:42.211425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609403+0000","last_change":"2024-09-10T07:48:50.619068+0000","last_active":"2024-09-10T07:48:51.609403+0000","last_peered":"2024-09-10T07:48:51.609403+0000","last_clean":"2024-09-10T07:48:51.609403+0000","last_became_active":"2024-09-10T07:48:50.618824+0000","last_became_peered":"2024-09-10T07:48:50.618824+0000","last_unstale":"2024-09-10T07:48:51.609403+0000","last_undegraded":"2024-09-10T07:48:51.609403+0000","last_fullsized":"2024-09-10T07:48:51.609403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:24.707876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611851+0000","last_change":"2024-09-10T07:48:50.611851+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611851+0000","last_undegraded":"2024-09-10T07:48:50.611851+0000","last_fullsized":"2024-09-10T07:48:50.611851+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606363+0000","last_change":"2024-09-10T07:48:50.614453+0000","last_active":"2024-09-10T07:48:51.606363+0000","last_peered":"2024-09-10T07:48:51.606363+0000","last_clean":"2024-09-10T07:48:51.606363+0000","last_became_active":"2024-09-10T07:48:50.614159+0000","last_became_peered":"2024-09-10T07:48:50.614159+0000","last_unstale":"2024-09-10T07:48:51.606363+0000","last_undegraded":"2024-09-10T07:48:51.606363+0000","last_fullsized":"2024-09-10T07:48:51.606363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.792276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631226+0000","last_change":"2024-09-10T07:48:51.628493+0000","last_active":"2024-09-10T07:48:51.631226+0000","last_peered":"2024-09-10T07:48:51.631226+0000","last_clean":"2024-09-10T07:48:51.631226+0000","last_became_active":"2024-09-10T07:48:51.628229+0000","last_became_peered":"2024-09-10T07:48:51.628229+0000","last_unstale":"2024-09-10T07:48:51.631226+0000","last_undegraded":"2024-09-10T07:48:51.631226+0000","last_fullsized":"2024-09-10T07:48:51.631226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:04.162971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624400+0000","last_change":"2024-09-10T07:48:50.620900+0000","last_active":"2024-09-10T07:48:50.624400+0000","last_peered":"2024-09-10T07:48:50.624400+0000","last_clean":"2024-09-10T07:48:50.624400+0000","last_became_active":"2024-09-10T07:48:50.620412+0000","last_became_peered":"2024-09-10T07:48:50.620412+0000","last_unstale":"2024-09-10T07:48:50.624400+0000","last_undegraded":"2024-09-10T07:48:50.624400+0000","last_fullsized":"2024-09-10T07:48:50.624400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:58.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635716+0000","last_change":"2024-09-10T07:48:51.632214+0000","last_active":"2024-09-10T07:48:51.635716+0000","last_peered":"2024-09-10T07:48:51.635716+0000","last_clean":"2024-09-10T07:48:51.635716+0000","last_became_active":"2024-09-10T07:48:51.631932+0000","last_became_peered":"2024-09-10T07:48:51.631932+0000","last_unstale":"2024-09-10T07:48:51.635716+0000","last_undegraded":"2024-09-10T07:48:51.635716+0000","last_fullsized":"2024-09-10T07:48:51.635716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.618234+0000","last_change":"2024-09-10T07:48:50.615382+0000","last_active":"2024-09-10T07:48:50.618234+0000","last_peered":"2024-09-10T07:48:50.618234+0000","last_clean":"2024-09-10T07:48:50.618234+0000","last_became_active":"2024-09-10T07:48:50.614269+0000","last_became_peered":"2024-09-10T07:48:50.614269+0000","last_unstale":"2024-09-10T07:48:50.618234+0000","last_undegraded":"2024-09-10T07:48:50.618234+0000","last_fullsized":"2024-09-10T07:48:50.618234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612430+0000","last_change":"2024-09-10T07:48:50.612430+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612430+0000","last_undegraded":"2024-09-10T07:48:50.612430+0000","last_fullsized":"2024-09-10T07:48:50.612430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.645507+0000","last_change":"2024-09-10T07:48:50.624209+0000","last_active":"2024-09-10T07:48:51.645507+0000","last_peered":"2024-09-10T07:48:51.645507+0000","last_clean":"2024-09-10T07:48:51.645507+0000","last_became_active":"2024-09-10T07:48:50.623406+0000","last_became_peered":"2024-09-10T07:48:50.623406+0000","last_unstale":"2024-09-10T07:48:51.645507+0000","last_undegraded":"2024-09-10T07:48:51.645507+0000","last_fullsized":"2024-09-10T07:48:51.645507+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:29.913985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":217,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.603655+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:48:51.603655+0000","last_peered":"2024-09-10T07:48:51.603655+0000","last_clean":"2024-09-10T07:48:51.603655+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:48:51.603655+0000","last_undegraded":"2024-09-10T07:48:51.603655+0000","last_fullsized":"2024-09-10T07:48:51.603655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343688,"statfs":{"total":96636764160,"available":96607936512,"internally_reserved":0,"allocated":1146880,"data_stored":814559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":808284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:48:56.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:48:56.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.434+0000 7fa013a00640 1 Processor -- start 2024-09-10T07:48:56.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.434+0000 7fa013a00640 1 -- start start 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.435+0000 7fa013a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.435+0000 7fa013a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa00c0cf1f0 0x7fa00c0d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.435+0000 7fa013a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00c0b7920 con 0x7fa00c15e8e0 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.435+0000 7fa013a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa00c0cf6a0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.435+0000 7fa013a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c0cf940 con 0x7fa00c0d1ba0 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa00c15e8e0 0x7fa00c0d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:40670/0 (socket says 172.21.15.3:40670) 2024-09-10T07:48:56.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011000640 1 -- 172.21.15.3:0/2983191430 learned_addr learned my addr 172.21.15.3:0/2983191430 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa00be00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011a00640 1 --2- 172.21.15.3:0/2983191430 >> v2:172.21.15.70:3300/0 conn(0x7fa00c0cf1f0 0x7fa00c0d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa00b400640 1 -- 172.21.15.3:0/2983191430 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654159799 0 0) 0x7fa000001110 con 0x7fa00c15e8e0 2024-09-10T07:48:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa00b400640 1 -- 172.21.15.3:0/2983191430 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ff4003520 con 0x7fa00c15e8e0 2024-09-10T07:48:56.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011a00640 1 -- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 msgr2=0x7fa00c162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:56.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011a00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011a00640 1 -- 172.21.15.3:0/2983191430 >> v1:172.21.15.70:6789/0 conn(0x7fa00c15e8e0 legacy=0x7fa00c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa011a00640 1 -- 172.21.15.3:0/2983191430 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c0cfc20 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.436+0000 7fa00be00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.437+0000 7fa011a00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c0d32e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa004009d90 tx=0x7fa00402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80100364365f75cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa00b400640 1 -- 172.21.15.3:0/2983191430 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004031600 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa00b400640 1 -- 172.21.15.3:0/2983191430 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0040317a0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa00b400640 1 -- 172.21.15.3:0/2983191430 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004031b50 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 -- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c0cf1f0 msgr2=0x7fa00c0d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c0d32e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa004009d90 tx=0x7fa00402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 -- 172.21.15.3:0/2983191430 shutdown_connections 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c0d32e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 --2- 172.21.15.3:0/2983191430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 -- 172.21.15.3:0/2983191430 >> 172.21.15.3:0/2983191430 conn(0x7fa00c06bd00 msgr2=0x7fa00c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 -- 172.21.15.3:0/2983191430 shutdown_connections 2024-09-10T07:48:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.438+0000 7fa013a00640 1 -- 172.21.15.3:0/2983191430 wait complete. 2024-09-10T07:48:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.440+0000 7fa013a00640 1 Processor -- start 2024-09-10T07:48:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.440+0000 7fa013a00640 1 -- start start 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c1f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c15e8e0 0x7fa00c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0d3820 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0d2dc0 con 0x7fa00c15e8e0 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa013a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c0d39a0 con 0x7fa00c0d1ba0 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa011000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa011000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54648/0 (socket says 172.21.15.3:54648) 2024-09-10T07:48:56.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.441+0000 7fa011000640 1 -- 172.21.15.3:0/168703503 learned_addr learned my addr 172.21.15.3:0/168703503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa00be00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c1f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c15e8e0 0x7fa00c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 -- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 msgr2=0x7fa00c1f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c1f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 -- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c15e8e0 msgr2=0x7fa00c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c15e8e0 0x7fa00c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 -- 172.21.15.3:0/168703503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c0ce820 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa011000640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c400 tx=0x7fa00000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000017070 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa000005ce0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c15fda0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.442+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0000072d0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.443+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c160360 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.443+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00c0d32e0 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.444+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa00001a070 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.445+0000 7fa008c00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.445+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa00008a110 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.445+0000 7fa00be00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.446+0000 7fa00be00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 0x7f9fec06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009560 tx=0x7f9ff8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.450+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa00c0b7920 con 0x7fa00c0cf1f0 2024-09-10T07:48:56.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.633+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa00c0d23d0 con 0x7f9fec06be30 2024-09-10T07:48:56.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.640+0000 7fa008c00640 1 -- 172.21.15.3:0/168703503 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249857 (secure 0 0 0) 0x7fa00c0d23d0 con 0x7f9fec06be30 2024-09-10T07:48:56.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:48:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 msgr2=0x7f9fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 0x7f9fec06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009560 tx=0x7f9ff8008040 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 msgr2=0x7fa00c1444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c400 tx=0x7fa00000c8c0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 shutdown_connections 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa00c15e8e0 0x7fa00c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa00c0d1ba0 0x7fa00c1f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 --2- 172.21.15.3:0/168703503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa00c0cf1f0 0x7fa00c1444d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 >> 172.21.15.3:0/168703503 conn(0x7fa00c06bd00 msgr2=0x7fa00c0b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.643+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 shutdown_connections 2024-09-10T07:48:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.644+0000 7fa013a00640 1 -- 172.21.15.3:0/168703503 wait complete. 2024-09-10T07:48:56.659 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T07:48:55.520865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238360,"kb_used_data":5476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736360,"statfs":{"total":773094113280,"available":772850032640,"internally_reserved":0,"allocated":5607424,"data_stored":2963918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927936"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621887+0000","last_change":"2024-09-10T07:48:51.617751+0000","last_active":"2024-09-10T07:48:51.621887+0000","last_peered":"2024-09-10T07:48:51.621887+0000","last_clean":"2024-09-10T07:48:51.621887+0000","last_became_active":"2024-09-10T07:48:51.617450+0000","last_became_peered":"2024-09-10T07:48:51.617450+0000","last_unstale":"2024-09-10T07:48:51.621887+0000","last_undegraded":"2024-09-10T07:48:51.621887+0000","last_fullsized":"2024-09-10T07:48:51.621887+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632949+0000","last_change":"2024-09-10T07:48:51.620056+0000","last_active":"2024-09-10T07:48:51.632949+0000","last_peered":"2024-09-10T07:48:51.632949+0000","last_clean":"2024-09-10T07:48:51.632949+0000","last_became_active":"2024-09-10T07:48:51.619801+0000","last_became_peered":"2024-09-10T07:48:51.619801+0000","last_unstale":"2024-09-10T07:48:51.632949+0000","last_undegraded":"2024-09-10T07:48:51.632949+0000","last_fullsized":"2024-09-10T07:48:51.632949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:12.498291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617258+0000","last_change":"2024-09-10T07:48:51.612764+0000","last_active":"2024-09-10T07:48:51.617258+0000","last_peered":"2024-09-10T07:48:51.617258+0000","last_clean":"2024-09-10T07:48:51.617258+0000","last_became_active":"2024-09-10T07:48:51.611550+0000","last_became_peered":"2024-09-10T07:48:51.611550+0000","last_unstale":"2024-09-10T07:48:51.617258+0000","last_undegraded":"2024-09-10T07:48:51.617258+0000","last_fullsized":"2024-09-10T07:48:51.617258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:07.243280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611003+0000","last_change":"2024-09-10T07:48:50.611003+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611003+0000","last_undegraded":"2024-09-10T07:48:50.611003+0000","last_fullsized":"2024-09-10T07:48:50.611003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631377+0000","last_change":"2024-09-10T07:48:51.624149+0000","last_active":"2024-09-10T07:48:51.631377+0000","last_peered":"2024-09-10T07:48:51.631377+0000","last_clean":"2024-09-10T07:48:51.631377+0000","last_became_active":"2024-09-10T07:48:51.623849+0000","last_became_peered":"2024-09-10T07:48:51.623849+0000","last_unstale":"2024-09-10T07:48:51.631377+0000","last_undegraded":"2024-09-10T07:48:51.631377+0000","last_fullsized":"2024-09-10T07:48:51.631377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.308850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608067+0000","last_change":"2024-09-10T07:48:50.608067+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608067+0000","last_undegraded":"2024-09-10T07:48:50.608067+0000","last_fullsized":"2024-09-10T07:48:50.608067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639319+0000","last_change":"2024-09-10T07:48:51.628760+0000","last_active":"2024-09-10T07:48:51.639319+0000","last_peered":"2024-09-10T07:48:51.639319+0000","last_clean":"2024-09-10T07:48:51.639319+0000","last_became_active":"2024-09-10T07:48:51.628462+0000","last_became_peered":"2024-09-10T07:48:51.628462+0000","last_unstale":"2024-09-10T07:48:51.639319+0000","last_undegraded":"2024-09-10T07:48:51.639319+0000","last_fullsized":"2024-09-10T07:48:51.639319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:19.292466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614856+0000","last_change":"2024-09-10T07:48:50.614856+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614856+0000","last_undegraded":"2024-09-10T07:48:50.614856+0000","last_fullsized":"2024-09-10T07:48:50.614856+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609844+0000","last_change":"2024-09-10T07:48:50.609844+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609844+0000","last_undegraded":"2024-09-10T07:48:50.609844+0000","last_fullsized":"2024-09-10T07:48:50.609844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607402+0000","last_change":"2024-09-10T07:48:51.617674+0000","last_active":"2024-09-10T07:48:52.607402+0000","last_peered":"2024-09-10T07:48:52.607402+0000","last_clean":"2024-09-10T07:48:52.607402+0000","last_became_active":"2024-09-10T07:48:51.617378+0000","last_became_peered":"2024-09-10T07:48:51.617378+0000","last_unstale":"2024-09-10T07:48:52.607402+0000","last_undegraded":"2024-09-10T07:48:52.607402+0000","last_fullsized":"2024-09-10T07:48:52.607402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:02.393228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611643+0000","last_change":"2024-09-10T07:48:50.611643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611643+0000","last_undegraded":"2024-09-10T07:48:50.611643+0000","last_fullsized":"2024-09-10T07:48:50.611643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633458+0000","last_change":"2024-09-10T07:48:51.620489+0000","last_active":"2024-09-10T07:48:51.633458+0000","last_peered":"2024-09-10T07:48:51.633458+0000","last_clean":"2024-09-10T07:48:51.633458+0000","last_became_active":"2024-09-10T07:48:51.620237+0000","last_became_peered":"2024-09-10T07:48:51.620237+0000","last_unstale":"2024-09-10T07:48:51.633458+0000","last_undegraded":"2024-09-10T07:48:51.633458+0000","last_fullsized":"2024-09-10T07:48:51.633458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:25.714474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609643+0000","last_change":"2024-09-10T07:48:50.609643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609643+0000","last_undegraded":"2024-09-10T07:48:50.609643+0000","last_fullsized":"2024-09-10T07:48:50.609643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608771+0000","last_change":"2024-09-10T07:48:50.608771+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608771+0000","last_undegraded":"2024-09-10T07:48:50.608771+0000","last_fullsized":"2024-09-10T07:48:50.608771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609579+0000","last_change":"2024-09-10T07:48:50.609579+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609579+0000","last_undegraded":"2024-09-10T07:48:50.609579+0000","last_fullsized":"2024-09-10T07:48:50.609579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637116+0000","last_change":"2024-09-10T07:48:51.631668+0000","last_active":"2024-09-10T07:48:51.637116+0000","last_peered":"2024-09-10T07:48:51.637116+0000","last_clean":"2024-09-10T07:48:51.637116+0000","last_became_active":"2024-09-10T07:48:51.631459+0000","last_became_peered":"2024-09-10T07:48:51.631459+0000","last_unstale":"2024-09-10T07:48:51.637116+0000","last_undegraded":"2024-09-10T07:48:51.637116+0000","last_fullsized":"2024-09-10T07:48:51.637116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:12.993962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632892+0000","last_change":"2024-09-10T07:48:51.622041+0000","last_active":"2024-09-10T07:48:51.632892+0000","last_peered":"2024-09-10T07:48:51.632892+0000","last_clean":"2024-09-10T07:48:51.632892+0000","last_became_active":"2024-09-10T07:48:51.621622+0000","last_became_peered":"2024-09-10T07:48:51.621622+0000","last_unstale":"2024-09-10T07:48:51.632892+0000","last_undegraded":"2024-09-10T07:48:51.632892+0000","last_fullsized":"2024-09-10T07:48:51.632892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:50.994249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638066+0000","last_change":"2024-09-10T07:48:51.632824+0000","last_active":"2024-09-10T07:48:51.638066+0000","last_peered":"2024-09-10T07:48:51.638066+0000","last_clean":"2024-09-10T07:48:51.638066+0000","last_became_active":"2024-09-10T07:48:51.629313+0000","last_became_peered":"2024-09-10T07:48:51.629313+0000","last_unstale":"2024-09-10T07:48:51.638066+0000","last_undegraded":"2024-09-10T07:48:51.638066+0000","last_fullsized":"2024-09-10T07:48:51.638066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:25.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630089+0000","last_change":"2024-09-10T07:48:51.623911+0000","last_active":"2024-09-10T07:48:51.630089+0000","last_peered":"2024-09-10T07:48:51.630089+0000","last_clean":"2024-09-10T07:48:51.630089+0000","last_became_active":"2024-09-10T07:48:51.623645+0000","last_became_peered":"2024-09-10T07:48:51.623645+0000","last_unstale":"2024-09-10T07:48:51.630089+0000","last_undegraded":"2024-09-10T07:48:51.630089+0000","last_fullsized":"2024-09-10T07:48:51.630089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:54.995968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606836+0000","last_change":"2024-09-10T07:48:51.624623+0000","last_active":"2024-09-10T07:48:52.606836+0000","last_peered":"2024-09-10T07:48:52.606836+0000","last_clean":"2024-09-10T07:48:52.606836+0000","last_became_active":"2024-09-10T07:48:51.621627+0000","last_became_peered":"2024-09-10T07:48:51.621627+0000","last_unstale":"2024-09-10T07:48:52.606836+0000","last_undegraded":"2024-09-10T07:48:52.606836+0000","last_fullsized":"2024-09-10T07:48:52.606836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:19.844611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608827+0000","last_change":"2024-09-10T07:48:50.608827+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608827+0000","last_undegraded":"2024-09-10T07:48:50.608827+0000","last_fullsized":"2024-09-10T07:48:50.608827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605350+0000","last_change":"2024-09-10T07:48:50.605350+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605350+0000","last_undegraded":"2024-09-10T07:48:50.605350+0000","last_fullsized":"2024-09-10T07:48:50.605350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.624565+0000","last_change":"2024-09-10T07:48:51.616950+0000","last_active":"2024-09-10T07:48:51.624565+0000","last_peered":"2024-09-10T07:48:51.624565+0000","last_clean":"2024-09-10T07:48:51.624565+0000","last_became_active":"2024-09-10T07:48:51.614672+0000","last_became_peered":"2024-09-10T07:48:51.614672+0000","last_unstale":"2024-09-10T07:48:51.624565+0000","last_undegraded":"2024-09-10T07:48:51.624565+0000","last_fullsized":"2024-09-10T07:48:51.624565+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:48.008128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635731+0000","last_change":"2024-09-10T07:48:51.630986+0000","last_active":"2024-09-10T07:48:51.635731+0000","last_peered":"2024-09-10T07:48:51.635731+0000","last_clean":"2024-09-10T07:48:51.635731+0000","last_became_active":"2024-09-10T07:48:51.630754+0000","last_became_peered":"2024-09-10T07:48:51.630754+0000","last_unstale":"2024-09-10T07:48:51.635731+0000","last_undegraded":"2024-09-10T07:48:51.635731+0000","last_fullsized":"2024-09-10T07:48:51.635731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621864+0000","last_change":"2024-09-10T07:48:51.617701+0000","last_active":"2024-09-10T07:48:51.621864+0000","last_peered":"2024-09-10T07:48:51.621864+0000","last_clean":"2024-09-10T07:48:51.621864+0000","last_became_active":"2024-09-10T07:48:51.617418+0000","last_became_peered":"2024-09-10T07:48:51.617418+0000","last_unstale":"2024-09-10T07:48:51.621864+0000","last_undegraded":"2024-09-10T07:48:51.621864+0000","last_fullsized":"2024-09-10T07:48:51.621864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612838+0000","last_change":"2024-09-10T07:48:50.612838+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612838+0000","last_undegraded":"2024-09-10T07:48:50.612838+0000","last_fullsized":"2024-09-10T07:48:50.612838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630570+0000","last_change":"2024-09-10T07:48:51.620624+0000","last_active":"2024-09-10T07:48:51.630570+0000","last_peered":"2024-09-10T07:48:51.630570+0000","last_clean":"2024-09-10T07:48:51.630570+0000","last_became_active":"2024-09-10T07:48:51.619782+0000","last_became_peered":"2024-09-10T07:48:51.619782+0000","last_unstale":"2024-09-10T07:48:51.630570+0000","last_undegraded":"2024-09-10T07:48:51.630570+0000","last_fullsized":"2024-09-10T07:48:51.630570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.304141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623981+0000","last_change":"2024-09-10T07:48:51.616307+0000","last_active":"2024-09-10T07:48:51.623981+0000","last_peered":"2024-09-10T07:48:51.623981+0000","last_clean":"2024-09-10T07:48:51.623981+0000","last_became_active":"2024-09-10T07:48:51.615851+0000","last_became_peered":"2024-09-10T07:48:51.615851+0000","last_unstale":"2024-09-10T07:48:51.623981+0000","last_undegraded":"2024-09-10T07:48:51.623981+0000","last_fullsized":"2024-09-10T07:48:51.623981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:05.714041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623649+0000","last_change":"2024-09-10T07:48:51.616551+0000","last_active":"2024-09-10T07:48:51.623649+0000","last_peered":"2024-09-10T07:48:51.623649+0000","last_clean":"2024-09-10T07:48:51.623649+0000","last_became_active":"2024-09-10T07:48:51.616241+0000","last_became_peered":"2024-09-10T07:48:51.616241+0000","last_unstale":"2024-09-10T07:48:51.623649+0000","last_undegraded":"2024-09-10T07:48:51.623649+0000","last_fullsized":"2024-09-10T07:48:51.623649+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633988+0000","last_change":"2024-09-10T07:48:51.628969+0000","last_active":"2024-09-10T07:48:51.633988+0000","last_peered":"2024-09-10T07:48:51.633988+0000","last_clean":"2024-09-10T07:48:51.633988+0000","last_became_active":"2024-09-10T07:48:51.628200+0000","last_became_peered":"2024-09-10T07:48:51.628200+0000","last_unstale":"2024-09-10T07:48:51.633988+0000","last_undegraded":"2024-09-10T07:48:51.633988+0000","last_fullsized":"2024-09-10T07:48:51.633988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:12.551381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.613199+0000","last_change":"2024-09-10T07:48:50.613199+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.613199+0000","last_undegraded":"2024-09-10T07:48:50.613199+0000","last_fullsized":"2024-09-10T07:48:50.613199+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614337+0000","last_change":"2024-09-10T07:48:50.614337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614337+0000","last_undegraded":"2024-09-10T07:48:50.614337+0000","last_fullsized":"2024-09-10T07:48:50.614337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607331+0000","last_change":"2024-09-10T07:48:50.607331+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607331+0000","last_undegraded":"2024-09-10T07:48:50.607331+0000","last_fullsized":"2024-09-10T07:48:50.607331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.628793+0000","last_change":"2024-09-10T07:48:50.627953+0000","last_active":"2024-09-10T07:48:50.628793+0000","last_peered":"2024-09-10T07:48:50.628793+0000","last_clean":"2024-09-10T07:48:50.628793+0000","last_became_active":"2024-09-10T07:48:50.627666+0000","last_became_peered":"2024-09-10T07:48:50.627666+0000","last_unstale":"2024-09-10T07:48:50.628793+0000","last_undegraded":"2024-09-10T07:48:50.628793+0000","last_fullsized":"2024-09-10T07:48:50.628793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:57.026249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630003+0000","last_change":"2024-09-10T07:48:51.621847+0000","last_active":"2024-09-10T07:48:51.630003+0000","last_peered":"2024-09-10T07:48:51.630003+0000","last_clean":"2024-09-10T07:48:51.630003+0000","last_became_active":"2024-09-10T07:48:51.621554+0000","last_became_peered":"2024-09-10T07:48:51.621554+0000","last_unstale":"2024-09-10T07:48:51.630003+0000","last_undegraded":"2024-09-10T07:48:51.630003+0000","last_fullsized":"2024-09-10T07:48:51.630003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634481+0000","last_change":"2024-09-10T07:48:50.619796+0000","last_active":"2024-09-10T07:48:51.634481+0000","last_peered":"2024-09-10T07:48:51.634481+0000","last_clean":"2024-09-10T07:48:51.634481+0000","last_became_active":"2024-09-10T07:48:50.619535+0000","last_became_peered":"2024-09-10T07:48:50.619535+0000","last_unstale":"2024-09-10T07:48:51.634481+0000","last_undegraded":"2024-09-10T07:48:51.634481+0000","last_fullsized":"2024-09-10T07:48:51.634481+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:44.514806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633602+0000","last_change":"2024-09-10T07:48:51.629435+0000","last_active":"2024-09-10T07:48:51.633602+0000","last_peered":"2024-09-10T07:48:51.633602+0000","last_clean":"2024-09-10T07:48:51.633602+0000","last_became_active":"2024-09-10T07:48:51.628928+0000","last_became_peered":"2024-09-10T07:48:51.628928+0000","last_unstale":"2024-09-10T07:48:51.633602+0000","last_undegraded":"2024-09-10T07:48:51.633602+0000","last_fullsized":"2024-09-10T07:48:51.633602+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622265+0000","last_change":"2024-09-10T07:48:50.619441+0000","last_active":"2024-09-10T07:48:50.622265+0000","last_peered":"2024-09-10T07:48:50.622265+0000","last_clean":"2024-09-10T07:48:50.622265+0000","last_became_active":"2024-09-10T07:48:50.619148+0000","last_became_peered":"2024-09-10T07:48:50.619148+0000","last_unstale":"2024-09-10T07:48:50.622265+0000","last_undegraded":"2024-09-10T07:48:50.622265+0000","last_fullsized":"2024-09-10T07:48:50.622265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.560270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610493+0000","last_change":"2024-09-10T07:48:50.610493+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610493+0000","last_undegraded":"2024-09-10T07:48:50.610493+0000","last_fullsized":"2024-09-10T07:48:50.610493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639518+0000","last_change":"2024-09-10T07:48:50.617904+0000","last_active":"2024-09-10T07:48:51.639518+0000","last_peered":"2024-09-10T07:48:51.639518+0000","last_clean":"2024-09-10T07:48:51.639518+0000","last_became_active":"2024-09-10T07:48:50.617656+0000","last_became_peered":"2024-09-10T07:48:50.617656+0000","last_unstale":"2024-09-10T07:48:51.639518+0000","last_undegraded":"2024-09-10T07:48:51.639518+0000","last_fullsized":"2024-09-10T07:48:51.639518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.305959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.627946+0000","last_change":"2024-09-10T07:48:51.620554+0000","last_active":"2024-09-10T07:48:51.627946+0000","last_peered":"2024-09-10T07:48:51.627946+0000","last_clean":"2024-09-10T07:48:51.627946+0000","last_became_active":"2024-09-10T07:48:51.619526+0000","last_became_peered":"2024-09-10T07:48:51.619526+0000","last_unstale":"2024-09-10T07:48:51.627946+0000","last_undegraded":"2024-09-10T07:48:51.627946+0000","last_fullsized":"2024-09-10T07:48:51.627946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:33.856786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.625037+0000","last_change":"2024-09-10T07:48:50.623162+0000","last_active":"2024-09-10T07:48:50.625037+0000","last_peered":"2024-09-10T07:48:50.625037+0000","last_clean":"2024-09-10T07:48:50.625037+0000","last_became_active":"2024-09-10T07:48:50.622765+0000","last_became_peered":"2024-09-10T07:48:50.622765+0000","last_unstale":"2024-09-10T07:48:50.625037+0000","last_undegraded":"2024-09-10T07:48:50.625037+0000","last_fullsized":"2024-09-10T07:48:50.625037+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:24.341239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606939+0000","last_change":"2024-09-10T07:48:51.628666+0000","last_active":"2024-09-10T07:48:52.606939+0000","last_peered":"2024-09-10T07:48:52.606939+0000","last_clean":"2024-09-10T07:48:52.606939+0000","last_became_active":"2024-09-10T07:48:51.628411+0000","last_became_peered":"2024-09-10T07:48:51.628411+0000","last_unstale":"2024-09-10T07:48:52.606939+0000","last_undegraded":"2024-09-10T07:48:52.606939+0000","last_fullsized":"2024-09-10T07:48:52.606939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:13.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635593+0000","last_change":"2024-09-10T07:48:50.616830+0000","last_active":"2024-09-10T07:48:51.635593+0000","last_peered":"2024-09-10T07:48:51.635593+0000","last_clean":"2024-09-10T07:48:51.635593+0000","last_became_active":"2024-09-10T07:48:50.616586+0000","last_became_peered":"2024-09-10T07:48:50.616586+0000","last_unstale":"2024-09-10T07:48:51.635593+0000","last_undegraded":"2024-09-10T07:48:51.635593+0000","last_fullsized":"2024-09-10T07:48:51.635593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:25.042914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606676+0000","last_change":"2024-09-10T07:48:51.629828+0000","last_active":"2024-09-10T07:48:52.606676+0000","last_peered":"2024-09-10T07:48:52.606676+0000","last_clean":"2024-09-10T07:48:52.606676+0000","last_became_active":"2024-09-10T07:48:51.628172+0000","last_became_peered":"2024-09-10T07:48:51.628172+0000","last_unstale":"2024-09-10T07:48:52.606676+0000","last_undegraded":"2024-09-10T07:48:52.606676+0000","last_fullsized":"2024-09-10T07:48:52.606676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609352+0000","last_change":"2024-09-10T07:48:50.627131+0000","last_active":"2024-09-10T07:48:51.609352+0000","last_peered":"2024-09-10T07:48:51.609352+0000","last_clean":"2024-09-10T07:48:51.609352+0000","last_became_active":"2024-09-10T07:48:50.626968+0000","last_became_peered":"2024-09-10T07:48:50.626968+0000","last_unstale":"2024-09-10T07:48:51.609352+0000","last_undegraded":"2024-09-10T07:48:51.609352+0000","last_fullsized":"2024-09-10T07:48:51.609352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:11.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634368+0000","last_change":"2024-09-10T07:48:51.624704+0000","last_active":"2024-09-10T07:48:51.634368+0000","last_peered":"2024-09-10T07:48:51.634368+0000","last_clean":"2024-09-10T07:48:51.634368+0000","last_became_active":"2024-09-10T07:48:51.624391+0000","last_became_peered":"2024-09-10T07:48:51.624391+0000","last_unstale":"2024-09-10T07:48:51.634368+0000","last_undegraded":"2024-09-10T07:48:51.634368+0000","last_fullsized":"2024-09-10T07:48:51.634368+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:39.576381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606654+0000","last_change":"2024-09-10T07:48:50.618315+0000","last_active":"2024-09-10T07:48:52.606654+0000","last_peered":"2024-09-10T07:48:52.606654+0000","last_clean":"2024-09-10T07:48:52.606654+0000","last_became_active":"2024-09-10T07:48:50.618022+0000","last_became_peered":"2024-09-10T07:48:50.618022+0000","last_unstale":"2024-09-10T07:48:52.606654+0000","last_undegraded":"2024-09-10T07:48:52.606654+0000","last_fullsized":"2024-09-10T07:48:52.606654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:24.272006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.622150+0000","last_change":"2024-09-10T07:48:51.616784+0000","last_active":"2024-09-10T07:48:51.622150+0000","last_peered":"2024-09-10T07:48:51.622150+0000","last_clean":"2024-09-10T07:48:51.622150+0000","last_became_active":"2024-09-10T07:48:51.616520+0000","last_became_peered":"2024-09-10T07:48:51.616520+0000","last_unstale":"2024-09-10T07:48:51.622150+0000","last_undegraded":"2024-09-10T07:48:51.622150+0000","last_fullsized":"2024-09-10T07:48:51.622150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:47.124772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.604652+0000","last_change":"2024-09-10T07:48:50.615394+0000","last_active":"2024-09-10T07:48:51.604652+0000","last_peered":"2024-09-10T07:48:51.604652+0000","last_clean":"2024-09-10T07:48:51.604652+0000","last_became_active":"2024-09-10T07:48:50.615139+0000","last_became_peered":"2024-09-10T07:48:50.615139+0000","last_unstale":"2024-09-10T07:48:51.604652+0000","last_undegraded":"2024-09-10T07:48:51.604652+0000","last_fullsized":"2024-09-10T07:48:51.604652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623257+0000","last_change":"2024-09-10T07:48:51.619010+0000","last_active":"2024-09-10T07:48:51.623257+0000","last_peered":"2024-09-10T07:48:51.623257+0000","last_clean":"2024-09-10T07:48:51.623257+0000","last_became_active":"2024-09-10T07:48:51.618676+0000","last_became_peered":"2024-09-10T07:48:51.618676+0000","last_unstale":"2024-09-10T07:48:51.623257+0000","last_undegraded":"2024-09-10T07:48:51.623257+0000","last_fullsized":"2024-09-10T07:48:51.623257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:47.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607323+0000","last_change":"2024-09-10T07:48:50.620699+0000","last_active":"2024-09-10T07:48:52.607323+0000","last_peered":"2024-09-10T07:48:52.607323+0000","last_clean":"2024-09-10T07:48:52.607323+0000","last_became_active":"2024-09-10T07:48:50.620417+0000","last_became_peered":"2024-09-10T07:48:50.620417+0000","last_unstale":"2024-09-10T07:48:52.607323+0000","last_undegraded":"2024-09-10T07:48:52.607323+0000","last_fullsized":"2024-09-10T07:48:52.607323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:31.155316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610526+0000","last_change":"2024-09-10T07:48:50.610526+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610526+0000","last_undegraded":"2024-09-10T07:48:50.610526+0000","last_fullsized":"2024-09-10T07:48:50.610526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644403+0000","last_change":"2024-09-10T07:48:50.620839+0000","last_active":"2024-09-10T07:48:51.644403+0000","last_peered":"2024-09-10T07:48:51.644403+0000","last_clean":"2024-09-10T07:48:51.644403+0000","last_became_active":"2024-09-10T07:48:50.620581+0000","last_became_peered":"2024-09-10T07:48:50.620581+0000","last_unstale":"2024-09-10T07:48:51.644403+0000","last_undegraded":"2024-09-10T07:48:51.644403+0000","last_fullsized":"2024-09-10T07:48:51.644403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639187+0000","last_change":"2024-09-10T07:48:51.630681+0000","last_active":"2024-09-10T07:48:51.639187+0000","last_peered":"2024-09-10T07:48:51.639187+0000","last_clean":"2024-09-10T07:48:51.639187+0000","last_became_active":"2024-09-10T07:48:51.630321+0000","last_became_peered":"2024-09-10T07:48:51.630321+0000","last_unstale":"2024-09-10T07:48:51.639187+0000","last_undegraded":"2024-09-10T07:48:51.639187+0000","last_fullsized":"2024-09-10T07:48:51.639187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:11.469475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606961+0000","last_change":"2024-09-10T07:48:50.620804+0000","last_active":"2024-09-10T07:48:52.606961+0000","last_peered":"2024-09-10T07:48:52.606961+0000","last_clean":"2024-09-10T07:48:52.606961+0000","last_became_active":"2024-09-10T07:48:50.620518+0000","last_became_peered":"2024-09-10T07:48:50.620518+0000","last_unstale":"2024-09-10T07:48:52.606961+0000","last_undegraded":"2024-09-10T07:48:52.606961+0000","last_fullsized":"2024-09-10T07:48:52.606961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:26.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637082+0000","last_change":"2024-09-10T07:48:51.632456+0000","last_active":"2024-09-10T07:48:51.637082+0000","last_peered":"2024-09-10T07:48:51.637082+0000","last_clean":"2024-09-10T07:48:51.637082+0000","last_became_active":"2024-09-10T07:48:51.632175+0000","last_became_peered":"2024-09-10T07:48:51.632175+0000","last_unstale":"2024-09-10T07:48:51.637082+0000","last_undegraded":"2024-09-10T07:48:51.637082+0000","last_fullsized":"2024-09-10T07:48:51.637082+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:59.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624685+0000","last_change":"2024-09-10T07:48:50.623794+0000","last_active":"2024-09-10T07:48:50.624685+0000","last_peered":"2024-09-10T07:48:50.624685+0000","last_clean":"2024-09-10T07:48:50.624685+0000","last_became_active":"2024-09-10T07:48:50.623196+0000","last_became_peered":"2024-09-10T07:48:50.623196+0000","last_unstale":"2024-09-10T07:48:50.624685+0000","last_undegraded":"2024-09-10T07:48:50.624685+0000","last_fullsized":"2024-09-10T07:48:50.624685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605758+0000","last_change":"2024-09-10T07:48:50.605758+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605758+0000","last_undegraded":"2024-09-10T07:48:50.605758+0000","last_fullsized":"2024-09-10T07:48:50.605758+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.608400+0000","last_change":"2024-09-10T07:48:50.617395+0000","last_active":"2024-09-10T07:48:51.608400+0000","last_peered":"2024-09-10T07:48:51.608400+0000","last_clean":"2024-09-10T07:48:51.608400+0000","last_became_active":"2024-09-10T07:48:50.615852+0000","last_became_peered":"2024-09-10T07:48:50.615852+0000","last_unstale":"2024-09-10T07:48:51.608400+0000","last_undegraded":"2024-09-10T07:48:51.608400+0000","last_fullsized":"2024-09-10T07:48:51.608400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605337+0000","last_change":"2024-09-10T07:48:50.605337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605337+0000","last_undegraded":"2024-09-10T07:48:50.605337+0000","last_fullsized":"2024-09-10T07:48:50.605337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624312+0000","last_change":"2024-09-10T07:48:50.620280+0000","last_active":"2024-09-10T07:48:50.624312+0000","last_peered":"2024-09-10T07:48:50.624312+0000","last_clean":"2024-09-10T07:48:50.624312+0000","last_became_active":"2024-09-10T07:48:50.619988+0000","last_became_peered":"2024-09-10T07:48:50.619988+0000","last_unstale":"2024-09-10T07:48:50.624312+0000","last_undegraded":"2024-09-10T07:48:50.624312+0000","last_fullsized":"2024-09-10T07:48:50.624312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:03.156373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632347+0000","last_change":"2024-09-10T07:48:51.616228+0000","last_active":"2024-09-10T07:48:51.632347+0000","last_peered":"2024-09-10T07:48:51.632347+0000","last_clean":"2024-09-10T07:48:51.632347+0000","last_became_active":"2024-09-10T07:48:51.615946+0000","last_became_peered":"2024-09-10T07:48:51.615946+0000","last_unstale":"2024-09-10T07:48:51.632347+0000","last_undegraded":"2024-09-10T07:48:51.632347+0000","last_fullsized":"2024-09-10T07:48:51.632347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606882+0000","last_change":"2024-09-10T07:48:50.615137+0000","last_active":"2024-09-10T07:48:52.606882+0000","last_peered":"2024-09-10T07:48:52.606882+0000","last_clean":"2024-09-10T07:48:52.606882+0000","last_became_active":"2024-09-10T07:48:50.614618+0000","last_became_peered":"2024-09-10T07:48:50.614618+0000","last_unstale":"2024-09-10T07:48:52.606882+0000","last_undegraded":"2024-09-10T07:48:52.606882+0000","last_fullsized":"2024-09-10T07:48:52.606882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:16.791861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.606876+0000","last_change":"2024-09-10T07:48:50.606876+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.606876+0000","last_undegraded":"2024-09-10T07:48:50.606876+0000","last_fullsized":"2024-09-10T07:48:50.606876+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624888+0000","last_change":"2024-09-10T07:48:50.622636+0000","last_active":"2024-09-10T07:48:50.624888+0000","last_peered":"2024-09-10T07:48:50.624888+0000","last_clean":"2024-09-10T07:48:50.624888+0000","last_became_active":"2024-09-10T07:48:50.621702+0000","last_became_peered":"2024-09-10T07:48:50.621702+0000","last_unstale":"2024-09-10T07:48:50.624888+0000","last_undegraded":"2024-09-10T07:48:50.624888+0000","last_fullsized":"2024-09-10T07:48:50.624888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:21.875313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607227+0000","last_change":"2024-09-10T07:48:50.607227+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607227+0000","last_undegraded":"2024-09-10T07:48:50.607227+0000","last_fullsized":"2024-09-10T07:48:50.607227+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644901+0000","last_change":"2024-09-10T07:48:50.621912+0000","last_active":"2024-09-10T07:48:51.644901+0000","last_peered":"2024-09-10T07:48:51.644901+0000","last_clean":"2024-09-10T07:48:51.644901+0000","last_became_active":"2024-09-10T07:48:50.620580+0000","last_became_peered":"2024-09-10T07:48:50.620580+0000","last_unstale":"2024-09-10T07:48:51.644901+0000","last_undegraded":"2024-09-10T07:48:51.644901+0000","last_fullsized":"2024-09-10T07:48:51.644901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637323+0000","last_change":"2024-09-10T07:48:51.634091+0000","last_active":"2024-09-10T07:48:51.637323+0000","last_peered":"2024-09-10T07:48:51.637323+0000","last_clean":"2024-09-10T07:48:51.637323+0000","last_became_active":"2024-09-10T07:48:51.633805+0000","last_became_peered":"2024-09-10T07:48:51.633805+0000","last_unstale":"2024-09-10T07:48:51.637323+0000","last_undegraded":"2024-09-10T07:48:51.637323+0000","last_fullsized":"2024-09-10T07:48:51.637323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.641630+0000","last_change":"2024-09-10T07:48:50.616978+0000","last_active":"2024-09-10T07:48:51.641630+0000","last_peered":"2024-09-10T07:48:51.641630+0000","last_clean":"2024-09-10T07:48:51.641630+0000","last_became_active":"2024-09-10T07:48:50.616723+0000","last_became_peered":"2024-09-10T07:48:50.616723+0000","last_unstale":"2024-09-10T07:48:51.641630+0000","last_undegraded":"2024-09-10T07:48:51.641630+0000","last_fullsized":"2024-09-10T07:48:51.641630+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617278+0000","last_change":"2024-09-10T07:48:51.612987+0000","last_active":"2024-09-10T07:48:51.617278+0000","last_peered":"2024-09-10T07:48:51.617278+0000","last_clean":"2024-09-10T07:48:51.617278+0000","last_became_active":"2024-09-10T07:48:51.611732+0000","last_became_peered":"2024-09-10T07:48:51.611732+0000","last_unstale":"2024-09-10T07:48:51.617278+0000","last_undegraded":"2024-09-10T07:48:51.617278+0000","last_fullsized":"2024-09-10T07:48:51.617278+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.626733+0000","last_change":"2024-09-10T07:48:50.625853+0000","last_active":"2024-09-10T07:48:50.626733+0000","last_peered":"2024-09-10T07:48:50.626733+0000","last_clean":"2024-09-10T07:48:50.626733+0000","last_became_active":"2024-09-10T07:48:50.625576+0000","last_became_peered":"2024-09-10T07:48:50.625576+0000","last_unstale":"2024-09-10T07:48:50.626733+0000","last_undegraded":"2024-09-10T07:48:50.626733+0000","last_fullsized":"2024-09-10T07:48:50.626733+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:54.365655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606693+0000","last_change":"2024-09-10T07:48:51.622658+0000","last_active":"2024-09-10T07:48:52.606693+0000","last_peered":"2024-09-10T07:48:52.606693+0000","last_clean":"2024-09-10T07:48:52.606693+0000","last_became_active":"2024-09-10T07:48:51.622387+0000","last_became_peered":"2024-09-10T07:48:51.622387+0000","last_unstale":"2024-09-10T07:48:52.606693+0000","last_undegraded":"2024-09-10T07:48:52.606693+0000","last_fullsized":"2024-09-10T07:48:52.606693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:20.188379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607033+0000","last_change":"2024-09-10T07:48:50.617463+0000","last_active":"2024-09-10T07:48:52.607033+0000","last_peered":"2024-09-10T07:48:52.607033+0000","last_clean":"2024-09-10T07:48:52.607033+0000","last_became_active":"2024-09-10T07:48:50.617117+0000","last_became_peered":"2024-09-10T07:48:50.617117+0000","last_unstale":"2024-09-10T07:48:52.607033+0000","last_undegraded":"2024-09-10T07:48:52.607033+0000","last_fullsized":"2024-09-10T07:48:52.607033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:12.913436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609194+0000","last_change":"2024-09-10T07:48:50.609194+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609194+0000","last_undegraded":"2024-09-10T07:48:50.609194+0000","last_fullsized":"2024-09-10T07:48:50.609194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609587+0000","last_change":"2024-09-10T07:48:50.624873+0000","last_active":"2024-09-10T07:48:51.609587+0000","last_peered":"2024-09-10T07:48:51.609587+0000","last_clean":"2024-09-10T07:48:51.609587+0000","last_became_active":"2024-09-10T07:48:50.622063+0000","last_became_peered":"2024-09-10T07:48:50.622063+0000","last_unstale":"2024-09-10T07:48:51.609587+0000","last_undegraded":"2024-09-10T07:48:51.609587+0000","last_fullsized":"2024-09-10T07:48:51.609587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:49.987651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608768+0000","last_change":"2024-09-10T07:48:50.608768+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608768+0000","last_undegraded":"2024-09-10T07:48:50.608768+0000","last_fullsized":"2024-09-10T07:48:50.608768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606490+0000","last_change":"2024-09-10T07:48:50.619456+0000","last_active":"2024-09-10T07:48:52.606490+0000","last_peered":"2024-09-10T07:48:52.606490+0000","last_clean":"2024-09-10T07:48:52.606490+0000","last_became_active":"2024-09-10T07:48:50.618866+0000","last_became_peered":"2024-09-10T07:48:50.618866+0000","last_unstale":"2024-09-10T07:48:52.606490+0000","last_undegraded":"2024-09-10T07:48:52.606490+0000","last_fullsized":"2024-09-10T07:48:52.606490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:58.806978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609027+0000","last_change":"2024-09-10T07:48:50.609027+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609027+0000","last_undegraded":"2024-09-10T07:48:50.609027+0000","last_fullsized":"2024-09-10T07:48:50.609027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606169+0000","last_change":"2024-09-10T07:48:50.617552+0000","last_active":"2024-09-10T07:48:51.606169+0000","last_peered":"2024-09-10T07:48:51.606169+0000","last_clean":"2024-09-10T07:48:51.606169+0000","last_became_active":"2024-09-10T07:48:50.617260+0000","last_became_peered":"2024-09-10T07:48:50.617260+0000","last_unstale":"2024-09-10T07:48:51.606169+0000","last_undegraded":"2024-09-10T07:48:51.606169+0000","last_fullsized":"2024-09-10T07:48:51.606169+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:03.137368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634402+0000","last_change":"2024-09-10T07:48:51.625611+0000","last_active":"2024-09-10T07:48:51.634402+0000","last_peered":"2024-09-10T07:48:51.634402+0000","last_clean":"2024-09-10T07:48:51.634402+0000","last_became_active":"2024-09-10T07:48:51.625352+0000","last_became_peered":"2024-09-10T07:48:51.625352+0000","last_unstale":"2024-09-10T07:48:51.634402+0000","last_undegraded":"2024-09-10T07:48:51.634402+0000","last_fullsized":"2024-09-10T07:48:51.634402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:49.068278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622838+0000","last_change":"2024-09-10T07:48:50.621422+0000","last_active":"2024-09-10T07:48:50.622838+0000","last_peered":"2024-09-10T07:48:50.622838+0000","last_clean":"2024-09-10T07:48:50.622838+0000","last_became_active":"2024-09-10T07:48:50.621142+0000","last_became_peered":"2024-09-10T07:48:50.621142+0000","last_unstale":"2024-09-10T07:48:50.622838+0000","last_undegraded":"2024-09-10T07:48:50.622838+0000","last_fullsized":"2024-09-10T07:48:50.622838+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:25.494380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606946+0000","last_change":"2024-09-10T07:48:51.632002+0000","last_active":"2024-09-10T07:48:52.606946+0000","last_peered":"2024-09-10T07:48:52.606946+0000","last_clean":"2024-09-10T07:48:52.606946+0000","last_became_active":"2024-09-10T07:48:51.629180+0000","last_became_peered":"2024-09-10T07:48:51.629180+0000","last_unstale":"2024-09-10T07:48:52.606946+0000","last_undegraded":"2024-09-10T07:48:52.606946+0000","last_fullsized":"2024-09-10T07:48:52.606946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.610075+0000","last_change":"2024-09-10T07:48:50.622779+0000","last_active":"2024-09-10T07:48:51.610075+0000","last_peered":"2024-09-10T07:48:51.610075+0000","last_clean":"2024-09-10T07:48:51.610075+0000","last_became_active":"2024-09-10T07:48:50.622556+0000","last_became_peered":"2024-09-10T07:48:50.622556+0000","last_unstale":"2024-09-10T07:48:51.610075+0000","last_undegraded":"2024-09-10T07:48:51.610075+0000","last_fullsized":"2024-09-10T07:48:51.610075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:11.544783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638933+0000","last_change":"2024-09-10T07:48:51.628390+0000","last_active":"2024-09-10T07:48:51.638933+0000","last_peered":"2024-09-10T07:48:51.638933+0000","last_clean":"2024-09-10T07:48:51.638933+0000","last_became_active":"2024-09-10T07:48:51.628095+0000","last_became_peered":"2024-09-10T07:48:51.628095+0000","last_unstale":"2024-09-10T07:48:51.638933+0000","last_undegraded":"2024-09-10T07:48:51.638933+0000","last_fullsized":"2024-09-10T07:48:51.638933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:48.707363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622111+0000","last_change":"2024-09-10T07:48:50.619655+0000","last_active":"2024-09-10T07:48:50.622111+0000","last_peered":"2024-09-10T07:48:50.622111+0000","last_clean":"2024-09-10T07:48:50.622111+0000","last_became_active":"2024-09-10T07:48:50.619360+0000","last_became_peered":"2024-09-10T07:48:50.619360+0000","last_unstale":"2024-09-10T07:48:50.622111+0000","last_undegraded":"2024-09-10T07:48:50.622111+0000","last_fullsized":"2024-09-10T07:48:50.622111+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606864+0000","last_change":"2024-09-10T07:48:51.623695+0000","last_active":"2024-09-10T07:48:52.606864+0000","last_peered":"2024-09-10T07:48:52.606864+0000","last_clean":"2024-09-10T07:48:52.606864+0000","last_became_active":"2024-09-10T07:48:51.623049+0000","last_became_peered":"2024-09-10T07:48:51.623049+0000","last_unstale":"2024-09-10T07:48:52.606864+0000","last_undegraded":"2024-09-10T07:48:52.606864+0000","last_fullsized":"2024-09-10T07:48:52.606864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:42.211425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609403+0000","last_change":"2024-09-10T07:48:50.619068+0000","last_active":"2024-09-10T07:48:51.609403+0000","last_peered":"2024-09-10T07:48:51.609403+0000","last_clean":"2024-09-10T07:48:51.609403+0000","last_became_active":"2024-09-10T07:48:50.618824+0000","last_became_peered":"2024-09-10T07:48:50.618824+0000","last_unstale":"2024-09-10T07:48:51.609403+0000","last_undegraded":"2024-09-10T07:48:51.609403+0000","last_fullsized":"2024-09-10T07:48:51.609403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:24.707876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611851+0000","last_change":"2024-09-10T07:48:50.611851+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611851+0000","last_undegraded":"2024-09-10T07:48:50.611851+0000","last_fullsized":"2024-09-10T07:48:50.611851+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606363+0000","last_change":"2024-09-10T07:48:50.614453+0000","last_active":"2024-09-10T07:48:51.606363+0000","last_peered":"2024-09-10T07:48:51.606363+0000","last_clean":"2024-09-10T07:48:51.606363+0000","last_became_active":"2024-09-10T07:48:50.614159+0000","last_became_peered":"2024-09-10T07:48:50.614159+0000","last_unstale":"2024-09-10T07:48:51.606363+0000","last_undegraded":"2024-09-10T07:48:51.606363+0000","last_fullsized":"2024-09-10T07:48:51.606363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.792276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631226+0000","last_change":"2024-09-10T07:48:51.628493+0000","last_active":"2024-09-10T07:48:51.631226+0000","last_peered":"2024-09-10T07:48:51.631226+0000","last_clean":"2024-09-10T07:48:51.631226+0000","last_became_active":"2024-09-10T07:48:51.628229+0000","last_became_peered":"2024-09-10T07:48:51.628229+0000","last_unstale":"2024-09-10T07:48:51.631226+0000","last_undegraded":"2024-09-10T07:48:51.631226+0000","last_fullsized":"2024-09-10T07:48:51.631226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:04.162971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624400+0000","last_change":"2024-09-10T07:48:50.620900+0000","last_active":"2024-09-10T07:48:50.624400+0000","last_peered":"2024-09-10T07:48:50.624400+0000","last_clean":"2024-09-10T07:48:50.624400+0000","last_became_active":"2024-09-10T07:48:50.620412+0000","last_became_peered":"2024-09-10T07:48:50.620412+0000","last_unstale":"2024-09-10T07:48:50.624400+0000","last_undegraded":"2024-09-10T07:48:50.624400+0000","last_fullsized":"2024-09-10T07:48:50.624400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:58.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635716+0000","last_change":"2024-09-10T07:48:51.632214+0000","last_active":"2024-09-10T07:48:51.635716+0000","last_peered":"2024-09-10T07:48:51.635716+0000","last_clean":"2024-09-10T07:48:51.635716+0000","last_became_active":"2024-09-10T07:48:51.631932+0000","last_became_peered":"2024-09-10T07:48:51.631932+0000","last_unstale":"2024-09-10T07:48:51.635716+0000","last_undegraded":"2024-09-10T07:48:51.635716+0000","last_fullsized":"2024-09-10T07:48:51.635716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.618234+0000","last_change":"2024-09-10T07:48:50.615382+0000","last_active":"2024-09-10T07:48:50.618234+0000","last_peered":"2024-09-10T07:48:50.618234+0000","last_clean":"2024-09-10T07:48:50.618234+0000","last_became_active":"2024-09-10T07:48:50.614269+0000","last_became_peered":"2024-09-10T07:48:50.614269+0000","last_unstale":"2024-09-10T07:48:50.618234+0000","last_undegraded":"2024-09-10T07:48:50.618234+0000","last_fullsized":"2024-09-10T07:48:50.618234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612430+0000","last_change":"2024-09-10T07:48:50.612430+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612430+0000","last_undegraded":"2024-09-10T07:48:50.612430+0000","last_fullsized":"2024-09-10T07:48:50.612430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.645507+0000","last_change":"2024-09-10T07:48:50.624209+0000","last_active":"2024-09-10T07:48:51.645507+0000","last_peered":"2024-09-10T07:48:51.645507+0000","last_clean":"2024-09-10T07:48:51.645507+0000","last_became_active":"2024-09-10T07:48:50.623406+0000","last_became_peered":"2024-09-10T07:48:50.623406+0000","last_unstale":"2024-09-10T07:48:51.645507+0000","last_undegraded":"2024-09-10T07:48:51.645507+0000","last_fullsized":"2024-09-10T07:48:51.645507+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:29.913985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":217,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.603655+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:48:51.603655+0000","last_peered":"2024-09-10T07:48:51.603655+0000","last_clean":"2024-09-10T07:48:51.603655+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:48:51.603655+0000","last_undegraded":"2024-09-10T07:48:51.603655+0000","last_fullsized":"2024-09-10T07:48:51.603655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343688,"statfs":{"total":96636764160,"available":96607936512,"internally_reserved":0,"allocated":1146880,"data_stored":814559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":808284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:48:56.664 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3c is not active+clean 2024-09-10T07:48:56.664 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.611003+0000', 'last_change': '2024-09-10T07:48:50.611003+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.611003+0000', 'last_undegraded': '2024-09-10T07:48:50.611003+0000', 'last_fullsized': '2024-09-10T07:48:50.611003+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.664 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-10T07:48:56.664 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.608067+0000', 'last_change': '2024-09-10T07:48:50.608067+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.608067+0000', 'last_undegraded': '2024-09-10T07:48:50.608067+0000', 'last_fullsized': '2024-09-10T07:48:50.608067+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:PG 7.38 is not active+clean 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.614856+0000', 'last_change': '2024-09-10T07:48:50.614856+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.614856+0000', 'last_undegraded': '2024-09-10T07:48:50.614856+0000', 'last_fullsized': '2024-09-10T07:48:50.614856+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.609844+0000', 'last_change': '2024-09-10T07:48:50.609844+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.609844+0000', 'last_undegraded': '2024-09-10T07:48:50.609844+0000', 'last_fullsized': '2024-09-10T07:48:50.609844+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:PG 7.35 is not active+clean 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.611643+0000', 'last_change': '2024-09-10T07:48:50.611643+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.611643+0000', 'last_undegraded': '2024-09-10T07:48:50.611643+0000', 'last_fullsized': '2024-09-10T07:48:50.611643+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.609643+0000', 'last_change': '2024-09-10T07:48:50.609643+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.609643+0000', 'last_undegraded': '2024-09-10T07:48:50.609643+0000', 'last_fullsized': '2024-09-10T07:48:50.609643+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:PG 7.32 is not active+clean 2024-09-10T07:48:56.665 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.608771+0000', 'last_change': '2024-09-10T07:48:50.608771+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.608771+0000', 'last_undegraded': '2024-09-10T07:48:50.608771+0000', 'last_fullsized': '2024-09-10T07:48:50.608771+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.609579+0000', 'last_change': '2024-09-10T07:48:50.609579+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.609579+0000', 'last_undegraded': '2024-09-10T07:48:50.609579+0000', 'last_fullsized': '2024-09-10T07:48:50.609579+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.608827+0000', 'last_change': '2024-09-10T07:48:50.608827+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.608827+0000', 'last_undegraded': '2024-09-10T07:48:50.608827+0000', 'last_fullsized': '2024-09-10T07:48:50.608827+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2a is not active+clean 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.605350+0000', 'last_change': '2024-09-10T07:48:50.605350+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.605350+0000', 'last_undegraded': '2024-09-10T07:48:50.605350+0000', 'last_fullsized': '2024-09-10T07:48:50.605350+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:PG 7.26 is not active+clean 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.612838+0000', 'last_change': '2024-09-10T07:48:50.612838+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.612838+0000', 'last_undegraded': '2024-09-10T07:48:50.612838+0000', 'last_fullsized': '2024-09-10T07:48:50.612838+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-10T07:48:56.666 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.613199+0000', 'last_change': '2024-09-10T07:48:50.613199+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.613199+0000', 'last_undegraded': '2024-09-10T07:48:50.613199+0000', 'last_fullsized': '2024-09-10T07:48:50.613199+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.20 is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.614337+0000', 'last_change': '2024-09-10T07:48:50.614337+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.614337+0000', 'last_undegraded': '2024-09-10T07:48:50.614337+0000', 'last_fullsized': '2024-09-10T07:48:50.614337+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.607331+0000', 'last_change': '2024-09-10T07:48:50.607331+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.607331+0000', 'last_undegraded': '2024-09-10T07:48:50.607331+0000', 'last_fullsized': '2024-09-10T07:48:50.607331+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1d is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.610493+0000', 'last_change': '2024-09-10T07:48:50.610493+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.610493+0000', 'last_undegraded': '2024-09-10T07:48:50.610493+0000', 'last_fullsized': '2024-09-10T07:48:50.610493+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.610526+0000', 'last_change': '2024-09-10T07:48:50.610526+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.610526+0000', 'last_undegraded': '2024-09-10T07:48:50.610526+0000', 'last_fullsized': '2024-09-10T07:48:50.610526+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.605758+0000', 'last_change': '2024-09-10T07:48:50.605758+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.605758+0000', 'last_undegraded': '2024-09-10T07:48:50.605758+0000', 'last_fullsized': '2024-09-10T07:48:50.605758+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-10T07:48:56.667 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.605337+0000', 'last_change': '2024-09-10T07:48:50.605337+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.605337+0000', 'last_undegraded': '2024-09-10T07:48:50.605337+0000', 'last_fullsized': '2024-09-10T07:48:50.605337+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.606876+0000', 'last_change': '2024-09-10T07:48:50.606876+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.606876+0000', 'last_undegraded': '2024-09-10T07:48:50.606876+0000', 'last_fullsized': '2024-09-10T07:48:50.606876+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.607227+0000', 'last_change': '2024-09-10T07:48:50.607227+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.607227+0000', 'last_undegraded': '2024-09-10T07:48:50.607227+0000', 'last_fullsized': '2024-09-10T07:48:50.607227+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.609194+0000', 'last_change': '2024-09-10T07:48:50.609194+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.609194+0000', 'last_undegraded': '2024-09-10T07:48:50.609194+0000', 'last_fullsized': '2024-09-10T07:48:50.609194+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.608768+0000', 'last_change': '2024-09-10T07:48:50.608768+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.608768+0000', 'last_undegraded': '2024-09-10T07:48:50.608768+0000', 'last_fullsized': '2024-09-10T07:48:50.608768+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-09-10T07:48:56.668 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.609027+0000', 'last_change': '2024-09-10T07:48:50.609027+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.609027+0000', 'last_undegraded': '2024-09-10T07:48:50.609027+0000', 'last_fullsized': '2024-09-10T07:48:50.609027+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.669 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-10T07:48:56.669 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.611851+0000', 'last_change': '2024-09-10T07:48:50.611851+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.611851+0000', 'last_undegraded': '2024-09-10T07:48:50.611851+0000', 'last_fullsized': '2024-09-10T07:48:50.611851+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.669 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-09-10T07:48:56.669 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:48:50.612430+0000', 'last_change': '2024-09-10T07:48:50.612430+0000', 'last_active': '2024-09-10T07:48:50.592745+0000', 'last_peered': '2024-09-10T07:48:50.592745+0000', 'last_clean': '2024-09-10T07:48:50.592745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:48:50.612430+0000', 'last_undegraded': '2024-09-10T07:48:50.612430+0000', 'last_fullsized': '2024-09-10T07:48:50.612430+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'last_clean_scrub_stamp': '2024-09-10T07:48:50.592745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:48:56.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:48:56.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.855+0000 7fe606800640 1 Processor -- start 2024-09-10T07:48:56.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.855+0000 7fe606800640 1 -- start start 2024-09-10T07:48:56.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe606800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6000b4aa0 0x7fe60006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe606800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6000cfac0 0x7fe6000b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe606800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6000ccd40 con 0x7fe60015b410 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe606800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe60006f250 con 0x7fe6000cfac0 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe606800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe6000cd020 con 0x7fe6000b4aa0 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe5ff400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe60015b410 0x7fe6000cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37078/0 (socket says 172.21.15.3:37078) 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe5ff400640 1 -- 172.21.15.3:0/1773106908 learned_addr learned my addr 172.21.15.3:0/1773106908 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.856+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/1773106908 >> v2:172.21.15.3:3300/0 conn(0x7fe6000cfac0 0x7fe6000b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5f7e00640 1 --2- 172.21.15.3:0/1773106908 >> v2:172.21.15.70:3300/0 conn(0x7fe6000b4aa0 0x7fe60006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5fea00640 1 -- 172.21.15.3:0/1773106908 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220042614 0 0) 0x7fe6000ccd40 con 0x7fe60015b410 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5ffe00640 1 -- 172.21.15.3:0/1773106908 >> v2:172.21.15.70:3300/0 conn(0x7fe6000b4aa0 msgr2=0x7fe60006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/1773106908 >> v2:172.21.15.70:3300/0 conn(0x7fe6000b4aa0 0x7fe60006ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5ffe00640 1 -- 172.21.15.3:0/1773106908 >> v1:172.21.15.3:6789/0 conn(0x7fe60015b410 legacy=0x7fe6000cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.857+0000 7fe5ffe00640 1 -- 172.21.15.3:0/1773106908 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6000cd300 con 0x7fe6000cfac0 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.858+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/1773106908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6000cfac0 0x7fe6000b3690 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ccba0 tx=0x7fe5f802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b96931306caa109 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.858+0000 7fe5fea00640 1 -- 172.21.15.3:0/1773106908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8032790 con 0x7fe6000cfac0 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.858+0000 7fe5fea00640 1 -- 172.21.15.3:0/1773106908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8038070 con 0x7fe6000cfac0 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.858+0000 7fe5fea00640 1 -- 172.21.15.3:0/1773106908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8032b50 con 0x7fe6000cfac0 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 -- 172.21.15.3:0/1773106908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6000cfac0 msgr2=0x7fe6000b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 --2- 172.21.15.3:0/1773106908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6000cfac0 0x7fe6000b3690 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ccba0 tx=0x7fe5f802c890 comp rx=0 tx=0).stop 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 -- 172.21.15.3:0/1773106908 shutdown_connections 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 --2- 172.21.15.3:0/1773106908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6000cfac0 0x7fe6000b3690 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 --2- 172.21.15.3:0/1773106908 >> v2:172.21.15.70:3300/0 conn(0x7fe6000b4aa0 0x7fe60006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 -- 172.21.15.3:0/1773106908 >> 172.21.15.3:0/1773106908 conn(0x7fe60006cfc0 msgr2=0x7fe60006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:56.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 -- 172.21.15.3:0/1773106908 shutdown_connections 2024-09-10T07:48:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.859+0000 7fe606800640 1 -- 172.21.15.3:0/1773106908 wait complete. 2024-09-10T07:48:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.861+0000 7fe606800640 1 Processor -- start 2024-09-10T07:48:56.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.861+0000 7fe606800640 1 -- start start 2024-09-10T07:48:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6000b4aa0 0x7fe60015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6000cfac0 0x7fe60015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000cdba0 con 0x7fe60015b410 2024-09-10T07:48:56.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000cd8c0 con 0x7fe6000cfac0 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe606800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe600072540 con 0x7fe6000b4aa0 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54656/0 (socket says 172.21.15.3:54656) 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ffe00640 1 -- 172.21.15.3:0/4179129489 learned_addr learned my addr 172.21.15.3:0/4179129489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ff400640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6000b4aa0 0x7fe60015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5f7e00640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6000cfac0 0x7fe60015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ffe00640 1 -- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6000b4aa0 msgr2=0x7fe60015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.862+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6000b4aa0 0x7fe60015dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5ffe00640 1 -- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6000cfac0 msgr2=0x7fe60015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6000cfac0 0x7fe60015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5ffe00640 1 -- 172.21.15.3:0/4179129489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6001479f0 con 0x7fe60015b410 2024-09-10T07:48:56.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5ffe00640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8035040 tx=0x7fe5f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8033620 con 0x7fe60015b410 2024-09-10T07:48:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8032e50 con 0x7fe60015b410 2024-09-10T07:48:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60014a8b0 con 0x7fe60015b410 2024-09-10T07:48:56.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80339e0 con 0x7fe60015b410 2024-09-10T07:48:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.863+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe600147f90 con 0x7fe60015b410 2024-09-10T07:48:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.864+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6000ce3d0 con 0x7fe60015b410 2024-09-10T07:48:56.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.865+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5f8033b80 con 0x7fe60015b410 2024-09-10T07:48:56.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.866+0000 7fe5f7400640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 0x7fe5dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:56.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.867+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fe5f80abc30 con 0x7fe60015b410 2024-09-10T07:48:56.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.867+0000 7fe5ff400640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 0x7fe5dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:56.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.867+0000 7fe5ff400640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 0x7fe5dc06e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f60 tx=0x7fe5f0007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:56.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:56.871+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe5f8079f80 con 0x7fe60015b410 2024-09-10T07:48:57.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.121+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe600075440 con 0x7fe60015b410 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.121+0000 7fe5f7400640 1 -- 172.21.15.3:0/4179129489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7fe5f807e280 con 0x7fe60015b410 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:57.121 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":170,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":72},{"state_name":"creating+peering","count":25}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592140,"bytes_used":244080640,"bytes_avail":772850032640,"bytes_total":773094113280,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":863,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":34,"btime":"2024-09-10T07:48:54:622365+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5433}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T07:48:33.516485+0000","services":{}},"progress_events":{}} 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 msgr2=0x7fe5dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 0x7fe5dc06e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0000f60 tx=0x7fe5f0007d50 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 msgr2=0x7fe600149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8035040 tx=0x7fe5f8004ab0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 shutdown_connections 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe5dc06be30 0x7fe5dc06e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe60015b410 0x7fe600149b10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6000cfac0 0x7fe60015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 --2- 172.21.15.3:0/4179129489 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6000b4aa0 0x7fe60015dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.123+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 >> 172.21.15.3:0/4179129489 conn(0x7fe60006cfc0 msgr2=0x7fe6000b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.124+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 shutdown_connections 2024-09-10T07:48:57.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.124+0000 7fe606800640 1 -- 172.21.15.3:0/4179129489 wait complete. 2024-09-10T07:48:57.140 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:48:57.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:48:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.296+0000 7fd5aec00640 1 Processor -- start 2024-09-10T07:48:57.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.296+0000 7fd5aec00640 1 -- start start 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5aec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 0x7fd5a8073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5aec00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a80cfdc0 con 0x7fd5a815e860 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5aec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a80cee40 con 0x7fd5a80d4050 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5aec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a80d3780 con 0x7fd5a80d3c10 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5ad800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5a815e860 0x7fd5a80cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37086/0 (socket says 172.21.15.3:37086) 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5ad800640 1 -- 172.21.15.3:0/3062893446 learned_addr learned my addr 172.21.15.3:0/3062893446 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:48:57.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.297+0000 7fd5ace00640 1 --2- 172.21.15.3:0/3062893446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 0x7fd5a8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725140173 0 0) 0x7fd5a80cfdc0 con 0x7fd5a815e860 2024-09-10T07:48:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd590003520 con 0x7fd5a815e860 2024-09-10T07:48:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5ace00640 1 -- 172.21.15.3:0/3062893446 >> v1:172.21.15.70:6789/0 conn(0x7fd5a80d4050 legacy=0x7fd5a80b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203501730 0 0) 0x7fd5a80cee40 con 0x7fd5a80d4050 2024-09-10T07:48:57.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5ace00640 1 -- 172.21.15.3:0/3062893446 >> v1:172.21.15.3:6789/0 conn(0x7fd5a815e860 legacy=0x7fd5a80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.298+0000 7fd5ace00640 1 -- 172.21.15.3:0/3062893446 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a80cf120 con 0x7fd5a80d3c10 2024-09-10T07:48:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.299+0000 7fd5ace00640 1 --2- 172.21.15.3:0/3062893446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 0x7fd5a8073150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5940097d0 tx=0x7fd59402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de325a55fedf93e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.300+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594037070 con 0x7fd5a80d3c10 2024-09-10T07:48:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.300+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd594032440 con 0x7fd5a80d3c10 2024-09-10T07:48:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.300+0000 7fd5a7e00640 1 -- 172.21.15.3:0/3062893446 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594032810 con 0x7fd5a80d3c10 2024-09-10T07:48:57.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 -- 172.21.15.3:0/3062893446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 msgr2=0x7fd5a8073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 --2- 172.21.15.3:0/3062893446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 0x7fd5a8073150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5940097d0 tx=0x7fd59402a210 comp rx=0 tx=0).stop 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 -- 172.21.15.3:0/3062893446 shutdown_connections 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 --2- 172.21.15.3:0/3062893446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a80d3c10 0x7fd5a8073150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 -- 172.21.15.3:0/3062893446 >> 172.21.15.3:0/3062893446 conn(0x7fd5a80d4650 msgr2=0x7fd5a80d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 -- 172.21.15.3:0/3062893446 shutdown_connections 2024-09-10T07:48:57.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.301+0000 7fd5aec00640 1 -- 172.21.15.3:0/3062893446 wait complete. 2024-09-10T07:48:57.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.302+0000 7fd5aec00640 1 Processor -- start 2024-09-10T07:48:57.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.302+0000 7fd5aec00640 1 -- start start 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a80d3c10 0x7fd5a8150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 0x7fd5a814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a80d2460 con 0x7fd5a80d4050 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a80cf400 con 0x7fd5a80d3c10 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.303+0000 7fd5aec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a80b7f90 con 0x7fd5a815e860 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54660/0 (socket says 172.21.15.3:54660) 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 -- 172.21.15.3:0/1819068200 learned_addr learned my addr 172.21.15.3:0/1819068200 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:48:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ad800640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a80d3c10 0x7fd5a8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ae200640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 0x7fd5a814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 -- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 msgr2=0x7fd5a814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 0x7fd5a814e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 -- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a80d3c10 msgr2=0x7fd5a8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a80d3c10 0x7fd5a8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 -- 172.21.15.3:0/1819068200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a81505a0 con 0x7fd5a80d4050 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ae200640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 0x7fd5a814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:48:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.304+0000 7fd5ace00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5940097d0 tx=0x7fd5940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.305+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594037050 con 0x7fd5a80d4050 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.305+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd594038040 con 0x7fd5a80d4050 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.305+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594004450 con 0x7fd5a80d4050 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.305+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a814efc0 con 0x7fd5a80d4050 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.305+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a814f580 con 0x7fd5a80d4050 2024-09-10T07:48:57.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.306+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a80cee40 con 0x7fd5a80d4050 2024-09-10T07:48:57.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.307+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd59403c6d0 con 0x7fd5a80d4050 2024-09-10T07:48:57.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.308+0000 7fd5a5600640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 0x7fd58806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:48:57.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.308+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fd5940ab8f0 con 0x7fd5a80d4050 2024-09-10T07:48:57.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.308+0000 7fd5ad800640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 0x7fd58806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:48:57.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.308+0000 7fd5ad800640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 0x7fd58806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0011e0 tx=0x7fd59c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:48:57.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.312+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd59407a200 con 0x7fd5a80d4050 2024-09-10T07:48:57.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.493+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5a80d2b00 con 0x7fd58806be30 2024-09-10T07:48:57.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.499+0000 7fd5a5600640 1 -- 172.21.15.3:0/1819068200 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249857 (secure 0 0 0) 0x7fd5a80d2b00 con 0x7fd58806be30 2024-09-10T07:48:57.502 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:48:57.504 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:48:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.503+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 msgr2=0x7fd58806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.503+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 0x7fd58806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0011e0 tx=0x7fd59c009040 comp rx=0 tx=0).stop 2024-09-10T07:48:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.503+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 msgr2=0x7fd5a81f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.503+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5940097d0 tx=0x7fd5940046c0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 shutdown_connections 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd58806be30 0x7fd58806e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5a815e860 0x7fd5a814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d4050 0x7fd5a81f7600 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 --2- 172.21.15.3:0/1819068200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a80d3c10 0x7fd5a8150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 >> 172.21.15.3:0/1819068200 conn(0x7fd5a80d4650 msgr2=0x7fd5a80b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 shutdown_connections 2024-09-10T07:48:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:48:57.504+0000 7fd5aec00640 1 -- 172.21.15.3:0/1819068200 wait complete. 2024-09-10T07:48:57.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T07:48:55.520865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592140,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238360,"kb_used_data":5476,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736360,"statfs":{"total":773094113280,"available":772850032640,"internally_reserved":0,"allocated":5607424,"data_stored":2963918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927936"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621887+0000","last_change":"2024-09-10T07:48:51.617751+0000","last_active":"2024-09-10T07:48:51.621887+0000","last_peered":"2024-09-10T07:48:51.621887+0000","last_clean":"2024-09-10T07:48:51.621887+0000","last_became_active":"2024-09-10T07:48:51.617450+0000","last_became_peered":"2024-09-10T07:48:51.617450+0000","last_unstale":"2024-09-10T07:48:51.621887+0000","last_undegraded":"2024-09-10T07:48:51.621887+0000","last_fullsized":"2024-09-10T07:48:51.621887+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632949+0000","last_change":"2024-09-10T07:48:51.620056+0000","last_active":"2024-09-10T07:48:51.632949+0000","last_peered":"2024-09-10T07:48:51.632949+0000","last_clean":"2024-09-10T07:48:51.632949+0000","last_became_active":"2024-09-10T07:48:51.619801+0000","last_became_peered":"2024-09-10T07:48:51.619801+0000","last_unstale":"2024-09-10T07:48:51.632949+0000","last_undegraded":"2024-09-10T07:48:51.632949+0000","last_fullsized":"2024-09-10T07:48:51.632949+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:12.498291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617258+0000","last_change":"2024-09-10T07:48:51.612764+0000","last_active":"2024-09-10T07:48:51.617258+0000","last_peered":"2024-09-10T07:48:51.617258+0000","last_clean":"2024-09-10T07:48:51.617258+0000","last_became_active":"2024-09-10T07:48:51.611550+0000","last_became_peered":"2024-09-10T07:48:51.611550+0000","last_unstale":"2024-09-10T07:48:51.617258+0000","last_undegraded":"2024-09-10T07:48:51.617258+0000","last_fullsized":"2024-09-10T07:48:51.617258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:07.243280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611003+0000","last_change":"2024-09-10T07:48:50.611003+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611003+0000","last_undegraded":"2024-09-10T07:48:50.611003+0000","last_fullsized":"2024-09-10T07:48:50.611003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631377+0000","last_change":"2024-09-10T07:48:51.624149+0000","last_active":"2024-09-10T07:48:51.631377+0000","last_peered":"2024-09-10T07:48:51.631377+0000","last_clean":"2024-09-10T07:48:51.631377+0000","last_became_active":"2024-09-10T07:48:51.623849+0000","last_became_peered":"2024-09-10T07:48:51.623849+0000","last_unstale":"2024-09-10T07:48:51.631377+0000","last_undegraded":"2024-09-10T07:48:51.631377+0000","last_fullsized":"2024-09-10T07:48:51.631377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.308850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608067+0000","last_change":"2024-09-10T07:48:50.608067+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608067+0000","last_undegraded":"2024-09-10T07:48:50.608067+0000","last_fullsized":"2024-09-10T07:48:50.608067+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639319+0000","last_change":"2024-09-10T07:48:51.628760+0000","last_active":"2024-09-10T07:48:51.639319+0000","last_peered":"2024-09-10T07:48:51.639319+0000","last_clean":"2024-09-10T07:48:51.639319+0000","last_became_active":"2024-09-10T07:48:51.628462+0000","last_became_peered":"2024-09-10T07:48:51.628462+0000","last_unstale":"2024-09-10T07:48:51.639319+0000","last_undegraded":"2024-09-10T07:48:51.639319+0000","last_fullsized":"2024-09-10T07:48:51.639319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:19.292466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614856+0000","last_change":"2024-09-10T07:48:50.614856+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614856+0000","last_undegraded":"2024-09-10T07:48:50.614856+0000","last_fullsized":"2024-09-10T07:48:50.614856+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609844+0000","last_change":"2024-09-10T07:48:50.609844+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609844+0000","last_undegraded":"2024-09-10T07:48:50.609844+0000","last_fullsized":"2024-09-10T07:48:50.609844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607402+0000","last_change":"2024-09-10T07:48:51.617674+0000","last_active":"2024-09-10T07:48:52.607402+0000","last_peered":"2024-09-10T07:48:52.607402+0000","last_clean":"2024-09-10T07:48:52.607402+0000","last_became_active":"2024-09-10T07:48:51.617378+0000","last_became_peered":"2024-09-10T07:48:51.617378+0000","last_unstale":"2024-09-10T07:48:52.607402+0000","last_undegraded":"2024-09-10T07:48:52.607402+0000","last_fullsized":"2024-09-10T07:48:52.607402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:02.393228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611643+0000","last_change":"2024-09-10T07:48:50.611643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611643+0000","last_undegraded":"2024-09-10T07:48:50.611643+0000","last_fullsized":"2024-09-10T07:48:50.611643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633458+0000","last_change":"2024-09-10T07:48:51.620489+0000","last_active":"2024-09-10T07:48:51.633458+0000","last_peered":"2024-09-10T07:48:51.633458+0000","last_clean":"2024-09-10T07:48:51.633458+0000","last_became_active":"2024-09-10T07:48:51.620237+0000","last_became_peered":"2024-09-10T07:48:51.620237+0000","last_unstale":"2024-09-10T07:48:51.633458+0000","last_undegraded":"2024-09-10T07:48:51.633458+0000","last_fullsized":"2024-09-10T07:48:51.633458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:25.714474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609643+0000","last_change":"2024-09-10T07:48:50.609643+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609643+0000","last_undegraded":"2024-09-10T07:48:50.609643+0000","last_fullsized":"2024-09-10T07:48:50.609643+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608771+0000","last_change":"2024-09-10T07:48:50.608771+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608771+0000","last_undegraded":"2024-09-10T07:48:50.608771+0000","last_fullsized":"2024-09-10T07:48:50.608771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609579+0000","last_change":"2024-09-10T07:48:50.609579+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609579+0000","last_undegraded":"2024-09-10T07:48:50.609579+0000","last_fullsized":"2024-09-10T07:48:50.609579+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637116+0000","last_change":"2024-09-10T07:48:51.631668+0000","last_active":"2024-09-10T07:48:51.637116+0000","last_peered":"2024-09-10T07:48:51.637116+0000","last_clean":"2024-09-10T07:48:51.637116+0000","last_became_active":"2024-09-10T07:48:51.631459+0000","last_became_peered":"2024-09-10T07:48:51.631459+0000","last_unstale":"2024-09-10T07:48:51.637116+0000","last_undegraded":"2024-09-10T07:48:51.637116+0000","last_fullsized":"2024-09-10T07:48:51.637116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:12.993962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632892+0000","last_change":"2024-09-10T07:48:51.622041+0000","last_active":"2024-09-10T07:48:51.632892+0000","last_peered":"2024-09-10T07:48:51.632892+0000","last_clean":"2024-09-10T07:48:51.632892+0000","last_became_active":"2024-09-10T07:48:51.621622+0000","last_became_peered":"2024-09-10T07:48:51.621622+0000","last_unstale":"2024-09-10T07:48:51.632892+0000","last_undegraded":"2024-09-10T07:48:51.632892+0000","last_fullsized":"2024-09-10T07:48:51.632892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:50.994249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638066+0000","last_change":"2024-09-10T07:48:51.632824+0000","last_active":"2024-09-10T07:48:51.638066+0000","last_peered":"2024-09-10T07:48:51.638066+0000","last_clean":"2024-09-10T07:48:51.638066+0000","last_became_active":"2024-09-10T07:48:51.629313+0000","last_became_peered":"2024-09-10T07:48:51.629313+0000","last_unstale":"2024-09-10T07:48:51.638066+0000","last_undegraded":"2024-09-10T07:48:51.638066+0000","last_fullsized":"2024-09-10T07:48:51.638066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:25.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630089+0000","last_change":"2024-09-10T07:48:51.623911+0000","last_active":"2024-09-10T07:48:51.630089+0000","last_peered":"2024-09-10T07:48:51.630089+0000","last_clean":"2024-09-10T07:48:51.630089+0000","last_became_active":"2024-09-10T07:48:51.623645+0000","last_became_peered":"2024-09-10T07:48:51.623645+0000","last_unstale":"2024-09-10T07:48:51.630089+0000","last_undegraded":"2024-09-10T07:48:51.630089+0000","last_fullsized":"2024-09-10T07:48:51.630089+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:54.995968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606836+0000","last_change":"2024-09-10T07:48:51.624623+0000","last_active":"2024-09-10T07:48:52.606836+0000","last_peered":"2024-09-10T07:48:52.606836+0000","last_clean":"2024-09-10T07:48:52.606836+0000","last_became_active":"2024-09-10T07:48:51.621627+0000","last_became_peered":"2024-09-10T07:48:51.621627+0000","last_unstale":"2024-09-10T07:48:52.606836+0000","last_undegraded":"2024-09-10T07:48:52.606836+0000","last_fullsized":"2024-09-10T07:48:52.606836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:19.844611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608827+0000","last_change":"2024-09-10T07:48:50.608827+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608827+0000","last_undegraded":"2024-09-10T07:48:50.608827+0000","last_fullsized":"2024-09-10T07:48:50.608827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605350+0000","last_change":"2024-09-10T07:48:50.605350+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605350+0000","last_undegraded":"2024-09-10T07:48:50.605350+0000","last_fullsized":"2024-09-10T07:48:50.605350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.624565+0000","last_change":"2024-09-10T07:48:51.616950+0000","last_active":"2024-09-10T07:48:51.624565+0000","last_peered":"2024-09-10T07:48:51.624565+0000","last_clean":"2024-09-10T07:48:51.624565+0000","last_became_active":"2024-09-10T07:48:51.614672+0000","last_became_peered":"2024-09-10T07:48:51.614672+0000","last_unstale":"2024-09-10T07:48:51.624565+0000","last_undegraded":"2024-09-10T07:48:51.624565+0000","last_fullsized":"2024-09-10T07:48:51.624565+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:48.008128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635731+0000","last_change":"2024-09-10T07:48:51.630986+0000","last_active":"2024-09-10T07:48:51.635731+0000","last_peered":"2024-09-10T07:48:51.635731+0000","last_clean":"2024-09-10T07:48:51.635731+0000","last_became_active":"2024-09-10T07:48:51.630754+0000","last_became_peered":"2024-09-10T07:48:51.630754+0000","last_unstale":"2024-09-10T07:48:51.635731+0000","last_undegraded":"2024-09-10T07:48:51.635731+0000","last_fullsized":"2024-09-10T07:48:51.635731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.621864+0000","last_change":"2024-09-10T07:48:51.617701+0000","last_active":"2024-09-10T07:48:51.621864+0000","last_peered":"2024-09-10T07:48:51.621864+0000","last_clean":"2024-09-10T07:48:51.621864+0000","last_became_active":"2024-09-10T07:48:51.617418+0000","last_became_peered":"2024-09-10T07:48:51.617418+0000","last_unstale":"2024-09-10T07:48:51.621864+0000","last_undegraded":"2024-09-10T07:48:51.621864+0000","last_fullsized":"2024-09-10T07:48:51.621864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612838+0000","last_change":"2024-09-10T07:48:50.612838+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612838+0000","last_undegraded":"2024-09-10T07:48:50.612838+0000","last_fullsized":"2024-09-10T07:48:50.612838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630570+0000","last_change":"2024-09-10T07:48:51.620624+0000","last_active":"2024-09-10T07:48:51.630570+0000","last_peered":"2024-09-10T07:48:51.630570+0000","last_clean":"2024-09-10T07:48:51.630570+0000","last_became_active":"2024-09-10T07:48:51.619782+0000","last_became_peered":"2024-09-10T07:48:51.619782+0000","last_unstale":"2024-09-10T07:48:51.630570+0000","last_undegraded":"2024-09-10T07:48:51.630570+0000","last_fullsized":"2024-09-10T07:48:51.630570+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:44.304141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623981+0000","last_change":"2024-09-10T07:48:51.616307+0000","last_active":"2024-09-10T07:48:51.623981+0000","last_peered":"2024-09-10T07:48:51.623981+0000","last_clean":"2024-09-10T07:48:51.623981+0000","last_became_active":"2024-09-10T07:48:51.615851+0000","last_became_peered":"2024-09-10T07:48:51.615851+0000","last_unstale":"2024-09-10T07:48:51.623981+0000","last_undegraded":"2024-09-10T07:48:51.623981+0000","last_fullsized":"2024-09-10T07:48:51.623981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:05.714041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623649+0000","last_change":"2024-09-10T07:48:51.616551+0000","last_active":"2024-09-10T07:48:51.623649+0000","last_peered":"2024-09-10T07:48:51.623649+0000","last_clean":"2024-09-10T07:48:51.623649+0000","last_became_active":"2024-09-10T07:48:51.616241+0000","last_became_peered":"2024-09-10T07:48:51.616241+0000","last_unstale":"2024-09-10T07:48:51.623649+0000","last_undegraded":"2024-09-10T07:48:51.623649+0000","last_fullsized":"2024-09-10T07:48:51.623649+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633988+0000","last_change":"2024-09-10T07:48:51.628969+0000","last_active":"2024-09-10T07:48:51.633988+0000","last_peered":"2024-09-10T07:48:51.633988+0000","last_clean":"2024-09-10T07:48:51.633988+0000","last_became_active":"2024-09-10T07:48:51.628200+0000","last_became_peered":"2024-09-10T07:48:51.628200+0000","last_unstale":"2024-09-10T07:48:51.633988+0000","last_undegraded":"2024-09-10T07:48:51.633988+0000","last_fullsized":"2024-09-10T07:48:51.633988+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:12.551381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.613199+0000","last_change":"2024-09-10T07:48:50.613199+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.613199+0000","last_undegraded":"2024-09-10T07:48:50.613199+0000","last_fullsized":"2024-09-10T07:48:50.613199+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.614337+0000","last_change":"2024-09-10T07:48:50.614337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.614337+0000","last_undegraded":"2024-09-10T07:48:50.614337+0000","last_fullsized":"2024-09-10T07:48:50.614337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607331+0000","last_change":"2024-09-10T07:48:50.607331+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607331+0000","last_undegraded":"2024-09-10T07:48:50.607331+0000","last_fullsized":"2024-09-10T07:48:50.607331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.628793+0000","last_change":"2024-09-10T07:48:50.627953+0000","last_active":"2024-09-10T07:48:50.628793+0000","last_peered":"2024-09-10T07:48:50.628793+0000","last_clean":"2024-09-10T07:48:50.628793+0000","last_became_active":"2024-09-10T07:48:50.627666+0000","last_became_peered":"2024-09-10T07:48:50.627666+0000","last_unstale":"2024-09-10T07:48:50.628793+0000","last_undegraded":"2024-09-10T07:48:50.628793+0000","last_fullsized":"2024-09-10T07:48:50.628793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:57.026249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.630003+0000","last_change":"2024-09-10T07:48:51.621847+0000","last_active":"2024-09-10T07:48:51.630003+0000","last_peered":"2024-09-10T07:48:51.630003+0000","last_clean":"2024-09-10T07:48:51.630003+0000","last_became_active":"2024-09-10T07:48:51.621554+0000","last_became_peered":"2024-09-10T07:48:51.621554+0000","last_unstale":"2024-09-10T07:48:51.630003+0000","last_undegraded":"2024-09-10T07:48:51.630003+0000","last_fullsized":"2024-09-10T07:48:51.630003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634481+0000","last_change":"2024-09-10T07:48:50.619796+0000","last_active":"2024-09-10T07:48:51.634481+0000","last_peered":"2024-09-10T07:48:51.634481+0000","last_clean":"2024-09-10T07:48:51.634481+0000","last_became_active":"2024-09-10T07:48:50.619535+0000","last_became_peered":"2024-09-10T07:48:50.619535+0000","last_unstale":"2024-09-10T07:48:51.634481+0000","last_undegraded":"2024-09-10T07:48:51.634481+0000","last_fullsized":"2024-09-10T07:48:51.634481+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:44.514806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.633602+0000","last_change":"2024-09-10T07:48:51.629435+0000","last_active":"2024-09-10T07:48:51.633602+0000","last_peered":"2024-09-10T07:48:51.633602+0000","last_clean":"2024-09-10T07:48:51.633602+0000","last_became_active":"2024-09-10T07:48:51.628928+0000","last_became_peered":"2024-09-10T07:48:51.628928+0000","last_unstale":"2024-09-10T07:48:51.633602+0000","last_undegraded":"2024-09-10T07:48:51.633602+0000","last_fullsized":"2024-09-10T07:48:51.633602+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622265+0000","last_change":"2024-09-10T07:48:50.619441+0000","last_active":"2024-09-10T07:48:50.622265+0000","last_peered":"2024-09-10T07:48:50.622265+0000","last_clean":"2024-09-10T07:48:50.622265+0000","last_became_active":"2024-09-10T07:48:50.619148+0000","last_became_peered":"2024-09-10T07:48:50.619148+0000","last_unstale":"2024-09-10T07:48:50.622265+0000","last_undegraded":"2024-09-10T07:48:50.622265+0000","last_fullsized":"2024-09-10T07:48:50.622265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.560270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610493+0000","last_change":"2024-09-10T07:48:50.610493+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610493+0000","last_undegraded":"2024-09-10T07:48:50.610493+0000","last_fullsized":"2024-09-10T07:48:50.610493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639518+0000","last_change":"2024-09-10T07:48:50.617904+0000","last_active":"2024-09-10T07:48:51.639518+0000","last_peered":"2024-09-10T07:48:51.639518+0000","last_clean":"2024-09-10T07:48:51.639518+0000","last_became_active":"2024-09-10T07:48:50.617656+0000","last_became_peered":"2024-09-10T07:48:50.617656+0000","last_unstale":"2024-09-10T07:48:51.639518+0000","last_undegraded":"2024-09-10T07:48:51.639518+0000","last_fullsized":"2024-09-10T07:48:51.639518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.305959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.627946+0000","last_change":"2024-09-10T07:48:51.620554+0000","last_active":"2024-09-10T07:48:51.627946+0000","last_peered":"2024-09-10T07:48:51.627946+0000","last_clean":"2024-09-10T07:48:51.627946+0000","last_became_active":"2024-09-10T07:48:51.619526+0000","last_became_peered":"2024-09-10T07:48:51.619526+0000","last_unstale":"2024-09-10T07:48:51.627946+0000","last_undegraded":"2024-09-10T07:48:51.627946+0000","last_fullsized":"2024-09-10T07:48:51.627946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:33.856786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.625037+0000","last_change":"2024-09-10T07:48:50.623162+0000","last_active":"2024-09-10T07:48:50.625037+0000","last_peered":"2024-09-10T07:48:50.625037+0000","last_clean":"2024-09-10T07:48:50.625037+0000","last_became_active":"2024-09-10T07:48:50.622765+0000","last_became_peered":"2024-09-10T07:48:50.622765+0000","last_unstale":"2024-09-10T07:48:50.625037+0000","last_undegraded":"2024-09-10T07:48:50.625037+0000","last_fullsized":"2024-09-10T07:48:50.625037+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:24.341239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606939+0000","last_change":"2024-09-10T07:48:51.628666+0000","last_active":"2024-09-10T07:48:52.606939+0000","last_peered":"2024-09-10T07:48:52.606939+0000","last_clean":"2024-09-10T07:48:52.606939+0000","last_became_active":"2024-09-10T07:48:51.628411+0000","last_became_peered":"2024-09-10T07:48:51.628411+0000","last_unstale":"2024-09-10T07:48:52.606939+0000","last_undegraded":"2024-09-10T07:48:52.606939+0000","last_fullsized":"2024-09-10T07:48:52.606939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:13.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635593+0000","last_change":"2024-09-10T07:48:50.616830+0000","last_active":"2024-09-10T07:48:51.635593+0000","last_peered":"2024-09-10T07:48:51.635593+0000","last_clean":"2024-09-10T07:48:51.635593+0000","last_became_active":"2024-09-10T07:48:50.616586+0000","last_became_peered":"2024-09-10T07:48:50.616586+0000","last_unstale":"2024-09-10T07:48:51.635593+0000","last_undegraded":"2024-09-10T07:48:51.635593+0000","last_fullsized":"2024-09-10T07:48:51.635593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:25.042914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606676+0000","last_change":"2024-09-10T07:48:51.629828+0000","last_active":"2024-09-10T07:48:52.606676+0000","last_peered":"2024-09-10T07:48:52.606676+0000","last_clean":"2024-09-10T07:48:52.606676+0000","last_became_active":"2024-09-10T07:48:51.628172+0000","last_became_peered":"2024-09-10T07:48:51.628172+0000","last_unstale":"2024-09-10T07:48:52.606676+0000","last_undegraded":"2024-09-10T07:48:52.606676+0000","last_fullsized":"2024-09-10T07:48:52.606676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609352+0000","last_change":"2024-09-10T07:48:50.627131+0000","last_active":"2024-09-10T07:48:51.609352+0000","last_peered":"2024-09-10T07:48:51.609352+0000","last_clean":"2024-09-10T07:48:51.609352+0000","last_became_active":"2024-09-10T07:48:50.626968+0000","last_became_peered":"2024-09-10T07:48:50.626968+0000","last_unstale":"2024-09-10T07:48:51.609352+0000","last_undegraded":"2024-09-10T07:48:51.609352+0000","last_fullsized":"2024-09-10T07:48:51.609352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:11.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634368+0000","last_change":"2024-09-10T07:48:51.624704+0000","last_active":"2024-09-10T07:48:51.634368+0000","last_peered":"2024-09-10T07:48:51.634368+0000","last_clean":"2024-09-10T07:48:51.634368+0000","last_became_active":"2024-09-10T07:48:51.624391+0000","last_became_peered":"2024-09-10T07:48:51.624391+0000","last_unstale":"2024-09-10T07:48:51.634368+0000","last_undegraded":"2024-09-10T07:48:51.634368+0000","last_fullsized":"2024-09-10T07:48:51.634368+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:39.576381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606654+0000","last_change":"2024-09-10T07:48:50.618315+0000","last_active":"2024-09-10T07:48:52.606654+0000","last_peered":"2024-09-10T07:48:52.606654+0000","last_clean":"2024-09-10T07:48:52.606654+0000","last_became_active":"2024-09-10T07:48:50.618022+0000","last_became_peered":"2024-09-10T07:48:50.618022+0000","last_unstale":"2024-09-10T07:48:52.606654+0000","last_undegraded":"2024-09-10T07:48:52.606654+0000","last_fullsized":"2024-09-10T07:48:52.606654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:24.272006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.622150+0000","last_change":"2024-09-10T07:48:51.616784+0000","last_active":"2024-09-10T07:48:51.622150+0000","last_peered":"2024-09-10T07:48:51.622150+0000","last_clean":"2024-09-10T07:48:51.622150+0000","last_became_active":"2024-09-10T07:48:51.616520+0000","last_became_peered":"2024-09-10T07:48:51.616520+0000","last_unstale":"2024-09-10T07:48:51.622150+0000","last_undegraded":"2024-09-10T07:48:51.622150+0000","last_fullsized":"2024-09-10T07:48:51.622150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:47.124772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.604652+0000","last_change":"2024-09-10T07:48:50.615394+0000","last_active":"2024-09-10T07:48:51.604652+0000","last_peered":"2024-09-10T07:48:51.604652+0000","last_clean":"2024-09-10T07:48:51.604652+0000","last_became_active":"2024-09-10T07:48:50.615139+0000","last_became_peered":"2024-09-10T07:48:50.615139+0000","last_unstale":"2024-09-10T07:48:51.604652+0000","last_undegraded":"2024-09-10T07:48:51.604652+0000","last_fullsized":"2024-09-10T07:48:51.604652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.623257+0000","last_change":"2024-09-10T07:48:51.619010+0000","last_active":"2024-09-10T07:48:51.623257+0000","last_peered":"2024-09-10T07:48:51.623257+0000","last_clean":"2024-09-10T07:48:51.623257+0000","last_became_active":"2024-09-10T07:48:51.618676+0000","last_became_peered":"2024-09-10T07:48:51.618676+0000","last_unstale":"2024-09-10T07:48:51.623257+0000","last_undegraded":"2024-09-10T07:48:51.623257+0000","last_fullsized":"2024-09-10T07:48:51.623257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:47.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607323+0000","last_change":"2024-09-10T07:48:50.620699+0000","last_active":"2024-09-10T07:48:52.607323+0000","last_peered":"2024-09-10T07:48:52.607323+0000","last_clean":"2024-09-10T07:48:52.607323+0000","last_became_active":"2024-09-10T07:48:50.620417+0000","last_became_peered":"2024-09-10T07:48:50.620417+0000","last_unstale":"2024-09-10T07:48:52.607323+0000","last_undegraded":"2024-09-10T07:48:52.607323+0000","last_fullsized":"2024-09-10T07:48:52.607323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:31.155316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.610526+0000","last_change":"2024-09-10T07:48:50.610526+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.610526+0000","last_undegraded":"2024-09-10T07:48:50.610526+0000","last_fullsized":"2024-09-10T07:48:50.610526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644403+0000","last_change":"2024-09-10T07:48:50.620839+0000","last_active":"2024-09-10T07:48:51.644403+0000","last_peered":"2024-09-10T07:48:51.644403+0000","last_clean":"2024-09-10T07:48:51.644403+0000","last_became_active":"2024-09-10T07:48:50.620581+0000","last_became_peered":"2024-09-10T07:48:50.620581+0000","last_unstale":"2024-09-10T07:48:51.644403+0000","last_undegraded":"2024-09-10T07:48:51.644403+0000","last_fullsized":"2024-09-10T07:48:51.644403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.639187+0000","last_change":"2024-09-10T07:48:51.630681+0000","last_active":"2024-09-10T07:48:51.639187+0000","last_peered":"2024-09-10T07:48:51.639187+0000","last_clean":"2024-09-10T07:48:51.639187+0000","last_became_active":"2024-09-10T07:48:51.630321+0000","last_became_peered":"2024-09-10T07:48:51.630321+0000","last_unstale":"2024-09-10T07:48:51.639187+0000","last_undegraded":"2024-09-10T07:48:51.639187+0000","last_fullsized":"2024-09-10T07:48:51.639187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:11.469475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606961+0000","last_change":"2024-09-10T07:48:50.620804+0000","last_active":"2024-09-10T07:48:52.606961+0000","last_peered":"2024-09-10T07:48:52.606961+0000","last_clean":"2024-09-10T07:48:52.606961+0000","last_became_active":"2024-09-10T07:48:50.620518+0000","last_became_peered":"2024-09-10T07:48:50.620518+0000","last_unstale":"2024-09-10T07:48:52.606961+0000","last_undegraded":"2024-09-10T07:48:52.606961+0000","last_fullsized":"2024-09-10T07:48:52.606961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:26.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637082+0000","last_change":"2024-09-10T07:48:51.632456+0000","last_active":"2024-09-10T07:48:51.637082+0000","last_peered":"2024-09-10T07:48:51.637082+0000","last_clean":"2024-09-10T07:48:51.637082+0000","last_became_active":"2024-09-10T07:48:51.632175+0000","last_became_peered":"2024-09-10T07:48:51.632175+0000","last_unstale":"2024-09-10T07:48:51.637082+0000","last_undegraded":"2024-09-10T07:48:51.637082+0000","last_fullsized":"2024-09-10T07:48:51.637082+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:59.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624685+0000","last_change":"2024-09-10T07:48:50.623794+0000","last_active":"2024-09-10T07:48:50.624685+0000","last_peered":"2024-09-10T07:48:50.624685+0000","last_clean":"2024-09-10T07:48:50.624685+0000","last_became_active":"2024-09-10T07:48:50.623196+0000","last_became_peered":"2024-09-10T07:48:50.623196+0000","last_unstale":"2024-09-10T07:48:50.624685+0000","last_undegraded":"2024-09-10T07:48:50.624685+0000","last_fullsized":"2024-09-10T07:48:50.624685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605758+0000","last_change":"2024-09-10T07:48:50.605758+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605758+0000","last_undegraded":"2024-09-10T07:48:50.605758+0000","last_fullsized":"2024-09-10T07:48:50.605758+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.608400+0000","last_change":"2024-09-10T07:48:50.617395+0000","last_active":"2024-09-10T07:48:51.608400+0000","last_peered":"2024-09-10T07:48:51.608400+0000","last_clean":"2024-09-10T07:48:51.608400+0000","last_became_active":"2024-09-10T07:48:50.615852+0000","last_became_peered":"2024-09-10T07:48:50.615852+0000","last_unstale":"2024-09-10T07:48:51.608400+0000","last_undegraded":"2024-09-10T07:48:51.608400+0000","last_fullsized":"2024-09-10T07:48:51.608400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.605337+0000","last_change":"2024-09-10T07:48:50.605337+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.605337+0000","last_undegraded":"2024-09-10T07:48:50.605337+0000","last_fullsized":"2024-09-10T07:48:50.605337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624312+0000","last_change":"2024-09-10T07:48:50.620280+0000","last_active":"2024-09-10T07:48:50.624312+0000","last_peered":"2024-09-10T07:48:50.624312+0000","last_clean":"2024-09-10T07:48:50.624312+0000","last_became_active":"2024-09-10T07:48:50.619988+0000","last_became_peered":"2024-09-10T07:48:50.619988+0000","last_unstale":"2024-09-10T07:48:50.624312+0000","last_undegraded":"2024-09-10T07:48:50.624312+0000","last_fullsized":"2024-09-10T07:48:50.624312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:03.156373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.632347+0000","last_change":"2024-09-10T07:48:51.616228+0000","last_active":"2024-09-10T07:48:51.632347+0000","last_peered":"2024-09-10T07:48:51.632347+0000","last_clean":"2024-09-10T07:48:51.632347+0000","last_became_active":"2024-09-10T07:48:51.615946+0000","last_became_peered":"2024-09-10T07:48:51.615946+0000","last_unstale":"2024-09-10T07:48:51.632347+0000","last_undegraded":"2024-09-10T07:48:51.632347+0000","last_fullsized":"2024-09-10T07:48:51.632347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606882+0000","last_change":"2024-09-10T07:48:50.615137+0000","last_active":"2024-09-10T07:48:52.606882+0000","last_peered":"2024-09-10T07:48:52.606882+0000","last_clean":"2024-09-10T07:48:52.606882+0000","last_became_active":"2024-09-10T07:48:50.614618+0000","last_became_peered":"2024-09-10T07:48:50.614618+0000","last_unstale":"2024-09-10T07:48:52.606882+0000","last_undegraded":"2024-09-10T07:48:52.606882+0000","last_fullsized":"2024-09-10T07:48:52.606882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:16.791861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.606876+0000","last_change":"2024-09-10T07:48:50.606876+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.606876+0000","last_undegraded":"2024-09-10T07:48:50.606876+0000","last_fullsized":"2024-09-10T07:48:50.606876+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624888+0000","last_change":"2024-09-10T07:48:50.622636+0000","last_active":"2024-09-10T07:48:50.624888+0000","last_peered":"2024-09-10T07:48:50.624888+0000","last_clean":"2024-09-10T07:48:50.624888+0000","last_became_active":"2024-09-10T07:48:50.621702+0000","last_became_peered":"2024-09-10T07:48:50.621702+0000","last_unstale":"2024-09-10T07:48:50.624888+0000","last_undegraded":"2024-09-10T07:48:50.624888+0000","last_fullsized":"2024-09-10T07:48:50.624888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:21.875313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.607227+0000","last_change":"2024-09-10T07:48:50.607227+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.607227+0000","last_undegraded":"2024-09-10T07:48:50.607227+0000","last_fullsized":"2024-09-10T07:48:50.607227+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.644901+0000","last_change":"2024-09-10T07:48:50.621912+0000","last_active":"2024-09-10T07:48:51.644901+0000","last_peered":"2024-09-10T07:48:51.644901+0000","last_clean":"2024-09-10T07:48:51.644901+0000","last_became_active":"2024-09-10T07:48:50.620580+0000","last_became_peered":"2024-09-10T07:48:50.620580+0000","last_unstale":"2024-09-10T07:48:51.644901+0000","last_undegraded":"2024-09-10T07:48:51.644901+0000","last_fullsized":"2024-09-10T07:48:51.644901+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.637323+0000","last_change":"2024-09-10T07:48:51.634091+0000","last_active":"2024-09-10T07:48:51.637323+0000","last_peered":"2024-09-10T07:48:51.637323+0000","last_clean":"2024-09-10T07:48:51.637323+0000","last_became_active":"2024-09-10T07:48:51.633805+0000","last_became_peered":"2024-09-10T07:48:51.633805+0000","last_unstale":"2024-09-10T07:48:51.637323+0000","last_undegraded":"2024-09-10T07:48:51.637323+0000","last_fullsized":"2024-09-10T07:48:51.637323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.641630+0000","last_change":"2024-09-10T07:48:50.616978+0000","last_active":"2024-09-10T07:48:51.641630+0000","last_peered":"2024-09-10T07:48:51.641630+0000","last_clean":"2024-09-10T07:48:51.641630+0000","last_became_active":"2024-09-10T07:48:50.616723+0000","last_became_peered":"2024-09-10T07:48:50.616723+0000","last_unstale":"2024-09-10T07:48:51.641630+0000","last_undegraded":"2024-09-10T07:48:51.641630+0000","last_fullsized":"2024-09-10T07:48:51.641630+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.617278+0000","last_change":"2024-09-10T07:48:51.612987+0000","last_active":"2024-09-10T07:48:51.617278+0000","last_peered":"2024-09-10T07:48:51.617278+0000","last_clean":"2024-09-10T07:48:51.617278+0000","last_became_active":"2024-09-10T07:48:51.611732+0000","last_became_peered":"2024-09-10T07:48:51.611732+0000","last_unstale":"2024-09-10T07:48:51.617278+0000","last_undegraded":"2024-09-10T07:48:51.617278+0000","last_fullsized":"2024-09-10T07:48:51.617278+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.626733+0000","last_change":"2024-09-10T07:48:50.625853+0000","last_active":"2024-09-10T07:48:50.626733+0000","last_peered":"2024-09-10T07:48:50.626733+0000","last_clean":"2024-09-10T07:48:50.626733+0000","last_became_active":"2024-09-10T07:48:50.625576+0000","last_became_peered":"2024-09-10T07:48:50.625576+0000","last_unstale":"2024-09-10T07:48:50.626733+0000","last_undegraded":"2024-09-10T07:48:50.626733+0000","last_fullsized":"2024-09-10T07:48:50.626733+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:54.365655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606693+0000","last_change":"2024-09-10T07:48:51.622658+0000","last_active":"2024-09-10T07:48:52.606693+0000","last_peered":"2024-09-10T07:48:52.606693+0000","last_clean":"2024-09-10T07:48:52.606693+0000","last_became_active":"2024-09-10T07:48:51.622387+0000","last_became_peered":"2024-09-10T07:48:51.622387+0000","last_unstale":"2024-09-10T07:48:52.606693+0000","last_undegraded":"2024-09-10T07:48:52.606693+0000","last_fullsized":"2024-09-10T07:48:52.606693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:20.188379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607033+0000","last_change":"2024-09-10T07:48:50.617463+0000","last_active":"2024-09-10T07:48:52.607033+0000","last_peered":"2024-09-10T07:48:52.607033+0000","last_clean":"2024-09-10T07:48:52.607033+0000","last_became_active":"2024-09-10T07:48:50.617117+0000","last_became_peered":"2024-09-10T07:48:50.617117+0000","last_unstale":"2024-09-10T07:48:52.607033+0000","last_undegraded":"2024-09-10T07:48:52.607033+0000","last_fullsized":"2024-09-10T07:48:52.607033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:12.913436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609194+0000","last_change":"2024-09-10T07:48:50.609194+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609194+0000","last_undegraded":"2024-09-10T07:48:50.609194+0000","last_fullsized":"2024-09-10T07:48:50.609194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609587+0000","last_change":"2024-09-10T07:48:50.624873+0000","last_active":"2024-09-10T07:48:51.609587+0000","last_peered":"2024-09-10T07:48:51.609587+0000","last_clean":"2024-09-10T07:48:51.609587+0000","last_became_active":"2024-09-10T07:48:50.622063+0000","last_became_peered":"2024-09-10T07:48:50.622063+0000","last_unstale":"2024-09-10T07:48:51.609587+0000","last_undegraded":"2024-09-10T07:48:51.609587+0000","last_fullsized":"2024-09-10T07:48:51.609587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:49.987651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.608768+0000","last_change":"2024-09-10T07:48:50.608768+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.608768+0000","last_undegraded":"2024-09-10T07:48:50.608768+0000","last_fullsized":"2024-09-10T07:48:50.608768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606490+0000","last_change":"2024-09-10T07:48:50.619456+0000","last_active":"2024-09-10T07:48:52.606490+0000","last_peered":"2024-09-10T07:48:52.606490+0000","last_clean":"2024-09-10T07:48:52.606490+0000","last_became_active":"2024-09-10T07:48:50.618866+0000","last_became_peered":"2024-09-10T07:48:50.618866+0000","last_unstale":"2024-09-10T07:48:52.606490+0000","last_undegraded":"2024-09-10T07:48:52.606490+0000","last_fullsized":"2024-09-10T07:48:52.606490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:58.806978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.609027+0000","last_change":"2024-09-10T07:48:50.609027+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.609027+0000","last_undegraded":"2024-09-10T07:48:50.609027+0000","last_fullsized":"2024-09-10T07:48:50.609027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606169+0000","last_change":"2024-09-10T07:48:50.617552+0000","last_active":"2024-09-10T07:48:51.606169+0000","last_peered":"2024-09-10T07:48:51.606169+0000","last_clean":"2024-09-10T07:48:51.606169+0000","last_became_active":"2024-09-10T07:48:50.617260+0000","last_became_peered":"2024-09-10T07:48:50.617260+0000","last_unstale":"2024-09-10T07:48:51.606169+0000","last_undegraded":"2024-09-10T07:48:51.606169+0000","last_fullsized":"2024-09-10T07:48:51.606169+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:03.137368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.634402+0000","last_change":"2024-09-10T07:48:51.625611+0000","last_active":"2024-09-10T07:48:51.634402+0000","last_peered":"2024-09-10T07:48:51.634402+0000","last_clean":"2024-09-10T07:48:51.634402+0000","last_became_active":"2024-09-10T07:48:51.625352+0000","last_became_peered":"2024-09-10T07:48:51.625352+0000","last_unstale":"2024-09-10T07:48:51.634402+0000","last_undegraded":"2024-09-10T07:48:51.634402+0000","last_fullsized":"2024-09-10T07:48:51.634402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:49.068278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622838+0000","last_change":"2024-09-10T07:48:50.621422+0000","last_active":"2024-09-10T07:48:50.622838+0000","last_peered":"2024-09-10T07:48:50.622838+0000","last_clean":"2024-09-10T07:48:50.622838+0000","last_became_active":"2024-09-10T07:48:50.621142+0000","last_became_peered":"2024-09-10T07:48:50.621142+0000","last_unstale":"2024-09-10T07:48:50.622838+0000","last_undegraded":"2024-09-10T07:48:50.622838+0000","last_fullsized":"2024-09-10T07:48:50.622838+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:25.494380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606946+0000","last_change":"2024-09-10T07:48:51.632002+0000","last_active":"2024-09-10T07:48:52.606946+0000","last_peered":"2024-09-10T07:48:52.606946+0000","last_clean":"2024-09-10T07:48:52.606946+0000","last_became_active":"2024-09-10T07:48:51.629180+0000","last_became_peered":"2024-09-10T07:48:51.629180+0000","last_unstale":"2024-09-10T07:48:52.606946+0000","last_undegraded":"2024-09-10T07:48:52.606946+0000","last_fullsized":"2024-09-10T07:48:52.606946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.610075+0000","last_change":"2024-09-10T07:48:50.622779+0000","last_active":"2024-09-10T07:48:51.610075+0000","last_peered":"2024-09-10T07:48:51.610075+0000","last_clean":"2024-09-10T07:48:51.610075+0000","last_became_active":"2024-09-10T07:48:50.622556+0000","last_became_peered":"2024-09-10T07:48:50.622556+0000","last_unstale":"2024-09-10T07:48:51.610075+0000","last_undegraded":"2024-09-10T07:48:51.610075+0000","last_fullsized":"2024-09-10T07:48:51.610075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:11.544783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.638933+0000","last_change":"2024-09-10T07:48:51.628390+0000","last_active":"2024-09-10T07:48:51.638933+0000","last_peered":"2024-09-10T07:48:51.638933+0000","last_clean":"2024-09-10T07:48:51.638933+0000","last_became_active":"2024-09-10T07:48:51.628095+0000","last_became_peered":"2024-09-10T07:48:51.628095+0000","last_unstale":"2024-09-10T07:48:51.638933+0000","last_undegraded":"2024-09-10T07:48:51.638933+0000","last_fullsized":"2024-09-10T07:48:51.638933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:48.707363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.622111+0000","last_change":"2024-09-10T07:48:50.619655+0000","last_active":"2024-09-10T07:48:50.622111+0000","last_peered":"2024-09-10T07:48:50.622111+0000","last_clean":"2024-09-10T07:48:50.622111+0000","last_became_active":"2024-09-10T07:48:50.619360+0000","last_became_peered":"2024-09-10T07:48:50.619360+0000","last_unstale":"2024-09-10T07:48:50.622111+0000","last_undegraded":"2024-09-10T07:48:50.622111+0000","last_fullsized":"2024-09-10T07:48:50.622111+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606864+0000","last_change":"2024-09-10T07:48:51.623695+0000","last_active":"2024-09-10T07:48:52.606864+0000","last_peered":"2024-09-10T07:48:52.606864+0000","last_clean":"2024-09-10T07:48:52.606864+0000","last_became_active":"2024-09-10T07:48:51.623049+0000","last_became_peered":"2024-09-10T07:48:51.623049+0000","last_unstale":"2024-09-10T07:48:52.606864+0000","last_undegraded":"2024-09-10T07:48:52.606864+0000","last_fullsized":"2024-09-10T07:48:52.606864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:42.211425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.609403+0000","last_change":"2024-09-10T07:48:50.619068+0000","last_active":"2024-09-10T07:48:51.609403+0000","last_peered":"2024-09-10T07:48:51.609403+0000","last_clean":"2024-09-10T07:48:51.609403+0000","last_became_active":"2024-09-10T07:48:50.618824+0000","last_became_peered":"2024-09-10T07:48:50.618824+0000","last_unstale":"2024-09-10T07:48:51.609403+0000","last_undegraded":"2024-09-10T07:48:51.609403+0000","last_fullsized":"2024-09-10T07:48:51.609403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:24.707876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.611851+0000","last_change":"2024-09-10T07:48:50.611851+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.611851+0000","last_undegraded":"2024-09-10T07:48:50.611851+0000","last_fullsized":"2024-09-10T07:48:50.611851+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.606363+0000","last_change":"2024-09-10T07:48:50.614453+0000","last_active":"2024-09-10T07:48:51.606363+0000","last_peered":"2024-09-10T07:48:51.606363+0000","last_clean":"2024-09-10T07:48:51.606363+0000","last_became_active":"2024-09-10T07:48:50.614159+0000","last_became_peered":"2024-09-10T07:48:50.614159+0000","last_unstale":"2024-09-10T07:48:51.606363+0000","last_undegraded":"2024-09-10T07:48:51.606363+0000","last_fullsized":"2024-09-10T07:48:51.606363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.792276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.631226+0000","last_change":"2024-09-10T07:48:51.628493+0000","last_active":"2024-09-10T07:48:51.631226+0000","last_peered":"2024-09-10T07:48:51.631226+0000","last_clean":"2024-09-10T07:48:51.631226+0000","last_became_active":"2024-09-10T07:48:51.628229+0000","last_became_peered":"2024-09-10T07:48:51.628229+0000","last_unstale":"2024-09-10T07:48:51.631226+0000","last_undegraded":"2024-09-10T07:48:51.631226+0000","last_fullsized":"2024-09-10T07:48:51.631226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:04.162971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.624400+0000","last_change":"2024-09-10T07:48:50.620900+0000","last_active":"2024-09-10T07:48:50.624400+0000","last_peered":"2024-09-10T07:48:50.624400+0000","last_clean":"2024-09-10T07:48:50.624400+0000","last_became_active":"2024-09-10T07:48:50.620412+0000","last_became_peered":"2024-09-10T07:48:50.620412+0000","last_unstale":"2024-09-10T07:48:50.624400+0000","last_undegraded":"2024-09-10T07:48:50.624400+0000","last_fullsized":"2024-09-10T07:48:50.624400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:58.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.635716+0000","last_change":"2024-09-10T07:48:51.632214+0000","last_active":"2024-09-10T07:48:51.635716+0000","last_peered":"2024-09-10T07:48:51.635716+0000","last_clean":"2024-09-10T07:48:51.635716+0000","last_became_active":"2024-09-10T07:48:51.631932+0000","last_became_peered":"2024-09-10T07:48:51.631932+0000","last_unstale":"2024-09-10T07:48:51.635716+0000","last_undegraded":"2024-09-10T07:48:51.635716+0000","last_fullsized":"2024-09-10T07:48:51.635716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T07:48:50.618234+0000","last_change":"2024-09-10T07:48:50.615382+0000","last_active":"2024-09-10T07:48:50.618234+0000","last_peered":"2024-09-10T07:48:50.618234+0000","last_clean":"2024-09-10T07:48:50.618234+0000","last_became_active":"2024-09-10T07:48:50.614269+0000","last_became_peered":"2024-09-10T07:48:50.614269+0000","last_unstale":"2024-09-10T07:48:50.618234+0000","last_undegraded":"2024-09-10T07:48:50.618234+0000","last_fullsized":"2024-09-10T07:48:50.618234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-09-10T07:48:50.612430+0000","last_change":"2024-09-10T07:48:50.612430+0000","last_active":"2024-09-10T07:48:50.592745+0000","last_peered":"2024-09-10T07:48:50.592745+0000","last_clean":"2024-09-10T07:48:50.592745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:48:50.612430+0000","last_undegraded":"2024-09-10T07:48:50.612430+0000","last_fullsized":"2024-09-10T07:48:50.612430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.645507+0000","last_change":"2024-09-10T07:48:50.624209+0000","last_active":"2024-09-10T07:48:51.645507+0000","last_peered":"2024-09-10T07:48:51.645507+0000","last_clean":"2024-09-10T07:48:51.645507+0000","last_became_active":"2024-09-10T07:48:50.623406+0000","last_became_peered":"2024-09-10T07:48:50.623406+0000","last_unstale":"2024-09-10T07:48:51.645507+0000","last_undegraded":"2024-09-10T07:48:51.645507+0000","last_fullsized":"2024-09-10T07:48:51.645507+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:29.913985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":217,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T07:48:51.603655+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:48:51.603655+0000","last_peered":"2024-09-10T07:48:51.603655+0000","last_clean":"2024-09-10T07:48:51.603655+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:48:51.603655+0000","last_undegraded":"2024-09-10T07:48:51.603655+0000","last_fullsized":"2024-09-10T07:48:51.603655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1772,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476770,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28152,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343688,"statfs":{"total":96636764160,"available":96607936512,"internally_reserved":0,"allocated":1146880,"data_stored":814559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":224191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":808284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:49:00.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:49:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.738+0000 7fe032e00640 1 Processor -- start 2024-09-10T07:49:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.739+0000 7fe032e00640 1 -- start start 2024-09-10T07:49:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe032e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe02c161b30 0x7fe02c15f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe032e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 0x7fe02c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe032e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02c15fbb0 con 0x7fe02c15fdb0 2024-09-10T07:49:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe032e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe02c0b36a0 con 0x7fe02c161b30 2024-09-10T07:49:00.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe032e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe02c0b3980 con 0x7fe02c06c230 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe02be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe02c161b30 0x7fe02c15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe02be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe02c161b30 0x7fe02c15f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54668/0 (socket says 172.21.15.3:54668) 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe02be00640 1 -- 172.21.15.3:0/775348231 learned_addr learned my addr 172.21.15.3:0/775348231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe030e00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 0x7fe02c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.740+0000 7fe02be00640 1 -- 172.21.15.3:0/775348231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 msgr2=0x7fe02c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.741+0000 7fe02be00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 0x7fe02c0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.741+0000 7fe02be00640 1 -- 172.21.15.3:0/775348231 >> v1:172.21.15.70:6789/0 conn(0x7fe02c15fdb0 legacy=0x7fe02c06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.741+0000 7fe02be00640 1 -- 172.21.15.3:0/775348231 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c0b3c60 con 0x7fe02c161b30 2024-09-10T07:49:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.741+0000 7fe030e00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 0x7fe02c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe02be00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15f670 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009870 tx=0x7fe01c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf2bf594c1fb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe02b400640 1 -- 172.21.15.3:0/775348231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c037070 con 0x7fe02c161b30 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe02b400640 1 -- 172.21.15.3:0/775348231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01c032440 con 0x7fe02c161b30 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe02b400640 1 -- 172.21.15.3:0/775348231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c032810 con 0x7fe02c161b30 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe032e00640 1 -- 172.21.15.3:0/775348231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 msgr2=0x7fe02c15f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.742+0000 7fe032e00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15f670 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009870 tx=0x7fe01c02a250 comp rx=0 tx=0).stop 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 -- 172.21.15.3:0/775348231 shutdown_connections 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c06c230 0x7fe02c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 --2- 172.21.15.3:0/775348231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15f670 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 -- 172.21.15.3:0/775348231 >> 172.21.15.3:0/775348231 conn(0x7fe02c06d000 msgr2=0x7fe02c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 -- 172.21.15.3:0/775348231 shutdown_connections 2024-09-10T07:49:00.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.743+0000 7fe032e00640 1 -- 172.21.15.3:0/775348231 wait complete. 2024-09-10T07:49:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.744+0000 7fe032e00640 1 Processor -- start 2024-09-10T07:49:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.744+0000 7fe032e00640 1 -- start start 2024-09-10T07:49:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe02c06c230 0x7fe02c14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c15fdb0 0x7fe02c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c161f10 con 0x7fe02c161b30 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c0b4220 con 0x7fe02c06c230 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe032e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe02c0aefa0 con 0x7fe02c15fdb0 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.745+0000 7fe030e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54676/0 (socket says 172.21.15.3:54676) 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 -- 172.21.15.3:0/209714991 learned_addr learned my addr 172.21.15.3:0/209714991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe023e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c15fdb0 0x7fe02c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe02be00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe02c06c230 0x7fe02c14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 -- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c15fdb0 msgr2=0x7fe02c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c15fdb0 0x7fe02c15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 -- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe02c06c230 msgr2=0x7fe02c14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe02c06c230 0x7fe02c14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 -- 172.21.15.3:0/209714991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c0cddc0 con 0x7fe02c161b30 2024-09-10T07:49:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.746+0000 7fe030e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe02400cd50 tx=0x7fe02400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.747+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02400f680 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.747+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02400f840 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.747+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02400fc30 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.747+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02c0ce0a0 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.747+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02c0ce660 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.748+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02c0ce860 con 0x7fe02c161b30 2024-09-10T07:49:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.749+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0240213f0 con 0x7fe02c161b30 2024-09-10T07:49:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.750+0000 7fe028c00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 0x7fe00806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.750+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fe02408bc30 con 0x7fe02c161b30 2024-09-10T07:49:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.750+0000 7fe02be00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 0x7fe00806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.751+0000 7fe02be00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 0x7fe00806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe01c034000 tx=0x7fe01c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.754+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe02c0ce860 con 0x7fe02c161b30 2024-09-10T07:49:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.935+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe02c0cc6c0 con 0x7fe00806be30 2024-09-10T07:49:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.942+0000 7fe028c00640 1 -- 172.21.15.3:0/209714991 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fe01c032780 con 0x7fe00806be30 2024-09-10T07:49:00.944 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 msgr2=0x7fe00806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 0x7fe00806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe01c034000 tx=0x7fe01c02e000 comp rx=0 tx=0).stop 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 msgr2=0x7fe02c15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe02400cd50 tx=0x7fe02400a650 comp rx=0 tx=0).stop 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 shutdown_connections 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe00806be30 0x7fe00806e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe02c161b30 0x7fe02c15adc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe02c15fdb0 0x7fe02c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 --2- 172.21.15.3:0/209714991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe02c06c230 0x7fe02c14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 >> 172.21.15.3:0/209714991 conn(0x7fe02c06d000 msgr2=0x7fe02c15ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 shutdown_connections 2024-09-10T07:49:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:00.946+0000 7fe032e00640 1 -- 172.21.15.3:0/209714991 wait complete. 2024-09-10T07:49:00.961 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-10T07:48:59.521826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238616,"kb_used_data":5736,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736104,"statfs":{"total":773094113280,"available":772849770496,"internally_reserved":0,"allocated":5873664,"data_stored":3083684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001957"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083060+0000","last_change":"2024-09-10T07:48:51.617751+0000","last_active":"2024-09-10T07:48:53.083060+0000","last_peered":"2024-09-10T07:48:53.083060+0000","last_clean":"2024-09-10T07:48:53.083060+0000","last_became_active":"2024-09-10T07:48:51.617450+0000","last_became_peered":"2024-09-10T07:48:51.617450+0000","last_unstale":"2024-09-10T07:48:53.083060+0000","last_undegraded":"2024-09-10T07:48:53.083060+0000","last_fullsized":"2024-09-10T07:48:53.083060+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:56.573657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083493+0000","last_change":"2024-09-10T07:48:51.620056+0000","last_active":"2024-09-10T07:48:53.083493+0000","last_peered":"2024-09-10T07:48:53.083493+0000","last_clean":"2024-09-10T07:48:53.083493+0000","last_became_active":"2024-09-10T07:48:51.619801+0000","last_became_peered":"2024-09-10T07:48:51.619801+0000","last_unstale":"2024-09-10T07:48:53.083493+0000","last_undegraded":"2024-09-10T07:48:53.083493+0000","last_fullsized":"2024-09-10T07:48:53.083493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.365454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606518+0000","last_change":"2024-09-10T07:48:51.612764+0000","last_active":"2024-09-10T07:48:52.606518+0000","last_peered":"2024-09-10T07:48:52.606518+0000","last_clean":"2024-09-10T07:48:52.606518+0000","last_became_active":"2024-09-10T07:48:51.611550+0000","last_became_peered":"2024-09-10T07:48:51.611550+0000","last_unstale":"2024-09-10T07:48:52.606518+0000","last_undegraded":"2024-09-10T07:48:52.606518+0000","last_fullsized":"2024-09-10T07:48:52.606518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:26.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629873+0000","last_change":"2024-09-10T07:48:51.629616+0000","last_active":"2024-09-10T07:48:52.629873+0000","last_peered":"2024-09-10T07:48:52.629873+0000","last_clean":"2024-09-10T07:48:52.629873+0000","last_became_active":"2024-09-10T07:48:51.629073+0000","last_became_peered":"2024-09-10T07:48:51.629073+0000","last_unstale":"2024-09-10T07:48:52.629873+0000","last_undegraded":"2024-09-10T07:48:52.629873+0000","last_fullsized":"2024-09-10T07:48:52.629873+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:28.302892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082569+0000","last_change":"2024-09-10T07:48:51.624149+0000","last_active":"2024-09-10T07:48:53.082569+0000","last_peered":"2024-09-10T07:48:53.082569+0000","last_clean":"2024-09-10T07:48:53.082569+0000","last_became_active":"2024-09-10T07:48:51.623849+0000","last_became_peered":"2024-09-10T07:48:51.623849+0000","last_unstale":"2024-09-10T07:48:53.082569+0000","last_undegraded":"2024-09-10T07:48:53.082569+0000","last_fullsized":"2024-09-10T07:48:53.082569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:39.923143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083424+0000","last_change":"2024-09-10T07:48:51.629348+0000","last_active":"2024-09-10T07:48:53.083424+0000","last_peered":"2024-09-10T07:48:53.083424+0000","last_clean":"2024-09-10T07:48:53.083424+0000","last_became_active":"2024-09-10T07:48:51.628738+0000","last_became_peered":"2024-09-10T07:48:51.628738+0000","last_unstale":"2024-09-10T07:48:53.083424+0000","last_undegraded":"2024-09-10T07:48:53.083424+0000","last_fullsized":"2024-09-10T07:48:53.083424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:29.182845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606824+0000","last_change":"2024-09-10T07:48:51.628760+0000","last_active":"2024-09-10T07:48:52.606824+0000","last_peered":"2024-09-10T07:48:52.606824+0000","last_clean":"2024-09-10T07:48:52.606824+0000","last_became_active":"2024-09-10T07:48:51.628462+0000","last_became_peered":"2024-09-10T07:48:51.628462+0000","last_unstale":"2024-09-10T07:48:52.606824+0000","last_undegraded":"2024-09-10T07:48:52.606824+0000","last_fullsized":"2024-09-10T07:48:52.606824+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:25.406609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629644+0000","last_change":"2024-09-10T07:48:51.629968+0000","last_active":"2024-09-10T07:48:52.629644+0000","last_peered":"2024-09-10T07:48:52.629644+0000","last_clean":"2024-09-10T07:48:52.629644+0000","last_became_active":"2024-09-10T07:48:51.625817+0000","last_became_peered":"2024-09-10T07:48:51.625817+0000","last_unstale":"2024-09-10T07:48:52.629644+0000","last_undegraded":"2024-09-10T07:48:52.629644+0000","last_fullsized":"2024-09-10T07:48:52.629644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:39.935680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.084364+0000","last_change":"2024-09-10T07:48:51.621773+0000","last_active":"2024-09-10T07:48:53.084364+0000","last_peered":"2024-09-10T07:48:53.084364+0000","last_clean":"2024-09-10T07:48:53.084364+0000","last_became_active":"2024-09-10T07:48:51.621463+0000","last_became_peered":"2024-09-10T07:48:51.621463+0000","last_unstale":"2024-09-10T07:48:53.084364+0000","last_undegraded":"2024-09-10T07:48:53.084364+0000","last_fullsized":"2024-09-10T07:48:53.084364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:54.995968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607402+0000","last_change":"2024-09-10T07:48:51.617674+0000","last_active":"2024-09-10T07:48:52.607402+0000","last_peered":"2024-09-10T07:48:52.607402+0000","last_clean":"2024-09-10T07:48:52.607402+0000","last_became_active":"2024-09-10T07:48:51.617378+0000","last_became_peered":"2024-09-10T07:48:51.617378+0000","last_unstale":"2024-09-10T07:48:52.607402+0000","last_undegraded":"2024-09-10T07:48:52.607402+0000","last_fullsized":"2024-09-10T07:48:52.607402+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:02.393228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629977+0000","last_change":"2024-09-10T07:48:51.626725+0000","last_active":"2024-09-10T07:48:52.629977+0000","last_peered":"2024-09-10T07:48:52.629977+0000","last_clean":"2024-09-10T07:48:52.629977+0000","last_became_active":"2024-09-10T07:48:51.626325+0000","last_became_peered":"2024-09-10T07:48:51.626325+0000","last_unstale":"2024-09-10T07:48:52.629977+0000","last_undegraded":"2024-09-10T07:48:52.629977+0000","last_fullsized":"2024-09-10T07:48:52.629977+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:13.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083097+0000","last_change":"2024-09-10T07:48:51.620489+0000","last_active":"2024-09-10T07:48:53.083097+0000","last_peered":"2024-09-10T07:48:53.083097+0000","last_clean":"2024-09-10T07:48:53.083097+0000","last_became_active":"2024-09-10T07:48:51.620237+0000","last_became_peered":"2024-09-10T07:48:51.620237+0000","last_unstale":"2024-09-10T07:48:53.083097+0000","last_undegraded":"2024-09-10T07:48:53.083097+0000","last_fullsized":"2024-09-10T07:48:53.083097+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:38.424994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083671+0000","last_change":"2024-09-10T07:48:51.629162+0000","last_active":"2024-09-10T07:48:53.083671+0000","last_peered":"2024-09-10T07:48:53.083671+0000","last_clean":"2024-09-10T07:48:53.083671+0000","last_became_active":"2024-09-10T07:48:51.628708+0000","last_became_peered":"2024-09-10T07:48:51.628708+0000","last_unstale":"2024-09-10T07:48:53.083671+0000","last_undegraded":"2024-09-10T07:48:53.083671+0000","last_fullsized":"2024-09-10T07:48:53.083671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:42.160979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629665+0000","last_change":"2024-09-10T07:48:51.626761+0000","last_active":"2024-09-10T07:48:52.629665+0000","last_peered":"2024-09-10T07:48:52.629665+0000","last_clean":"2024-09-10T07:48:52.629665+0000","last_became_active":"2024-09-10T07:48:51.623842+0000","last_became_peered":"2024-09-10T07:48:51.623842+0000","last_unstale":"2024-09-10T07:48:52.629665+0000","last_undegraded":"2024-09-10T07:48:52.629665+0000","last_fullsized":"2024-09-10T07:48:52.629665+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.365454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629587+0000","last_change":"2024-09-10T07:48:51.625412+0000","last_active":"2024-09-10T07:48:52.629587+0000","last_peered":"2024-09-10T07:48:52.629587+0000","last_clean":"2024-09-10T07:48:52.629587+0000","last_became_active":"2024-09-10T07:48:51.625012+0000","last_became_peered":"2024-09-10T07:48:51.625012+0000","last_unstale":"2024-09-10T07:48:52.629587+0000","last_undegraded":"2024-09-10T07:48:52.629587+0000","last_fullsized":"2024-09-10T07:48:52.629587+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:18.477897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082333+0000","last_change":"2024-09-10T07:48:51.631668+0000","last_active":"2024-09-10T07:48:53.082333+0000","last_peered":"2024-09-10T07:48:53.082333+0000","last_clean":"2024-09-10T07:48:53.082333+0000","last_became_active":"2024-09-10T07:48:51.631459+0000","last_became_peered":"2024-09-10T07:48:51.631459+0000","last_unstale":"2024-09-10T07:48:53.082333+0000","last_undegraded":"2024-09-10T07:48:53.082333+0000","last_fullsized":"2024-09-10T07:48:53.082333+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:27.291991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083595+0000","last_change":"2024-09-10T07:48:51.622041+0000","last_active":"2024-09-10T07:48:53.083595+0000","last_peered":"2024-09-10T07:48:53.083595+0000","last_clean":"2024-09-10T07:48:53.083595+0000","last_became_active":"2024-09-10T07:48:51.621622+0000","last_became_peered":"2024-09-10T07:48:51.621622+0000","last_unstale":"2024-09-10T07:48:53.083595+0000","last_undegraded":"2024-09-10T07:48:53.083595+0000","last_fullsized":"2024-09-10T07:48:53.083595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:15.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606493+0000","last_change":"2024-09-10T07:48:51.632824+0000","last_active":"2024-09-10T07:48:52.606493+0000","last_peered":"2024-09-10T07:48:52.606493+0000","last_clean":"2024-09-10T07:48:52.606493+0000","last_became_active":"2024-09-10T07:48:51.629313+0000","last_became_peered":"2024-09-10T07:48:51.629313+0000","last_unstale":"2024-09-10T07:48:52.606493+0000","last_undegraded":"2024-09-10T07:48:52.606493+0000","last_fullsized":"2024-09-10T07:48:52.606493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:01:03.036248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083045+0000","last_change":"2024-09-10T07:48:51.623911+0000","last_active":"2024-09-10T07:48:53.083045+0000","last_peered":"2024-09-10T07:48:53.083045+0000","last_clean":"2024-09-10T07:48:53.083045+0000","last_became_active":"2024-09-10T07:48:51.623645+0000","last_became_peered":"2024-09-10T07:48:51.623645+0000","last_unstale":"2024-09-10T07:48:53.083045+0000","last_undegraded":"2024-09-10T07:48:53.083045+0000","last_fullsized":"2024-09-10T07:48:53.083045+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:18.477897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606836+0000","last_change":"2024-09-10T07:48:51.624623+0000","last_active":"2024-09-10T07:48:52.606836+0000","last_peered":"2024-09-10T07:48:52.606836+0000","last_clean":"2024-09-10T07:48:52.606836+0000","last_became_active":"2024-09-10T07:48:51.621627+0000","last_became_peered":"2024-09-10T07:48:51.621627+0000","last_unstale":"2024-09-10T07:48:52.606836+0000","last_undegraded":"2024-09-10T07:48:52.606836+0000","last_fullsized":"2024-09-10T07:48:52.606836+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:19.844611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629956+0000","last_change":"2024-09-10T07:48:51.633904+0000","last_active":"2024-09-10T07:48:52.629956+0000","last_peered":"2024-09-10T07:48:52.629956+0000","last_clean":"2024-09-10T07:48:52.629956+0000","last_became_active":"2024-09-10T07:48:51.633517+0000","last_became_peered":"2024-09-10T07:48:51.633517+0000","last_unstale":"2024-09-10T07:48:52.629956+0000","last_undegraded":"2024-09-10T07:48:52.629956+0000","last_fullsized":"2024-09-10T07:48:52.629956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:30.985308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083480+0000","last_change":"2024-09-10T07:48:51.618115+0000","last_active":"2024-09-10T07:48:53.083480+0000","last_peered":"2024-09-10T07:48:53.083480+0000","last_clean":"2024-09-10T07:48:53.083480+0000","last_became_active":"2024-09-10T07:48:51.617132+0000","last_became_peered":"2024-09-10T07:48:51.617132+0000","last_unstale":"2024-09-10T07:48:53.083480+0000","last_undegraded":"2024-09-10T07:48:53.083480+0000","last_fullsized":"2024-09-10T07:48:53.083480+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607094+0000","last_change":"2024-09-10T07:48:51.616950+0000","last_active":"2024-09-10T07:48:52.607094+0000","last_peered":"2024-09-10T07:48:52.607094+0000","last_clean":"2024-09-10T07:48:52.607094+0000","last_became_active":"2024-09-10T07:48:51.614672+0000","last_became_peered":"2024-09-10T07:48:51.614672+0000","last_unstale":"2024-09-10T07:48:52.607094+0000","last_undegraded":"2024-09-10T07:48:52.607094+0000","last_fullsized":"2024-09-10T07:48:52.607094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:53.291761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082695+0000","last_change":"2024-09-10T07:48:51.630986+0000","last_active":"2024-09-10T07:48:53.082695+0000","last_peered":"2024-09-10T07:48:53.082695+0000","last_clean":"2024-09-10T07:48:53.082695+0000","last_became_active":"2024-09-10T07:48:51.630754+0000","last_became_peered":"2024-09-10T07:48:51.630754+0000","last_unstale":"2024-09-10T07:48:53.082695+0000","last_undegraded":"2024-09-10T07:48:53.082695+0000","last_fullsized":"2024-09-10T07:48:53.082695+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.308850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083152+0000","last_change":"2024-09-10T07:48:51.617701+0000","last_active":"2024-09-10T07:48:53.083152+0000","last_peered":"2024-09-10T07:48:53.083152+0000","last_clean":"2024-09-10T07:48:53.083152+0000","last_became_active":"2024-09-10T07:48:51.617418+0000","last_became_peered":"2024-09-10T07:48:51.617418+0000","last_unstale":"2024-09-10T07:48:53.083152+0000","last_undegraded":"2024-09-10T07:48:53.083152+0000","last_fullsized":"2024-09-10T07:48:53.083152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.365454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.630181+0000","last_change":"2024-09-10T07:48:51.626151+0000","last_active":"2024-09-10T07:48:52.630181+0000","last_peered":"2024-09-10T07:48:52.630181+0000","last_clean":"2024-09-10T07:48:52.630181+0000","last_became_active":"2024-09-10T07:48:51.625851+0000","last_became_peered":"2024-09-10T07:48:51.625851+0000","last_unstale":"2024-09-10T07:48:52.630181+0000","last_undegraded":"2024-09-10T07:48:52.630181+0000","last_fullsized":"2024-09-10T07:48:52.630181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:45.521403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607545+0000","last_change":"2024-09-10T07:48:51.620624+0000","last_active":"2024-09-10T07:48:52.607545+0000","last_peered":"2024-09-10T07:48:52.607545+0000","last_clean":"2024-09-10T07:48:52.607545+0000","last_became_active":"2024-09-10T07:48:51.619782+0000","last_became_peered":"2024-09-10T07:48:51.619782+0000","last_unstale":"2024-09-10T07:48:52.607545+0000","last_undegraded":"2024-09-10T07:48:52.607545+0000","last_fullsized":"2024-09-10T07:48:52.607545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:35.074432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607381+0000","last_change":"2024-09-10T07:48:51.616307+0000","last_active":"2024-09-10T07:48:52.607381+0000","last_peered":"2024-09-10T07:48:52.607381+0000","last_clean":"2024-09-10T07:48:52.607381+0000","last_became_active":"2024-09-10T07:48:51.615851+0000","last_became_peered":"2024-09-10T07:48:51.615851+0000","last_unstale":"2024-09-10T07:48:52.607381+0000","last_undegraded":"2024-09-10T07:48:52.607381+0000","last_fullsized":"2024-09-10T07:48:52.607381+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:14.603796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083019+0000","last_change":"2024-09-10T07:48:51.616551+0000","last_active":"2024-09-10T07:48:53.083019+0000","last_peered":"2024-09-10T07:48:53.083019+0000","last_clean":"2024-09-10T07:48:53.083019+0000","last_became_active":"2024-09-10T07:48:51.616241+0000","last_became_peered":"2024-09-10T07:48:51.616241+0000","last_unstale":"2024-09-10T07:48:53.083019+0000","last_undegraded":"2024-09-10T07:48:53.083019+0000","last_fullsized":"2024-09-10T07:48:53.083019+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:38.424994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082880+0000","last_change":"2024-09-10T07:48:51.628969+0000","last_active":"2024-09-10T07:48:53.082880+0000","last_peered":"2024-09-10T07:48:53.082880+0000","last_clean":"2024-09-10T07:48:53.082880+0000","last_became_active":"2024-09-10T07:48:51.628200+0000","last_became_peered":"2024-09-10T07:48:51.628200+0000","last_unstale":"2024-09-10T07:48:53.082880+0000","last_undegraded":"2024-09-10T07:48:53.082880+0000","last_fullsized":"2024-09-10T07:48:53.082880+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:27.291991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082839+0000","last_change":"2024-09-10T07:48:51.630113+0000","last_active":"2024-09-10T07:48:53.082839+0000","last_peered":"2024-09-10T07:48:53.082839+0000","last_clean":"2024-09-10T07:48:53.082839+0000","last_became_active":"2024-09-10T07:48:51.629842+0000","last_became_peered":"2024-09-10T07:48:51.629842+0000","last_unstale":"2024-09-10T07:48:53.082839+0000","last_undegraded":"2024-09-10T07:48:53.082839+0000","last_fullsized":"2024-09-10T07:48:53.082839+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:53.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629622+0000","last_change":"2024-09-10T07:48:51.626166+0000","last_active":"2024-09-10T07:48:52.629622+0000","last_peered":"2024-09-10T07:48:52.629622+0000","last_clean":"2024-09-10T07:48:52.629622+0000","last_became_active":"2024-09-10T07:48:51.625197+0000","last_became_peered":"2024-09-10T07:48:51.625197+0000","last_unstale":"2024-09-10T07:48:52.629622+0000","last_undegraded":"2024-09-10T07:48:52.629622+0000","last_fullsized":"2024-09-10T07:48:52.629622+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:56.573657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083407+0000","last_change":"2024-09-10T07:48:51.624067+0000","last_active":"2024-09-10T07:48:53.083407+0000","last_peered":"2024-09-10T07:48:53.083407+0000","last_clean":"2024-09-10T07:48:53.083407+0000","last_became_active":"2024-09-10T07:48:51.623812+0000","last_became_peered":"2024-09-10T07:48:51.623812+0000","last_unstale":"2024-09-10T07:48:53.083407+0000","last_undegraded":"2024-09-10T07:48:53.083407+0000","last_fullsized":"2024-09-10T07:48:53.083407+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:39.498098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.084294+0000","last_change":"2024-09-10T07:48:50.627953+0000","last_active":"2024-09-10T07:48:53.084294+0000","last_peered":"2024-09-10T07:48:53.084294+0000","last_clean":"2024-09-10T07:48:53.084294+0000","last_became_active":"2024-09-10T07:48:50.627666+0000","last_became_peered":"2024-09-10T07:48:50.627666+0000","last_unstale":"2024-09-10T07:48:53.084294+0000","last_undegraded":"2024-09-10T07:48:53.084294+0000","last_fullsized":"2024-09-10T07:48:53.084294+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083220+0000","last_change":"2024-09-10T07:48:51.621847+0000","last_active":"2024-09-10T07:48:53.083220+0000","last_peered":"2024-09-10T07:48:53.083220+0000","last_clean":"2024-09-10T07:48:53.083220+0000","last_became_active":"2024-09-10T07:48:51.621554+0000","last_became_peered":"2024-09-10T07:48:51.621554+0000","last_unstale":"2024-09-10T07:48:53.083220+0000","last_undegraded":"2024-09-10T07:48:53.083220+0000","last_fullsized":"2024-09-10T07:48:53.083220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:15.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607727+0000","last_change":"2024-09-10T07:48:50.619796+0000","last_active":"2024-09-10T07:48:52.607727+0000","last_peered":"2024-09-10T07:48:52.607727+0000","last_clean":"2024-09-10T07:48:52.607727+0000","last_became_active":"2024-09-10T07:48:50.619535+0000","last_became_peered":"2024-09-10T07:48:50.619535+0000","last_unstale":"2024-09-10T07:48:52.607727+0000","last_undegraded":"2024-09-10T07:48:52.607727+0000","last_fullsized":"2024-09-10T07:48:52.607727+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:44.514806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083248+0000","last_change":"2024-09-10T07:48:51.629435+0000","last_active":"2024-09-10T07:48:53.083248+0000","last_peered":"2024-09-10T07:48:53.083248+0000","last_clean":"2024-09-10T07:48:53.083248+0000","last_became_active":"2024-09-10T07:48:51.628928+0000","last_became_peered":"2024-09-10T07:48:51.628928+0000","last_unstale":"2024-09-10T07:48:53.083248+0000","last_undegraded":"2024-09-10T07:48:53.083248+0000","last_fullsized":"2024-09-10T07:48:53.083248+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:18.477897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.084265+0000","last_change":"2024-09-10T07:48:50.619441+0000","last_active":"2024-09-10T07:48:53.084265+0000","last_peered":"2024-09-10T07:48:53.084265+0000","last_clean":"2024-09-10T07:48:53.084265+0000","last_became_active":"2024-09-10T07:48:50.619148+0000","last_became_peered":"2024-09-10T07:48:50.619148+0000","last_unstale":"2024-09-10T07:48:53.084265+0000","last_undegraded":"2024-09-10T07:48:53.084265+0000","last_fullsized":"2024-09-10T07:48:53.084265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:56.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.630132+0000","last_change":"2024-09-10T07:48:51.643329+0000","last_active":"2024-09-10T07:48:52.630132+0000","last_peered":"2024-09-10T07:48:52.630132+0000","last_clean":"2024-09-10T07:48:52.630132+0000","last_became_active":"2024-09-10T07:48:51.642961+0000","last_became_peered":"2024-09-10T07:48:51.642961+0000","last_unstale":"2024-09-10T07:48:52.630132+0000","last_undegraded":"2024-09-10T07:48:52.630132+0000","last_fullsized":"2024-09-10T07:48:52.630132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:57.158671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607334+0000","last_change":"2024-09-10T07:48:50.617904+0000","last_active":"2024-09-10T07:48:52.607334+0000","last_peered":"2024-09-10T07:48:52.607334+0000","last_clean":"2024-09-10T07:48:52.607334+0000","last_became_active":"2024-09-10T07:48:50.617656+0000","last_became_peered":"2024-09-10T07:48:50.617656+0000","last_unstale":"2024-09-10T07:48:52.607334+0000","last_undegraded":"2024-09-10T07:48:52.607334+0000","last_fullsized":"2024-09-10T07:48:52.607334+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.305959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607161+0000","last_change":"2024-09-10T07:48:51.620554+0000","last_active":"2024-09-10T07:48:52.607161+0000","last_peered":"2024-09-10T07:48:52.607161+0000","last_clean":"2024-09-10T07:48:52.607161+0000","last_became_active":"2024-09-10T07:48:51.619526+0000","last_became_peered":"2024-09-10T07:48:51.619526+0000","last_unstale":"2024-09-10T07:48:52.607161+0000","last_undegraded":"2024-09-10T07:48:52.607161+0000","last_fullsized":"2024-09-10T07:48:52.607161+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.301723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083816+0000","last_change":"2024-09-10T07:48:50.623162+0000","last_active":"2024-09-10T07:48:53.083816+0000","last_peered":"2024-09-10T07:48:53.083816+0000","last_clean":"2024-09-10T07:48:53.083816+0000","last_became_active":"2024-09-10T07:48:50.622765+0000","last_became_peered":"2024-09-10T07:48:50.622765+0000","last_unstale":"2024-09-10T07:48:53.083816+0000","last_undegraded":"2024-09-10T07:48:53.083816+0000","last_fullsized":"2024-09-10T07:48:53.083816+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:01:02.029650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606939+0000","last_change":"2024-09-10T07:48:51.628666+0000","last_active":"2024-09-10T07:48:52.606939+0000","last_peered":"2024-09-10T07:48:52.606939+0000","last_clean":"2024-09-10T07:48:52.606939+0000","last_became_active":"2024-09-10T07:48:51.628411+0000","last_became_peered":"2024-09-10T07:48:51.628411+0000","last_unstale":"2024-09-10T07:48:52.606939+0000","last_undegraded":"2024-09-10T07:48:52.606939+0000","last_fullsized":"2024-09-10T07:48:52.606939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:13.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607177+0000","last_change":"2024-09-10T07:48:50.616830+0000","last_active":"2024-09-10T07:48:52.607177+0000","last_peered":"2024-09-10T07:48:52.607177+0000","last_clean":"2024-09-10T07:48:52.607177+0000","last_became_active":"2024-09-10T07:48:50.616586+0000","last_became_peered":"2024-09-10T07:48:50.616586+0000","last_unstale":"2024-09-10T07:48:52.607177+0000","last_undegraded":"2024-09-10T07:48:52.607177+0000","last_fullsized":"2024-09-10T07:48:52.607177+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:25.042914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606676+0000","last_change":"2024-09-10T07:48:51.629828+0000","last_active":"2024-09-10T07:48:52.606676+0000","last_peered":"2024-09-10T07:48:52.606676+0000","last_clean":"2024-09-10T07:48:52.606676+0000","last_became_active":"2024-09-10T07:48:51.628172+0000","last_became_peered":"2024-09-10T07:48:51.628172+0000","last_unstale":"2024-09-10T07:48:52.606676+0000","last_undegraded":"2024-09-10T07:48:52.606676+0000","last_fullsized":"2024-09-10T07:48:52.606676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:20.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082411+0000","last_change":"2024-09-10T07:48:50.627131+0000","last_active":"2024-09-10T07:48:53.082411+0000","last_peered":"2024-09-10T07:48:53.082411+0000","last_clean":"2024-09-10T07:48:53.082411+0000","last_became_active":"2024-09-10T07:48:50.626968+0000","last_became_peered":"2024-09-10T07:48:50.626968+0000","last_unstale":"2024-09-10T07:48:53.082411+0000","last_undegraded":"2024-09-10T07:48:53.082411+0000","last_fullsized":"2024-09-10T07:48:53.082411+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:14:11.491693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606925+0000","last_change":"2024-09-10T07:48:51.624704+0000","last_active":"2024-09-10T07:48:52.606925+0000","last_peered":"2024-09-10T07:48:52.606925+0000","last_clean":"2024-09-10T07:48:52.606925+0000","last_became_active":"2024-09-10T07:48:51.624391+0000","last_became_peered":"2024-09-10T07:48:51.624391+0000","last_unstale":"2024-09-10T07:48:52.606925+0000","last_undegraded":"2024-09-10T07:48:52.606925+0000","last_fullsized":"2024-09-10T07:48:52.606925+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:08.622863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606654+0000","last_change":"2024-09-10T07:48:50.618315+0000","last_active":"2024-09-10T07:48:52.606654+0000","last_peered":"2024-09-10T07:48:52.606654+0000","last_clean":"2024-09-10T07:48:52.606654+0000","last_became_active":"2024-09-10T07:48:50.618022+0000","last_became_peered":"2024-09-10T07:48:50.618022+0000","last_unstale":"2024-09-10T07:48:52.606654+0000","last_undegraded":"2024-09-10T07:48:52.606654+0000","last_fullsized":"2024-09-10T07:48:52.606654+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:24.272006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606608+0000","last_change":"2024-09-10T07:48:51.616784+0000","last_active":"2024-09-10T07:48:52.606608+0000","last_peered":"2024-09-10T07:48:52.606608+0000","last_clean":"2024-09-10T07:48:52.606608+0000","last_became_active":"2024-09-10T07:48:51.616520+0000","last_became_peered":"2024-09-10T07:48:51.616520+0000","last_unstale":"2024-09-10T07:48:52.606608+0000","last_undegraded":"2024-09-10T07:48:52.606608+0000","last_fullsized":"2024-09-10T07:48:52.606608+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:57.578020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606724+0000","last_change":"2024-09-10T07:48:50.615394+0000","last_active":"2024-09-10T07:48:52.606724+0000","last_peered":"2024-09-10T07:48:52.606724+0000","last_clean":"2024-09-10T07:48:52.606724+0000","last_became_active":"2024-09-10T07:48:50.615139+0000","last_became_peered":"2024-09-10T07:48:50.615139+0000","last_unstale":"2024-09-10T07:48:52.606724+0000","last_undegraded":"2024-09-10T07:48:52.606724+0000","last_fullsized":"2024-09-10T07:48:52.606724+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:15.017199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606635+0000","last_change":"2024-09-10T07:48:51.619010+0000","last_active":"2024-09-10T07:48:52.606635+0000","last_peered":"2024-09-10T07:48:52.606635+0000","last_clean":"2024-09-10T07:48:52.606635+0000","last_became_active":"2024-09-10T07:48:51.618676+0000","last_became_peered":"2024-09-10T07:48:51.618676+0000","last_unstale":"2024-09-10T07:48:52.606635+0000","last_undegraded":"2024-09-10T07:48:52.606635+0000","last_fullsized":"2024-09-10T07:48:52.606635+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:39.498098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607323+0000","last_change":"2024-09-10T07:48:50.620699+0000","last_active":"2024-09-10T07:48:52.607323+0000","last_peered":"2024-09-10T07:48:52.607323+0000","last_clean":"2024-09-10T07:48:52.607323+0000","last_became_active":"2024-09-10T07:48:50.620417+0000","last_became_peered":"2024-09-10T07:48:50.620417+0000","last_unstale":"2024-09-10T07:48:52.607323+0000","last_undegraded":"2024-09-10T07:48:52.607323+0000","last_fullsized":"2024-09-10T07:48:52.607323+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:31.155316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629941+0000","last_change":"2024-09-10T07:48:51.629405+0000","last_active":"2024-09-10T07:48:52.629941+0000","last_peered":"2024-09-10T07:48:52.629941+0000","last_clean":"2024-09-10T07:48:52.629941+0000","last_became_active":"2024-09-10T07:48:51.628767+0000","last_became_peered":"2024-09-10T07:48:51.628767+0000","last_unstale":"2024-09-10T07:48:52.629941+0000","last_undegraded":"2024-09-10T07:48:52.629941+0000","last_fullsized":"2024-09-10T07:48:52.629941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:44.715168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083364+0000","last_change":"2024-09-10T07:48:50.620839+0000","last_active":"2024-09-10T07:48:53.083364+0000","last_peered":"2024-09-10T07:48:53.083364+0000","last_clean":"2024-09-10T07:48:53.083364+0000","last_became_active":"2024-09-10T07:48:50.620581+0000","last_became_peered":"2024-09-10T07:48:50.620581+0000","last_unstale":"2024-09-10T07:48:53.083364+0000","last_undegraded":"2024-09-10T07:48:53.083364+0000","last_fullsized":"2024-09-10T07:48:53.083364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607566+0000","last_change":"2024-09-10T07:48:51.630681+0000","last_active":"2024-09-10T07:48:52.607566+0000","last_peered":"2024-09-10T07:48:52.607566+0000","last_clean":"2024-09-10T07:48:52.607566+0000","last_became_active":"2024-09-10T07:48:51.630321+0000","last_became_peered":"2024-09-10T07:48:51.630321+0000","last_unstale":"2024-09-10T07:48:52.607566+0000","last_undegraded":"2024-09-10T07:48:52.607566+0000","last_fullsized":"2024-09-10T07:48:52.607566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:17.708919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606961+0000","last_change":"2024-09-10T07:48:50.620804+0000","last_active":"2024-09-10T07:48:52.606961+0000","last_peered":"2024-09-10T07:48:52.606961+0000","last_clean":"2024-09-10T07:48:52.606961+0000","last_became_active":"2024-09-10T07:48:50.620518+0000","last_became_peered":"2024-09-10T07:48:50.620518+0000","last_unstale":"2024-09-10T07:48:52.606961+0000","last_undegraded":"2024-09-10T07:48:52.606961+0000","last_fullsized":"2024-09-10T07:48:52.606961+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:26.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606323+0000","last_change":"2024-09-10T07:48:51.632456+0000","last_active":"2024-09-10T07:48:52.606323+0000","last_peered":"2024-09-10T07:48:52.606323+0000","last_clean":"2024-09-10T07:48:52.606323+0000","last_became_active":"2024-09-10T07:48:51.632175+0000","last_became_peered":"2024-09-10T07:48:51.632175+0000","last_unstale":"2024-09-10T07:48:52.606323+0000","last_undegraded":"2024-09-10T07:48:52.606323+0000","last_fullsized":"2024-09-10T07:48:52.606323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:35.522091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083134+0000","last_change":"2024-09-10T07:48:50.623794+0000","last_active":"2024-09-10T07:48:53.083134+0000","last_peered":"2024-09-10T07:48:53.083134+0000","last_clean":"2024-09-10T07:48:53.083134+0000","last_became_active":"2024-09-10T07:48:50.623196+0000","last_became_peered":"2024-09-10T07:48:50.623196+0000","last_unstale":"2024-09-10T07:48:53.083134+0000","last_undegraded":"2024-09-10T07:48:53.083134+0000","last_fullsized":"2024-09-10T07:48:53.083134+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:46.118174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083173+0000","last_change":"2024-09-10T07:48:51.621603+0000","last_active":"2024-09-10T07:48:53.083173+0000","last_peered":"2024-09-10T07:48:53.083173+0000","last_clean":"2024-09-10T07:48:53.083173+0000","last_became_active":"2024-09-10T07:48:51.621306+0000","last_became_peered":"2024-09-10T07:48:51.621306+0000","last_unstale":"2024-09-10T07:48:53.083173+0000","last_undegraded":"2024-09-10T07:48:53.083173+0000","last_fullsized":"2024-09-10T07:48:53.083173+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:26.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082322+0000","last_change":"2024-09-10T07:48:50.617395+0000","last_active":"2024-09-10T07:48:53.082322+0000","last_peered":"2024-09-10T07:48:53.082322+0000","last_clean":"2024-09-10T07:48:53.082322+0000","last_became_active":"2024-09-10T07:48:50.615852+0000","last_became_peered":"2024-09-10T07:48:50.615852+0000","last_unstale":"2024-09-10T07:48:53.082322+0000","last_undegraded":"2024-09-10T07:48:53.082322+0000","last_fullsized":"2024-09-10T07:48:53.082322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082846+0000","last_change":"2024-09-10T07:48:51.614922+0000","last_active":"2024-09-10T07:48:53.082846+0000","last_peered":"2024-09-10T07:48:53.082846+0000","last_clean":"2024-09-10T07:48:53.082846+0000","last_became_active":"2024-09-10T07:48:51.614654+0000","last_became_peered":"2024-09-10T07:48:51.614654+0000","last_unstale":"2024-09-10T07:48:53.082846+0000","last_undegraded":"2024-09-10T07:48:53.082846+0000","last_fullsized":"2024-09-10T07:48:53.082846+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:01:03.036248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083044+0000","last_change":"2024-09-10T07:48:50.620280+0000","last_active":"2024-09-10T07:48:53.083044+0000","last_peered":"2024-09-10T07:48:53.083044+0000","last_clean":"2024-09-10T07:48:53.083044+0000","last_became_active":"2024-09-10T07:48:50.619988+0000","last_became_peered":"2024-09-10T07:48:50.619988+0000","last_unstale":"2024-09-10T07:48:53.083044+0000","last_undegraded":"2024-09-10T07:48:53.083044+0000","last_fullsized":"2024-09-10T07:48:53.083044+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:25.494380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082751+0000","last_change":"2024-09-10T07:48:51.616228+0000","last_active":"2024-09-10T07:48:53.082751+0000","last_peered":"2024-09-10T07:48:53.082751+0000","last_clean":"2024-09-10T07:48:53.082751+0000","last_became_active":"2024-09-10T07:48:51.615946+0000","last_became_peered":"2024-09-10T07:48:51.615946+0000","last_unstale":"2024-09-10T07:48:53.082751+0000","last_undegraded":"2024-09-10T07:48:53.082751+0000","last_fullsized":"2024-09-10T07:48:53.082751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:12.993962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606882+0000","last_change":"2024-09-10T07:48:50.615137+0000","last_active":"2024-09-10T07:48:52.606882+0000","last_peered":"2024-09-10T07:48:52.606882+0000","last_clean":"2024-09-10T07:48:52.606882+0000","last_became_active":"2024-09-10T07:48:50.614618+0000","last_became_peered":"2024-09-10T07:48:50.614618+0000","last_unstale":"2024-09-10T07:48:52.606882+0000","last_undegraded":"2024-09-10T07:48:52.606882+0000","last_fullsized":"2024-09-10T07:48:52.606882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:16.791861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083398+0000","last_change":"2024-09-10T07:48:51.631532+0000","last_active":"2024-09-10T07:48:53.083398+0000","last_peered":"2024-09-10T07:48:53.083398+0000","last_clean":"2024-09-10T07:48:53.083398+0000","last_became_active":"2024-09-10T07:48:51.631204+0000","last_became_peered":"2024-09-10T07:48:51.631204+0000","last_unstale":"2024-09-10T07:48:53.083398+0000","last_undegraded":"2024-09-10T07:48:53.083398+0000","last_fullsized":"2024-09-10T07:48:53.083398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:57.578020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.630112+0000","last_change":"2024-09-10T07:48:50.622636+0000","last_active":"2024-09-10T07:48:52.630112+0000","last_peered":"2024-09-10T07:48:52.630112+0000","last_clean":"2024-09-10T07:48:52.630112+0000","last_became_active":"2024-09-10T07:48:50.621702+0000","last_became_peered":"2024-09-10T07:48:50.621702+0000","last_unstale":"2024-09-10T07:48:52.630112+0000","last_undegraded":"2024-09-10T07:48:52.630112+0000","last_fullsized":"2024-09-10T07:48:52.630112+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:38.491500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083378+0000","last_change":"2024-09-10T07:48:51.626683+0000","last_active":"2024-09-10T07:48:53.083378+0000","last_peered":"2024-09-10T07:48:53.083378+0000","last_clean":"2024-09-10T07:48:53.083378+0000","last_became_active":"2024-09-10T07:48:51.626399+0000","last_became_peered":"2024-09-10T07:48:51.626399+0000","last_unstale":"2024-09-10T07:48:53.083378+0000","last_undegraded":"2024-09-10T07:48:53.083378+0000","last_fullsized":"2024-09-10T07:48:53.083378+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:22.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606696+0000","last_change":"2024-09-10T07:48:50.621912+0000","last_active":"2024-09-10T07:48:52.606696+0000","last_peered":"2024-09-10T07:48:52.606696+0000","last_clean":"2024-09-10T07:48:52.606696+0000","last_became_active":"2024-09-10T07:48:50.620580+0000","last_became_peered":"2024-09-10T07:48:50.620580+0000","last_unstale":"2024-09-10T07:48:52.606696+0000","last_undegraded":"2024-09-10T07:48:52.606696+0000","last_fullsized":"2024-09-10T07:48:52.606696+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:40.772054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083385+0000","last_change":"2024-09-10T07:48:51.634091+0000","last_active":"2024-09-10T07:48:53.083385+0000","last_peered":"2024-09-10T07:48:53.083385+0000","last_clean":"2024-09-10T07:48:53.083385+0000","last_became_active":"2024-09-10T07:48:51.633805+0000","last_became_peered":"2024-09-10T07:48:51.633805+0000","last_unstale":"2024-09-10T07:48:53.083385+0000","last_undegraded":"2024-09-10T07:48:53.083385+0000","last_fullsized":"2024-09-10T07:48:53.083385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:56.573657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606615+0000","last_change":"2024-09-10T07:48:50.616978+0000","last_active":"2024-09-10T07:48:52.606615+0000","last_peered":"2024-09-10T07:48:52.606615+0000","last_clean":"2024-09-10T07:48:52.606615+0000","last_became_active":"2024-09-10T07:48:50.616723+0000","last_became_peered":"2024-09-10T07:48:50.616723+0000","last_unstale":"2024-09-10T07:48:52.606615+0000","last_undegraded":"2024-09-10T07:48:52.606615+0000","last_fullsized":"2024-09-10T07:48:52.606615+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:59.200441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606578+0000","last_change":"2024-09-10T07:48:51.612987+0000","last_active":"2024-09-10T07:48:52.606578+0000","last_peered":"2024-09-10T07:48:52.606578+0000","last_clean":"2024-09-10T07:48:52.606578+0000","last_became_active":"2024-09-10T07:48:51.611732+0000","last_became_peered":"2024-09-10T07:48:51.611732+0000","last_unstale":"2024-09-10T07:48:52.606578+0000","last_undegraded":"2024-09-10T07:48:52.606578+0000","last_fullsized":"2024-09-10T07:48:52.606578+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:22.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.084082+0000","last_change":"2024-09-10T07:48:50.625853+0000","last_active":"2024-09-10T07:48:53.084082+0000","last_peered":"2024-09-10T07:48:53.084082+0000","last_clean":"2024-09-10T07:48:53.084082+0000","last_became_active":"2024-09-10T07:48:50.625576+0000","last_became_peered":"2024-09-10T07:48:50.625576+0000","last_unstale":"2024-09-10T07:48:53.084082+0000","last_undegraded":"2024-09-10T07:48:53.084082+0000","last_fullsized":"2024-09-10T07:48:53.084082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:34.515493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606693+0000","last_change":"2024-09-10T07:48:51.622658+0000","last_active":"2024-09-10T07:48:52.606693+0000","last_peered":"2024-09-10T07:48:52.606693+0000","last_clean":"2024-09-10T07:48:52.606693+0000","last_became_active":"2024-09-10T07:48:51.622387+0000","last_became_peered":"2024-09-10T07:48:51.622387+0000","last_unstale":"2024-09-10T07:48:52.606693+0000","last_undegraded":"2024-09-10T07:48:52.606693+0000","last_fullsized":"2024-09-10T07:48:52.606693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:20.188379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607033+0000","last_change":"2024-09-10T07:48:50.617463+0000","last_active":"2024-09-10T07:48:52.607033+0000","last_peered":"2024-09-10T07:48:52.607033+0000","last_clean":"2024-09-10T07:48:52.607033+0000","last_became_active":"2024-09-10T07:48:50.617117+0000","last_became_peered":"2024-09-10T07:48:50.617117+0000","last_unstale":"2024-09-10T07:48:52.607033+0000","last_undegraded":"2024-09-10T07:48:52.607033+0000","last_fullsized":"2024-09-10T07:48:52.607033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:12.913436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083762+0000","last_change":"2024-09-10T07:48:51.635276+0000","last_active":"2024-09-10T07:48:53.083762+0000","last_peered":"2024-09-10T07:48:53.083762+0000","last_clean":"2024-09-10T07:48:53.083762+0000","last_became_active":"2024-09-10T07:48:51.634932+0000","last_became_peered":"2024-09-10T07:48:51.634932+0000","last_unstale":"2024-09-10T07:48:53.083762+0000","last_undegraded":"2024-09-10T07:48:53.083762+0000","last_fullsized":"2024-09-10T07:48:53.083762+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:03:05.804237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082435+0000","last_change":"2024-09-10T07:48:50.624873+0000","last_active":"2024-09-10T07:48:53.082435+0000","last_peered":"2024-09-10T07:48:53.082435+0000","last_clean":"2024-09-10T07:48:53.082435+0000","last_became_active":"2024-09-10T07:48:50.622063+0000","last_became_peered":"2024-09-10T07:48:50.622063+0000","last_unstale":"2024-09-10T07:48:53.082435+0000","last_undegraded":"2024-09-10T07:48:53.082435+0000","last_fullsized":"2024-09-10T07:48:53.082435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:49.987651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629844+0000","last_change":"2024-09-10T07:48:51.628756+0000","last_active":"2024-09-10T07:48:52.629844+0000","last_peered":"2024-09-10T07:48:52.629844+0000","last_clean":"2024-09-10T07:48:52.629844+0000","last_became_active":"2024-09-10T07:48:51.625501+0000","last_became_peered":"2024-09-10T07:48:51.625501+0000","last_unstale":"2024-09-10T07:48:52.629844+0000","last_undegraded":"2024-09-10T07:48:52.629844+0000","last_fullsized":"2024-09-10T07:48:52.629844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:23.545227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606490+0000","last_change":"2024-09-10T07:48:50.619456+0000","last_active":"2024-09-10T07:48:52.606490+0000","last_peered":"2024-09-10T07:48:52.606490+0000","last_clean":"2024-09-10T07:48:52.606490+0000","last_became_active":"2024-09-10T07:48:50.618866+0000","last_became_peered":"2024-09-10T07:48:50.618866+0000","last_unstale":"2024-09-10T07:48:52.606490+0000","last_undegraded":"2024-09-10T07:48:52.606490+0000","last_fullsized":"2024-09-10T07:48:52.606490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:58.806978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629721+0000","last_change":"2024-09-10T07:48:51.621971+0000","last_active":"2024-09-10T07:48:52.629721+0000","last_peered":"2024-09-10T07:48:52.629721+0000","last_clean":"2024-09-10T07:48:52.629721+0000","last_became_active":"2024-09-10T07:48:51.621729+0000","last_became_peered":"2024-09-10T07:48:51.621729+0000","last_unstale":"2024-09-10T07:48:52.629721+0000","last_undegraded":"2024-09-10T07:48:52.629721+0000","last_fullsized":"2024-09-10T07:48:52.629721+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:15.009924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606755+0000","last_change":"2024-09-10T07:48:50.617552+0000","last_active":"2024-09-10T07:48:52.606755+0000","last_peered":"2024-09-10T07:48:52.606755+0000","last_clean":"2024-09-10T07:48:52.606755+0000","last_became_active":"2024-09-10T07:48:50.617260+0000","last_became_peered":"2024-09-10T07:48:50.617260+0000","last_unstale":"2024-09-10T07:48:52.606755+0000","last_undegraded":"2024-09-10T07:48:52.606755+0000","last_fullsized":"2024-09-10T07:48:52.606755+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:03.137368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607841+0000","last_change":"2024-09-10T07:48:51.625611+0000","last_active":"2024-09-10T07:48:52.607841+0000","last_peered":"2024-09-10T07:48:52.607841+0000","last_clean":"2024-09-10T07:48:52.607841+0000","last_became_active":"2024-09-10T07:48:51.625352+0000","last_became_peered":"2024-09-10T07:48:51.625352+0000","last_unstale":"2024-09-10T07:48:52.607841+0000","last_undegraded":"2024-09-10T07:48:52.607841+0000","last_fullsized":"2024-09-10T07:48:52.607841+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:24.417387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.630257+0000","last_change":"2024-09-10T07:48:50.621422+0000","last_active":"2024-09-10T07:48:52.630257+0000","last_peered":"2024-09-10T07:48:52.630257+0000","last_clean":"2024-09-10T07:48:52.630257+0000","last_became_active":"2024-09-10T07:48:50.621142+0000","last_became_peered":"2024-09-10T07:48:50.621142+0000","last_unstale":"2024-09-10T07:48:52.630257+0000","last_undegraded":"2024-09-10T07:48:52.630257+0000","last_fullsized":"2024-09-10T07:48:52.630257+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:09.377321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606946+0000","last_change":"2024-09-10T07:48:51.632002+0000","last_active":"2024-09-10T07:48:52.606946+0000","last_peered":"2024-09-10T07:48:52.606946+0000","last_clean":"2024-09-10T07:48:52.606946+0000","last_became_active":"2024-09-10T07:48:51.629180+0000","last_became_peered":"2024-09-10T07:48:51.629180+0000","last_unstale":"2024-09-10T07:48:52.606946+0000","last_undegraded":"2024-09-10T07:48:52.606946+0000","last_fullsized":"2024-09-10T07:48:52.606946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082824+0000","last_change":"2024-09-10T07:48:50.622779+0000","last_active":"2024-09-10T07:48:53.082824+0000","last_peered":"2024-09-10T07:48:53.082824+0000","last_clean":"2024-09-10T07:48:53.082824+0000","last_became_active":"2024-09-10T07:48:50.622556+0000","last_became_peered":"2024-09-10T07:48:50.622556+0000","last_unstale":"2024-09-10T07:48:53.082824+0000","last_undegraded":"2024-09-10T07:48:53.082824+0000","last_fullsized":"2024-09-10T07:48:53.082824+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:11.544783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607674+0000","last_change":"2024-09-10T07:48:51.628390+0000","last_active":"2024-09-10T07:48:52.607674+0000","last_peered":"2024-09-10T07:48:52.607674+0000","last_clean":"2024-09-10T07:48:52.607674+0000","last_became_active":"2024-09-10T07:48:51.628095+0000","last_became_peered":"2024-09-10T07:48:51.628095+0000","last_unstale":"2024-09-10T07:48:52.607674+0000","last_undegraded":"2024-09-10T07:48:52.607674+0000","last_fullsized":"2024-09-10T07:48:52.607674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083283+0000","last_change":"2024-09-10T07:48:50.619655+0000","last_active":"2024-09-10T07:48:53.083283+0000","last_peered":"2024-09-10T07:48:53.083283+0000","last_clean":"2024-09-10T07:48:53.083283+0000","last_became_active":"2024-09-10T07:48:50.619360+0000","last_became_peered":"2024-09-10T07:48:50.619360+0000","last_unstale":"2024-09-10T07:48:53.083283+0000","last_undegraded":"2024-09-10T07:48:53.083283+0000","last_fullsized":"2024-09-10T07:48:53.083283+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.560270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606864+0000","last_change":"2024-09-10T07:48:51.623695+0000","last_active":"2024-09-10T07:48:52.606864+0000","last_peered":"2024-09-10T07:48:52.606864+0000","last_clean":"2024-09-10T07:48:52.606864+0000","last_became_active":"2024-09-10T07:48:51.623049+0000","last_became_peered":"2024-09-10T07:48:51.623049+0000","last_unstale":"2024-09-10T07:48:52.606864+0000","last_undegraded":"2024-09-10T07:48:52.606864+0000","last_fullsized":"2024-09-10T07:48:52.606864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:42.211425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082885+0000","last_change":"2024-09-10T07:48:50.619068+0000","last_active":"2024-09-10T07:48:53.082885+0000","last_peered":"2024-09-10T07:48:53.082885+0000","last_clean":"2024-09-10T07:48:53.082885+0000","last_became_active":"2024-09-10T07:48:50.618824+0000","last_became_peered":"2024-09-10T07:48:50.618824+0000","last_unstale":"2024-09-10T07:48:53.082885+0000","last_undegraded":"2024-09-10T07:48:53.082885+0000","last_fullsized":"2024-09-10T07:48:53.082885+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:24.707876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.629772+0000","last_change":"2024-09-10T07:48:51.634038+0000","last_active":"2024-09-10T07:48:52.629772+0000","last_peered":"2024-09-10T07:48:52.629772+0000","last_clean":"2024-09-10T07:48:52.629772+0000","last_became_active":"2024-09-10T07:48:51.633783+0000","last_became_peered":"2024-09-10T07:48:51.633783+0000","last_unstale":"2024-09-10T07:48:52.629772+0000","last_undegraded":"2024-09-10T07:48:52.629772+0000","last_fullsized":"2024-09-10T07:48:52.629772+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:53.547800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607255+0000","last_change":"2024-09-10T07:48:50.614453+0000","last_active":"2024-09-10T07:48:52.607255+0000","last_peered":"2024-09-10T07:48:52.607255+0000","last_clean":"2024-09-10T07:48:52.607255+0000","last_became_active":"2024-09-10T07:48:50.614159+0000","last_became_peered":"2024-09-10T07:48:50.614159+0000","last_unstale":"2024-09-10T07:48:52.607255+0000","last_undegraded":"2024-09-10T07:48:52.607255+0000","last_fullsized":"2024-09-10T07:48:52.607255+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:54.792276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606183+0000","last_change":"2024-09-10T07:48:51.628493+0000","last_active":"2024-09-10T07:48:52.606183+0000","last_peered":"2024-09-10T07:48:52.606183+0000","last_clean":"2024-09-10T07:48:52.606183+0000","last_became_active":"2024-09-10T07:48:51.628229+0000","last_became_peered":"2024-09-10T07:48:51.628229+0000","last_unstale":"2024-09-10T07:48:52.606183+0000","last_undegraded":"2024-09-10T07:48:52.606183+0000","last_fullsized":"2024-09-10T07:48:52.606183+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:58.032846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083110+0000","last_change":"2024-09-10T07:48:50.620900+0000","last_active":"2024-09-10T07:48:53.083110+0000","last_peered":"2024-09-10T07:48:53.083110+0000","last_clean":"2024-09-10T07:48:53.083110+0000","last_became_active":"2024-09-10T07:48:50.620412+0000","last_became_peered":"2024-09-10T07:48:50.620412+0000","last_unstale":"2024-09-10T07:48:53.083110+0000","last_undegraded":"2024-09-10T07:48:53.083110+0000","last_fullsized":"2024-09-10T07:48:53.083110+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:28:21.875313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.083078+0000","last_change":"2024-09-10T07:48:51.632214+0000","last_active":"2024-09-10T07:48:53.083078+0000","last_peered":"2024-09-10T07:48:53.083078+0000","last_clean":"2024-09-10T07:48:53.083078+0000","last_became_active":"2024-09-10T07:48:51.631932+0000","last_became_peered":"2024-09-10T07:48:51.631932+0000","last_unstale":"2024-09-10T07:48:53.083078+0000","last_undegraded":"2024-09-10T07:48:53.083078+0000","last_fullsized":"2024-09-10T07:48:53.083078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:39.923143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:53.082932+0000","last_change":"2024-09-10T07:48:50.615382+0000","last_active":"2024-09-10T07:48:53.082932+0000","last_peered":"2024-09-10T07:48:53.082932+0000","last_clean":"2024-09-10T07:48:53.082932+0000","last_became_active":"2024-09-10T07:48:50.614269+0000","last_became_peered":"2024-09-10T07:48:50.614269+0000","last_unstale":"2024-09-10T07:48:53.082932+0000","last_undegraded":"2024-09-10T07:48:53.082932+0000","last_fullsized":"2024-09-10T07:48:53.082932+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:37.526628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.630160+0000","last_change":"2024-09-10T07:48:51.630882+0000","last_active":"2024-09-10T07:48:52.630160+0000","last_peered":"2024-09-10T07:48:52.630160+0000","last_clean":"2024-09-10T07:48:52.630160+0000","last_became_active":"2024-09-10T07:48:51.630606+0000","last_became_peered":"2024-09-10T07:48:51.630606+0000","last_unstale":"2024-09-10T07:48:52.630160+0000","last_undegraded":"2024-09-10T07:48:52.630160+0000","last_fullsized":"2024-09-10T07:48:52.630160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:50.592745+0000","last_clean_scrub_stamp":"2024-09-10T07:48:50.592745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:06.893540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.607840+0000","last_change":"2024-09-10T07:48:50.624209+0000","last_active":"2024-09-10T07:48:52.607840+0000","last_peered":"2024-09-10T07:48:52.607840+0000","last_clean":"2024-09-10T07:48:52.607840+0000","last_became_active":"2024-09-10T07:48:50.623406+0000","last_became_peered":"2024-09-10T07:48:50.623406+0000","last_unstale":"2024-09-10T07:48:52.607840+0000","last_undegraded":"2024-09-10T07:48:52.607840+0000","last_fullsized":"2024-09-10T07:48:52.607840+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:48:49.586147+0000","last_clean_scrub_stamp":"2024-09-10T07:48:49.586147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:29.913985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T07:48:52.606204+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:48:52.606204+0000","last_peered":"2024-09-10T07:48:52.606204+0000","last_clean":"2024-09-10T07:48:52.606204+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:48:52.606204+0000","last_undegraded":"2024-09-10T07:48:52.606204+0000","last_fullsized":"2024-09-10T07:48:52.606204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476771,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":238878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":237356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":827704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476770,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":237938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":581632,"data_stored":237300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":585728,"data_stored":237882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1175552,"data_stored":827656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":598016,"data_stored":238970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:49:00.965 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:49:00.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:49:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.165+0000 7fbcf9800640 1 Processor -- start 2024-09-10T07:49:01.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.165+0000 7fbcf9800640 1 -- start start 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf9800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcf41451d0 0x7fbcf406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf40cf270 0x7fbcf40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf9800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf40cd900 con 0x7fbcf4145730 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf9800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf406d7c0 con 0x7fbcf41451d0 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf9800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf40cdbe0 con 0x7fbcf40cf270 2024-09-10T07:49:01.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcf41451d0 0x7fbcf406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbcf41451d0 0x7fbcf406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54688/0 (socket says 172.21.15.3:54688) 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 -- 172.21.15.3:0/2714754097 learned_addr learned my addr 172.21.15.3:0/2714754097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf3400640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf40cf270 0x7fbcf40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 -- 172.21.15.3:0/2714754097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf40cf270 msgr2=0x7fbcf40cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf40cf270 0x7fbcf40cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 -- 172.21.15.3:0/2714754097 >> v1:172.21.15.70:6789/0 conn(0x7fbcf4145730 legacy=0x7fbcf40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.167+0000 7fbcf2a00640 1 -- 172.21.15.3:0/2714754097 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf406da10 con 0x7fbcf41451d0 2024-09-10T07:49:01.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.168+0000 7fbcf2a00640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf406cfc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009870 tx=0x7fbce402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaf6413a0ec498e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf2000640 1 -- 172.21.15.3:0/2714754097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4037070 con 0x7fbcf41451d0 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf2000640 1 -- 172.21.15.3:0/2714754097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce4032440 con 0x7fbcf41451d0 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf2000640 1 -- 172.21.15.3:0/2714754097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4032810 con 0x7fbcf41451d0 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 -- 172.21.15.3:0/2714754097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 msgr2=0x7fbcf406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf406cfc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009870 tx=0x7fbce402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 -- 172.21.15.3:0/2714754097 shutdown_connections 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf40cf270 0x7fbcf40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 --2- 172.21.15.3:0/2714754097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf406cfc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 -- 172.21.15.3:0/2714754097 >> 172.21.15.3:0/2714754097 conn(0x7fbcf40b5cb0 msgr2=0x7fbcf406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 -- 172.21.15.3:0/2714754097 shutdown_connections 2024-09-10T07:49:01.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.169+0000 7fbcf9800640 1 -- 172.21.15.3:0/2714754097 wait complete. 2024-09-10T07:49:01.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.171+0000 7fbcf9800640 1 Processor -- start 2024-09-10T07:49:01.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.171+0000 7fbcf9800640 1 -- start start 2024-09-10T07:49:01.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf40cf270 0x7fbcf41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf4145730 0x7fbcf4072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf406df30 con 0x7fbcf41451d0 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf40cfc10 con 0x7fbcf40cf270 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcf9800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf406e0b0 con 0x7fbcf4145730 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.172+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54700/0 (socket says 172.21.15.3:54700) 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcf2a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf40cf270 0x7fbcf41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 -- 172.21.15.3:0/114496841 learned_addr learned my addr 172.21.15.3:0/114496841 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcf3400640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf4145730 0x7fbcf4072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 -- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf4145730 msgr2=0x7fbcf4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf4145730 0x7fbcf4072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 -- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf40cf270 msgr2=0x7fbcf41406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf40cf270 0x7fbcf41406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 -- 172.21.15.3:0/114496841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf40715e0 con 0x7fbcf41451d0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbcebe00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c340 tx=0x7fbcdc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.173+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc017070 con 0x7fbcf41451d0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.174+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc0052d0 con 0x7fbcf41451d0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.174+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003850 con 0x7fbcf41451d0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.174+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf40718c0 con 0x7fbcf41451d0 2024-09-10T07:49:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.174+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf4071e80 con 0x7fbcf41451d0 2024-09-10T07:49:01.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.175+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf40cf6d0 con 0x7fbcf41451d0 2024-09-10T07:49:01.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.175+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcdc01a070 con 0x7fbcf41451d0 2024-09-10T07:49:01.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.176+0000 7fbceaa00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 0x7fbcd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.176+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fbcdc08a1b0 con 0x7fbcf41451d0 2024-09-10T07:49:01.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.177+0000 7fbcf2a00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 0x7fbcd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.177+0000 7fbcf2a00640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 0x7fbcd006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbce4034000 tx=0x7fbce402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.181+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcdc058470 con 0x7fbcf41451d0 2024-09-10T07:49:01.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.434+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbcf406f0f0 con 0x7fbcf41451d0 2024-09-10T07:49:01.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.437+0000 7fbceaa00640 1 -- 172.21.15.3:0/114496841 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7fbcdc05c770 con 0x7fbcf41451d0 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:49:01.437 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 msgr2=0x7fbcd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 0x7fbcd006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbce4034000 tx=0x7fbce402e000 comp rx=0 tx=0).stop 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 msgr2=0x7fbcf4144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c340 tx=0x7fbcdc00c800 comp rx=0 tx=0).stop 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 shutdown_connections 2024-09-10T07:49:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbcd006be30 0x7fbcd006e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbcf4145730 0x7fbcf4072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcf41451d0 0x7fbcf4144430 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 --2- 172.21.15.3:0/114496841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf40cf270 0x7fbcf41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 >> 172.21.15.3:0/114496841 conn(0x7fbcf40b5cb0 msgr2=0x7fbcf4074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 shutdown_connections 2024-09-10T07:49:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.440+0000 7fbcf9800640 1 -- 172.21.15.3:0/114496841 wait complete. 2024-09-10T07:49:01.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:01.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.613+0000 7fd519000640 1 Processor -- start 2024-09-10T07:49:01.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.614+0000 7fd519000640 1 -- start start 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd519000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd5140ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd519000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5140ccd80 con 0x7fd51415b110 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd519000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5140cd060 con 0x7fd51415b4f0 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd519000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5140b44f0 con 0x7fd5140d08f0 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd511e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd51415b110 0x7fd5140b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37102/0 (socket says 172.21.15.3:37102) 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd511e00640 1 -- 172.21.15.3:0/3284270890 learned_addr learned my addr 172.21.15.3:0/3284270890 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.615+0000 7fd512800640 1 --2- 172.21.15.3:0/3284270890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd5140ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373056579 0 0) 0x7fd5140ccd80 con 0x7fd51415b110 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4f8005730 con 0x7fd51415b110 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313257104 0 0) 0x7fd5140cd060 con 0x7fd51415b4f0 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5140ccd80 con 0x7fd51415b4f0 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3593445436 0 0) 0x7fd4f8005730 con 0x7fd51415b110 2024-09-10T07:49:01.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd512800640 1 -- 172.21.15.3:0/3284270890 >> v1:172.21.15.70:6789/0 conn(0x7fd51415b4f0 legacy=0x7fd5140d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd512800640 1 -- 172.21.15.3:0/3284270890 >> v1:172.21.15.3:6789/0 conn(0x7fd51415b110 legacy=0x7fd5140b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.616+0000 7fd512800640 1 -- 172.21.15.3:0/3284270890 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5140cd340 con 0x7fd5140d08f0 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.617+0000 7fd512800640 1 --2- 172.21.15.3:0/3284270890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd5140ceb50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009dc0 tx=0x7fd50c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6c512deeda2fe40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.617+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504001670 con 0x7fd5140d08f0 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.617+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50c007540 con 0x7fd5140d08f0 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.617+0000 7fd511400640 1 -- 172.21.15.3:0/3284270890 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c007910 con 0x7fd5140d08f0 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 -- 172.21.15.3:0/3284270890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 msgr2=0x7fd5140ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 --2- 172.21.15.3:0/3284270890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd5140ceb50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009dc0 tx=0x7fd50c02a660 comp rx=0 tx=0).stop 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 -- 172.21.15.3:0/3284270890 shutdown_connections 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 --2- 172.21.15.3:0/3284270890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd5140ceb50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 -- 172.21.15.3:0/3284270890 >> 172.21.15.3:0/3284270890 conn(0x7fd51406d000 msgr2=0x7fd51406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 -- 172.21.15.3:0/3284270890 shutdown_connections 2024-09-10T07:49:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.618+0000 7fd519000640 1 -- 172.21.15.3:0/3284270890 wait complete. 2024-09-10T07:49:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.620+0000 7fd519000640 1 Processor -- start 2024-09-10T07:49:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.620+0000 7fd519000640 1 -- start start 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.620+0000 7fd519000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd51415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd519000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd519000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51415b4f0 0x7fd51415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd519000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd514073e90 con 0x7fd51415b110 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd519000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd514073d10 con 0x7fd51415b4f0 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd519000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd514074010 con 0x7fd5140d08f0 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54708/0 (socket says 172.21.15.3:54708) 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 -- 172.21.15.3:0/2033569666 learned_addr learned my addr 172.21.15.3:0/2033569666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd512800640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51415b4f0 0x7fd51415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd511e00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd51415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 -- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 msgr2=0x7fd51415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd51415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 -- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51415b4f0 msgr2=0x7fd51415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51415b4f0 0x7fd51415d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.621+0000 7fd50be00640 1 -- 172.21.15.3:0/2033569666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51415cb00 con 0x7fd51415b110 2024-09-10T07:49:01.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd50be00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c430 tx=0x7fd4fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc017070 con 0x7fd51415b110 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4fc0052f0 con 0x7fd51415b110 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc003830 con 0x7fd51415b110 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51415de60 con 0x7fd51415b110 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.622+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51415bdc0 con 0x7fd51415b110 2024-09-10T07:49:01.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.623+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51415b8d0 con 0x7fd51415b110 2024-09-10T07:49:01.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.624+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4fc01a070 con 0x7fd51415b110 2024-09-10T07:49:01.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.625+0000 7fd50a000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 0x7fd4f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:01.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.625+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fd4fc08a1a0 con 0x7fd51415b110 2024-09-10T07:49:01.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.625+0000 7fd511e00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 0x7fd4f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:01.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.626+0000 7fd511e00640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 0x7fd4f006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd504001eb0 tx=0x7fd5040016d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:01.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.630+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4fc0584f0 con 0x7fd51415b110 2024-09-10T07:49:01.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.851+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd51415fa20 con 0x7fd51415b110 2024-09-10T07:49:01.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.852+0000 7fd50a000640 1 -- 172.21.15.3:0/2033569666 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5211 (secure 0 0 0) 0x7fd4fc05c7f0 con 0x7fd51415b110 2024-09-10T07:49:01.852 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:01.852 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":34,"btime":"2024-09-10T07:48:54:622365+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:48:54.051392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-10T07:49:01.852 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 34 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 msgr2=0x7fd4f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 0x7fd4f006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd504001eb0 tx=0x7fd5040016d0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 msgr2=0x7fd51414ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c430 tx=0x7fd4fc00c8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 shutdown_connections 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd4f006be30 0x7fd4f006e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd51415b4f0 0x7fd51415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd51415b110 0x7fd51414ace0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 --2- 172.21.15.3:0/2033569666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5140d08f0 0x7fd51415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 >> 172.21.15.3:0/2033569666 conn(0x7fd51406d000 msgr2=0x7fd5140b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:01.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 shutdown_connections 2024-09-10T07:49:01.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:01.854+0000 7fd519000640 1 -- 172.21.15.3:0/2033569666 wait complete. 2024-09-10T07:49:02.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:48:51.031977+0000', 'modified': '2024-09-10T07:48:54.051392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5433': {'gid': 5433, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6836/29592196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 29592196}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 29592196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5433, 'qdb_cluster': [5433]} 2024-09-10T07:49:02.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:49:02.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:49:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.433+0000 7f11a8600640 1 Processor -- start 2024-09-10T07:49:02.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.433+0000 7f11a8600640 1 -- start start 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a8600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 0x7f11a00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a8600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a00ccd80 con 0x7f11a015b110 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a8600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a00cd060 con 0x7f11a0073d10 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a8600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11a00b44f0 con 0x7f11a015b4f0 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a5c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f11a015b110 0x7f11a00b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37116/0 (socket says 172.21.15.3:37116) 2024-09-10T07:49:02.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.434+0000 7f11a5c00640 1 -- 172.21.15.3:0/3099611611 learned_addr learned my addr 172.21.15.3:0/3099611611 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f11a5200640 1 --2- 172.21.15.3:0/3099611611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 0x7f11a00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3485940686 0 0) 0x7f11a00ccd80 con 0x7f11a015b110 2024-09-10T07:49:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1188005730 con 0x7f11a015b110 2024-09-10T07:49:02.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 15114286 0 0) 0x7f1188005730 con 0x7f11a015b110 2024-09-10T07:49:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f11a5200640 1 -- 172.21.15.3:0/3099611611 >> v1:172.21.15.70:6789/0 conn(0x7f11a0073d10 legacy=0x7f11a00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f11a5200640 1 -- 172.21.15.3:0/3099611611 >> v1:172.21.15.3:6789/0 conn(0x7f11a015b110 legacy=0x7f11a00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.435+0000 7f11a5200640 1 -- 172.21.15.3:0/3099611611 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a00cd340 con 0x7f11a015b4f0 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.436+0000 7f11a5200640 1 --2- 172.21.15.3:0/3099611611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 0x7f11a00d01e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f11a00b4670 tx=0x7f118c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed549251e1560748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.436+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c037070 con 0x7f11a015b4f0 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.436+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c02ac10 con 0x7f11a015b4f0 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.436+0000 7f119fe00640 1 -- 172.21.15.3:0/3099611611 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194001670 con 0x7f11a015b4f0 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 -- 172.21.15.3:0/3099611611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 msgr2=0x7f11a00d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 --2- 172.21.15.3:0/3099611611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 0x7f11a00d01e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f11a00b4670 tx=0x7f118c02a180 comp rx=0 tx=0).stop 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 -- 172.21.15.3:0/3099611611 shutdown_connections 2024-09-10T07:49:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 --2- 172.21.15.3:0/3099611611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b4f0 0x7f11a00d01e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 -- 172.21.15.3:0/3099611611 >> 172.21.15.3:0/3099611611 conn(0x7f11a006d000 msgr2=0x7f11a006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 -- 172.21.15.3:0/3099611611 shutdown_connections 2024-09-10T07:49:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.437+0000 7f11a8600640 1 -- 172.21.15.3:0/3099611611 wait complete. 2024-09-10T07:49:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.439+0000 7f11a8600640 1 Processor -- start 2024-09-10T07:49:02.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.439+0000 7f11a8600640 1 -- start start 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b110 0x7f11a014bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11a015b4f0 0x7f11a00aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00d08a0 con 0x7f11a0073d10 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00d0720 con 0x7f11a015b4f0 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a8600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11a00d0a20 con 0x7f11a015b110 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54724/0 (socket says 172.21.15.3:54724) 2024-09-10T07:49:02.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a5c00640 1 -- 172.21.15.3:0/1449639865 learned_addr learned my addr 172.21.15.3:0/1449639865 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a5200640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b110 0x7f11a014bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.440+0000 7f11a6600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11a015b4f0 0x7f11a00aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 -- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b110 msgr2=0x7f11a014bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b110 0x7f11a014bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 -- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11a015b4f0 msgr2=0x7f11a00aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11a015b4f0 0x7f11a00aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 -- 172.21.15.3:0/1449639865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a0157890 con 0x7f11a0073d10 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a5c00640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1194001eb0 tx=0x7f119400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f119d600640 1 -- 172.21.15.3:0/1449639865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194017070 con 0x7f11a0073d10 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f119d600640 1 -- 172.21.15.3:0/1449639865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1194005ad0 con 0x7f11a0073d10 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f119d600640 1 -- 172.21.15.3:0/1449639865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194007070 con 0x7f11a0073d10 2024-09-10T07:49:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a01598f0 con 0x7f11a0073d10 2024-09-10T07:49:02.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.441+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a00af440 con 0x7f11a0073d10 2024-09-10T07:49:02.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.443+0000 7f119d600640 1 -- 172.21.15.3:0/1449639865 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f119401a070 con 0x7f11a0073d10 2024-09-10T07:49:02.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.444+0000 7f119d600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 0x7f118006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.444+0000 7f119d600640 1 -- 172.21.15.3:0/1449639865 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f119408a330 con 0x7f11a0073d10 2024-09-10T07:49:02.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.445+0000 7f11a5200640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 0x7f118006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.445+0000 7f11ab67e740 1 Processor -- start 2024-09-10T07:49:02.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.445+0000 7f11a5200640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 0x7f118006e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f118c034000 tx=0x7f118c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.445+0000 7f11ab67e740 1 -- start start 2024-09-10T07:49:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e21490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014dd6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562014d3a3b0 con 0x562014dd94b0 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562014d3f410 con 0x562014ddbcf0 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562014ce7620 con 0x562014ddd480 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.446+0000 7f11a5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54732/0 (socket says 172.21.15.3:54732) 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 -- 172.21.15.3:0/766029333 learned_addr learned my addr 172.21.15.3:0/766029333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5c00640 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e21490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a6600640 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014dd6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 -- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 msgr2=0x562014dd6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014dd6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 -- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 msgr2=0x562014e21490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e21490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 -- 172.21.15.3:0/766029333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562014a522d0 con 0x562014dd94b0 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.447+0000 7f11a5200640 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f118c02a770 tx=0x7f118c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a49e067c7e73a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f1192000640 1 -- 172.21.15.3:0/766029333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c04e070 con 0x562014dd94b0 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f1192000640 1 -- 172.21.15.3:0/766029333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c032d10 con 0x562014dd94b0 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f1192000640 1 -- 172.21.15.3:0/766029333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c037040 con 0x562014dd94b0 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 -- 172.21.15.3:0/766029333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 msgr2=0x562014ddc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f118c02a770 tx=0x7f118c02c000 comp rx=0 tx=0).stop 2024-09-10T07:49:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 -- 172.21.15.3:0/766029333 shutdown_connections 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014dd6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dd94b0 0x562014ddc0d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 --2- 172.21.15.3:0/766029333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e21490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 -- 172.21.15.3:0/766029333 >> 172.21.15.3:0/766029333 conn(0x562014dd8740 msgr2=0x562014e20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 -- 172.21.15.3:0/766029333 shutdown_connections 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.448+0000 7f11ab67e740 1 -- 172.21.15.3:0/766029333 wait complete. 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.449+0000 7f11ab67e740 1 -- start start 2024-09-10T07:49:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.449+0000 7f11ab67e740 1 Processor -- start 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014dfbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014dda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562014cfa780 con 0x562014ddd480 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562014d3b3d0 con 0x562014ddbcf0 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562014dd7be0 con 0x562014dfc9b0 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.450+0000 7f11ab67e740 10 client.0 _ll_register_callbacks cb 0x562014d385d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54734/0 (socket says 172.21.15.3:54734) 2024-09-10T07:49:02.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 -- 172.21.15.3:0/2612856405 learned_addr learned my addr 172.21.15.3:0/2612856405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a6600640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014dda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014dfbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 -- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 msgr2=0x562014dda3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014dda3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 -- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 msgr2=0x562014dfbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014dfbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 -- 172.21.15.3:0/2612856405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562014b1c6b0 con 0x562014ddd480 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f11a5200640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f118c009560 tx=0x7f118c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757a18166fa03ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.451+0000 7f1186a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c037050 con 0x562014ddd480 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118c032d10 con 0x562014ddd480 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f118c03eab0 con 0x562014ddd480 2024-09-10T07:49:02.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f118c066050 con 0x562014ddd480 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 10 client.5592 fetch_fsmap learned FSMap version 34 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x562014dd7310 con 0x562014ddd480 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c032d10 con 0x562014ddd480 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f118c069020 con 0x562014ddd480 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 10 client.5592 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 10 client.5592 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 20 client.5592 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e05cf0 0x562014e060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 4 client.5592 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.452+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562014d4cb40 con 0x562014e05cf0 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.453+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e05cf0 0x562014e060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.453+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e05cf0 0x562014e060d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5711395afb8e733 server_cookie=8c84499acfe9f37c in_seq=0 out_seq=0 2024-09-10T07:49:02.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.453+0000 7f1186a00640 10 client.5592 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f1186a00640 1 -- 172.21.15.3:0/2612856405 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562014d4cb40 con 0x562014e05cf0 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f1186a00640 10 client.5592 handle_command_reply: tid=0 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 client.5592 shutdown 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 msgr2=0x562014dfa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f118c009560 tx=0x7f118c004730 comp rx=0 tx=0).stop 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 shutdown_connections 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e05cf0 0x562014e060d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014dda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddd480 0x562014dfa730 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2612856405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014dfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f1186a00640 0 client.5592 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 >> 172.21.15.3:0/2612856405 conn(0x562014dd94b0 msgr2=0x562014e1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 shutdown_connections 2024-09-10T07:49:02.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.454+0000 7f11ab67e740 1 -- 172.21.15.3:0/2612856405 wait complete. 2024-09-10T07:49:02.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.455+0000 7f11ab67e740 20 client.5592 trim_cache size 0 max 16384 2024-09-10T07:49:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.473+0000 7f11ab67e740 1 Processor -- start 2024-09-10T07:49:02.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.473+0000 7f11ab67e740 1 -- start start 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddd480 0x562014e00360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014e45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00aed60 con 0x562014ddbcf0 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00ae0c0 con 0x562014ddd480 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f11a00ae710 con 0x562014dfc9b0 2024-09-10T07:49:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54744/0 (socket says 172.21.15.3:54744) 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.474+0000 7f11a5c00640 1 -- 172.21.15.3:0/511467432 learned_addr learned my addr 172.21.15.3:0/511467432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5200640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddd480 0x562014e00360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a6600640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014e45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 -- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 msgr2=0x562014e45080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014e45080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 -- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddd480 msgr2=0x562014e00360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddd480 0x562014e00360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 -- 172.21.15.3:0/511467432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562014e06270 con 0x562014ddbcf0 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a6600640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014e45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f11a5c00640 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f11940585c0 tx=0x7f1194014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b6e28d52f3996 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f1191600640 1 -- 172.21.15.3:0/511467432 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119408a0b0 con 0x562014ddbcf0 2024-09-10T07:49:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f1191600640 1 -- 172.21.15.3:0/511467432 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1194070070 con 0x562014ddbcf0 2024-09-10T07:49:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.475+0000 7f1191600640 1 -- 172.21.15.3:0/511467432 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194054cf0 con 0x562014ddbcf0 2024-09-10T07:49:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 -- 172.21.15.3:0/511467432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 msgr2=0x562014e048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f11940585c0 tx=0x7f1194014040 comp rx=0 tx=0).stop 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 -- 172.21.15.3:0/511467432 shutdown_connections 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014dfc9b0 0x562014e45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddd480 0x562014e00360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 --2- 172.21.15.3:0/511467432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014ddbcf0 0x562014e048e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 -- 172.21.15.3:0/511467432 >> 172.21.15.3:0/511467432 conn(0x562014dd94b0 msgr2=0x562014e30170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 -- 172.21.15.3:0/511467432 shutdown_connections 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.476+0000 7f11ab67e740 1 -- 172.21.15.3:0/511467432 wait complete. 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.477+0000 7f11ab67e740 1 -- start start 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.477+0000 7f11ab67e740 1 Processor -- start 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014e9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562014e067e0 con 0x562014dfc9b0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562014dfac70 con 0x562014ddbcf0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562014dd5e70 con 0x562014ddd480 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11ab67e740 10 client.0 _ll_register_callbacks cb 0x7f118c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11a6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11a6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54760/0 (socket says 172.21.15.3:54760) 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11a6600640 1 -- 172.21.15.3:0/2663243258 learned_addr learned my addr 172.21.15.3:0/2663243258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.478+0000 7f11a5200640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014e9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 -- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 msgr2=0x562014e9f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014e9f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 -- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 msgr2=0x562014e01b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e01b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 -- 172.21.15.3:0/2663243258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562014cff550 con 0x562014dfc9b0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11a6600640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1198005700 tx=0x7f1198001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c2021af357d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f1186a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198017700 con 0x562014dfc9b0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f11980178a0 con 0x562014dfc9b0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194070070 con 0x562014dfc9b0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1198027050 con 0x562014dfc9b0 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11ab67e740 10 client.5598 fetch_fsmap learned FSMap version 34 2024-09-10T07:49:02.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.479+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x562014dd6580 con 0x562014dfc9b0 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1194070070 con 0x562014dfc9b0 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f119802a020 con 0x562014dfc9b0 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 10 client.5598 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 10 client.5598 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 20 client.5598 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e1f070 0x562014e1ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 4 client.5598 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "status"}) -- 0x562014d46e90 con 0x562014e1f070 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e1f070 0x562014e1ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f11a5c00640 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e1f070 0x562014e1ff50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d4d38725c589b7 server_cookie=38133f720f0869c5 in_seq=0 out_seq=0 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.480+0000 7f1186a00640 10 client.5598 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f1186a00640 1 -- 172.21.15.3:0/2663243258 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x562014d46e90 con 0x562014e1f070 2024-09-10T07:49:02.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f1186a00640 10 client.5598 handle_command_reply: tid=0 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 client.5598 shutdown 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 msgr2=0x562014dfadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1198005700 tx=0x7f1198001cf0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 shutdown_connections 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x562014e1f070 0x562014e1ff50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562014dfc9b0 0x562014dfadf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562014ddd480 0x562014e9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 --2- 172.21.15.3:0/2663243258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562014ddbcf0 0x562014e01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f1186a00640 0 client.5598 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 >> 172.21.15.3:0/2663243258 conn(0x562014dd94b0 msgr2=0x562014e30420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 shutdown_connections 2024-09-10T07:49:02.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 1 -- 172.21.15.3:0/2663243258 wait complete. 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.481+0000 7f11ab67e740 20 client.5598 trim_cache size 0 max 16384 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5433, 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:49:02.487 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.868499278, 2024-09-10T07:49:02.488 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 34, 2024-09-10T07:49:02.488 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 50, 2024-09-10T07:49:02.488 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 50, 2024-09-10T07:49:02.488 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 22.226995446 2024-09-10T07:49:02.488 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 msgr2=0x7f118006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 0x7f118006e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f118c034000 tx=0x7f118c02e000 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 msgr2=0x7f11a01593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1194001eb0 tx=0x7f119400cbe0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 shutdown_connections 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f118006be30 0x7f118006e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11a015b4f0 0x7f11a00aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f11a015b110 0x7f11a014bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 --2- 172.21.15.3:0/1449639865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a0073d10 0x7f11a01593b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 >> 172.21.15.3:0/1449639865 conn(0x7f11a006d000 msgr2=0x7f11a00b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 shutdown_connections 2024-09-10T07:49:02.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.482+0000 7f11a8600640 1 -- 172.21.15.3:0/1449639865 wait complete. 2024-09-10T07:49:02.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:49:02.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.654+0000 7fd3cd400640 1 Processor -- start 2024-09-10T07:49:02.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.654+0000 7fd3cd400640 1 -- start start 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3c815a1e0 0x7fd3c806f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c806faf0 0x7fd3c8072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3c8072990 0x7fd3c8062630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c815cfa0 con 0x7fd3c815a1e0 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c815d280 con 0x7fd3c8072990 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3cd400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c815d560 con 0x7fd3c806faf0 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3c6800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3c815a1e0 0x7fd3c806f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.655+0000 7fd3c6800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3c815a1e0 0x7fd3c806f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54774/0 (socket says 172.21.15.3:54774) 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 -- 172.21.15.3:0/1255698330 learned_addr learned my addr 172.21.15.3:0/1255698330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c7200640 1 --2- 172.21.15.3:0/1255698330 >> v2:172.21.15.70:3300/0 conn(0x7fd3c8072990 0x7fd3c8062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c5e00640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c806faf0 0x7fd3c8072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 -- 172.21.15.3:0/1255698330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c806faf0 msgr2=0x7fd3c8072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c806faf0 0x7fd3c8072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 -- 172.21.15.3:0/1255698330 >> v2:172.21.15.70:3300/0 conn(0x7fd3c8072990 msgr2=0x7fd3c8062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 --2- 172.21.15.3:0/1255698330 >> v2:172.21.15.70:3300/0 conn(0x7fd3c8072990 0x7fd3c8062630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.656+0000 7fd3c6800640 1 -- 172.21.15.3:0/1255698330 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c815d840 con 0x7fd3c815a1e0 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.657+0000 7fd3c6800640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c815a1e0 0x7fd3c806f5b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80731f0 tx=0x7fd3b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82da797f9f7f3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.657+0000 7fd3c5400640 1 -- 172.21.15.3:0/1255698330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037070 con 0x7fd3c815a1e0 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.657+0000 7fd3c5400640 1 -- 172.21.15.3:0/1255698330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8032440 con 0x7fd3c815a1e0 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.657+0000 7fd3c5400640 1 -- 172.21.15.3:0/1255698330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032810 con 0x7fd3c815a1e0 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 -- 172.21.15.3:0/1255698330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c815a1e0 msgr2=0x7fd3c806f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c815a1e0 0x7fd3c806f5b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80731f0 tx=0x7fd3b802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 -- 172.21.15.3:0/1255698330 shutdown_connections 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 --2- 172.21.15.3:0/1255698330 >> v2:172.21.15.70:3300/0 conn(0x7fd3c8072990 0x7fd3c8062630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c806faf0 0x7fd3c8072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 --2- 172.21.15.3:0/1255698330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c815a1e0 0x7fd3c806f5b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 -- 172.21.15.3:0/1255698330 >> 172.21.15.3:0/1255698330 conn(0x7fd3c80b3540 msgr2=0x7fd3c80b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 -- 172.21.15.3:0/1255698330 shutdown_connections 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.658+0000 7fd3cd400640 1 -- 172.21.15.3:0/1255698330 wait complete. 2024-09-10T07:49:02.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.660+0000 7fd3cd400640 1 Processor -- start 2024-09-10T07:49:02.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.660+0000 7fd3cd400640 1 -- start start 2024-09-10T07:49:02.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.660+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c8072990 0x7fd3c814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c815a1e0 0x7fd3c80af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3cd400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c815db20 con 0x7fd3c806faf0 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3cd400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c815cc50 con 0x7fd3c8072990 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3cd400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80cf490 con 0x7fd3c815a1e0 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54784/0 (socket says 172.21.15.3:54784) 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 -- 172.21.15.3:0/3523893212 learned_addr learned my addr 172.21.15.3:0/3523893212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c7200640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c815a1e0 0x7fd3c80af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c5e00640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c8072990 0x7fd3c814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 -- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c815a1e0 msgr2=0x7fd3c80af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c815a1e0 0x7fd3c80af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 -- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c8072990 msgr2=0x7fd3c814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c8072990 0x7fd3c814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 -- 172.21.15.3:0/3523893212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c815e8c0 con 0x7fd3c806faf0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.661+0000 7fd3c6800640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000c00 tx=0x7fd3b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.662+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8041020 con 0x7fd3c806faf0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.662+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b80306c0 con 0x7fd3c806faf0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.662+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037040 con 0x7fd3c806faf0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.662+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c815eba0 con 0x7fd3c806faf0 2024-09-10T07:49:02.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.662+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c815f160 con 0x7fd3c806faf0 2024-09-10T07:49:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.663+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c8072450 con 0x7fd3c806faf0 2024-09-10T07:49:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.664+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3b8044020 con 0x7fd3c806faf0 2024-09-10T07:49:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.665+0000 7fd3bea00640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 0x7fd3a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.665+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fd3b80ab5f0 con 0x7fd3c806faf0 2024-09-10T07:49:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.665+0000 7fd3c5e00640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 0x7fd3a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.666+0000 7fd3c5e00640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 0x7fd3a806e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006fd0 tx=0x7fd3b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.669+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd3b804b030 con 0x7fd3c806faf0 2024-09-10T07:49:02.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.891+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3c806e0e0 con 0x7fd3c806faf0 2024-09-10T07:49:02.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.891+0000 7fd3bea00640 1 -- 172.21.15.3:0/3523893212 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7fd3b807a030 con 0x7fd3c806faf0 2024-09-10T07:49:02.891 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:49:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 msgr2=0x7fd3a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 0x7fd3a806e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006fd0 tx=0x7fd3b4008040 comp rx=0 tx=0).stop 2024-09-10T07:49:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 msgr2=0x7fd3c815f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:02.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000c00 tx=0x7fd3b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 shutdown_connections 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd3a806be30 0x7fd3a806e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c815a1e0 0x7fd3c80af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c8072990 0x7fd3c814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 --2- 172.21.15.3:0/3523893212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3c806faf0 0x7fd3c815f960 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 >> 172.21.15.3:0/3523893212 conn(0x7fd3c80b3540 msgr2=0x7fd3c815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.893+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 shutdown_connections 2024-09-10T07:49:02.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:02.894+0000 7fd3cd400640 1 -- 172.21.15.3:0/3523893212 wait complete. 2024-09-10T07:49:02.908 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:49:02.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:49:02.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:02.909 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:49:02.945 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:49:02.945 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:49:02.945 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:49:02.946 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.947 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:49:02.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:02.948 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:49:03.038 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:49:03.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:03.042 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:49:03.077 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:49:03.078 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:49:03.078 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:49:03.078 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:49:03.078 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:49:03.078 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:49:03.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:49:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:03.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:03.275 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:49:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:49:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:49:03.310 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:49:03.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:03.310 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:49:03.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:49:03.406 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:49:03.442 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:49:03.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:49:03.442 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:49:03.517 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:49:03.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:03.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:49:03.647 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:49:03.647 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:49:03.647 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:49:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:49:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:49:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:49:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:49:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:49:03.686 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:49:03.687 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:49:03.689 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:49:03.689 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:49:03.689 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:49:03.689 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:49:03.689 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:49:03.692 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:49:03.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:03.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:49:03.749 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:49:03.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:49:03.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:03.846 INFO:teuthology.orchestra.run.smithi003.stdout:client5138 2024-09-10T07:49:03.847 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:03.847 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:03.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:49:04.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.083+0000 7fd3a4c00640 1 Processor -- start 2024-09-10T07:49:04.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.083+0000 7fd3a4c00640 1 -- start start 2024-09-10T07:49:04.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.084+0000 7fd3a4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a0147ce0 0x7fd3a00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.084+0000 7fd3a4c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3a0072aa0 0x7fd3a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.084+0000 7fd3a4c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a015faf0 con 0x7fd3a00aede0 2024-09-10T07:49:04.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.084+0000 7fd3a4c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a015eaf0 con 0x7fd3a0072aa0 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.084+0000 7fd3a4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00730c0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3a0072aa0 0x7fd3a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3a0072aa0 0x7fd3a006e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59334/0 (socket says 172.21.15.3:59334) 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 -- 172.21.15.3:0/1212216755 learned_addr learned my addr 172.21.15.3:0/1212216755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39de00640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a0147ce0 0x7fd3a00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 -- 172.21.15.3:0/1212216755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a0147ce0 msgr2=0x7fd3a00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a0147ce0 0x7fd3a00ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 -- 172.21.15.3:0/1212216755 >> v1:172.21.15.70:6789/0 conn(0x7fd3a00aede0 legacy=0x7fd3a0072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.085+0000 7fd39e800640 1 -- 172.21.15.3:0/1212216755 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a015ed30 con 0x7fd3a0072aa0 2024-09-10T07:49:04.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.086+0000 7fd39e800640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0072aa0 0x7fd3a006e090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e50 tx=0x7fd39802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c898765880a4dd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.086+0000 7fd39ca00640 1 -- 172.21.15.3:0/1212216755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398036070 con 0x7fd3a0072aa0 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.086+0000 7fd39ca00640 1 -- 172.21.15.3:0/1212216755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd398031600 con 0x7fd3a0072aa0 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.086+0000 7fd39ca00640 1 -- 172.21.15.3:0/1212216755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980319d0 con 0x7fd3a0072aa0 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 -- 172.21.15.3:0/1212216755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0072aa0 msgr2=0x7fd3a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0072aa0 0x7fd3a006e090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e50 tx=0x7fd39802a710 comp rx=0 tx=0).stop 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 -- 172.21.15.3:0/1212216755 shutdown_connections 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0072aa0 0x7fd3a006e090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/1212216755 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a0147ce0 0x7fd3a00ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 -- 172.21.15.3:0/1212216755 >> 172.21.15.3:0/1212216755 conn(0x7fd3a00b5cb0 msgr2=0x7fd3a006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 -- 172.21.15.3:0/1212216755 shutdown_connections 2024-09-10T07:49:04.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.087+0000 7fd3a4c00640 1 -- 172.21.15.3:0/1212216755 wait complete. 2024-09-10T07:49:04.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.089+0000 7fd3a4c00640 1 Processor -- start 2024-09-10T07:49:04.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.089+0000 7fd3a4c00640 1 -- start start 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a0072aa0 0x7fd3a00b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00aede0 0x7fd3a01f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00af220 con 0x7fd3a0147ce0 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a015f660 con 0x7fd3a0072aa0 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd3a4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a015f2f0 con 0x7fd3a00aede0 2024-09-10T07:49:04.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd39e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd39e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59338/0 (socket says 172.21.15.3:59338) 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd39e800640 1 -- 172.21.15.3:0/377827059 learned_addr learned my addr 172.21.15.3:0/377827059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd39de00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a0072aa0 0x7fd3a00b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.090+0000 7fd39d400640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00aede0 0x7fd3a01f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 -- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00aede0 msgr2=0x7fd3a01f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00aede0 0x7fd3a01f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 -- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a0072aa0 msgr2=0x7fd3a00b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a0072aa0 0x7fd3a00b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 -- 172.21.15.3:0/377827059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a0144040 con 0x7fd3a0147ce0 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd39e800640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd398034040 tx=0x7fd398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398036050 con 0x7fd3a0147ce0 2024-09-10T07:49:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd398042070 con 0x7fd3a0147ce0 2024-09-10T07:49:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398009510 con 0x7fd3a0147ce0 2024-09-10T07:49:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a0144320 con 0x7fd3a0147ce0 2024-09-10T07:49:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.091+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01448e0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.092+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a00ae8a0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.093+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd398007bf0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.095+0000 7fd396000640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 0x7fd38006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.095+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fd3980abbf0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.095+0000 7fd39de00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 0x7fd38006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.095+0000 7fd39de00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 0x7fd38006e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01fbf50 tx=0x7fd390009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.099+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd398079eb0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.319+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fd3a00cdce0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.585+0000 7fd396000640 1 -- 172.21.15.3:0/377827059 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v35) ==== 105+0+0 (secure 0 0 0) 0x7fd39807e1b0 con 0x7fd3a0147ce0 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 msgr2=0x7fd38006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 0x7fd38006e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01fbf50 tx=0x7fd390009190 comp rx=0 tx=0).stop 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 msgr2=0x7fd3a01fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd398034040 tx=0x7fd398004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 shutdown_connections 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd38006be30 0x7fd38006e2f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3a0147ce0 0x7fd3a01fbff0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3a00aede0 0x7fd3a01f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 --2- 172.21.15.3:0/377827059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3a0072aa0 0x7fd3a00b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 >> 172.21.15.3:0/377827059 conn(0x7fd3a00b5cb0 msgr2=0x7fd3a0160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 shutdown_connections 2024-09-10T07:49:04.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.588+0000 7fd3a4c00640 1 -- 172.21.15.3:0/377827059 wait complete. 2024-09-10T07:49:04.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.766+0000 7f7b49400640 1 Processor -- start 2024-09-10T07:49:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.766+0000 7f7b49400640 1 -- start start 2024-09-10T07:49:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b49400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b4415b4f0 0x7f7b440d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b49400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b44073d10 0x7f7b440d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b49400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b440ccd80 con 0x7f7b4415b110 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b49400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7b440b44f0 con 0x7f7b4415b4f0 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b49400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7b440cd060 con 0x7f7b44073d10 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b4415b4f0 0x7f7b440d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b4415b4f0 0x7f7b440d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59342/0 (socket says 172.21.15.3:59342) 2024-09-10T07:49:04.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 -- 172.21.15.3:0/3414752108 learned_addr learned my addr 172.21.15.3:0/3414752108 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b43200640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b44073d10 0x7f7b440d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 -- 172.21.15.3:0/3414752108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b44073d10 msgr2=0x7f7b440d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b44073d10 0x7f7b440d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 -- 172.21.15.3:0/3414752108 >> v1:172.21.15.70:6789/0 conn(0x7f7b4415b110 legacy=0x7f7b440b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.768+0000 7f7b39e00640 1 -- 172.21.15.3:0/3414752108 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b440cd340 con 0x7f7b4415b4f0 2024-09-10T07:49:04.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.769+0000 7f7b39e00640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4415b4f0 0x7f7b440d01e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00bb10 tx=0x7f7b2c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13c6bb58698bc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b41e00640 1 -- 172.21.15.3:0/3414752108 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c032790 con 0x7f7b4415b4f0 2024-09-10T07:49:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b41e00640 1 -- 172.21.15.3:0/3414752108 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2c038070 con 0x7f7b4415b4f0 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b41e00640 1 -- 172.21.15.3:0/3414752108 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c032b50 con 0x7f7b4415b4f0 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 -- 172.21.15.3:0/3414752108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4415b4f0 msgr2=0x7f7b440d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4415b4f0 0x7f7b440d01e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00bb10 tx=0x7f7b2c02c890 comp rx=0 tx=0).stop 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 -- 172.21.15.3:0/3414752108 shutdown_connections 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b44073d10 0x7f7b440d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 --2- 172.21.15.3:0/3414752108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4415b4f0 0x7f7b440d01e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 -- 172.21.15.3:0/3414752108 >> 172.21.15.3:0/3414752108 conn(0x7f7b4406d000 msgr2=0x7f7b4406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 -- 172.21.15.3:0/3414752108 shutdown_connections 2024-09-10T07:49:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.770+0000 7f7b49400640 1 -- 172.21.15.3:0/3414752108 wait complete. 2024-09-10T07:49:04.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.772+0000 7f7b49400640 1 Processor -- start 2024-09-10T07:49:04.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.772+0000 7f7b49400640 1 -- start start 2024-09-10T07:49:04.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b4415b110 0x7f7b4415e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b4415b4f0 0x7f7b441f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4406bca0 con 0x7f7b44073d10 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4406bb20 con 0x7f7b4415b4f0 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b49400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4406be20 con 0x7f7b4415b110 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b42800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b42800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59354/0 (socket says 172.21.15.3:59354) 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.773+0000 7f7b42800640 1 -- 172.21.15.3:0/2679795394 learned_addr learned my addr 172.21.15.3:0/2679795394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b43200640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b4415b4f0 0x7f7b441f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b39e00640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b4415b110 0x7f7b4415e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 -- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b4415b110 msgr2=0x7f7b4415e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b4415b110 0x7f7b4415e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 -- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b4415b4f0 msgr2=0x7f7b441f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:04.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b4415b4f0 0x7f7b441f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 -- 172.21.15.3:0/2679795394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4415be40 con 0x7f7b44073d10 2024-09-10T07:49:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b42800640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c400 tx=0x7f7b3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340052d0 con 0x7f7b44073d10 2024-09-10T07:49:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34005490 con 0x7f7b44073d10 2024-09-10T07:49:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34007260 con 0x7f7b44073d10 2024-09-10T07:49:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.774+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4415c120 con 0x7f7b44073d10 2024-09-10T07:49:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.775+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4415c6e0 con 0x7f7b44073d10 2024-09-10T07:49:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.776+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b440b3d10 con 0x7f7b44073d10 2024-09-10T07:49:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.777+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b34007400 con 0x7f7b44073d10 2024-09-10T07:49:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.778+0000 7f7b3b400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 0x7f7b2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.778+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f7b3408a2f0 con 0x7f7b44073d10 2024-09-10T07:49:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.778+0000 7f7b39e00640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 0x7f7b2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.779+0000 7f7b39e00640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 0x7f7b2006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c037000 tx=0x7f7b2c032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:04.782+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b34058650 con 0x7f7b44073d10 2024-09-10T07:49:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.001+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b440cdbe0 con 0x7f7b44073d10 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.002+0000 7f7b3b400640 1 -- 172.21.15.3:0/2679795394 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7f7b3405c950 con 0x7f7b44073d10 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:05.001 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-10T07:49:04:589063+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:04.589046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5433,"mds_1":5430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-10T07:49:05.002 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 msgr2=0x7f7b2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 0x7f7b2006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c037000 tx=0x7f7b2c032000 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 msgr2=0x7f7b4415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c400 tx=0x7f7b3400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 shutdown_connections 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b2006be30 0x7f7b2006e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b4415b4f0 0x7f7b441f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b4415b110 0x7f7b4415e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 --2- 172.21.15.3:0/2679795394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b44073d10 0x7f7b4415e130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 >> 172.21.15.3:0/2679795394 conn(0x7f7b4406d000 msgr2=0x7f7b4415ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 shutdown_connections 2024-09-10T07:49:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:05.004+0000 7f7b49400640 1 -- 172.21.15.3:0/2679795394 wait complete. 2024-09-10T07:49:05.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:48:51.031977+0000', 'modified': '2024-09-10T07:49:04.589046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5433, 'mds_1': 5430}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5430': {'gid': 5430, 'name': 'c', 'rank': 1, 'incarnation': 36, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6837/2353998589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 2353998589}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2353998589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5433': {'gid': 5433, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6836/29592196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 29592196}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 29592196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5433, 'qdb_cluster': [5433]} 2024-09-10T07:49:05.422 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5430:up:creating 2024-09-10T07:49:05.897 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:49:05.897+0000 7f6862c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:49:06.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.594+0000 7f689f200640 1 Processor -- start 2024-09-10T07:49:06.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.594+0000 7f689f200640 1 -- start start 2024-09-10T07:49:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.595+0000 7f689f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68a015e860 0x7f68a00cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a00cee40 con 0x7f68a00cfbf0 2024-09-10T07:49:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689f200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a00cf120 con 0x7f68a0073150 2024-09-10T07:49:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f68a00d3780 con 0x7f68a015e860 2024-09-10T07:49:06.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689d400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f68a00cfbf0 0x7f68a00d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41880/0 (socket says 172.21.15.3:41880) 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689d400640 1 -- 172.21.15.3:0/307973543 learned_addr learned my addr 172.21.15.3:0/307973543 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689de00640 1 --2- 172.21.15.3:0/307973543 >> v2:172.21.15.70:3300/0 conn(0x7f68a015e860 0x7f68a00cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.596+0000 7f689ca00640 1 -- 172.21.15.3:0/307973543 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716972643 0 0) 0x7f68a00cee40 con 0x7f68a00cfbf0 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.597+0000 7f689ca00640 1 -- 172.21.15.3:0/307973543 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6884003520 con 0x7f68a00cfbf0 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.597+0000 7f689de00640 1 -- 172.21.15.3:0/307973543 >> v1:172.21.15.70:6789/0 conn(0x7f68a0073150 legacy=0x7f68a00b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.597+0000 7f689de00640 1 -- 172.21.15.3:0/307973543 >> v1:172.21.15.3:6789/0 conn(0x7f68a00cfbf0 legacy=0x7f68a00d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.597+0000 7f689de00640 1 -- 172.21.15.3:0/307973543 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a00d3410 con 0x7f68a015e860 2024-09-10T07:49:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.598+0000 7f689de00640 1 --2- 172.21.15.3:0/307973543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a015e860 0x7f68a00cf6b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68900097a0 tx=0x7f689002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174556bf356f6b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:06.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.601+0000 7f689ca00640 1 -- 172.21.15.3:0/307973543 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890037070 con 0x7f68a015e860 2024-09-10T07:49:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.601+0000 7f689ca00640 1 -- 172.21.15.3:0/307973543 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6890032440 con 0x7f68a015e860 2024-09-10T07:49:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.601+0000 7f689ca00640 1 -- 172.21.15.3:0/307973543 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890032810 con 0x7f68a015e860 2024-09-10T07:49:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 -- 172.21.15.3:0/307973543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a015e860 msgr2=0x7f68a00cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 --2- 172.21.15.3:0/307973543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a015e860 0x7f68a00cf6b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68900097a0 tx=0x7f689002a230 comp rx=0 tx=0).stop 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 -- 172.21.15.3:0/307973543 shutdown_connections 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 --2- 172.21.15.3:0/307973543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a015e860 0x7f68a00cf6b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 -- 172.21.15.3:0/307973543 >> 172.21.15.3:0/307973543 conn(0x7f68a00d4650 msgr2=0x7f68a00d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 -- 172.21.15.3:0/307973543 shutdown_connections 2024-09-10T07:49:06.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.602+0000 7f689f200640 1 -- 172.21.15.3:0/307973543 wait complete. 2024-09-10T07:49:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.604+0000 7f689f200640 1 Processor -- start 2024-09-10T07:49:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 -- start start 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 0x7f68a00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68a015e860 0x7f68a015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68a00d2460 con 0x7f68a0073150 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f68a00cf400 con 0x7f68a00cfbf0 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.605+0000 7f689f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f68a00b7f90 con 0x7f68a015e860 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59362/0 (socket says 172.21.15.3:59362) 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 -- 172.21.15.3:0/1081353547 learned_addr learned my addr 172.21.15.3:0/1081353547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689d400640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 0x7f68a00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689e800640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68a015e860 0x7f68a015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 -- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68a015e860 msgr2=0x7f68a015a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68a015e860 0x7f68a015a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 -- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 msgr2=0x7f68a00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 0x7f68a00cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689de00640 1 -- 172.21.15.3:0/1081353547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a0159680 con 0x7f68a0073150 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.606+0000 7f689d400640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 0x7f68a00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f689de00640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6890035040 tx=0x7f68900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890037040 con 0x7f68a0073150 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6890030500 con 0x7f68a0073150 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890030910 con 0x7f68a0073150 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a0159960 con 0x7f68a0073150 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.607+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a0159f20 con 0x7f68a0073150 2024-09-10T07:49:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.608+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a00d4a30 con 0x7f68a0073150 2024-09-10T07:49:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.609+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6890044020 con 0x7f68a0073150 2024-09-10T07:49:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.610+0000 7f6896000640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.610+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f68900ab160 con 0x7f68a0073150 2024-09-10T07:49:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.610+0000 7f689d400640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.611+0000 7f689d400640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b4c60 tx=0x7f688c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.615+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f68a00d4a30 con 0x7f68a0073150 2024-09-10T07:49:06.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.838+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68a00b5000 con 0x7f68a0073150 2024-09-10T07:49:06.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.838+0000 7f6896000640 1 -- 172.21.15.3:0/1081353547 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5249 (secure 0 0 0) 0x7f6890079b10 con 0x7f68a0073150 2024-09-10T07:49:06.838 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:06.838 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"btime":"2024-09-10T07:49:06:601178+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:06.601173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:06.838 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 39 2024-09-10T07:49:06.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.840+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 msgr2=0x7f688006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.840+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b4c60 tx=0x7f688c009040 comp rx=0 tx=0).stop 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.840+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 msgr2=0x7f68a00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.840+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6890035040 tx=0x7f68900046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 shutdown_connections 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68a015e860 0x7f68a015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a00cfbf0 0x7f68a00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 --2- 172.21.15.3:0/1081353547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0073150 0x7f68a00ca3c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:06.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 >> 172.21.15.3:0/1081353547 conn(0x7f68a00d4650 msgr2=0x7f68a00b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:06.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 shutdown_connections 2024-09-10T07:49:06.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:06.841+0000 7f689f200640 1 -- 172.21.15.3:0/1081353547 wait complete. 2024-09-10T07:49:07.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:48:51.031977+0000', 'modified': '2024-09-10T07:49:06.601173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5433, 'mds_1': 5430, 'mds_2': 5266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5266': {'gid': 5266, 'name': 'b', 'rank': 2, 'incarnation': 38, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.70:6833/1001818622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1001818622}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 1001818622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5430': {'gid': 5430, 'name': 'c', 'rank': 1, 'incarnation': 36, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6837/2353998589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 2353998589}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2353998589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5433': {'gid': 5433, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6836/29592196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 29592196}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 29592196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5433, 'qdb_cluster': [5266, 5430, 5433]} 2024-09-10T07:49:07.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:49:07.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T07:49:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.418+0000 7f0644a00640 1 Processor -- start 2024-09-10T07:49:07.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.418+0000 7f0644a00640 1 -- start start 2024-09-10T07:49:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0644a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0644a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06400d05f0 0x7f0640072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0644a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06400cfdd0 con 0x7f0640160710 2024-09-10T07:49:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0644a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f064015cc90 con 0x7f06400d05f0 2024-09-10T07:49:07.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0644a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f064015cf70 con 0x7f0640160af0 2024-09-10T07:49:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0637e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f063e800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06400d05f0 0x7f0640072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f063de00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0640160710 0x7f06400cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:55494/0 (socket says 172.21.15.3:55494) 2024-09-10T07:49:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f0637e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:40200/0 (socket says 172.21.15.3:40200) 2024-09-10T07:49:07.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.420+0000 7f063de00640 1 -- 172.21.15.3:0/917983613 learned_addr learned my addr 172.21.15.3:0/917983613 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.421+0000 7f063e800640 1 -- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 msgr2=0x7f06400d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.421+0000 7f063e800640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.421+0000 7f063e800640 1 -- 172.21.15.3:0/917983613 >> v1:172.21.15.70:6789/0 conn(0x7f0640160710 legacy=0x7f06400cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.421+0000 7f063e800640 1 -- 172.21.15.3:0/917983613 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064015d250 con 0x7f06400d05f0 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.421+0000 7f0637e00640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f063e800640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f0640072070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43f1921a41d976 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f063d400640 1 -- 172.21.15.3:0/917983613 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638007740 con 0x7f06400d05f0 2024-09-10T07:49:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f063d400640 1 -- 172.21.15.3:0/917983613 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06380078e0 con 0x7f06400d05f0 2024-09-10T07:49:07.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f063d400640 1 -- 172.21.15.3:0/917983613 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638007cb0 con 0x7f06400d05f0 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f0644a00640 1 -- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 msgr2=0x7f0640072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.422+0000 7f0644a00640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f0640072070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063802a660 comp rx=0 tx=0).stop 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 -- 172.21.15.3:0/917983613 shutdown_connections 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f0640072070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 --2- 172.21.15.3:0/917983613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f06400d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 -- 172.21.15.3:0/917983613 >> 172.21.15.3:0/917983613 conn(0x7f064006d000 msgr2=0x7f064006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 -- 172.21.15.3:0/917983613 shutdown_connections 2024-09-10T07:49:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.423+0000 7f0644a00640 1 -- 172.21.15.3:0/917983613 wait complete. 2024-09-10T07:49:07.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.424+0000 7f0644a00640 1 Processor -- start 2024-09-10T07:49:07.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.424+0000 7f0644a00640 1 -- start start 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f06400aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f064006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f064015d770 con 0x7f0640160710 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f06400d0a10 con 0x7f06400d05f0 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.425+0000 7f0644a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0640144bd0 con 0x7f0640160af0 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59372/0 (socket says 172.21.15.3:59372) 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 -- 172.21.15.3:0/2345376813 learned_addr learned my addr 172.21.15.3:0/2345376813 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f063de00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f06400aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f063e800640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f064006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 -- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 msgr2=0x7f064006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f064006bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 -- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 msgr2=0x7f06400aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f06400aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 -- 172.21.15.3:0/2345376813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06400aace0 con 0x7f0640160710 2024-09-10T07:49:07.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.426+0000 7f0637e00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f062800e960 tx=0x7f062800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.427+0000 7f0636000640 1 -- 172.21.15.3:0/2345376813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628017070 con 0x7f0640160710 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.427+0000 7f0636000640 1 -- 172.21.15.3:0/2345376813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062800ca00 con 0x7f0640160710 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.427+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401442a0 con 0x7f0640160710 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.427+0000 7f0636000640 1 -- 172.21.15.3:0/2345376813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800cdd0 con 0x7f0640160710 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.427+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0640144860 con 0x7f0640160710 2024-09-10T07:49:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.429+0000 7f0636000640 1 -- 172.21.15.3:0/2345376813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f062801a070 con 0x7f0640160710 2024-09-10T07:49:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.430+0000 7f0636000640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 0x7f061c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.430+0000 7f0636000640 1 -- 172.21.15.3:0/2345376813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f062808ad20 con 0x7f0640160710 2024-09-10T07:49:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.430+0000 7f063de00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 0x7f061c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.430+0000 7f0647808740 1 Processor -- start 2024-09-10T07:49:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.430+0000 7f0647808740 1 -- start start 2024-09-10T07:49:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f063de00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 0x7f061c06e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0630007fd0 tx=0x7f0630007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 0x5649a92ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5649a922c3b0 con 0x5649a92cdcf0 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5649a9231410 con 0x5649a92cf480 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.431+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5649a91d9620 con 0x5649a92cb4b0 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59388/0 (socket says 172.21.15.3:59388) 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 -- 172.21.15.3:0/787613734 learned_addr learned my addr 172.21.15.3:0/787613734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063e800640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f0637e00640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 0x5649a92ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 -- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 msgr2=0x5649a92ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 0x5649a92ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 -- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 msgr2=0x5649a92c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 -- 172.21.15.3:0/787613734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a8f442d0 con 0x5649a92cdcf0 2024-09-10T07:49:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f0637e00640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 0x5649a92ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:07.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.432+0000 7f063de00640 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0630001880 tx=0x7f0630013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4543f2bc4adbabfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f062e600640 1 -- 172.21.15.3:0/787613734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630026070 con 0x5649a92cdcf0 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f062e600640 1 -- 172.21.15.3:0/787613734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630013b80 con 0x5649a92cdcf0 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f062e600640 1 -- 172.21.15.3:0/787613734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630012350 con 0x5649a92cdcf0 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 -- 172.21.15.3:0/787613734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 msgr2=0x5649a9313490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0630001880 tx=0x7f0630013000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 -- 172.21.15.3:0/787613734 shutdown_connections 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cb4b0 0x5649a92ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 --2- 172.21.15.3:0/787613734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9313490 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 -- 172.21.15.3:0/787613734 >> 172.21.15.3:0/787613734 conn(0x5649a92ca740 msgr2=0x5649a9312960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 -- 172.21.15.3:0/787613734 shutdown_connections 2024-09-10T07:49:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.433+0000 7f0647808740 1 -- 172.21.15.3:0/787613734 wait complete. 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.434+0000 7f0647808740 1 -- start start 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.434+0000 7f0647808740 1 Processor -- start 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a92c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5649a91ec780 con 0x5649a92cca80 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5649a922d3d0 con 0x5649a92cf480 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5649a92c9be0 con 0x5649a92cdcf0 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.435+0000 7f0647808740 10 client.0 _ll_register_callbacks cb 0x5649a922a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59398/0 (socket says 172.21.15.3:59398) 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 -- 172.21.15.3:0/3721787922 learned_addr learned my addr 172.21.15.3:0/3721787922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063de00640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a92c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f0637e00640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 -- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 msgr2=0x5649a92c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a92c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 -- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 msgr2=0x5649a92cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 -- 172.21.15.3:0/3721787922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a900e6b0 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f063e800640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0638009eb0 tx=0x7f0638004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9795ce1af9c7cbd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.436+0000 7f0622a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638009450 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f063803c420 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0630013b80 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0638050050 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 10 client.5631 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5649a9326e30 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630013b80 con 0x5649a92cca80 2024-09-10T07:49:07.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7f0638053020 con 0x5649a92cca80 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 10 client.5631 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 10 client.5631 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 20 client.5631 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a92c78e0 0x5649a93233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 4 client.5631 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.437+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a923eb40 con 0x5649a92c78e0 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.438+0000 7f063de00640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a92c78e0 0x5649a93233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.438+0000 7f063de00640 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a92c78e0 0x5649a93233f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=416a031aa83cbedb server_cookie=c9c28e726125240b in_seq=0 out_seq=0 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.438+0000 7f0622a00640 10 client.5631 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.439+0000 7f0622a00640 1 -- 172.21.15.3:0/3721787922 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649a923eb40 con 0x5649a92c78e0 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.439+0000 7f0622a00640 10 client.5631 handle_command_reply: tid=0 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.439+0000 7f0647808740 1 client.5631 shutdown 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 msgr2=0x5649a92ec210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0638009eb0 tx=0x7f0638004ab0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 shutdown_connections 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a92c78e0 0x5649a93233f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a92ec210 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a92cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 --2- 172.21.15.3:0/3721787922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a92c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0622a00640 0 client.5631 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 >> 172.21.15.3:0/3721787922 conn(0x5649a92cb4b0 msgr2=0x5649a93109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 shutdown_connections 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 1 -- 172.21.15.3:0/3721787922 wait complete. 2024-09-10T07:49:07.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.440+0000 7f0647808740 20 client.5631 trim_cache size 0 max 16384 2024-09-10T07:49:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.459+0000 7f0647808740 1 Processor -- start 2024-09-10T07:49:07.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.459+0000 7f0647808740 1 -- start start 2024-09-10T07:49:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.459+0000 7f0647808740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a9311d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a9312250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0647808740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06401fff50 con 0x5649a92cca80 2024-09-10T07:49:07.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0640200620 con 0x5649a92cf480 2024-09-10T07:49:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0640145e20 con 0x5649a92cdcf0 2024-09-10T07:49:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59402/0 (socket says 172.21.15.3:59402) 2024-09-10T07:49:07.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 -- 172.21.15.3:0/4079253892 learned_addr learned my addr 172.21.15.3:0/4079253892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063e800640 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a9312250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f0637e00640 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a9311d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 -- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 msgr2=0x5649a9311d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a9311d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 -- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 msgr2=0x5649a9312250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a9312250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.460+0000 7f063de00640 1 -- 172.21.15.3:0/4079253892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a93114b0 con 0x5649a92cca80 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f063de00640 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0630013d10 tx=0x7f0630014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607664b677c3e9cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f062dc00640 1 -- 172.21.15.3:0/4079253892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630026040 con 0x5649a92cca80 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f062dc00640 1 -- 172.21.15.3:0/4079253892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630013df0 con 0x5649a92cca80 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f062dc00640 1 -- 172.21.15.3:0/4079253892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063002e640 con 0x5649a92cca80 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 -- 172.21.15.3:0/4079253892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 msgr2=0x5649a9337020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0630013d10 tx=0x7f0630014000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 -- 172.21.15.3:0/4079253892 shutdown_connections 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cf480 0x5649a9312250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cdcf0 0x5649a9311d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 --2- 172.21.15.3:0/4079253892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cca80 0x5649a9337020 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 -- 172.21.15.3:0/4079253892 >> 172.21.15.3:0/4079253892 conn(0x5649a92cb4b0 msgr2=0x5649a92f80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 -- 172.21.15.3:0/4079253892 shutdown_connections 2024-09-10T07:49:07.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.461+0000 7f0647808740 1 -- 172.21.15.3:0/4079253892 wait complete. 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.462+0000 7f0647808740 1 -- start start 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.462+0000 7f0647808740 1 Processor -- start 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cca80 0x5649a9390f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cf480 0x5649a9312110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5649a92c7760 con 0x5649a92cdcf0 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5649a92c84a0 con 0x5649a92cca80 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5649a92c7db0 con 0x5649a92cf480 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0647808740 10 client.0 _ll_register_callbacks cb 0x7f0638009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59412/0 (socket says 172.21.15.3:59412) 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.463+0000 7f0637e00640 1 -- 172.21.15.3:0/2867235771 learned_addr learned my addr 172.21.15.3:0/2867235771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 -- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cf480 msgr2=0x5649a9312110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cf480 0x5649a9312110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 -- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cca80 msgr2=0x5649a9390f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cca80 0x5649a9390f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 -- 172.21.15.3:0/2867235771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a92cc660 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0637e00640 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0628058f80 tx=0x7f06280590b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc0715689a2d54e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0622a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628021040 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f0628061690 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0630013df0 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0628061890 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0647808740 10 client.5637 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.464+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5649a93367c0 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630013df0 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7f0628024060 con 0x5649a92cdcf0 2024-09-10T07:49:07.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 10 client.5637 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 10 client.5637 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 20 client.5637 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a930ebb0 0x5649a930ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 4 client.5637 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "status"}) -- 0x5649a9238e90 con 0x5649a930ebb0 2024-09-10T07:49:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.465+0000 7f063de00640 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a930ebb0 0x5649a930ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f063de00640 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a930ebb0 0x5649a930ef90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8ce0d6f7c734f32 server_cookie=fd82fcb9cf3411f1 in_seq=0 out_seq=0 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0622a00640 10 client.5637 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0622a00640 1 -- 172.21.15.3:0/2867235771 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5649a9238e90 con 0x5649a930ebb0 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0622a00640 10 client.5637 handle_command_reply: tid=0 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 client.5637 shutdown 2024-09-10T07:49:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 msgr2=0x5649a9311bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0628058f80 tx=0x7f06280590b0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 shutdown_connections 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x5649a930ebb0 0x5649a930ef90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5649a92cf480 0x5649a9312110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5649a92cdcf0 0x5649a9311bd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0622a00640 0 client.5637 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 --2- 172.21.15.3:0/2867235771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a92cca80 0x5649a9390f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 >> 172.21.15.3:0/2867235771 conn(0x5649a92cb4b0 msgr2=0x5649a93223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.466+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 shutdown_connections 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0647808740 1 -- 172.21.15.3:0/2867235771 wait complete. 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0647808740 20 client.5637 trim_cache size 0 max 16384 2024-09-10T07:49:07.469 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5266, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.866094248, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 39, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 50, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 50, 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 27.208690123 2024-09-10T07:49:07.470 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 msgr2=0x7f061c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 0x7f061c06e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0630007fd0 tx=0x7f0630007e80 comp rx=0 tx=0).stop 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 msgr2=0x7f06401437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f062800e960 tx=0x7f062800ee20 comp rx=0 tx=0).stop 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 shutdown_connections 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f061c06be30 0x7f061c06e2f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0640160af0 0x7f064006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0640160710 0x7f06401437e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 --2- 172.21.15.3:0/2345376813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06400d05f0 0x7f06400aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 >> 172.21.15.3:0/2345376813 conn(0x7f064006d000 msgr2=0x7f06400cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 shutdown_connections 2024-09-10T07:49:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.467+0000 7f0644a00640 1 -- 172.21.15.3:0/2345376813 wait complete. 2024-09-10T07:49:07.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:49:07.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.646+0000 7f3e1ba00640 1 Processor -- start 2024-09-10T07:49:07.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.646+0000 7f3e1ba00640 1 -- start start 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e1415e8e0 0x7f3e14162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e1ba00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e140d1ba0 0x7f3e140d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e1ba00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e140b7920 con 0x7f3e140cf1f0 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e1ba00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3e140cf940 con 0x7f3e140d1ba0 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e140cfc20 con 0x7f3e1415e8e0 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e13e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e140cf1f0 0x7f3e140d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41882/0 (socket says 172.21.15.3:41882) 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e13e00640 1 -- 172.21.15.3:0/3708367674 learned_addr learned my addr 172.21.15.3:0/3708367674 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:07.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e19000640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e1415e8e0 0x7f3e14162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.647+0000 7f3e19a00640 1 --2- 172.21.15.3:0/3708367674 >> v2:172.21.15.70:3300/0 conn(0x7f3e140d1ba0 0x7f3e140d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e13400640 1 -- 172.21.15.3:0/3708367674 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404276414 0 0) 0x7f3e00001110 con 0x7f3e140cf1f0 2024-09-10T07:49:07.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e13400640 1 -- 172.21.15.3:0/3708367674 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dfc003520 con 0x7f3e140cf1f0 2024-09-10T07:49:07.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e19a00640 1 -- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e1415e8e0 msgr2=0x7f3e14162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e19a00640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e1415e8e0 0x7f3e14162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e19a00640 1 -- 172.21.15.3:0/3708367674 >> v1:172.21.15.3:6789/0 conn(0x7f3e140cf1f0 legacy=0x7f3e140d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.648+0000 7f3e19a00640 1 -- 172.21.15.3:0/3708367674 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e140d2ae0 con 0x7f3e140d1ba0 2024-09-10T07:49:07.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.649+0000 7f3e19a00640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e140d1ba0 0x7f3e140d32e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009d90 tx=0x7f3e0c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886ae82123b25403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.649+0000 7f3e13400640 1 -- 172.21.15.3:0/3708367674 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c031600 con 0x7f3e140d1ba0 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.649+0000 7f3e13400640 1 -- 172.21.15.3:0/3708367674 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0317a0 con 0x7f3e140d1ba0 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.649+0000 7f3e13400640 1 -- 172.21.15.3:0/3708367674 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c031b70 con 0x7f3e140d1ba0 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 -- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e140d1ba0 msgr2=0x7f3e140d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e140d1ba0 0x7f3e140d32e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009d90 tx=0x7f3e0c02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 -- 172.21.15.3:0/3708367674 shutdown_connections 2024-09-10T07:49:07.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e140d1ba0 0x7f3e140d32e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/3708367674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e1415e8e0 0x7f3e14162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 -- 172.21.15.3:0/3708367674 >> 172.21.15.3:0/3708367674 conn(0x7f3e1406bd00 msgr2=0x7f3e1406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 -- 172.21.15.3:0/3708367674 shutdown_connections 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.650+0000 7f3e1ba00640 1 -- 172.21.15.3:0/3708367674 wait complete. 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.652+0000 7f3e1ba00640 1 Processor -- start 2024-09-10T07:49:07.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.652+0000 7f3e1ba00640 1 -- start start 2024-09-10T07:49:07.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e140d1ba0 0x7f3e14161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e1415e8e0 0x7f3e14161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e140d3820 con 0x7f3e140cf1f0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e140cf630 con 0x7f3e1415e8e0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e140d39a0 con 0x7f3e140d1ba0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59418/0 (socket says 172.21.15.3:59418) 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 -- 172.21.15.3:0/135618062 learned_addr learned my addr 172.21.15.3:0/135618062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19a00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e1415e8e0 0x7f3e14161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e13e00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e140d1ba0 0x7f3e14161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 -- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e140d1ba0 msgr2=0x7f3e14161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e140d1ba0 0x7f3e14161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 -- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e1415e8e0 msgr2=0x7f3e14161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e1415e8e0 0x7f3e14161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.653+0000 7f3e19000640 1 -- 172.21.15.3:0/135618062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14160040 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e19000640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e8a0 tx=0x7f3e0800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e10c00640 1 -- 172.21.15.3:0/135618062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08017070 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e10c00640 1 -- 172.21.15.3:0/135618062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0800c940 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e10c00640 1 -- 172.21.15.3:0/135618062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800cd30 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1415efa0 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.654+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1415f560 con 0x7f3e140cf1f0 2024-09-10T07:49:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.656+0000 7f3e10c00640 1 -- 172.21.15.3:0/135618062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e0801a070 con 0x7f3e140cf1f0 2024-09-10T07:49:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.657+0000 7f3e10c00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 0x7f3df406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.657+0000 7f3e10c00640 1 -- 172.21.15.3:0/135618062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f3e0808b0c0 con 0x7f3e140cf1f0 2024-09-10T07:49:07.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.657+0000 7f3e13e00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 0x7f3df406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.658+0000 7f3e1ea6d740 1 Processor -- start 2024-09-10T07:49:07.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.658+0000 7f3e1ea6d740 1 -- start start 2024-09-10T07:49:07.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.658+0000 7f3e13e00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 0x7f3df406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5ac0 tx=0x7f3e00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.658+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e49cf0 0x561df7e8f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e474b0 0x561df7e4a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561df7da83b0 con 0x561df7e4b480 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561df7dad410 con 0x561df7e49cf0 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561df7d55620 con 0x561df7e474b0 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59420/0 (socket says 172.21.15.3:59420) 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 -- 172.21.15.3:0/2732107256 learned_addr learned my addr 172.21.15.3:0/2732107256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19000640 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e49cf0 0x561df7e8f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e13e00640 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e474b0 0x561df7e4a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 -- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e474b0 msgr2=0x561df7e4a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e474b0 0x561df7e4a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 -- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e49cf0 msgr2=0x561df7e8f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e49cf0 0x561df7e8f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.659+0000 7f3e19a00640 1 -- 172.21.15.3:0/2732107256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df7ac02d0 con 0x561df7e4b480 2024-09-10T07:49:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e80 tx=0x7f3e0c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ee2274fa7a9051 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e05600640 1 -- 172.21.15.3:0/2732107256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c031780 con 0x561df7e4b480 2024-09-10T07:49:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e05600640 1 -- 172.21.15.3:0/2732107256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c041070 con 0x561df7e4b480 2024-09-10T07:49:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e05600640 1 -- 172.21.15.3:0/2732107256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c031b60 con 0x561df7e4b480 2024-09-10T07:49:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2732107256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 msgr2=0x561df7e44570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.660+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e80 tx=0x7f3e0c02e000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2732107256 shutdown_connections 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e4b480 0x561df7e44570 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e474b0 0x561df7e4a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2732107256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e49cf0 0x561df7e8f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2732107256 >> 172.21.15.3:0/2732107256 conn(0x561df7e46740 msgr2=0x561df7e8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2732107256 shutdown_connections 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2732107256 wait complete. 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.661+0000 7f3e1ea6d740 1 -- start start 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.662+0000 7f3e1ea6d740 1 Processor -- start 2024-09-10T07:49:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e49cf0 0x561df7e681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7e72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561df7d68780 con 0x561df7e6b050 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561df7da93d0 con 0x561df7e4b480 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561df7e45be0 con 0x561df7e49cf0 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e1ea6d740 10 client.0 _ll_register_callbacks cb 0x561df7da65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59426/0 (socket says 172.21.15.3:59426) 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 -- 172.21.15.3:0/2473967464 learned_addr learned my addr 172.21.15.3:0/2473967464 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19000640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e49cf0 0x561df7e681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e13e00640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7e72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 -- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e49cf0 msgr2=0x561df7e681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e49cf0 0x561df7e681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 -- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 msgr2=0x561df7e72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7e72fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e19a00640 1 -- 172.21.15.3:0/2473967464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df7b8a6b0 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.663+0000 7f3e13e00640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7e72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e19a00640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009eb0 tx=0x7f3e0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7129a9970ad69a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004310 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f3e0c0044b0 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c041070 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c0046b0 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e1ea6d740 10 client.5649 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x561df7e45080 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c041070 con 0x561df7e6b050 2024-09-10T07:49:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7f3e0c031580 con 0x561df7e6b050 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e1ea6d740 10 client.5649 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.664+0000 7f3e1ea6d740 10 client.5649 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e1ea6d740 20 client.5649 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e43a00 0x561df7e43de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e1ea6d740 4 client.5649 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df7dbab40 con 0x561df7e43a00 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e19000640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e43a00 0x561df7e43de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3e19000640 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e43a00 0x561df7e43de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e4037233ac89273 server_cookie=f88e028b045dae45 in_seq=0 out_seq=0 2024-09-10T07:49:07.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.665+0000 7f3dfa000640 10 client.5649 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3dfa000640 1 -- 172.21.15.3:0/2473967464 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561df7dbab40 con 0x561df7e43a00 2024-09-10T07:49:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3dfa000640 10 client.5649 handle_command_reply: tid=0 2024-09-10T07:49:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 client.5649 shutdown 2024-09-10T07:49:07.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 msgr2=0x561df7e69920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009eb0 tx=0x7f3e0c004ab0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 shutdown_connections 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e43a00 0x561df7e43de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.666+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e6b050 0x561df7e69920 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3dfa000640 0 client.5649 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7e72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2473967464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e49cf0 0x561df7e681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 >> 172.21.15.3:0/2473967464 conn(0x561df7e474b0 msgr2=0x561df7e8df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 shutdown_connections 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2473967464 wait complete. 2024-09-10T07:49:07.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.667+0000 7f3e1ea6d740 20 client.5649 trim_cache size 0 max 16384 2024-09-10T07:49:07.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.686+0000 7f3e1ea6d740 1 Processor -- start 2024-09-10T07:49:07.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.686+0000 7f3e1ea6d740 1 -- start start 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.686+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7eb2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14200020 con 0x561df7e49cf0 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e142006f0 con 0x561df7e4b480 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e140c9f70 con 0x561df7e6b050 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59434/0 (socket says 172.21.15.3:59434) 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 -- 172.21.15.3:0/2581527318 learned_addr learned my addr 172.21.15.3:0/2581527318 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 -- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 msgr2=0x561df7eb2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7eb2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 -- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 msgr2=0x561df7eb24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 -- 172.21.15.3:0/2581527318 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df7e489c0 con 0x561df7e49cf0 2024-09-10T07:49:07.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.687+0000 7f3e19000640 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e080557c0 tx=0x7f3e0801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e25f7cc04c96252 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e04c00640 1 -- 172.21.15.3:0/2581527318 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08021040 con 0x561df7e49cf0 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e04c00640 1 -- 172.21.15.3:0/2581527318 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0805d710 con 0x561df7e49cf0 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e04c00640 1 -- 172.21.15.3:0/2581527318 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08061f40 con 0x561df7e49cf0 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2581527318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 msgr2=0x561df7e6a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e080557c0 tx=0x7f3e0801e040 comp rx=0 tx=0).stop 2024-09-10T07:49:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2581527318 shutdown_connections 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7eb2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/2581527318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e6a600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2581527318 >> 172.21.15.3:0/2581527318 conn(0x561df7e474b0 msgr2=0x561df7e45f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2581527318 shutdown_connections 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.688+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/2581527318 wait complete. 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.689+0000 7f3e1ea6d740 1 -- start start 2024-09-10T07:49:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.689+0000 7f3e1ea6d740 1 Processor -- start 2024-09-10T07:49:07.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7e8c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561df7e69400 con 0x561df7e49cf0 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561df7e45360 con 0x561df7e4b480 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561df7e44b40 con 0x561df7e6b050 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e1ea6d740 10 client.0 _ll_register_callbacks cb 0x7f3e080094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.690+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59442/0 (socket says 172.21.15.3:59442) 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 -- 172.21.15.3:0/3448545278 learned_addr learned my addr 172.21.15.3:0/3448545278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e13e00640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19a00640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7e8c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 -- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 msgr2=0x561df7e8c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7e8c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 -- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 msgr2=0x561df7eb2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 -- 172.21.15.3:0/3448545278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df7e73710 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e19000640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e080592f0 tx=0x7f3e0c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ad3809e9c2676c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08021070 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f3e0805bd40 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.691+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0805d710 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e080559d0 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 10 client.5655 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x561df7e6a600 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0805d710 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7f3e08071150 con 0x561df7e49cf0 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 10 client.5655 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 10 client.5655 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 20 client.5655 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e6ddd0 0x561df7f0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 4 client.5655 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "status"}) -- 0x561df7e91940 con 0x561df7e6ddd0 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.692+0000 7f3e13e00640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e6ddd0 0x561df7f0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e13e00640 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e6ddd0 0x561df7f0c010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f45f05714344741c server_cookie=e00714ec61c48826 in_seq=0 out_seq=0 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3dfa000640 10 client.5655 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3dfa000640 1 -- 172.21.15.3:0/3448545278 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561df7e91940 con 0x561df7e6ddd0 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3dfa000640 10 client.5655 handle_command_reply: tid=0 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 client.5655 shutdown 2024-09-10T07:49:07.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 msgr2=0x561df7e71ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e080592f0 tx=0x7f3e0c030000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 shutdown_connections 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x561df7e6ddd0 0x561df7f0c010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561df7e6b050 0x561df7e8c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561df7e4b480 0x561df7eb2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 --2- 172.21.15.3:0/3448545278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561df7e49cf0 0x561df7e71ed0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3dfa000640 0 client.5655 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 >> 172.21.15.3:0/3448545278 conn(0x561df7e474b0 msgr2=0x561df7e45f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 shutdown_connections 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.693+0000 7f3e1ea6d740 1 -- 172.21.15.3:0/3448545278 wait complete. 2024-09-10T07:49:07.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ea6d740 20 client.5655 trim_cache size 0 max 16384 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5430, 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.1031184930000002, 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 39, 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 50, 2024-09-10T07:49:07.698 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 50, 2024-09-10T07:49:07.699 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 27.439074401999999 2024-09-10T07:49:07.699 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:07.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 msgr2=0x7f3df406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 0x7f3df406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5ac0 tx=0x7f3e00009040 comp rx=0 tx=0).stop 2024-09-10T07:49:07.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 msgr2=0x7f3e1415cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e8a0 tx=0x7f3e0800ed60 comp rx=0 tx=0).stop 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 shutdown_connections 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3df406be30 0x7f3df406e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e1415e8e0 0x7f3e14161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e140d1ba0 0x7f3e14161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 --2- 172.21.15.3:0/135618062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e140cf1f0 0x7f3e1415cca0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 >> 172.21.15.3:0/135618062 conn(0x7f3e1406bd00 msgr2=0x7f3e1415dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.694+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 shutdown_connections 2024-09-10T07:49:07.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.695+0000 7f3e1ba00640 1 -- 172.21.15.3:0/135618062 wait complete. 2024-09-10T07:49:07.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:49:07.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.868+0000 7fc9e3200640 1 Processor -- start 2024-09-10T07:49:07.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.868+0000 7fc9e3200640 1 -- start start 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.869+0000 7fc9e3200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e40d3c10 0x7fc9e4073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.869+0000 7fc9e3200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e40cfdc0 con 0x7fc9e40d4050 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.869+0000 7fc9e3200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e40cee40 con 0x7fc9e415e860 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.869+0000 7fc9e3200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e40d3780 con 0x7fc9e40d3c10 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.869+0000 7fc9e1400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e40d3c10 0x7fc9e4073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e1400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc9e40d3c10 0x7fc9e4073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59448/0 (socket says 172.21.15.3:59448) 2024-09-10T07:49:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e1400640 1 -- 172.21.15.3:0/25372202 learned_addr learned my addr 172.21.15.3:0/25372202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e2800640 1 --1- 172.21.15.3:0/25372202 >> v1:172.21.15.3:6789/0 conn(0x7fc9e40d4050 0x7fc9e40b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41884/0 (socket says 172.21.15.3:41884) 2024-09-10T07:49:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e1400640 1 -- 172.21.15.3:0/25372202 >> v1:172.21.15.70:6789/0 conn(0x7fc9e415e860 legacy=0x7fc9e40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e1400640 1 -- 172.21.15.3:0/25372202 >> v1:172.21.15.3:6789/0 conn(0x7fc9e40d4050 legacy=0x7fc9e40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.870+0000 7fc9e1400640 1 -- 172.21.15.3:0/25372202 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40cf120 con 0x7fc9e40d3c10 2024-09-10T07:49:07.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.871+0000 7fc9e1400640 1 --2- 172.21.15.3:0/25372202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e40d3c10 0x7fc9e4073150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009870 tx=0x7fc9d002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ef5d7862091523 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.871+0000 7fc9e0a00640 1 -- 172.21.15.3:0/25372202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0037070 con 0x7fc9e40d3c10 2024-09-10T07:49:07.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.871+0000 7fc9e0a00640 1 -- 172.21.15.3:0/25372202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d002ae50 con 0x7fc9e40d3c10 2024-09-10T07:49:07.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.871+0000 7fc9e0a00640 1 -- 172.21.15.3:0/25372202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0032860 con 0x7fc9e40d3c10 2024-09-10T07:49:07.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 -- 172.21.15.3:0/25372202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e40d3c10 msgr2=0x7fc9e4073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 --2- 172.21.15.3:0/25372202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e40d3c10 0x7fc9e4073150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009870 tx=0x7fc9d002a300 comp rx=0 tx=0).stop 2024-09-10T07:49:07.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 -- 172.21.15.3:0/25372202 shutdown_connections 2024-09-10T07:49:07.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 --2- 172.21.15.3:0/25372202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e40d3c10 0x7fc9e4073150 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 -- 172.21.15.3:0/25372202 >> 172.21.15.3:0/25372202 conn(0x7fc9e40d4650 msgr2=0x7fc9e40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 -- 172.21.15.3:0/25372202 shutdown_connections 2024-09-10T07:49:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.872+0000 7fc9e3200640 1 -- 172.21.15.3:0/25372202 wait complete. 2024-09-10T07:49:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.874+0000 7fc9e3200640 1 Processor -- start 2024-09-10T07:49:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.874+0000 7fc9e3200640 1 -- start start 2024-09-10T07:49:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 0x7fc9e40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 0x7fc9e4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40d2460 con 0x7fc9e415e860 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40cf400 con 0x7fc9e40d4050 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e3200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40b7f90 con 0x7fc9e40d3c10 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59452/0 (socket says 172.21.15.3:59452) 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 -- 172.21.15.3:0/3079139961 learned_addr learned my addr 172.21.15.3:0/3079139961 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 0x7fc9e40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e1400640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 0x7fc9e4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 -- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 msgr2=0x7fc9e40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 0x7fc9e40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 -- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 msgr2=0x7fc9e4158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 0x7fc9e4158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e2800640 1 -- 172.21.15.3:0/3079139961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40cc2b0 con 0x7fc9e415e860 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 0x7fc9e40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.875+0000 7fc9e1400640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 0x7fc9e4158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9e2800640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c7b0 tx=0x7fc9dc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9da000640 1 -- 172.21.15.3:0/3079139961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017070 con 0x7fc9e415e860 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e40cc590 con 0x7fc9e415e860 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9da000640 1 -- 172.21.15.3:0/3079139961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00a640 con 0x7fc9e415e860 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e40ccb50 con 0x7fc9e415e860 2024-09-10T07:49:07.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.876+0000 7fc9da000640 1 -- 172.21.15.3:0/3079139961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00a9f0 con 0x7fc9e415e860 2024-09-10T07:49:07.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.878+0000 7fc9da000640 1 -- 172.21.15.3:0/3079139961 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9dc01a070 con 0x7fc9e415e860 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.879+0000 7fc9da000640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.879+0000 7fc9da000640 1 -- 172.21.15.3:0/3079139961 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fc9dc08a5e0 con 0x7fc9e415e860 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.879+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.879+0000 7fc9ebe39740 1 Processor -- start 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 -- start start 2024-09-10T07:49:07.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 0x7fc9c406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4007fd0 tx=0x7fc9d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a26660 0x559f34a29280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a23720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f349873b0 con 0x559f34a28ea0 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559f3498c410 con 0x559f34a26660 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.880+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f34934620 con 0x559f34a2a630 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59468/0 (socket says 172.21.15.3:59468) 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 -- 172.21.15.3:0/538985280 learned_addr learned my addr 172.21.15.3:0/538985280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e2800640 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a23720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1400640 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a26660 0x559f34a29280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 -- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 msgr2=0x559f34a23720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a23720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 -- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a26660 msgr2=0x559f34a29280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a26660 0x559f34a29280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 -- 172.21.15.3:0/538985280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3469f2d0 con 0x559f34a28ea0 2024-09-10T07:49:07.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.881+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4005130 tx=0x7fc9d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8002827449df15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9cea00640 1 -- 172.21.15.3:0/538985280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40035f0 con 0x559f34a28ea0 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9cea00640 1 -- 172.21.15.3:0/538985280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4003790 con 0x559f34a28ea0 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9cea00640 1 -- 172.21.15.3:0/538985280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4003b40 con 0x559f34a28ea0 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 -- 172.21.15.3:0/538985280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 msgr2=0x559f34a6e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4005130 tx=0x7fc9d4011000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 -- 172.21.15.3:0/538985280 shutdown_connections 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a23720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a26660 0x559f34a29280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/538985280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a6e640 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 -- 172.21.15.3:0/538985280 >> 172.21.15.3:0/538985280 conn(0x559f34a258f0 msgr2=0x559f34a6db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.882+0000 7fc9ebe39740 1 -- 172.21.15.3:0/538985280 shutdown_connections 2024-09-10T07:49:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.883+0000 7fc9ebe39740 1 -- 172.21.15.3:0/538985280 wait complete. 2024-09-10T07:49:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.883+0000 7fc9ebe39740 1 -- start start 2024-09-10T07:49:07.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.883+0000 7fc9ebe39740 1 Processor -- start 2024-09-10T07:49:07.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a28ea0 0x559f34a24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f34947780 con 0x559f34a27bf0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559f349883d0 con 0x559f34a28ea0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f34a24d90 con 0x559f34a2a630 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.884+0000 7fc9ebe39740 10 client.0 _ll_register_callbacks cb 0x559f349855d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59474/0 (socket says 172.21.15.3:59474) 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 -- 172.21.15.3:0/1377760129 learned_addr learned my addr 172.21.15.3:0/1377760129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e1400640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a28ea0 0x559f34a24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 -- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 msgr2=0x559f34a678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 -- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a28ea0 msgr2=0x559f34a24320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a28ea0 0x559f34a24320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 -- 172.21.15.3:0/1377760129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f347696b0 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9e2800640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c780 tx=0x7fc9dc0586e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde02c29d96a399 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.885+0000 7fc9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0610c0 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fc9dc00ab90 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d4003790 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc00ad90 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 10 client.5670 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x559f34a825f0 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4003790 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7fc9dc057b30 con 0x559f34a27bf0 2024-09-10T07:49:07.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 10 client.5670 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 10 client.5670 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 20 client.5670 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a51220 0x559f34a51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 4 client.5670 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.886+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f34999b40 con 0x559f34a51220 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.887+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a51220 0x559f34a51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.887+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a51220 0x559f34a51600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673cfde28d8dfd1d server_cookie=dc85f3484a39df26 in_seq=0 out_seq=0 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.887+0000 7fc9c3400640 10 client.5670 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9c3400640 1 -- 172.21.15.3:0/1377760129 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f34999b40 con 0x559f34a51220 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9c3400640 10 client.5670 handle_command_reply: tid=0 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 client.5670 shutdown 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 msgr2=0x559f34a28070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c780 tx=0x7fc9dc0586e0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 shutdown_connections 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a51220 0x559f34a51600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a28070 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/1377760129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a28ea0 0x559f34a24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 >> 172.21.15.3:0/1377760129 conn(0x559f34a26660 msgr2=0x559f34a6bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9c3400640 0 client.5670 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 shutdown_connections 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 1 -- 172.21.15.3:0/1377760129 wait complete. 2024-09-10T07:49:07.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.888+0000 7fc9ebe39740 20 client.5670 trim_cache size 0 max 16384 2024-09-10T07:49:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.907+0000 7fc9ebe39740 1 Processor -- start 2024-09-10T07:49:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 -- start start 2024-09-10T07:49:07.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a27bf0 0x559f34a674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4200ad0 con 0x559f34a28ea0 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41ffd30 con 0x559f34a27bf0 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.908+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e4200400 con 0x559f34a2a630 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59488/0 (socket says 172.21.15.3:59488) 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 -- 172.21.15.3:0/312807158 learned_addr learned my addr 172.21.15.3:0/312807158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a27bf0 0x559f34a674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e2800640 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 -- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 msgr2=0x559f34a516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 -- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a27bf0 msgr2=0x559f34a674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a27bf0 0x559f34a674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 -- 172.21.15.3:0/312807158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f34a6fec0 con 0x559f34a28ea0 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.909+0000 7fc9e1400640 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0000c00 tx=0x7fc9d002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ddc8b7cfd38bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ce000640 1 -- 172.21.15.3:0/312807158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d004c070 con 0x559f34a28ea0 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ce000640 1 -- 172.21.15.3:0/312807158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d002ae50 con 0x559f34a28ea0 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ce000640 1 -- 172.21.15.3:0/312807158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0037040 con 0x559f34a28ea0 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 -- 172.21.15.3:0/312807158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 msgr2=0x559f34a511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0000c00 tx=0x7fc9d002e000 comp rx=0 tx=0).stop 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 -- 172.21.15.3:0/312807158 shutdown_connections 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34a516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a28ea0 0x559f34a511a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/312807158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34a27bf0 0x559f34a674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 -- 172.21.15.3:0/312807158 >> 172.21.15.3:0/312807158 conn(0x559f34a26660 msgr2=0x559f34a4fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 -- 172.21.15.3:0/312807158 shutdown_connections 2024-09-10T07:49:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.910+0000 7fc9ebe39740 1 -- 172.21.15.3:0/312807158 wait complete. 2024-09-10T07:49:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.911+0000 7fc9ebe39740 1 -- start start 2024-09-10T07:49:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.911+0000 7fc9ebe39740 1 Processor -- start 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34ad52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34ad5e20 0x559f34a912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f34a82920 con 0x559f34a27bf0 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41f4c80 con 0x559f34ad5e20 2024-09-10T07:49:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f34a51d50 con 0x559f34a2a630 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9ebe39740 10 client.0 _ll_register_callbacks cb 0x7fc9dc054980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59496/0 (socket says 172.21.15.3:59496) 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9e1e00640 1 -- 172.21.15.3:0/487066390 learned_addr learned my addr 172.21.15.3:0/487066390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9e1400640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34ad52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.912+0000 7fc9e2800640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34ad5e20 0x559f34a912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 -- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 msgr2=0x559f34ad52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34ad52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 -- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34ad5e20 msgr2=0x559f34a912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34ad5e20 0x559f34a912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 -- 172.21.15.3:0/487066390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f34ad5a50 con 0x559f34a27bf0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9e1e00640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400eb80 tx=0x7fc9d4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd43bbbd9ee3e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:07.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40038f0 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fc9d4038ae0 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d002ae50 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9d403f050 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9ebe39740 10 client.5676 fetch_fsmap learned FSMap version 39 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.913+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x559f34a50a30 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d002ae50 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) ==== 3542+0+0 (secure 0 0 0) 0x7fc9d4042020 con 0x559f34a27bf0 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 10 client.5676 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 10 client.5676 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 20 client.5676 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a6c580 0x559f34a91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 4 client.5676 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "status"}) -- 0x559f34993e90 con 0x559f34a6c580 2024-09-10T07:49:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.914+0000 7fc9e1400640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a6c580 0x559f34a91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9e1400640 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a6c580 0x559f34a91800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3ab3e3e908586 server_cookie=100ab6781e7878cb in_seq=0 out_seq=0 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9c3400640 10 client.5676 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9c3400640 1 -- 172.21.15.3:0/487066390 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559f34993e90 con 0x559f34a6c580 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9c3400640 10 client.5676 handle_command_reply: tid=0 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 client.5676 shutdown 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 msgr2=0x559f34a6e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400eb80 tx=0x7fc9d4000ed0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 shutdown_connections 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x559f34a6c580 0x559f34a91800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f34ad5e20 0x559f34a912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f34a2a630 0x559f34ad52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9c3400640 0 client.5676 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 --2- 172.21.15.3:0/487066390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f34a27bf0 0x559f34a6e120 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 >> 172.21.15.3:0/487066390 conn(0x559f34a26660 msgr2=0x559f34a67490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 shutdown_connections 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 1 -- 172.21.15.3:0/487066390 wait complete. 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.915+0000 7fc9ebe39740 20 client.5676 trim_cache size 0 max 16384 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5433, 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:49:07.919 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 16.302599267000002, 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 39, 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 50, 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 50, 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 27.661095565 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:07.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 msgr2=0x7fc9c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 0x7fc9c406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4007fd0 tx=0x7fc9d4009040 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 msgr2=0x7fc9e415af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c7b0 tx=0x7fc9dc00cc70 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 shutdown_connections 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc9c406be30 0x7fc9c406e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e415e860 0x7fc9e415af40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9e40d4050 0x7fc9e4158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 --2- 172.21.15.3:0/3079139961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9e40d3c10 0x7fc9e40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 >> 172.21.15.3:0/3079139961 conn(0x7fc9e40d4650 msgr2=0x7fc9e40b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 shutdown_connections 2024-09-10T07:49:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:07.916+0000 7fc9e3200640 1 -- 172.21.15.3:0/3079139961 wait complete. 2024-09-10T07:49:07.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.087+0000 7f1c70000640 1 Processor -- start 2024-09-10T07:49:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.087+0000 7f1c70000640 1 -- start start 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.088+0000 7f1c70000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c6815b4f0 0x7f1c680d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.088+0000 7f1c70000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1c68073d10 0x7f1c680d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.088+0000 7f1c70000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c680ccd80 con 0x7f1c6815b110 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.088+0000 7f1c70000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1c680b44f0 con 0x7f1c6815b4f0 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.088+0000 7f1c70000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1c680cd060 con 0x7f1c68073d10 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c6815b4f0 0x7f1c680d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c6815b4f0 0x7f1c680d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59512/0 (socket says 172.21.15.3:59512) 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 -- 172.21.15.3:0/1145744863 learned_addr learned my addr 172.21.15.3:0/1145744863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6e000640 1 --2- 172.21.15.3:0/1145744863 >> v2:172.21.15.70:3300/0 conn(0x7f1c68073d10 0x7f1c680d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 -- 172.21.15.3:0/1145744863 >> v2:172.21.15.70:3300/0 conn(0x7f1c68073d10 msgr2=0x7f1c680d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 --2- 172.21.15.3:0/1145744863 >> v2:172.21.15.70:3300/0 conn(0x7f1c68073d10 0x7f1c680d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 -- 172.21.15.3:0/1145744863 >> v1:172.21.15.70:6789/0 conn(0x7f1c6815b110 legacy=0x7f1c680b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.089+0000 7f1c6cc00640 1 -- 172.21.15.3:0/1145744863 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c680cd340 con 0x7f1c6815b4f0 2024-09-10T07:49:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.090+0000 7f1c6cc00640 1 --2- 172.21.15.3:0/1145744863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c680d01e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1c680aff40 tx=0x7f1c5402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d03a8c5028e752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.090+0000 7f1c67e00640 1 -- 172.21.15.3:0/1145744863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54036020 con 0x7f1c6815b4f0 2024-09-10T07:49:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.090+0000 7f1c67e00640 1 -- 172.21.15.3:0/1145744863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54032640 con 0x7f1c6815b4f0 2024-09-10T07:49:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.090+0000 7f1c67e00640 1 -- 172.21.15.3:0/1145744863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54032a10 con 0x7f1c6815b4f0 2024-09-10T07:49:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 -- 172.21.15.3:0/1145744863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 msgr2=0x7f1c680d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 --2- 172.21.15.3:0/1145744863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c680d01e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1c680aff40 tx=0x7f1c5402c940 comp rx=0 tx=0).stop 2024-09-10T07:49:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 -- 172.21.15.3:0/1145744863 shutdown_connections 2024-09-10T07:49:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 --2- 172.21.15.3:0/1145744863 >> v2:172.21.15.70:3300/0 conn(0x7f1c68073d10 0x7f1c680d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 --2- 172.21.15.3:0/1145744863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c680d01e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 -- 172.21.15.3:0/1145744863 >> 172.21.15.3:0/1145744863 conn(0x7f1c6806d000 msgr2=0x7f1c6806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 -- 172.21.15.3:0/1145744863 shutdown_connections 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.091+0000 7f1c70000640 1 -- 172.21.15.3:0/1145744863 wait complete. 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.093+0000 7f1c70000640 1 Processor -- start 2024-09-10T07:49:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.093+0000 7f1c70000640 1 -- start start 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c68073d10 0x7f1c680aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 0x7f1c6815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6806bca0 con 0x7f1c6815b4f0 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6806bb20 con 0x7f1c6815b110 2024-09-10T07:49:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c70000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6806be20 con 0x7f1c68073d10 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c6e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c6e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59520/0 (socket says 172.21.15.3:59520) 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c6e000640 1 -- 172.21.15.3:0/1250678585 learned_addr learned my addr 172.21.15.3:0/1250678585 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c6d600640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c68073d10 0x7f1c680aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.094+0000 7f1c6cc00640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 0x7f1c6815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 -- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c68073d10 msgr2=0x7f1c680aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c68073d10 0x7f1c680aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 -- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 msgr2=0x7f1c6815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 0x7f1c6815e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 -- 172.21.15.3:0/1250678585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6815d3d0 con 0x7f1c6815b4f0 2024-09-10T07:49:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6cc00640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 0x7f1c6815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:08.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c6e000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1c60007e70 tx=0x7f1c6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60014620 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c60019070 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6815d670 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.095+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60014a20 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.096+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6815dc30 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.096+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c680b3d10 con 0x7f1c6815b4f0 2024-09-10T07:49:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.097+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c60014bc0 con 0x7f1c6815b4f0 2024-09-10T07:49:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.098+0000 7f1c65600640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.098+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f1c6008b6d0 con 0x7f1c6815b4f0 2024-09-10T07:49:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.098+0000 7f1c6d600640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.099+0000 7f1c6d600640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c68159540 tx=0x7f1c5c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:08.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.103+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1c6005a070 con 0x7f1c6815b4f0 2024-09-10T07:49:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.321+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c680b0390 con 0x7f1c6815b4f0 2024-09-10T07:49:08.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.322+0000 7f1c65600640 1 -- 172.21.15.3:0/1250678585 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5249 (secure 0 0 0) 0x7f1c60059970 con 0x7f1c6815b4f0 2024-09-10T07:49:08.322 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:08.322 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"btime":"2024-09-10T07:49:06:601178+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:06.601173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:08.322 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 39 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 msgr2=0x7f1c4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c68159540 tx=0x7f1c5c009040 comp rx=0 tx=0).stop 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 msgr2=0x7f1c6815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1c60007e70 tx=0x7f1c6000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 shutdown_connections 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c6815b4f0 0x7f1c6815ce90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6815b110 0x7f1c6815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 --2- 172.21.15.3:0/1250678585 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1c68073d10 0x7f1c680aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:08.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 >> 172.21.15.3:0/1250678585 conn(0x7f1c6806d000 msgr2=0x7f1c68072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:08.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 shutdown_connections 2024-09-10T07:49:08.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:08.324+0000 7f1c70000640 1 -- 172.21.15.3:0/1250678585 wait complete. 2024-09-10T07:49:08.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:08.339 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:49:08.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:08.389 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:49:08.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:08.430 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:49:08.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:08.507 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:49:08.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:08.584 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:49:08.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:08.875 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:49:08.876 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:08.876 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:10.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:11.063 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:49:11.064 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:11.064 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:13.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:13.251 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:49:13.251 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:13.251 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:15.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-09-10T07:49:15.442 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:15.442 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:17.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:17.632 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"exportbound":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0},"nref":4}}] 2024-09-10T07:49:17.633 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 1, "dirty": 1, "exportbound": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:17.633 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:19.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:19.822 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"exportbound":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0},"nref":4}}] 2024-09-10T07:49:19.823 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 1, "dirty": 1, "exportbound": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:19.823 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:21.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:49:21.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:49:21.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:49:22.002 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:49:22.003 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:49:22.003 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:49:22.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:49:22.178 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"0","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}}] 2024-09-10T07:49:22.179 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:49:22.179 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:49:22.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-10T07:49:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:49:22.353 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:49:22.353 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:49:22.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-10T07:49:22.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.512+0000 7f81d5a00640 1 Processor -- start 2024-09-10T07:49:22.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.512+0000 7f81d5a00640 1 -- start start 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.513+0000 7f81d5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 0x7f81d0072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.513+0000 7f81d5a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d015faf0 con 0x7f81d00728d0 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81d5a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d015ed30 con 0x7f81d0147ce0 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81d5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81d015eaf0 con 0x7f81d00aefb0 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81cf400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f81d00728d0 0x7f81d006e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34990/0 (socket says 172.21.15.3:34990) 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81cf400640 1 -- 172.21.15.3:0/2767767930 learned_addr learned my addr 172.21.15.3:0/2767767930 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2767767930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 0x7f81d0072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686293026 0 0) 0x7f81d015faf0 con 0x7f81d00728d0 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81b4003520 con 0x7f81d00728d0 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305059851 0 0) 0x7f81d015ed30 con 0x7f81d0147ce0 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.514+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81d015faf0 con 0x7f81d0147ce0 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.515+0000 7f81c7e00640 1 -- 172.21.15.3:0/2767767930 >> v1:172.21.15.70:6789/0 conn(0x7f81d0147ce0 legacy=0x7f81d00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.515+0000 7f81c7e00640 1 -- 172.21.15.3:0/2767767930 >> v1:172.21.15.3:6789/0 conn(0x7f81d00728d0 legacy=0x7f81d006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.515+0000 7f81c7e00640 1 -- 172.21.15.3:0/2767767930 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d015f010 con 0x7f81d00aefb0 2024-09-10T07:49:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2767767930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 0x7f81d0072390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81b80097d0 tx=0x7f81b802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2de926ad9467cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8037070 con 0x7f81d00aefb0 2024-09-10T07:49:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8032440 con 0x7f81d00aefb0 2024-09-10T07:49:22.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81ce000640 1 -- 172.21.15.3:0/2767767930 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b80327f0 con 0x7f81d00aefb0 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81d5a00640 1 -- 172.21.15.3:0/2767767930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 msgr2=0x7f81d0072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.516+0000 7f81d5a00640 1 --2- 172.21.15.3:0/2767767930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 0x7f81d0072390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81b80097d0 tx=0x7f81b802a210 comp rx=0 tx=0).stop 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.517+0000 7f81d5a00640 1 -- 172.21.15.3:0/2767767930 shutdown_connections 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.517+0000 7f81d5a00640 1 --2- 172.21.15.3:0/2767767930 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d00aefb0 0x7f81d0072390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.517+0000 7f81d5a00640 1 -- 172.21.15.3:0/2767767930 >> 172.21.15.3:0/2767767930 conn(0x7f81d00b5cb0 msgr2=0x7f81d006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.517+0000 7f81d5a00640 1 -- 172.21.15.3:0/2767767930 shutdown_connections 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.517+0000 7f81d5a00640 1 -- 172.21.15.3:0/2767767930 wait complete. 2024-09-10T07:49:22.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.518+0000 7f81d5a00640 1 Processor -- start 2024-09-10T07:49:22.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.518+0000 7f81d5a00640 1 -- start start 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 0x7f81d00cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d0147ce0 0x7f81d00b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81d015f660 con 0x7f81d00728d0 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81d00730c0 con 0x7f81d00aefb0 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81d5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81d01480c0 con 0x7f81d0147ce0 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81cea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81cea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49684/0 (socket says 172.21.15.3:49684) 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.519+0000 7f81cea00640 1 -- 172.21.15.3:0/3299449254 learned_addr learned my addr 172.21.15.3:0/3299449254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81c7e00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 0x7f81d00cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cf400640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d0147ce0 0x7f81d00b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 -- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d0147ce0 msgr2=0x7f81d00b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d0147ce0 0x7f81d00b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 -- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 msgr2=0x7f81d00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 0x7f81d00cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 -- 172.21.15.3:0/3299449254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d01474f0 con 0x7f81d00728d0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81c7e00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 0x7f81d00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81cea00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c430 tx=0x7f81c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81c6a00640 1 -- 172.21.15.3:0/3299449254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0017070 con 0x7f81d00728d0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81c6a00640 1 -- 172.21.15.3:0/3299449254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c00052f0 con 0x7f81d00728d0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81c6a00640 1 -- 172.21.15.3:0/3299449254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0003830 con 0x7f81d00728d0 2024-09-10T07:49:22.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.520+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d00cd900 con 0x7f81d00728d0 2024-09-10T07:49:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.521+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d00cee90 con 0x7f81d00728d0 2024-09-10T07:49:22.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.523+0000 7f81c6a00640 1 -- 172.21.15.3:0/3299449254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81c001a070 con 0x7f81d00728d0 2024-09-10T07:49:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.523+0000 7f81c6a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 0x7f81ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.523+0000 7f81c6a00640 1 -- 172.21.15.3:0/3299449254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f81c008a410 con 0x7f81d00728d0 2024-09-10T07:49:22.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.524+0000 7f81c7e00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 0x7f81ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.524+0000 7f81d8858740 1 Processor -- start 2024-09-10T07:49:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.524+0000 7f81d8858740 1 -- start start 2024-09-10T07:49:22.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.524+0000 7f81c7e00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 0x7f81ac06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034000 tx=0x7f81b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a95c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 0x55a260a96740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2609b3660 con 0x55a260a65800 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a26095b680 con 0x55a260a4b480 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a2609af620 con 0x55a260a65420 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49696/0 (socket says 172.21.15.3:49696) 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.525+0000 7f81c7e00640 1 -- 172.21.15.3:0/35132730 learned_addr learned my addr 172.21.15.3:0/35132730 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81cea00640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a95c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81cf400640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 0x55a260a96740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 -- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 msgr2=0x55a260a95c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a95c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 -- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 msgr2=0x55a260a96740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 0x55a260a96740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 -- 172.21.15.3:0/35132730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2607907d0 con 0x55a260a65800 2024-09-10T07:49:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81cf400640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 0x55a260a96740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:22.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81c7e00640 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009720 tx=0x7f81b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95218251419bb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81bf400640 1 -- 172.21.15.3:0/35132730 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b804e070 con 0x55a260a65800 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81bf400640 1 -- 172.21.15.3:0/35132730 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8003ae0 con 0x55a260a65800 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.526+0000 7f81bf400640 1 -- 172.21.15.3:0/35132730 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8037040 con 0x55a260a65800 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 -- 172.21.15.3:0/35132730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 msgr2=0x55a260a96200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009720 tx=0x7f81b802c000 comp rx=0 tx=0).stop 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 -- 172.21.15.3:0/35132730 shutdown_connections 2024-09-10T07:49:22.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a4b480 0x55a260a96740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a96200 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 --2- 172.21.15.3:0/35132730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a95c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 -- 172.21.15.3:0/35132730 >> 172.21.15.3:0/35132730 conn(0x55a260a4c220 msgr2=0x55a260a94840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 -- 172.21.15.3:0/35132730 shutdown_connections 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.527+0000 7f81d8858740 1 -- 172.21.15.3:0/35132730 wait complete. 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.528+0000 7f81d8858740 1 -- start start 2024-09-10T07:49:22.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.528+0000 7f81d8858740 1 Processor -- start 2024-09-10T07:49:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260aa8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a8d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a260a4a3e0 con 0x55a260a65800 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a26093c4c0 con 0x55a260a53f20 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a260a4a6c0 con 0x55a260a65420 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81d8858740 10 client.0 _ll_register_callbacks cb 0x55a2609abe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49706/0 (socket says 172.21.15.3:49706) 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81c7e00640 1 -- 172.21.15.3:0/2036613206 learned_addr learned my addr 172.21.15.3:0/2036613206 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81cea00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260aa8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.529+0000 7f81cf400640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 -- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 msgr2=0x55a260aa8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260aa8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 -- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 msgr2=0x55a260a8d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a8d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 -- 172.21.15.3:0/2036613206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2606c63f0 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81b80098f0 tx=0x7f81b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d093413824f3e89f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b804e070 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f81b8049c90 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b8003ae0 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.530+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f81b805a050 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 10 client.5694 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55a2606860f0 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8003ae0 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f81b805d020 con 0x55a260a65800 2024-09-10T07:49:22.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 10 client.5694 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 10 client.5694 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 20 client.5694 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a51920 0x55a260a51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 4 client.5694 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2609c0d90 con 0x55a260a51920 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.531+0000 7f81cea00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a51920 0x55a260a51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.532+0000 7f81cea00640 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a51920 0x55a260a51d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d728b5b606de3df server_cookie=b7f02ff413fa8265 in_seq=0 out_seq=0 2024-09-10T07:49:22.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.532+0000 7f81b3e00640 10 client.5694 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.532+0000 7f81b3e00640 1 -- 172.21.15.3:0/2036613206 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2609c0d90 con 0x55a260a51920 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.532+0000 7f81b3e00640 10 client.5694 handle_command_reply: tid=0 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.532+0000 7f81d8858740 1 client.5694 shutdown 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 msgr2=0x55a260a539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81b80098f0 tx=0x7f81b8004730 comp rx=0 tx=0).stop 2024-09-10T07:49:22.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 shutdown_connections 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a51920 0x55a260a51d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a8d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a539e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81b3e00640 0 client.5694 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 --2- 172.21.15.3:0/2036613206 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260aa8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 >> 172.21.15.3:0/2036613206 conn(0x55a260a4b480 msgr2=0x55a260a4f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 shutdown_connections 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 1 -- 172.21.15.3:0/2036613206 wait complete. 2024-09-10T07:49:22.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.533+0000 7f81d8858740 20 client.5694 trim_cache size 0 max 16384 2024-09-10T07:49:22.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.548+0000 7f81d8858740 1 Processor -- start 2024-09-10T07:49:22.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.548+0000 7f81d8858740 1 -- start start 2024-09-10T07:49:22.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a49970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a52bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0200ab0 con 0x55a260a65800 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01ffd00 con 0x55a260a53f20 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.549+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81d0200380 con 0x55a260a65420 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49714/0 (socket says 172.21.15.3:49714) 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 -- 172.21.15.3:0/274480769 learned_addr learned my addr 172.21.15.3:0/274480769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cea00640 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a49970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81c7e00640 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a52bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 -- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 msgr2=0x55a260a52bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a52bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 -- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 msgr2=0x55a260a49970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a49970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 -- 172.21.15.3:0/274480769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a260829980 con 0x55a260a65800 2024-09-10T07:49:22.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.550+0000 7f81cf400640 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001ff0 tx=0x7f81c80020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c44bd4118f3790 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81bea00640 1 -- 172.21.15.3:0/274480769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c801a020 con 0x55a260a65800 2024-09-10T07:49:22.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81bea00640 1 -- 172.21.15.3:0/274480769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c8007750 con 0x55a260a65800 2024-09-10T07:49:22.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81bea00640 1 -- 172.21.15.3:0/274480769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8016450 con 0x55a260a65800 2024-09-10T07:49:22.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 -- 172.21.15.3:0/274480769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 msgr2=0x55a260b0d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001ff0 tx=0x7f81c80020e0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 -- 172.21.15.3:0/274480769 shutdown_connections 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260b0d8e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a65420 0x55a260a52bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 --2- 172.21.15.3:0/274480769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a53f20 0x55a260a49970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 -- 172.21.15.3:0/274480769 >> 172.21.15.3:0/274480769 conn(0x55a260a4b480 msgr2=0x55a260a95e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 -- 172.21.15.3:0/274480769 shutdown_connections 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.551+0000 7f81d8858740 1 -- 172.21.15.3:0/274480769 wait complete. 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.552+0000 7f81d8858740 1 -- start start 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.552+0000 7f81d8858740 1 Processor -- start 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a53f20 0x55a260aa6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a65420 0x55a260af9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a260a966d0 con 0x55a260a65800 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01f79d0 con 0x55a260a65420 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a260ab9040 con 0x55a260a53f20 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81d8858740 10 client.0 _ll_register_callbacks cb 0x7f81b804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81cf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49724/0 (socket says 172.21.15.3:49724) 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.553+0000 7f81cf400640 1 -- 172.21.15.3:0/2975069824 learned_addr learned my addr 172.21.15.3:0/2975069824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cea00640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a53f20 0x55a260aa6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81c7e00640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a65420 0x55a260af9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 -- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a53f20 msgr2=0x55a260aa6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a53f20 0x55a260aa6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 -- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a65420 msgr2=0x55a260af9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a65420 0x55a260af9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 -- 172.21.15.3:0/2975069824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a260a79b90 con 0x55a260a65800 2024-09-10T07:49:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81cf400640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f81c8002120 tx=0x7f81c8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14bae2e9a057bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8016990 con 0x55a260a65800 2024-09-10T07:49:22.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f81c802ab50 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.554+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c8007750 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f81c8031050 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 10 client.5700 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55a260afa4c0 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c8007750 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f81c8034020 con 0x55a260a65800 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 10 client.5700 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 10 client.5700 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 20 client.5700 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a50170 0x55a260a639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 4 client.5700 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2609bb0e0 con 0x55a260a50170 2024-09-10T07:49:22.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.555+0000 7f81cea00640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a50170 0x55a260a639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.556+0000 7f81cea00640 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a50170 0x55a260a639f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1928301cd63d1ad7 server_cookie=20175ef93c5b8331 in_seq=0 out_seq=0 2024-09-10T07:49:22.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.556+0000 7f81b3e00640 10 client.5700 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81b3e00640 1 -- 172.21.15.3:0/2975069824 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2609bb0e0 con 0x55a260a50170 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81b3e00640 10 client.5700 handle_command_reply: tid=0 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 client.5700 shutdown 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 msgr2=0x55a260a93190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f81c8002120 tx=0x7f81c8001250 comp rx=0 tx=0).stop 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 shutdown_connections 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x55a260a50170 0x55a260a639f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a260a65800 0x55a260a93190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a260a65420 0x55a260af9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 --2- 172.21.15.3:0/2975069824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a260a53f20 0x55a260aa6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81b3e00640 0 client.5700 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 >> 172.21.15.3:0/2975069824 conn(0x55a260a4b480 msgr2=0x55a260a53610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 shutdown_connections 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 1 -- 172.21.15.3:0/2975069824 wait complete. 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.573+0000 7f81d8858740 20 client.5700 trim_cache size 0 max 16384 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:49:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 msgr2=0x7f81ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 0x7f81ac06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034000 tx=0x7f81b802e000 comp rx=0 tx=0).stop 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 msgr2=0x7f81d0146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c430 tx=0x7f81c000c8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 shutdown_connections 2024-09-10T07:49:22.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81ac06be30 0x7f81ac06e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81d0147ce0 0x7f81d00b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81d00aefb0 0x7f81d00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 --2- 172.21.15.3:0/3299449254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81d00728d0 0x7f81d0146fb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 >> 172.21.15.3:0/3299449254 conn(0x7f81d00b5cb0 msgr2=0x7f81d015dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 shutdown_connections 2024-09-10T07:49:22.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.574+0000 7f81d5a00640 1 -- 172.21.15.3:0/3299449254 wait complete. 2024-09-10T07:49:22.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2024-09-10T07:49:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.750+0000 7f65da800640 1 Processor -- start 2024-09-10T07:49:22.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.750+0000 7f65da800640 1 -- start start 2024-09-10T07:49:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65da800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d4161b70 0x7f65d40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65da800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65d40d3860 0x7f65d40d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65da800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d40d3de0 con 0x7f65d40d4ae0 2024-09-10T07:49:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65da800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f65d4161f50 con 0x7f65d40d3860 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65da800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f65d40d40c0 con 0x7f65d4161b70 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65d8a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65d40d3860 0x7f65d40d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65d8a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65d40d3860 0x7f65d40d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49734/0 (socket says 172.21.15.3:49734) 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.751+0000 7f65d8a00640 1 -- 172.21.15.3:0/590252382 learned_addr learned my addr 172.21.15.3:0/590252382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.752+0000 7f65d9400640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d4161b70 0x7f65d40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.752+0000 7f65d8a00640 1 -- 172.21.15.3:0/590252382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d4161b70 msgr2=0x7f65d40d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.752+0000 7f65d8a00640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d4161b70 0x7f65d40d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.752+0000 7f65d8a00640 1 -- 172.21.15.3:0/590252382 >> v1:172.21.15.70:6789/0 conn(0x7f65d40d4ae0 legacy=0x7f65d40d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.752+0000 7f65d8a00640 1 -- 172.21.15.3:0/590252382 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d40cf690 con 0x7f65d40d3860 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.753+0000 7f65d8a00640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d40d3860 0x7f65d40d45a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009870 tx=0x7f65c002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0efd49b1f38bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.753+0000 7f65d3e00640 1 -- 172.21.15.3:0/590252382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0037070 con 0x7f65d40d3860 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.753+0000 7f65d3e00640 1 -- 172.21.15.3:0/590252382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c0032440 con 0x7f65d40d3860 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.753+0000 7f65d3e00640 1 -- 172.21.15.3:0/590252382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0032810 con 0x7f65d40d3860 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 -- 172.21.15.3:0/590252382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d40d3860 msgr2=0x7f65d40d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d40d3860 0x7f65d40d45a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009870 tx=0x7f65c002a250 comp rx=0 tx=0).stop 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 -- 172.21.15.3:0/590252382 shutdown_connections 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d40d3860 0x7f65d40d45a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 --2- 172.21.15.3:0/590252382 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d4161b70 0x7f65d40d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 -- 172.21.15.3:0/590252382 >> 172.21.15.3:0/590252382 conn(0x7f65d4062630 msgr2=0x7f65d4062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 -- 172.21.15.3:0/590252382 shutdown_connections 2024-09-10T07:49:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.754+0000 7f65da800640 1 -- 172.21.15.3:0/590252382 wait complete. 2024-09-10T07:49:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.755+0000 7f65da800640 1 Processor -- start 2024-09-10T07:49:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.755+0000 7f65da800640 1 -- start start 2024-09-10T07:49:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d40d3860 0x7f65d414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4161b70 0x7f65d415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65d40cf2d0 con 0x7f65d4161b70 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f65d40c9fc0 con 0x7f65d40d4ae0 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.756+0000 7f65da800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f65d40cf450 con 0x7f65d40d3860 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4161b70 0x7f65d415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d9400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d40d3860 0x7f65d414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:50916/0 (socket says 172.21.15.3:50916) 2024-09-10T07:49:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 -- 172.21.15.3:0/1501136868 learned_addr learned my addr 172.21.15.3:0/1501136868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 -- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d40d3860 msgr2=0x7f65d414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d40d3860 0x7f65d414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 -- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4161b70 msgr2=0x7f65d415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4161b70 0x7f65d415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 -- 172.21.15.3:0/1501136868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d414d440 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.757+0000 7f65d8a00640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65c0000c00 tx=0x7f65c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.758+0000 7f65d1600640 1 -- 172.21.15.3:0/1501136868 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0041020 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.758+0000 7f65d1600640 1 -- 172.21.15.3:0/1501136868 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00306c0 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.758+0000 7f65d1600640 1 -- 172.21.15.3:0/1501136868 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0037040 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.758+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d40b40b0 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.758+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d40b4370 con 0x7f65d40d4ae0 2024-09-10T07:49:22.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.760+0000 7f65d1600640 1 -- 172.21.15.3:0/1501136868 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65c0044020 con 0x7f65d40d4ae0 2024-09-10T07:49:22.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.761+0000 7f65d1600640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 0x7f65b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.761+0000 7f65d1600640 1 -- 172.21.15.3:0/1501136868 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f65c00ab680 con 0x7f65d40d4ae0 2024-09-10T07:49:22.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.761+0000 7f65d9400640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 0x7f65b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.762+0000 7f65dec87740 1 Processor -- start 2024-09-10T07:49:22.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.762+0000 7f65d9400640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 0x7f65b406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65c80079a0 tx=0x7f65c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.762+0000 7f65dec87740 1 -- start start 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.762+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae0568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 0x5610ae03b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65dec87740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65dec87740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5610adfa4660 con 0x5610ae086ea0 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5610adf4c680 con 0x5610ae0564f0 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5610adfa0620 con 0x5610ae03c9e0 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49756/0 (socket says 172.21.15.3:49756) 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 0x5610ae03b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 -- 172.21.15.3:0/1422921053 learned_addr learned my addr 172.21.15.3:0/1422921053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9400640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae0568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 -- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 msgr2=0x5610ae03b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 0x5610ae03b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 -- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 msgr2=0x5610ae0568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae0568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9e00640 1 -- 172.21.15.3:0/1422921053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610add817d0 con 0x5610ae086ea0 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d8a00640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 0x5610ae03b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.763+0000 7f65d9400640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae0568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:22.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f65cc001f90 tx=0x7f65cc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18087d414a8d13bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65c5c00640 1 -- 172.21.15.3:0/1422921053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00fe50 con 0x5610ae086ea0 2024-09-10T07:49:22.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65c5c00640 1 -- 172.21.15.3:0/1422921053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65cc018070 con 0x5610ae086ea0 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65c5c00640 1 -- 172.21.15.3:0/1422921053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc003b70 con 0x5610ae086ea0 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65dec87740 1 -- 172.21.15.3:0/1422921053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 msgr2=0x5610ae0873f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.764+0000 7f65dec87740 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f65cc001f90 tx=0x7f65cc001670 comp rx=0 tx=0).stop 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 -- 172.21.15.3:0/1422921053 shutdown_connections 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae0873f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae03c9e0 0x5610ae03b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 --2- 172.21.15.3:0/1422921053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae0568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 -- 172.21.15.3:0/1422921053 >> 172.21.15.3:0/1422921053 conn(0x5610ae03d410 msgr2=0x5610ae085a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 -- 172.21.15.3:0/1422921053 shutdown_connections 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 -- 172.21.15.3:0/1422921053 wait complete. 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 -- start start 2024-09-10T07:49:22.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.765+0000 7f65dec87740 1 Processor -- start 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae07c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae07b350 0x5610ae07e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5610ae03b940 con 0x5610ae086ea0 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5610adf2d4c0 con 0x5610ae07b350 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5610ae03bc20 con 0x5610ae0564f0 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.766+0000 7f65dec87740 10 client.0 _ll_register_callbacks cb 0x5610adf9ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:22.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49766/0 (socket says 172.21.15.3:49766) 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 -- 172.21.15.3:0/3003261060 learned_addr learned my addr 172.21.15.3:0/3003261060 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 -- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 msgr2=0x5610ae07c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae07c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 -- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae07b350 msgr2=0x5610ae07e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae07b350 0x5610ae07e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 -- 172.21.15.3:0/3003261060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610adcb73f0 con 0x5610ae086ea0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65c002a290 tx=0x7f65c0079970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c0d8ff40c9e4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65ba000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0081f50 con 0x5610ae086ea0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f65c00820f0 con 0x5610ae086ea0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.767+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65cc018070 con 0x5610ae086ea0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f65c007ab30 con 0x5610ae086ea0 2024-09-10T07:49:22.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 10 client.5712 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5610adc770f0 con 0x5610ae086ea0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65cc018070 con 0x5610ae086ea0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f65c003a030 con 0x5610ae086ea0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 10 client.5712 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 10 client.5712 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 20 client.5712 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae03f9c0 0x5610ae03fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 4 client.5712 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610adfb1d90 con 0x5610ae03f9c0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.768+0000 7f65d9400640 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae03f9c0 0x5610ae03fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.769+0000 7f65d9400640 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae03f9c0 0x5610ae03fe50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d50a2811c7958c9 server_cookie=1259fb8359b752fd in_seq=0 out_seq=0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.769+0000 7f65ba000640 10 client.5712 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.769+0000 7f65ba000640 1 -- 172.21.15.3:0/3003261060 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610adfb1d90 con 0x5610ae03f9c0 2024-09-10T07:49:22.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.769+0000 7f65ba000640 10 client.5712 handle_command_reply: tid=0 2024-09-10T07:49:22.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 client.5712 shutdown 2024-09-10T07:49:22.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 msgr2=0x5610ae07f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65c002a290 tx=0x7f65c0079970 comp rx=0 tx=0).stop 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 shutdown_connections 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae03f9c0 0x5610ae03fe50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae07b350 0x5610ae07e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae086ea0 0x5610ae07f320 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 --2- 172.21.15.3:0/3003261060 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae07c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 >> 172.21.15.3:0/3003261060 conn(0x5610ae03c9e0 msgr2=0x5610ae03a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65ba000640 0 client.5712 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 shutdown_connections 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 1 -- 172.21.15.3:0/3003261060 wait complete. 2024-09-10T07:49:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.770+0000 7f65dec87740 20 client.5712 trim_cache size 0 max 16384 2024-09-10T07:49:22.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.785+0000 7f65dec87740 1 Processor -- start 2024-09-10T07:49:22.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.785+0000 7f65dec87740 1 -- start start 2024-09-10T07:49:22.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae0862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0a8d00 0x5610ae07bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4200da0 con 0x5610ae07b350 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f65d41fffa0 con 0x5610ae0a8d00 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.786+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f65d4200670 con 0x5610ae0564f0 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49776/0 (socket says 172.21.15.3:49776) 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 -- 172.21.15.3:0/3361753531 learned_addr learned my addr 172.21.15.3:0/3361753531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d9400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae0862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d9e00640 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0a8d00 0x5610ae07bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 -- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 msgr2=0x5610ae0862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae0862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 -- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0a8d00 msgr2=0x5610ae07bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0a8d00 0x5610ae07bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 -- 172.21.15.3:0/3361753531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610adfdab80 con 0x5610ae07b350 2024-09-10T07:49:22.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65d8a00640 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65c0075d10 tx=0x7f65c003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f697f3d889b81529 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65c5200640 1 -- 172.21.15.3:0/3361753531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0078a80 con 0x5610ae07b350 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65c5200640 1 -- 172.21.15.3:0/3361753531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c0078770 con 0x5610ae07b350 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.787+0000 7f65c5200640 1 -- 172.21.15.3:0/3361753531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0082150 con 0x5610ae07b350 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 -- 172.21.15.3:0/3361753531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 msgr2=0x5610ae07b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65c0075d10 tx=0x7f65c003e000 comp rx=0 tx=0).stop 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 -- 172.21.15.3:0/3361753531 shutdown_connections 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0a8d00 0x5610ae07bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae07b350 0x5610ae07b780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 --2- 172.21.15.3:0/3361753531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae0564f0 0x5610ae0862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 -- 172.21.15.3:0/3361753531 >> 172.21.15.3:0/3361753531 conn(0x5610ae03c9e0 msgr2=0x5610ae07e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 -- 172.21.15.3:0/3361753531 shutdown_connections 2024-09-10T07:49:22.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.788+0000 7f65dec87740 1 -- 172.21.15.3:0/3361753531 wait complete. 2024-09-10T07:49:22.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.789+0000 7f65dec87740 1 -- start start 2024-09-10T07:49:22.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.789+0000 7f65dec87740 1 Processor -- start 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae1041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae07b350 0x5610ae054ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5610adfb1d90 con 0x5610ae0a8d00 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4157cc0 con 0x5610ae0564f0 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5610ae040e00 con 0x5610ae07b350 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65dec87740 10 client.0 _ll_register_callbacks cb 0x7f65c00097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49780/0 (socket says 172.21.15.3:49780) 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 -- 172.21.15.3:0/1526937265 learned_addr learned my addr 172.21.15.3:0/1526937265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d8a00640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae07b350 0x5610ae054ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9400640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae1041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 -- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae07b350 msgr2=0x5610ae054ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae07b350 0x5610ae054ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 -- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 msgr2=0x5610ae1041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae1041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9e00640 1 -- 172.21.15.3:0/1526937265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ae0a8250 con 0x5610ae0a8d00 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.790+0000 7f65d9400640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65d9e00640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65cc001f60 tx=0x7f65cc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb31436451bdbc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65ba000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc026060 con 0x5610ae0a8d00 2024-09-10T07:49:22.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f65cc003d80 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c0078770 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f65cc02c050 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65dec87740 10 client.5718 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5610ae09e860 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.791+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c0078770 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f65cc02f020 con 0x5610ae0a8d00 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 10 client.5718 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 10 client.5718 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 20 client.5718 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae107f80 0x5610ae108360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 4 client.5718 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610adfac0e0 con 0x5610ae107f80 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65d9400640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae107f80 0x5610ae108360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65d9400640 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae107f80 0x5610ae108360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2d727324b43f76 server_cookie=dc7cb4ee0ca58973 in_seq=0 out_seq=0 2024-09-10T07:49:22.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.792+0000 7f65ba000640 10 client.5718 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65ba000640 1 -- 172.21.15.3:0/1526937265 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610adfac0e0 con 0x5610ae107f80 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65ba000640 10 client.5718 handle_command_reply: tid=0 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 client.5718 shutdown 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 msgr2=0x5610ae055000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65cc001f60 tx=0x7f65cc0014a0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 shutdown_connections 2024-09-10T07:49:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5610ae107f80 0x5610ae108360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610ae0a8d00 0x5610ae055000 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5610ae07b350 0x5610ae054ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65ba000640 0 client.5718 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 --2- 172.21.15.3:0/1526937265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5610ae0564f0 0x5610ae1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 >> 172.21.15.3:0/1526937265 conn(0x5610ae03c9e0 msgr2=0x5610ae07b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 shutdown_connections 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.810+0000 7f65dec87740 1 -- 172.21.15.3:0/1526937265 wait complete. 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65dec87740 20 client.5718 trim_cache size 0 max 16384 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:22.811 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:49:22.812 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:49:22.812 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 msgr2=0x7f65b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 0x7f65b406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65c80079a0 tx=0x7f65c8008040 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 msgr2=0x7f65d40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65c0000c00 tx=0x7f65c00046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 shutdown_connections 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65b406be30 0x7f65b406e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65d4161b70 0x7f65d415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40d4ae0 0x7f65d40b4e50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 --2- 172.21.15.3:0/1501136868 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65d40d3860 0x7f65d414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 >> 172.21.15.3:0/1501136868 conn(0x7f65d4062630 msgr2=0x7f65d415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.811+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 shutdown_connections 2024-09-10T07:49:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.812+0000 7f65da800640 1 -- 172.21.15.3:0/1501136868 wait complete. 2024-09-10T07:49:22.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2024-09-10T07:49:22.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.985+0000 7fbf1fe00640 1 Processor -- start 2024-09-10T07:49:22.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.985+0000 7fbf1fe00640 1 -- start start 2024-09-10T07:49:22.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.986+0000 7fbf1fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf200d45a0 0x7fbf200d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.986+0000 7fbf1fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf200cfe90 0x7fbf200d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf200d4f00 con 0x7fbf20161b70 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1fe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbf20161f50 con 0x7fbf200cfe90 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1fe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbf200d3de0 con 0x7fbf200d45a0 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1f400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbf200cfe90 0x7fbf200d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf20161b70 0x7fbf200d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35002/0 (socket says 172.21.15.3:35002) 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1ea00640 1 -- 172.21.15.3:0/2455496246 learned_addr learned my addr 172.21.15.3:0/2455496246 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:22.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf17e00640 1 --2- 172.21.15.3:0/2455496246 >> v2:172.21.15.70:3300/0 conn(0x7fbf200d45a0 0x7fbf200d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1f400640 1 -- 172.21.15.3:0/2455496246 >> v2:172.21.15.70:3300/0 conn(0x7fbf200d45a0 msgr2=0x7fbf200d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1f400640 1 --2- 172.21.15.3:0/2455496246 >> v2:172.21.15.70:3300/0 conn(0x7fbf200d45a0 0x7fbf200d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1f400640 1 -- 172.21.15.3:0/2455496246 >> v1:172.21.15.3:6789/0 conn(0x7fbf20161b70 legacy=0x7fbf200d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1f400640 1 -- 172.21.15.3:0/2455496246 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf200d40c0 con 0x7fbf200cfe90 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.987+0000 7fbf1e000640 1 -- 172.21.15.3:0/2455496246 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842710397 0 0) 0x7fbf200d4f00 con 0x7fbf20161b70 2024-09-10T07:49:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.988+0000 7fbf1f400640 1 --2- 172.21.15.3:0/2455496246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200cfe90 0x7fbf200d0300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf200cb4e0 tx=0x7fbf1802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f899976474ec43a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1e000640 1 -- 172.21.15.3:0/2455496246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18007740 con 0x7fbf200cfe90 2024-09-10T07:49:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1e000640 1 -- 172.21.15.3:0/2455496246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf180078e0 con 0x7fbf200cfe90 2024-09-10T07:49:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1e000640 1 -- 172.21.15.3:0/2455496246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18007cb0 con 0x7fbf200cfe90 2024-09-10T07:49:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 -- 172.21.15.3:0/2455496246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200cfe90 msgr2=0x7fbf200d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/2455496246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200cfe90 0x7fbf200d0300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf200cb4e0 tx=0x7fbf1802a680 comp rx=0 tx=0).stop 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 -- 172.21.15.3:0/2455496246 shutdown_connections 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/2455496246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200cfe90 0x7fbf200d0300 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/2455496246 >> v2:172.21.15.70:3300/0 conn(0x7fbf200d45a0 0x7fbf200d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 -- 172.21.15.3:0/2455496246 >> 172.21.15.3:0/2455496246 conn(0x7fbf20062630 msgr2=0x7fbf20062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 -- 172.21.15.3:0/2455496246 shutdown_connections 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.989+0000 7fbf1fe00640 1 -- 172.21.15.3:0/2455496246 wait complete. 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.991+0000 7fbf1fe00640 1 Processor -- start 2024-09-10T07:49:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.991+0000 7fbf1fe00640 1 -- start start 2024-09-10T07:49:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf200cfe90 0x7fbf200cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 0x7fbf20151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200c9fc0 con 0x7fbf200d45a0 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200cf970 con 0x7fbf20161b70 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbf200d3a30 con 0x7fbf200cfe90 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf17e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf17e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49790/0 (socket says 172.21.15.3:49790) 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf17e00640 1 -- 172.21.15.3:0/3428058056 learned_addr learned my addr 172.21.15.3:0/3428058056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.992+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf200cfe90 0x7fbf200cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf1f400640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 0x7fbf20151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 -- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf200cfe90 msgr2=0x7fbf200cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf200cfe90 0x7fbf200cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 -- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 msgr2=0x7fbf20151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 0x7fbf20151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 -- 172.21.15.3:0/3428058056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf200cd020 con 0x7fbf200d45a0 2024-09-10T07:49:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf1f400640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 0x7fbf20151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf17e00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800e9e0 tx=0x7fbf0800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf16a00640 1 -- 172.21.15.3:0/3428058056 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08017070 con 0x7fbf200d45a0 2024-09-10T07:49:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf16a00640 1 -- 172.21.15.3:0/3428058056 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0800c9d0 con 0x7fbf200d45a0 2024-09-10T07:49:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf200cd300 con 0x7fbf200d45a0 2024-09-10T07:49:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf16a00640 1 -- 172.21.15.3:0/3428058056 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800cdc0 con 0x7fbf200d45a0 2024-09-10T07:49:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.993+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf200cd8c0 con 0x7fbf200d45a0 2024-09-10T07:49:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.996+0000 7fbf16a00640 1 -- 172.21.15.3:0/3428058056 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf0801a070 con 0x7fbf200d45a0 2024-09-10T07:49:22.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.996+0000 7fbf16a00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 0x7fbefc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.997+0000 7fbf16a00640 1 -- 172.21.15.3:0/3428058056 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fbf0808b310 con 0x7fbf200d45a0 2024-09-10T07:49:22.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.997+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 0x7fbefc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.997+0000 7fbf285e7740 1 Processor -- start 2024-09-10T07:49:22.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.997+0000 7fbf285e7740 1 -- start start 2024-09-10T07:49:22.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.997+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 0x7fbefc06e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f60 tx=0x7fbf10007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 0x559be10c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be110c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559be1029660 con 0x559be10db4f0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559be0fd1680 con 0x559be10c19e0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559be1025620 con 0x559be110bea0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49792/0 (socket says 172.21.15.3:49792) 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.998+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1983818063 learned_addr learned my addr 172.21.15.3:0/1983818063 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf17e00640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 0x559be10c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1f400640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be110c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 msgr2=0x559be110c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be110c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 msgr2=0x559be10c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 0x559be10c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 -- 172.21.15.3:0/1983818063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be0e067d0 con 0x559be10db4f0 2024-09-10T07:49:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf17e00640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 0x559be10c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005de0 tx=0x7fbf10012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a169dbafd494d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf0f400640 1 -- 172.21.15.3:0/1983818063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10005e10 con 0x559be10db4f0 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf0f400640 1 -- 172.21.15.3:0/1983818063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf100071a0 con 0x559be10db4f0 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:22.999+0000 7fbf0f400640 1 -- 172.21.15.3:0/1983818063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10015bc0 con 0x559be10db4f0 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 -- 172.21.15.3:0/1983818063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 msgr2=0x559be10db8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005de0 tx=0x7fbf10012000 comp rx=0 tx=0).stop 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 -- 172.21.15.3:0/1983818063 shutdown_connections 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be110c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10c19e0 0x559be10c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 --2- 172.21.15.3:0/1983818063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be10db8d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 -- 172.21.15.3:0/1983818063 >> 172.21.15.3:0/1983818063 conn(0x559be10c2410 msgr2=0x559be110aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:23.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 -- 172.21.15.3:0/1983818063 shutdown_connections 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.000+0000 7fbf285e7740 1 -- 172.21.15.3:0/1983818063 wait complete. 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.001+0000 7fbf285e7740 1 -- start start 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.001+0000 7fbf285e7740 1 Processor -- start 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be1105cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be111b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559be10c0940 con 0x559be1100ef0 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559be0fb24c0 con 0x559be10db4f0 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559be10c0c20 con 0x559be110bea0 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf285e7740 10 client.0 _ll_register_callbacks cb 0x559be1021e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49806/0 (socket says 172.21.15.3:49806) 2024-09-10T07:49:23.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.002+0000 7fbf1f400640 1 -- 172.21.15.3:0/864632238 learned_addr learned my addr 172.21.15.3:0/864632238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf17e00640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be111b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be1105cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 -- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 msgr2=0x559be111b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be111b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 -- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 msgr2=0x559be1105cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be1105cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 -- 172.21.15.3:0/864632238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be0d3c3f0 con 0x559be1100ef0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf1f400640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbf1802ac70 tx=0x7fbf18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f4f71469414dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf03e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1804a060 con 0x559be1100ef0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf100071a0 con 0x559be1100ef0 2024-09-10T07:49:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.003+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fbf1803c420 con 0x559be1100ef0 2024-09-10T07:49:23.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbf18050050 con 0x559be1100ef0 2024-09-10T07:49:23.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 10 client.5733 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:23.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x559be0cfc0f0 con 0x559be1100ef0 2024-09-10T07:49:23.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf100071a0 con 0x559be1100ef0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7fbf18053020 con 0x559be1100ef0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 10 client.5733 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 10 client.5733 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 20 client.5733 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be1106bb0 0x559be1106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 4 client.5733 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be1036d90 con 0x559be1106bb0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.004+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be1106bb0 0x559be1106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.005+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be1106bb0 0x559be1106fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78400187034a499d server_cookie=a76be2fa9a5884d5 in_seq=0 out_seq=0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.005+0000 7fbf03e00640 10 client.5733 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf03e00640 1 -- 172.21.15.3:0/864632238 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559be1036d90 con 0x559be1106bb0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf03e00640 10 client.5733 handle_command_reply: tid=0 2024-09-10T07:49:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 client.5733 shutdown 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 msgr2=0x559be1101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbf1802ac70 tx=0x7fbf18004ab0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 shutdown_connections 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be1106bb0 0x559be1106fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be1100ef0 0x559be1101b50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be110bea0 0x559be111b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 --2- 172.21.15.3:0/864632238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be1105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 >> 172.21.15.3:0/864632238 conn(0x559be10c19e0 msgr2=0x559be10bf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf03e00640 0 client.5733 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 shutdown_connections 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 1 -- 172.21.15.3:0/864632238 wait complete. 2024-09-10T07:49:23.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.006+0000 7fbf285e7740 20 client.5733 trim_cache size 0 max 16384 2024-09-10T07:49:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.022+0000 7fbf285e7740 1 Processor -- start 2024-09-10T07:49:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.022+0000 7fbf285e7740 1 -- start start 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.022+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.022+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be1102a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.022+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 0x559be1102f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf285e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20200ab0 con 0x559be10db4f0 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201ffd10 con 0x559be110bea0 2024-09-10T07:49:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbf20200380 con 0x559be1100ef0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49822/0 (socket says 172.21.15.3:49822) 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 -- 172.21.15.3:0/2664105334 learned_addr learned my addr 172.21.15.3:0/2664105334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf17e00640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be1102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1f400640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 0x559be1102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 -- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 msgr2=0x559be1102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be1102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 -- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 msgr2=0x559be1102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 0x559be1102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1ea00640 1 -- 172.21.15.3:0/2664105334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be111b9d0 con 0x559be10db4f0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.023+0000 7fbf1f400640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 0x559be1102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbf10014000 tx=0x7fbf10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ddbb4676efa66e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf0ea00640 1 -- 172.21.15.3:0/2664105334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10010040 con 0x559be10db4f0 2024-09-10T07:49:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf0ea00640 1 -- 172.21.15.3:0/2664105334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf100071a0 con 0x559be10db4f0 2024-09-10T07:49:23.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf0ea00640 1 -- 172.21.15.3:0/2664105334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10021450 con 0x559be10db4f0 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 -- 172.21.15.3:0/2664105334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 msgr2=0x559be11082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbf10014000 tx=0x7fbf10013000 comp rx=0 tx=0).stop 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 -- 172.21.15.3:0/2664105334 shutdown_connections 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be110bea0 0x559be1102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be1102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 --2- 172.21.15.3:0/2664105334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be10db4f0 0x559be11082b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 -- 172.21.15.3:0/2664105334 >> 172.21.15.3:0/2664105334 conn(0x559be10c19e0 msgr2=0x559be111ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 -- 172.21.15.3:0/2664105334 shutdown_connections 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.024+0000 7fbf285e7740 1 -- 172.21.15.3:0/2664105334 wait complete. 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.025+0000 7fbf285e7740 1 -- start start 2024-09-10T07:49:23.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.025+0000 7fbf285e7740 1 Processor -- start 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be110d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be10c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559be111a410 con 0x559be110bea0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200cc400 con 0x559be10db4f0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559be10c1dc0 con 0x559be1100ef0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf285e7740 10 client.0 _ll_register_callbacks cb 0x7fbf18009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.026+0000 7fbf1f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49838/0 (socket says 172.21.15.3:49838) 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 -- 172.21.15.3:0/4277273424 learned_addr learned my addr 172.21.15.3:0/4277273424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf17e00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be10c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be110d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 -- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 msgr2=0x559be10c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be10c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 -- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 msgr2=0x559be110d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be110d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 -- 172.21.15.3:0/4277273424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be1103050 con 0x559be110bea0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf17e00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be10c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be110d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf1f400640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbf1803cfd0 tx=0x7fbf18030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1074c53482e41f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf03e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18058030 con 0x559be110bea0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fbf1803c420 con 0x559be110bea0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.027+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf100071a0 con 0x559be110bea0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbf18061050 con 0x559be110bea0 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 10 client.5739 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:23.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x559be118a5a0 con 0x559be110bea0 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf100071a0 con 0x559be110bea0 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7fbf18064020 con 0x559be110bea0 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 10 client.5739 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 10 client.5739 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 20 client.5739 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be110a6a0 0x559be110aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 4 client.5739 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:49:23.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.028+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559be10310e0 con 0x559be110a6a0 2024-09-10T07:49:23.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.029+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be110a6a0 0x559be110aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:23.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.029+0000 7fbf1ea00640 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be110a6a0 0x559be110aab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f13c9c673f91f601 server_cookie=1965104d0e8e78f5 in_seq=0 out_seq=0 2024-09-10T07:49:23.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.029+0000 7fbf03e00640 10 client.5739 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf03e00640 1 -- 172.21.15.3:0/4277273424 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559be10310e0 con 0x559be110a6a0 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf03e00640 10 client.5739 handle_command_reply: tid=0 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 client.5739 shutdown 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 msgr2=0x559be10c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbf1803cfd0 tx=0x7fbf18030000 comp rx=0 tx=0).stop 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 shutdown_connections 2024-09-10T07:49:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x559be110a6a0 0x559be110aab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559be110bea0 0x559be10c5dd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559be1100ef0 0x559be10c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 --2- 172.21.15.3:0/4277273424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559be10db4f0 0x559be110d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf03e00640 0 client.5739 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 >> 172.21.15.3:0/4277273424 conn(0x559be10c19e0 msgr2=0x559be1102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 shutdown_connections 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 1 -- 172.21.15.3:0/4277273424 wait complete. 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.039+0000 7fbf285e7740 20 client.5739 trim_cache size 0 max 16384 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:23.040 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:49:23.041 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:49:23.041 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 msgr2=0x7fbefc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 0x7fbefc06e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f60 tx=0x7fbf10007b50 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 msgr2=0x7fbf200ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800e9e0 tx=0x7fbf0800eea0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 shutdown_connections 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbefc06be60 0x7fbefc06e320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf20161b70 0x7fbf20151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf200d45a0 0x7fbf200ce0c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 --2- 172.21.15.3:0/3428058056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf200cfe90 0x7fbf200cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 >> 172.21.15.3:0/3428058056 conn(0x7fbf20062630 msgr2=0x7fbf2015cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 shutdown_connections 2024-09-10T07:49:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23.040+0000 7fbf1fe00640 1 -- 172.21.15.3:0/3428058056 wait complete. 2024-09-10T07:49:23.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:23.056 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-10T07:49:23.098 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/ 2024-09-10T07:49:23.099 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4 2024-09-10T07:49:23.099 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5 2024-09-10T07:49:23.099 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6 2024-09-10T07:49:23.099 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-10T07:49:23.099 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-10T07:49:23.100 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:23.240 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627778 2024-09-10T07:49:23.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:23.251 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.251 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.252 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.252 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.252 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-10T07:49:23.252 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:23.391 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627779 2024-09-10T07:49:23.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:23.402 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.402 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.403 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.403 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.403 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-10T07:49:23.403 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:23.543 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627780 2024-09-10T07:49:23.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-10T07:49:23.556 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:23.698 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627781 2024-09-10T07:49:23.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:23.709 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.709 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.709 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.710 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.710 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-10T07:49:23.710 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:23.850 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627782 2024-09-10T07:49:23.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:23.861 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:49:23.861 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:49:23.861 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:49:23.861 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:49:23.862 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-10T07:49:23.862 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:49:24.002 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627783 2024-09-10T07:49:24.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:49:24.014 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-10T07:49:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.130+0000 7f90f0d3ff40 1 Processor -- start 2024-09-10T07:49:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.130+0000 7f90f0d3ff40 1 -- start start 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba870507a0 0x55ba870e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87051e40 0x55ba8704ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ba87051150 con 0x55ba87052220 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ba86fc6b30 con 0x55ba870507a0 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ba870f7620 con 0x55ba87051e40 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90eea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90eea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45744/0 (socket says 172.21.15.3:45744) 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90eea00640 1 -- 172.21.15.3:0/3075721756 learned_addr learned my addr 172.21.15.3:0/3075721756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90ee000640 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba870507a0 0x55ba870e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.131+0000 7f90ed600640 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87051e40 0x55ba8704ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 -- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87051e40 msgr2=0x55ba8704ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87051e40 0x55ba8704ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 -- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba870507a0 msgr2=0x55ba870e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba870507a0 0x55ba870e9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 -- 172.21.15.3:0/3075721756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8704d130 con 0x55ba87052220 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90eea00640 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c480 tx=0x7f90e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a89cca34f8d3f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90e7e00640 1 -- 172.21.15.3:0/3075721756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80116b0 con 0x55ba87052220 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90e7e00640 1 -- 172.21.15.3:0/3075721756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e8017070 con 0x55ba87052220 2024-09-10T07:49:24.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.132+0000 7f90e7e00640 1 -- 172.21.15.3:0/3075721756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8011a90 con 0x55ba87052220 2024-09-10T07:49:24.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/3075721756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 msgr2=0x55ba870f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c480 tx=0x7f90e800c940 comp rx=0 tx=0).stop 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/3075721756 shutdown_connections 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba87052220 0x55ba870f90e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87051e40 0x55ba8704ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/3075721756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba870507a0 0x55ba870e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/3075721756 >> 172.21.15.3:0/3075721756 conn(0x55ba8704ed40 msgr2=0x55ba8704e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/3075721756 shutdown_connections 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.133+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/3075721756 wait complete. 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.134+0000 7f90f0d3ff40 1 Processor -- start 2024-09-10T07:49:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.134+0000 7f90f0d3ff40 1 -- start start 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.134+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 0x55ba870eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba87051e40 0x55ba870ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90f0d3ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87052220 0x55ba870f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ba870f9f90 con 0x55ba870507a0 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ba870f9cb0 con 0x55ba87051e40 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90f0d3ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ba870f08e0 con 0x55ba87052220 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 0x55ba870eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 0x55ba870eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45750/0 (socket says 172.21.15.3:45750) 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 -- 172.21.15.3:0/1571985916 learned_addr learned my addr 172.21.15.3:0/1571985916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 -- 172.21.15.3:0/1571985916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87052220 msgr2=0x55ba870f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ba87052220 0x55ba870f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 -- 172.21.15.3:0/1571985916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba87051e40 msgr2=0x55ba870ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba87051e40 0x55ba870ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 -- 172.21.15.3:0/1571985916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba870ed930 con 0x55ba870507a0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.135+0000 7f90ee000640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 0x55ba870eb7d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f90e0025ca0 tx=0x7f90e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.136+0000 7f90e6a00640 1 -- 172.21.15.3:0/1571985916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0056070 con 0x55ba870507a0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.136+0000 7f90e6a00640 1 -- 172.21.15.3:0/1571985916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0052910 con 0x55ba870507a0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.136+0000 7f90e6a00640 1 -- 172.21.15.3:0/1571985916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0052d20 con 0x55ba870507a0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.136+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba870edc10 con 0x55ba870507a0 2024-09-10T07:49:24.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.136+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba870ee1d0 con 0x55ba870507a0 2024-09-10T07:49:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.138+0000 7f90e6a00640 1 -- 172.21.15.3:0/1571985916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90e0059020 con 0x55ba870507a0 2024-09-10T07:49:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.138+0000 7f90e6a00640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90d006a810 0x7f90d006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.138+0000 7f90e6a00640 1 -- 172.21.15.3:0/1571985916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f90e00cdb10 con 0x55ba870507a0 2024-09-10T07:49:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.139+0000 7f90ed600640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90d006a810 0x7f90d006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.139+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55ba870e9a20 0x55ba870e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.139+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x55ba870ea3e0 con 0x55ba870e9a20 2024-09-10T07:49:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.139+0000 7f90eea00640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55ba870e9a20 0x55ba870e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.139+0000 7f90ed600640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90d006a810 0x7f90d006ccd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ba870f0700 tx=0x7f90dc003590 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.140+0000 7f90eea00640 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55ba870e9a20 0x55ba870e9ea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90eea00640 1 -- 172.21.15.3:0/1571985916 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v52'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90e8011730 con 0x55ba870e9a20 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55ba870e9a20 msgr2=0x55ba870e9ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55ba870e9a20 0x55ba870e9ea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90d006a810 msgr2=0x7f90d006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90d006a810 0x7f90d006ccd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ba870f0700 tx=0x7f90dc003590 comp rx=0 tx=0).stop 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 msgr2=0x55ba870eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 --2- 172.21.15.3:0/1571985916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ba870507a0 0x55ba870eb7d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f90e0025ca0 tx=0x7f90e0002fb0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90eea00640 1 -- 172.21.15.3:0/1571985916 reap_dead start 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 shutdown_connections 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 >> 172.21.15.3:0/1571985916 conn(0x55ba8704ed40 msgr2=0x55ba8704e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 shutdown_connections 2024-09-10T07:49:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.142+0000 7f90f0d3ff40 1 -- 172.21.15.3:0/1571985916 wait complete. 2024-09-10T07:49:24.146 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-10T07:49:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.222+0000 7f8102f45f40 1 Processor -- start 2024-09-10T07:49:24.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 -- start start 2024-09-10T07:49:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 0x559b7a505440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a501100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559b7a5ab900 con 0x559b7a51e130 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559b7a5ab620 con 0x559b7a506220 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.223+0000 7f8102f45f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559b7a5abbe0 con 0x559b7a505e40 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45766/0 (socket says 172.21.15.3:45766) 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 -- 172.21.15.3:0/3582378045 learned_addr learned my addr 172.21.15.3:0/3582378045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100200640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 0x559b7a505440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 -- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 msgr2=0x559b7a505440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 0x559b7a505440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 -- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 msgr2=0x559b7a501100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f80ff800640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a501100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a501100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 -- 172.21.15.3:0/3582378045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a3feb30 con 0x559b7a51e130 2024-09-10T07:49:24.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100200640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 0x559b7a505440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.224+0000 7f8100c00640 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c440 tx=0x7f80f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb39cbf53231917 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f80fe400640 1 -- 172.21.15.3:0/3582378045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8010650 con 0x559b7a51e130 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f80fe400640 1 -- 172.21.15.3:0/3582378045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f8016070 con 0x559b7a51e130 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f80fe400640 1 -- 172.21.15.3:0/3582378045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8010a50 con 0x559b7a51e130 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 -- 172.21.15.3:0/3582378045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 msgr2=0x559b7a5a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c440 tx=0x7f80f800c900 comp rx=0 tx=0).stop 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 -- 172.21.15.3:0/3582378045 shutdown_connections 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a51e130 0x559b7a5a1420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a501100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 --2- 172.21.15.3:0/3582378045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a505e40 0x559b7a505440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 -- 172.21.15.3:0/3582378045 >> 172.21.15.3:0/3582378045 conn(0x559b7a502d40 msgr2=0x559b7a4d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 -- 172.21.15.3:0/3582378045 shutdown_connections 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.225+0000 7f8102f45f40 1 -- 172.21.15.3:0/3582378045 wait complete. 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.226+0000 7f8102f45f40 1 Processor -- start 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.226+0000 7f8102f45f40 1 -- start start 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 0x559b7a5a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a58bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a51e130 0x559b7a59e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559b7a520530 con 0x559b7a505e40 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559b7a58e6e0 con 0x559b7a506220 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8102f45f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559b7a58ce10 con 0x559b7a51e130 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8100200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 0x559b7a5a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8100200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 0x559b7a5a9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45774/0 (socket says 172.21.15.3:45774) 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8100200640 1 -- 172.21.15.3:0/4049873233 learned_addr learned my addr 172.21.15.3:0/4049873233 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8100c00640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a51e130 0x559b7a59e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f80ff800640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a58bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.227+0000 7f8100200640 1 -- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a51e130 msgr2=0x559b7a59e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8100200640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559b7a51e130 0x559b7a59e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8100200640 1 -- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 msgr2=0x559b7a58bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8100200640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559b7a506220 0x559b7a58bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8100200640 1 -- 172.21.15.3:0/4049873233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a5a8ba0 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8100200640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 0x559b7a5a9640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f80f0058040 tx=0x7f80f0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f80fd000640 1 -- 172.21.15.3:0/4049873233 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f005a040 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f80fd000640 1 -- 172.21.15.3:0/4049873233 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f0055af0 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b7a5a4220 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f80fd000640 1 -- 172.21.15.3:0/4049873233 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0060470 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.228+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7a5a9120 con 0x559b7a505e40 2024-09-10T07:49:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.230+0000 7f80fd000640 1 -- 172.21.15.3:0/4049873233 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80f0060610 con 0x559b7a505e40 2024-09-10T07:49:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.231+0000 7f80fd000640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80e006a810 0x7f80e006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.231+0000 7f80fd000640 1 -- 172.21.15.3:0/4049873233 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f80f00ce2c0 con 0x559b7a505e40 2024-09-10T07:49:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.231+0000 7f80ff800640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80e006a810 0x7f80e006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.231+0000 7f8102f45f40 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x559b7a501a70 0x559b7a59f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.231+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 6.12 6:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x559b7a5a46a0 con 0x559b7a501a70 2024-09-10T07:49:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.232+0000 7f8100c00640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x559b7a501a70 0x559b7a59f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.232+0000 7f80ff800640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80e006a810 0x7f80e006ccd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559b7a5058a0 tx=0x7f80ec005c70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.232+0000 7f8100c00640 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x559b7a501a70 0x559b7a59f170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8100c00640 1 -- 172.21.15.3:0/4049873233 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v52'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f80f8010750 con 0x559b7a501a70 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x559b7a501a70 msgr2=0x559b7a59f170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x559b7a501a70 0x559b7a59f170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80e006a810 msgr2=0x7f80e006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80e006a810 0x7f80e006ccd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559b7a5058a0 tx=0x7f80ec005c70 comp rx=0 tx=0).stop 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 msgr2=0x559b7a5a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 --2- 172.21.15.3:0/4049873233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b7a505e40 0x559b7a5a9640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f80f0058040 tx=0x7f80f0002fb0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8100c00640 1 -- 172.21.15.3:0/4049873233 reap_dead start 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 shutdown_connections 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 >> 172.21.15.3:0/4049873233 conn(0x559b7a502d40 msgr2=0x559b7a503120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 shutdown_connections 2024-09-10T07:49:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.234+0000 7f8102f45f40 1 -- 172.21.15.3:0/4049873233 wait complete. 2024-09-10T07:49:24.238 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-10T07:49:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.316+0000 7f3c9823cf40 1 Processor -- start 2024-09-10T07:49:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.316+0000 7f3c9823cf40 1 -- start start 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601cce090a0 0x5601cce983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5601ccdbdf30 0x5601ccdbe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601ccd65b30 con 0x7f3c7c003ab0 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5601ccd69b70 con 0x5601ccdbdf30 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5601cce96900 con 0x5601cce090a0 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c95400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c95400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45800/0 (socket says 172.21.15.3:45800) 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.317+0000 7f3c95400640 1 -- 172.21.15.3:0/2526156878 learned_addr learned my addr 172.21.15.3:0/2526156878 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c94a00640 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601cce090a0 0x5601cce983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95e00640 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5601ccdbdf30 0x5601ccdbe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 -- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601cce090a0 msgr2=0x5601cce983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601cce090a0 0x5601cce983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 -- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5601ccdbdf30 msgr2=0x5601ccdbe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5601ccdbdf30 0x5601ccdbe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 -- 172.21.15.3:0/2526156878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601cce8e550 con 0x7f3c7c003ab0 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c95400640 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c880097b0 tx=0x7f3c88005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cf78d518ce96a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c8f400640 1 -- 172.21.15.3:0/2526156878 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007830 con 0x7f3c7c003ab0 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c8f400640 1 -- 172.21.15.3:0/2526156878 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c880079d0 con 0x7f3c7c003ab0 2024-09-10T07:49:24.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.318+0000 7f3c8f400640 1 -- 172.21.15.3:0/2526156878 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007d80 con 0x7f3c7c003ab0 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 -- 172.21.15.3:0/2526156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 msgr2=0x5601cce97870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c880097b0 tx=0x7f3c88005a00 comp rx=0 tx=0).stop 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 -- 172.21.15.3:0/2526156878 shutdown_connections 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5601ccdbdf30 0x5601ccdbe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601cce090a0 0x5601cce983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/2526156878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7c003ab0 0x5601cce97870 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 -- 172.21.15.3:0/2526156878 >> 172.21.15.3:0/2526156878 conn(0x5601ccdedd40 msgr2=0x5601cce970d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 -- 172.21.15.3:0/2526156878 shutdown_connections 2024-09-10T07:49:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.319+0000 7f3c9823cf40 1 -- 172.21.15.3:0/2526156878 wait complete. 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.320+0000 7f3c9823cf40 1 Processor -- start 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.320+0000 7f3c9823cf40 1 -- start start 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.320+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601ccdbdf30 0x5601cce86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 0x5601cce903a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c9823cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c003ab0 0x5601cce93f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601ccdbf5e0 con 0x5601cce090a0 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5601cce0b630 con 0x7f3c7c003ab0 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c9823cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5601cce8e830 con 0x5601ccdbdf30 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 0x5601cce903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 0x5601cce903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45802/0 (socket says 172.21.15.3:45802) 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 -- 172.21.15.3:0/344318987 learned_addr learned my addr 172.21.15.3:0/344318987 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c95e00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c003ab0 0x5601cce93f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c95400640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601ccdbdf30 0x5601cce86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 -- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601ccdbdf30 msgr2=0x5601cce86ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601ccdbdf30 0x5601cce86ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 -- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c003ab0 msgr2=0x5601cce93f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c7c003ab0 0x5601cce93f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 -- 172.21.15.3:0/344318987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ccf2c8e0 con 0x5601cce090a0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c95400640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5601ccdbdf30 0x5601cce86ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.321+0000 7f3c94a00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 0x5601cce903a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3c84001780 tx=0x7f3c84010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.322+0000 7f3c8e000640 1 -- 172.21.15.3:0/344318987 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400e880 con 0x5601cce090a0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.322+0000 7f3c8e000640 1 -- 172.21.15.3:0/344318987 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8400ea20 con 0x5601cce090a0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.322+0000 7f3c8e000640 1 -- 172.21.15.3:0/344318987 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400edf0 con 0x5601cce090a0 2024-09-10T07:49:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.322+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5601cce93480 con 0x5601cce090a0 2024-09-10T07:49:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.322+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601cce93a40 con 0x5601cce090a0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.324+0000 7f3c8e000640 1 -- 172.21.15.3:0/344318987 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c8400d3d0 con 0x5601cce090a0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.324+0000 7f3c8e000640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3c7806a810 0x7f3c7806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c8e000640 1 -- 172.21.15.3:0/344318987 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f3c8408aac0 con 0x5601cce090a0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c95400640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3c7806a810 0x7f3c7806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5601ccdbef10 0x5601cce975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 6.5 6:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x5601ccdbc6b0 con 0x5601ccdbef10 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c95e00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5601ccdbef10 0x5601cce975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.325+0000 7f3c95400640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3c7806a810 0x7f3c7806ccd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005a40 tx=0x7f3c8800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.326+0000 7f3c95e00640 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5601ccdbef10 0x5601cce975e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c95e00640 1 -- 172.21.15.3:0/344318987 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v52'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3c90051810 con 0x5601ccdbef10 2024-09-10T07:49:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5601ccdbef10 msgr2=0x5601cce975e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5601ccdbef10 0x5601cce975e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3c7806a810 msgr2=0x7f3c7806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3c7806a810 0x7f3c7806ccd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005a40 tx=0x7f3c8800b040 comp rx=0 tx=0).stop 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 msgr2=0x5601cce903a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 --2- 172.21.15.3:0/344318987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5601cce090a0 0x5601cce903a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3c84001780 tx=0x7f3c84010830 comp rx=0 tx=0).stop 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c95e00640 1 -- 172.21.15.3:0/344318987 reap_dead start 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.327+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 shutdown_connections 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.328+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 >> 172.21.15.3:0/344318987 conn(0x5601ccdedd40 msgr2=0x5601cce08690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.328+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 shutdown_connections 2024-09-10T07:49:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.328+0000 7f3c9823cf40 1 -- 172.21.15.3:0/344318987 wait complete. 2024-09-10T07:49:24.331 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-10T07:49:24.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.408+0000 7f4784c1cf40 1 Processor -- start 2024-09-10T07:49:24.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.408+0000 7f4784c1cf40 1 -- start start 2024-09-10T07:49:24.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e647918330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e647891b30 con 0x55e64791b7a0 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e647895b70 con 0x55e64791d220 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e6479c2900 con 0x55e64791ce40 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.409+0000 7f4781e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45820/0 (socket says 172.21.15.3:45820) 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 -- 172.21.15.3:0/1692451002 learned_addr learned my addr 172.21.15.3:0/1692451002 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4782800640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781400640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e647918330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 -- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 msgr2=0x55e647918330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e647918330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 -- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 msgr2=0x55e647918d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 -- 172.21.15.3:0/1692451002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e647937500 con 0x55e64791b7a0 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781400640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e647918330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f4781e00640 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4774001e00 tx=0x7f477400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3277de59eed693 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.410+0000 7f477be00640 1 -- 172.21.15.3:0/1692451002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477400be40 con 0x55e64791b7a0 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f477be00640 1 -- 172.21.15.3:0/1692451002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477400f400 con 0x55e64791b7a0 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f477be00640 1 -- 172.21.15.3:0/1692451002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477400f7d0 con 0x55e64791b7a0 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 -- 172.21.15.3:0/1692451002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 msgr2=0x55e6479a56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4774001e00 tx=0x7f477400b670 comp rx=0 tx=0).stop 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 -- 172.21.15.3:0/1692451002 shutdown_connections 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e647918330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/1692451002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a56b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 -- 172.21.15.3:0/1692451002 >> 172.21.15.3:0/1692451002 conn(0x55e647919d40 msgr2=0x55e6478e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 -- 172.21.15.3:0/1692451002 shutdown_connections 2024-09-10T07:49:24.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.411+0000 7f4784c1cf40 1 -- 172.21.15.3:0/1692451002 wait complete. 2024-09-10T07:49:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.412+0000 7f4784c1cf40 1 Processor -- start 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.412+0000 7f4784c1cf40 1 -- start start 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e6479c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e6478ea890 con 0x55e64791b7a0 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e6478ea640 con 0x55e64791d220 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4784c1cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e647917e10 con 0x55e64791ce40 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45830/0 (socket says 172.21.15.3:45830) 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 -- 172.21.15.3:0/4053354380 learned_addr learned my addr 172.21.15.3:0/4053354380 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4782800640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781400640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e6479c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 -- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 msgr2=0x55e6479c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e64791ce40 0x55e6479c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 -- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 msgr2=0x55e647918bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4781e00640 1 -- 172.21.15.3:0/4053354380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6479c0ef0 con 0x55e64791b7a0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.413+0000 7f4782800640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e64791d220 0x55e647918bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f4781e00640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a2b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f477aa00640 1 -- 172.21.15.3:0/4053354380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774020070 con 0x55e64791b7a0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f477aa00640 1 -- 172.21.15.3:0/4053354380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477401c440 con 0x55e64791b7a0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6479be4f0 con 0x55e64791b7a0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f477aa00640 1 -- 172.21.15.3:0/4053354380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477401c870 con 0x55e64791b7a0 2024-09-10T07:49:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.414+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6479beab0 con 0x55e64791b7a0 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.416+0000 7f477aa00640 1 -- 172.21.15.3:0/4053354380 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4774023070 con 0x55e64791b7a0 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.417+0000 7f477aa00640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f476406a810 0x7f476406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.417+0000 7f477aa00640 1 -- 172.21.15.3:0/4053354380 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f477408b5e0 con 0x55e64791b7a0 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.417+0000 7f4781400640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f476406a810 0x7f476406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.417+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e647a58450 0x55e647a58830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.417+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 6.9 6:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x55e6479c4e00 con 0x55e647a58450 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.418+0000 7f4781400640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f476406a810 0x7f476406ccd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4770057000 tx=0x7f477005a040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.418+0000 7f4782800640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e647a58450 0x55e647a58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.418+0000 7f4782800640 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e647a58450 0x55e647a58830 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4782800640 1 -- 172.21.15.3:0/4053354380 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v52'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f477c002660 con 0x55e647a58450 2024-09-10T07:49:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e647a58450 msgr2=0x55e647a58830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55e647a58450 0x55e647a58830 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f476406a810 msgr2=0x7f476406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f476406a810 0x7f476406ccd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4770057000 tx=0x7f477005a040 comp rx=0 tx=0).stop 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 msgr2=0x55e6479a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 --2- 172.21.15.3:0/4053354380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e64791b7a0 0x55e6479a2b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774000ed0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4782800640 1 -- 172.21.15.3:0/4053354380 reap_dead start 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 shutdown_connections 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.420+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 >> 172.21.15.3:0/4053354380 conn(0x55e647919d40 msgr2=0x55e6478e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.421+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 shutdown_connections 2024-09-10T07:49:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.421+0000 7f4784c1cf40 1 -- 172.21.15.3:0/4053354380 wait complete. 2024-09-10T07:49:24.424 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-10T07:49:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.502+0000 7f94da862f40 1 Processor -- start 2024-09-10T07:49:24.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.502+0000 7f94da862f40 1 -- start start 2024-09-10T07:49:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae477a0 0x56276aee0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae48e40 0x56276aef0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56276adbdb30 con 0x56276ae49220 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56276adc1b70 con 0x56276ae477a0 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94da862f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56276aeee900 con 0x56276ae48e40 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45838/0 (socket says 172.21.15.3:45838) 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.503+0000 7f94d8400640 1 -- 172.21.15.3:0/1495542555 learned_addr learned my addr 172.21.15.3:0/1495542555 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d7a00640 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae477a0 0x56276aee0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d7000640 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae48e40 0x56276aef0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 -- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae48e40 msgr2=0x56276aef0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae48e40 0x56276aef0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 -- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae477a0 msgr2=0x56276aee0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae477a0 0x56276aee0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 -- 172.21.15.3:0/1495542555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276ae44840 con 0x56276ae49220 2024-09-10T07:49:24.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d8400640 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f94d0001650 tx=0x7f94d0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed21f6b0ed49d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d5c00640 1 -- 172.21.15.3:0/1495542555 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0016070 con 0x56276ae49220 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d5c00640 1 -- 172.21.15.3:0/1495542555 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d000f710 con 0x56276ae49220 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.504+0000 7f94d5c00640 1 -- 172.21.15.3:0/1495542555 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d000fb20 con 0x56276ae49220 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 -- 172.21.15.3:0/1495542555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 msgr2=0x56276aee65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f94d0001650 tx=0x7f94d0001b10 comp rx=0 tx=0).stop 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 -- 172.21.15.3:0/1495542555 shutdown_connections 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae49220 0x56276aee65c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae48e40 0x56276aef0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 --2- 172.21.15.3:0/1495542555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae477a0 0x56276aee0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 -- 172.21.15.3:0/1495542555 >> 172.21.15.3:0/1495542555 conn(0x56276ae45d40 msgr2=0x56276ae612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 -- 172.21.15.3:0/1495542555 shutdown_connections 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.505+0000 7f94da862f40 1 -- 172.21.15.3:0/1495542555 wait complete. 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.506+0000 7f94da862f40 1 Processor -- start 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.506+0000 7f94da862f40 1 -- start start 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae477a0 0x56276aee5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 0x56276aedff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae49220 0x56276aef0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56276ae44280 con 0x56276ae48e40 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56276ae43fa0 con 0x56276ae49220 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94da862f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56276ae44560 con 0x56276ae477a0 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 0x56276aedff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 0x56276aedff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45850/0 (socket says 172.21.15.3:45850) 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 -- 172.21.15.3:0/1226438616 learned_addr learned my addr 172.21.15.3:0/1226438616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d8400640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae49220 0x56276aef0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7a00640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae477a0 0x56276aee5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 -- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae477a0 msgr2=0x56276aee5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae477a0 0x56276aee5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 -- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae49220 msgr2=0x56276aef0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56276ae49220 0x56276aef0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7000640 1 -- 172.21.15.3:0/1226438616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276aeea3d0 con 0x56276ae48e40 2024-09-10T07:49:24.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.507+0000 7f94d7a00640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56276ae477a0 0x56276aee5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94d7000640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 0x56276aedff90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000f60 tx=0x7f94c400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94cfe00640 1 -- 172.21.15.3:0/1226438616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400f650 con 0x56276ae48e40 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94cfe00640 1 -- 172.21.15.3:0/1226438616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c400f7f0 con 0x56276ae48e40 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94cfe00640 1 -- 172.21.15.3:0/1226438616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400fbe0 con 0x56276ae48e40 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56276aeea6b0 con 0x56276ae48e40 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.508+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276aee55f0 con 0x56276ae48e40 2024-09-10T07:49:24.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.510+0000 7f94cfe00640 1 -- 172.21.15.3:0/1226438616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94c4017400 con 0x56276ae48e40 2024-09-10T07:49:24.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.511+0000 7f94cfe00640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94b806a810 0x7f94b806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.511+0000 7f94cfe00640 1 -- 172.21.15.3:0/1226438616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f94c408a2a0 con 0x56276ae48e40 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.511+0000 7f94d7a00640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94b806a810 0x7f94b806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.511+0000 7f94da862f40 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x56276aee9f40 0x56276aee8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.511+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 6.5 6:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x56276aeed0d0 con 0x56276aee9f40 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.512+0000 7f94d8400640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x56276aee9f40 0x56276aee8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.512+0000 7f94d7a00640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94b806a810 0x7f94b806ccd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94c8056000 tx=0x7f94c8050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.512+0000 7f94d8400640 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x56276aee9f40 0x56276aee8460 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94d8400640 1 -- 172.21.15.3:0/1226438616 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v52'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f94d000f070 con 0x56276aee9f40 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x56276aee9f40 msgr2=0x56276aee8460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x56276aee9f40 0x56276aee8460 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94b806a810 msgr2=0x7f94b806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f94b806a810 0x7f94b806ccd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94c8056000 tx=0x7f94c8050000 comp rx=0 tx=0).stop 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 msgr2=0x56276aedff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 --2- 172.21.15.3:0/1226438616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56276ae48e40 0x56276aedff90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000f60 tx=0x7f94c400b6c0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94d8400640 1 -- 172.21.15.3:0/1226438616 reap_dead start 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 shutdown_connections 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 >> 172.21.15.3:0/1226438616 conn(0x56276ae45d40 msgr2=0x56276ae611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 shutdown_connections 2024-09-10T07:49:24.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.514+0000 7f94da862f40 1 -- 172.21.15.3:0/1226438616 wait complete. 2024-09-10T07:49:24.518 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-10T07:49:24.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 Processor -- start 2024-09-10T07:49:24.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 -- start start 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d8b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d88d99620 con 0x560d88d0c130 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d88c68b30 con 0x560d88cf3e40 2024-09-10T07:49:24.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.596+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d88d99900 con 0x560d88cf4220 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c0c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45864/0 (socket says 172.21.15.3:45864) 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 -- 172.21.15.3:0/4064601443 learned_addr learned my addr 172.21.15.3:0/4064601443 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51bbe00640 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d8b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 -- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 msgr2=0x560d88d8b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d8b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 -- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 msgr2=0x560d88d92060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d92060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.597+0000 7f51c1600640 1 -- 172.21.15.3:0/4064601443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d88d8b9f0 con 0x560d88d0c130 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c1600640 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0015f0 tx=0x7f51bc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3da3184846bd310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51baa00640 1 -- 172.21.15.3:0/4064601443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc004520 con 0x560d88d0c130 2024-09-10T07:49:24.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51baa00640 1 -- 172.21.15.3:0/4064601443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc0046c0 con 0x560d88d0c130 2024-09-10T07:49:24.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51baa00640 1 -- 172.21.15.3:0/4064601443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc004a90 con 0x560d88d0c130 2024-09-10T07:49:24.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 -- 172.21.15.3:0/4064601443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 msgr2=0x560d88e28860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0015f0 tx=0x7f51bc001ab0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 -- 172.21.15.3:0/4064601443 shutdown_connections 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88e28860 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/4064601443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d92060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 -- 172.21.15.3:0/4064601443 >> 172.21.15.3:0/4064601443 conn(0x560d88cf0d40 msgr2=0x560d88cc17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.598+0000 7f51c3a29f40 1 -- 172.21.15.3:0/4064601443 shutdown_connections 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.599+0000 7f51c3a29f40 1 -- 172.21.15.3:0/4064601443 wait complete. 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.599+0000 7f51c3a29f40 1 Processor -- start 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.599+0000 7f51c3a29f40 1 -- start start 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d85ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d0d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88d0dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d88d90410 con 0x560d88d0c130 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560d88d0bcd0 con 0x560d88cf3e40 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c3a29f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560d88d9bd90 con 0x560d88cf4220 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88d0dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88d0dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45870/0 (socket says 172.21.15.3:45870) 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.600+0000 7f51c1600640 1 -- 172.21.15.3:0/3576840214 learned_addr learned my addr 172.21.15.3:0/3576840214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c0c00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d85ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51bbe00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d0d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 -- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 msgr2=0x560d88d0d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560d88cf4220 0x560d88d0d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 -- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 msgr2=0x560d88d85ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560d88cf3e40 0x560d88d85ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 -- 172.21.15.3:0/3576840214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d88d8da30 con 0x560d88d0c130 2024-09-10T07:49:24.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51c1600640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88d0dcc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0012e0 tx=0x7f51bc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.601+0000 7f51b9600640 1 -- 172.21.15.3:0/3576840214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc018040 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.602+0000 7f51b9600640 1 -- 172.21.15.3:0/3576840214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc022070 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.602+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d88d9aac0 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.602+0000 7f51b9600640 1 -- 172.21.15.3:0/3576840214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc013620 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.602+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d88d8cc70 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.603+0000 7f51b9600640 1 -- 172.21.15.3:0/3576840214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51bc010070 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.604+0000 7f51b9600640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51a006a810 0x7f51a006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.604+0000 7f51b9600640 1 -- 172.21.15.3:0/3576840214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f51bc08aeb0 con 0x560d88d0c130 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.604+0000 7f51c0c00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51a006a810 0x7f51a006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.604+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x560d88d8ff20 0x560d88e2f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.604+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 6.1d 6:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x560d88e2fb60 con 0x560d88d8ff20 2024-09-10T07:49:24.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.605+0000 7f51bbe00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x560d88d8ff20 0x560d88e2f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.605+0000 7f51c0c00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51a006a810 0x7f51a006ccd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f51b4056000 tx=0x7f51b4050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.606+0000 7f51bbe00640 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x560d88d8ff20 0x560d88e2f620 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51bbe00640 1 -- 172.21.15.3:0/3576840214 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v52'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f51ac009b60 con 0x560d88d8ff20 2024-09-10T07:49:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x560d88d8ff20 msgr2=0x560d88e2f620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x560d88d8ff20 0x560d88e2f620 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51a006a810 msgr2=0x7f51a006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51a006a810 0x7f51a006ccd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f51b4056000 tx=0x7f51b4050000 comp rx=0 tx=0).stop 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 msgr2=0x560d88d0dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c3a29f40 1 --2- 172.21.15.3:0/3576840214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88d0c130 0x560d88d0dcc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0012e0 tx=0x7f51bc001250 comp rx=0 tx=0).stop 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.608+0000 7f51c1600640 1 -- 172.21.15.3:0/3576840214 reap_dead start 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.609+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 shutdown_connections 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.609+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 >> 172.21.15.3:0/3576840214 conn(0x560d88cf0d40 msgr2=0x560d88cf33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.609+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 shutdown_connections 2024-09-10T07:49:24.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.609+0000 7f51c3a29f40 1 -- 172.21.15.3:0/3576840214 wait complete. 2024-09-10T07:49:24.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:24.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2024-09-10T07:49:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.773+0000 7fa830600640 1 Processor -- start 2024-09-10T07:49:24.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.773+0000 7fa830600640 1 -- start start 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.774+0000 7fa830600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8281591b0 0x7fa82815c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.774+0000 7fa830600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8280b3e10 0x7fa8280b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.774+0000 7fa830600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8280cd8d0 con 0x7fa8280ccc50 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.774+0000 7fa830600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa82815b900 con 0x7fa8281591b0 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.774+0000 7fa830600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa8280cdbb0 con 0x7fa8280b3e10 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82dc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8281591b0 0x7fa82815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82e600640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa8280ccc50 0x7fa8280cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54892/0 (socket says 172.21.15.3:54892) 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82e600640 1 -- 172.21.15.3:0/2132979033 learned_addr learned my addr 172.21.15.3:0/2132979033 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82d200640 1 --2- 172.21.15.3:0/2132979033 >> v2:172.21.15.70:3300/0 conn(0x7fa8280b3e10 0x7fa8280b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82dc00640 1 -- 172.21.15.3:0/2132979033 >> v2:172.21.15.70:3300/0 conn(0x7fa8280b3e10 msgr2=0x7fa8280b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82dc00640 1 --2- 172.21.15.3:0/2132979033 >> v2:172.21.15.70:3300/0 conn(0x7fa8280b3e10 0x7fa8280b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82dc00640 1 -- 172.21.15.3:0/2132979033 >> v1:172.21.15.3:6789/0 conn(0x7fa8280ccc50 legacy=0x7fa8280cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa82dc00640 1 -- 172.21.15.3:0/2132979033 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82815bb20 con 0x7fa8281591b0 2024-09-10T07:49:24.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.775+0000 7fa827e00640 1 -- 172.21.15.3:0/2132979033 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848624772 0 0) 0x7fa8280cd8d0 con 0x7fa8280ccc50 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.776+0000 7fa82dc00640 1 --2- 172.21.15.3:0/2132979033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281591b0 0x7fa82815c600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8280631e0 tx=0x7fa81c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0ab3059eb9f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.776+0000 7fa827e00640 1 -- 172.21.15.3:0/2132979033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c037070 con 0x7fa8281591b0 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.776+0000 7fa827e00640 1 -- 172.21.15.3:0/2132979033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c032440 con 0x7fa8281591b0 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.776+0000 7fa827e00640 1 -- 172.21.15.3:0/2132979033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c032810 con 0x7fa8281591b0 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 -- 172.21.15.3:0/2132979033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281591b0 msgr2=0x7fa82815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 --2- 172.21.15.3:0/2132979033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281591b0 0x7fa82815c600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8280631e0 tx=0x7fa81c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 -- 172.21.15.3:0/2132979033 shutdown_connections 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 --2- 172.21.15.3:0/2132979033 >> v2:172.21.15.70:3300/0 conn(0x7fa8280b3e10 0x7fa8280b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 --2- 172.21.15.3:0/2132979033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281591b0 0x7fa82815c600 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 -- 172.21.15.3:0/2132979033 >> 172.21.15.3:0/2132979033 conn(0x7fa828062cc0 msgr2=0x7fa82806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 -- 172.21.15.3:0/2132979033 shutdown_connections 2024-09-10T07:49:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.777+0000 7fa830600640 1 -- 172.21.15.3:0/2132979033 wait complete. 2024-09-10T07:49:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.779+0000 7fa830600640 1 Processor -- start 2024-09-10T07:49:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.779+0000 7fa830600640 1 -- start start 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8280ccc50 0x7fa828156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa828156f80 0x7fa8280ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280ce410 con 0x7fa8280b3e10 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280ce290 con 0x7fa828156f80 2024-09-10T07:49:24.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa830600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa8280ce590 con 0x7fa8280ccc50 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45894/0 (socket says 172.21.15.3:45894) 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 -- 172.21.15.3:0/377910224 learned_addr learned my addr 172.21.15.3:0/377910224 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82d200640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8280ccc50 0x7fa828156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82e600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa828156f80 0x7fa8280ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 -- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8280ccc50 msgr2=0x7fa828156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8280ccc50 0x7fa828156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 -- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa828156f80 msgr2=0x7fa8280ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa828156f80 0x7fa8280ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.780+0000 7fa82dc00640 1 -- 172.21.15.3:0/377910224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa828150970 con 0x7fa8280b3e10 2024-09-10T07:49:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa82dc00640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009890 tx=0x7fa81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa825600640 1 -- 172.21.15.3:0/377910224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c037040 con 0x7fa8280b3e10 2024-09-10T07:49:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa825600640 1 -- 172.21.15.3:0/377910224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c042070 con 0x7fa8280b3e10 2024-09-10T07:49:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa825600640 1 -- 172.21.15.3:0/377910224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c030960 con 0x7fa8280b3e10 2024-09-10T07:49:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa828150c50 con 0x7fa8280b3e10 2024-09-10T07:49:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.781+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa828151210 con 0x7fa8280b3e10 2024-09-10T07:49:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.783+0000 7fa825600640 1 -- 172.21.15.3:0/377910224 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa81c030b00 con 0x7fa8280b3e10 2024-09-10T07:49:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.785+0000 7fa825600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.785+0000 7fa825600640 1 -- 172.21.15.3:0/377910224 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fa81c0ab8e0 con 0x7fa8280b3e10 2024-09-10T07:49:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.785+0000 7fa82d200640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.785+0000 7fa833471740 1 Processor -- start 2024-09-10T07:49:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 -- start start 2024-09-10T07:49:24.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa82d200640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 0x7fa80806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa8280d0ca0 tx=0x7fa814008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb48be0 0x561aecb48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 0x561aecb1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561aeca819c0 con 0x561aecb635a0 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561aeca298f0 con 0x561aecb1c540 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.786+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561aeca7d980 con 0x561aecb48be0 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45910/0 (socket says 172.21.15.3:45910) 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 -- 172.21.15.3:0/270825679 learned_addr learned my addr 172.21.15.3:0/270825679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82dc00640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb48be0 0x561aecb48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82e600640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 0x561aecb1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 -- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb48be0 msgr2=0x561aecb48fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb48be0 0x561aecb48fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 -- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 msgr2=0x561aecb1ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 0x561aecb1ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 -- 172.21.15.3:0/270825679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aec85ca50 con 0x561aecb635a0 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82e600640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 0x561aecb1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:24.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.787+0000 7fa82d200640 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa814011d00 tx=0x7fa814014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bec376552d384e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa81a000640 1 -- 172.21.15.3:0/270825679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814011d30 con 0x561aecb635a0 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa81a000640 1 -- 172.21.15.3:0/270825679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa814026070 con 0x561aecb635a0 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa81a000640 1 -- 172.21.15.3:0/270825679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814021640 con 0x561aecb635a0 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 -- 172.21.15.3:0/270825679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 msgr2=0x561aecb619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa814011d00 tx=0x7fa814014000 comp rx=0 tx=0).stop 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 -- 172.21.15.3:0/270825679 shutdown_connections 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb1c540 0x561aecb1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb635a0 0x561aecb619e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 --2- 172.21.15.3:0/270825679 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb48be0 0x561aecb48fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 -- 172.21.15.3:0/270825679 >> 172.21.15.3:0/270825679 conn(0x561aecb1b7d0 msgr2=0x561aecb639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 -- 172.21.15.3:0/270825679 shutdown_connections 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.788+0000 7fa833471740 1 -- 172.21.15.3:0/270825679 wait complete. 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.789+0000 7fa833471740 1 -- start start 2024-09-10T07:49:24.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.789+0000 7fa833471740 1 Processor -- start 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561aecb49500 con 0x561aecb48be0 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561aeca3cb40 con 0x561aecb635a0 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561aecb1ac00 con 0x561aecb5e1b0 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.790+0000 7fa833471740 10 client.0 _ll_register_callbacks cb 0x561aeca3ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45922/0 (socket says 172.21.15.3:45922) 2024-09-10T07:49:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 -- 172.21.15.3:0/322651676 learned_addr learned my addr 172.21.15.3:0/322651676 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82e600640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82d200640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 -- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 msgr2=0x561aecb5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 -- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 msgr2=0x561aecb5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 -- 172.21.15.3:0/322651676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aec7543e0 con 0x561aecb48be0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82e600640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82d200640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa82dc00640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0091d0 tx=0x7fa81c032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5b451cbac3a86d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.791+0000 7fa80ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0a8060 con 0x561aecb48be0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fa81c0a9940 con 0x561aecb48be0 2024-09-10T07:49:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa814026070 con 0x561aecb48be0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa81c096050 con 0x561aecb48be0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 10 client.5808 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x561aec8bd1e0 con 0x561aecb48be0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa814026070 con 0x561aecb48be0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7fa81c03a070 con 0x561aecb48be0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 10 client.5808 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 10 client.5808 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 20 client.5808 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb61f80 0x561aecb5af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 4 client.5808 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.792+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aeca8f0f0 con 0x561aecb61f80 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.793+0000 7fa82d200640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb61f80 0x561aecb5af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.793+0000 7fa80ea00640 10 client.5808 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.793+0000 7fa82d200640 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb61f80 0x561aecb5af10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9afedd8b3bd227 server_cookie=24950de4593258ba in_seq=0 out_seq=0 2024-09-10T07:49:24.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa80ea00640 1 -- 172.21.15.3:0/322651676 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561aeca8f0f0 con 0x561aecb61f80 2024-09-10T07:49:24.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa80ea00640 10 client.5808 handle_command_reply: tid=0 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 client.5808 shutdown 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 msgr2=0x561aecb595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0091d0 tx=0x7fa81c032440 comp rx=0 tx=0).stop 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 shutdown_connections 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb61f80 0x561aecb5af10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa80ea00640 0 client.5808 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 --2- 172.21.15.3:0/322651676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb595e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 >> 172.21.15.3:0/322651676 conn(0x561aecb1c540 msgr2=0x561aecb1ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 shutdown_connections 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.794+0000 7fa833471740 1 -- 172.21.15.3:0/322651676 wait complete. 2024-09-10T07:49:24.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.795+0000 7fa833471740 20 client.5808 trim_cache size 0 max 16384 2024-09-10T07:49:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.813+0000 7fa833471740 1 Processor -- start 2024-09-10T07:49:24.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.813+0000 7fa833471740 1 -- start start 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa828200b40 con 0x561aecb48be0 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281ffd80 con 0x561aecb635a0 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa828200410 con 0x561aecb5e1b0 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45936/0 (socket says 172.21.15.3:45936) 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 -- 172.21.15.3:0/306987555 learned_addr learned my addr 172.21.15.3:0/306987555 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 -- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 msgr2=0x561aecb5fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 -- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 msgr2=0x561aecb44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.814+0000 7fa82dc00640 1 -- 172.21.15.3:0/306987555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aecb49330 con 0x561aecb48be0 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa82dc00640 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef85c44d3021ed15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa819600640 1 -- 172.21.15.3:0/306987555 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09c020 con 0x561aecb48be0 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa819600640 1 -- 172.21.15.3:0/306987555 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c082640 con 0x561aecb48be0 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa819600640 1 -- 172.21.15.3:0/306987555 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c091030 con 0x561aecb48be0 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 -- 172.21.15.3:0/306987555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 msgr2=0x561aecb47d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c07c000 comp rx=0 tx=0).stop 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 -- 172.21.15.3:0/306987555 shutdown_connections 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb635a0 0x561aecb44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb5e1b0 0x561aecb5fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 --2- 172.21.15.3:0/306987555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb48be0 0x561aecb47d10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 -- 172.21.15.3:0/306987555 >> 172.21.15.3:0/306987555 conn(0x561aecb1c540 msgr2=0x561aecb610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.815+0000 7fa833471740 1 -- 172.21.15.3:0/306987555 shutdown_connections 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.816+0000 7fa833471740 1 -- 172.21.15.3:0/306987555 wait complete. 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.816+0000 7fa833471740 1 -- start start 2024-09-10T07:49:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.816+0000 7fa833471740 1 Processor -- start 2024-09-10T07:49:24.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb48be0 0x561aecb870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb635a0 0x561aecb851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281fee70 con 0x561aecb5e1b0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561aecb5d9b0 con 0x561aecb48be0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561aecbe68c0 con 0x561aecb635a0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa833471740 10 client.0 _ll_register_callbacks cb 0x7fa81c006c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa82d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa82d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45946/0 (socket says 172.21.15.3:45946) 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.817+0000 7fa82d200640 1 -- 172.21.15.3:0/3971958400 learned_addr learned my addr 172.21.15.3:0/3971958400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82e600640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb635a0 0x561aecb851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82dc00640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb48be0 0x561aecb870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 -- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb635a0 msgr2=0x561aecb851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb635a0 0x561aecb851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 -- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb48be0 msgr2=0x561aecb870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:24.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb48be0 0x561aecb870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 -- 172.21.15.3:0/3971958400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aec72d4a0 con 0x561aecb5e1b0 2024-09-10T07:49:24.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa82d200640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa814023040 tx=0x7fa814009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d2266c2269bedf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa80ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814011e30 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fa814036450 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.818+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81c082640 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa814046050 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 10 client.5814 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x561aecb20860 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c082640 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7fa814049020 con 0x561aecb5e1b0 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 10 client.5814 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 10 client.5814 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:24.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 20 client.5814 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb5af90 0x561aecbe5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 4 client.5814 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x561aeca89440 con 0x561aecb5af90 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.819+0000 7fa82dc00640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb5af90 0x561aecbe5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.820+0000 7fa82dc00640 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb5af90 0x561aecbe5910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9206a8431e094db2 server_cookie=d205ec023d7ee825 in_seq=0 out_seq=0 2024-09-10T07:49:24.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.820+0000 7fa80ea00640 10 client.5814 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:24.827 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-09-10T07:49:24.823+0000 7f9200e00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1) n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55784161f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.827 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-09-10T07:49:24.823+0000 7f9200e00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/AsLsXsFsx@3},l=5138 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55784161eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa80ea00640 1 -- 172.21.15.3:0/3971958400 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561aeca89440 con 0x561aecb5af90 2024-09-10T07:49:24.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa80ea00640 10 client.5814 handle_command_reply: tid=0 2024-09-10T07:49:24.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 client.5814 shutdown 2024-09-10T07:49:24.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 msgr2=0x561aecbe1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa814023040 tx=0x7fa814009190 comp rx=0 tx=0).stop 2024-09-10T07:49:24.828 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:49:24.821+0000 7f685dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1) n(v3 rc2024-09-10T07:49:08.659495+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x557178f19b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:49:08.659495+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 shutdown_connections 2024-09-10T07:49:24.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x561aecb5af90 0x561aecbe5910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561aecb635a0 0x561aecb851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561aecb5e1b0 0x561aecbe1c40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 --2- 172.21.15.3:0/3971958400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561aecb48be0 0x561aecb870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 >> 172.21.15.3:0/3971958400 conn(0x561aecb1c540 msgr2=0x561aecb5fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa80ea00640 0 client.5814 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.821+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 shutdown_connections 2024-09-10T07:49:24.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.822+0000 7fa833471740 1 -- 172.21.15.3:0/3971958400 wait complete. 2024-09-10T07:49:24.829 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:49:24.822+0000 7f48cf200640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1) n(v2 rc2024-09-10T07:49:08.659495+0000 4=0+4) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5639f92b4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:49:08.659495+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.822+0000 7fa833471740 20 client.5814 trim_cache size 0 max 16384 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "7158f455-bbda-49e6-8d65-b9654e3e4ce8", 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-10T07:49:24.830 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:24.831 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:49:24.823+0000 7f6865e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1) n(v4 rc2024-09-10T07:49:08.659495+0000 5=0+5) (inest sync->mix g=1,2) caps={5138=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x557178f1a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:49:08.659495+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 msgr2=0x7fa80806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 0x7fa80806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa8280d0ca0 tx=0x7fa814008040 comp rx=0 tx=0).stop 2024-09-10T07:49:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 msgr2=0x7fa82814f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:24.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009890 tx=0x7fa81c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 shutdown_connections 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa828156f80 0x7fa8280ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8280ccc50 0x7fa828156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 --2- 172.21.15.3:0/377910224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280b3e10 0x7fa82814f880 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 >> 172.21.15.3:0/377910224 conn(0x7fa828062cc0 msgr2=0x7fa8280cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 shutdown_connections 2024-09-10T07:49:24.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.823+0000 7fa830600640 1 -- 172.21.15.3:0/377910224 wait complete. 2024-09-10T07:49:24.832 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:49:24.824+0000 7f48d7400640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1) n(v1 rc2024-09-10T07:49:08.659495+0000 3=0+3) (inest sync->mix g=0,2) caps={5138=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x5639f92b4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:49:08.659495+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:49:24.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:24.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub abort 2024-09-10T07:49:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.997+0000 7f7460a00640 1 Processor -- start 2024-09-10T07:49:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.997+0000 7f7460a00640 1 -- start start 2024-09-10T07:49:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.998+0000 7f7460a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c1451d0 0x7f745c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7460a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f745c145730 0x7f745c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7460a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c0cd900 con 0x7f745c0cf270 2024-09-10T07:49:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7460a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f745c06d7c0 con 0x7f745c145730 2024-09-10T07:49:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7460a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f745c0cdbe0 con 0x7f745c1451d0 2024-09-10T07:49:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7451400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f745c145730 0x7f745c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f745a800640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f745c0cf270 0x7f745c0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:33024/0 (socket says 172.21.15.3:33024) 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f745a800640 1 -- 172.21.15.3:0/2073677146 learned_addr learned my addr 172.21.15.3:0/2073677146 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:24.999+0000 7f7459e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c1451d0 0x7f745c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.000+0000 7f7459400640 1 -- 172.21.15.3:0/2073677146 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 408783814 0 0) 0x7f745c0cd900 con 0x7f745c0cf270 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.000+0000 7f7451400640 1 -- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c1451d0 msgr2=0x7f745c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.000+0000 7f7451400640 1 --2- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c1451d0 0x7f745c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.000+0000 7f7451400640 1 -- 172.21.15.3:0/2073677146 >> v1:172.21.15.70:6789/0 conn(0x7f745c0cf270 legacy=0x7f745c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.000+0000 7f7451400640 1 -- 172.21.15.3:0/2073677146 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c06da10 con 0x7f745c145730 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7451400640 1 --2- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c0b3370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74440097a0 tx=0x7f744402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3056fafb7b93f415 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7459400640 1 -- 172.21.15.3:0/2073677146 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444037070 con 0x7f745c145730 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7459400640 1 -- 172.21.15.3:0/2073677146 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744402ae40 con 0x7f745c145730 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7459400640 1 -- 172.21.15.3:0/2073677146 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444032660 con 0x7f745c145730 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 -- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 msgr2=0x7f745c0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 --2- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c0b3370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74440097a0 tx=0x7f744402a0f0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 -- 172.21.15.3:0/2073677146 shutdown_connections 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 --2- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c0b3370 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 --2- 172.21.15.3:0/2073677146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c1451d0 0x7f745c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.001+0000 7f7460a00640 1 -- 172.21.15.3:0/2073677146 >> 172.21.15.3:0/2073677146 conn(0x7f745c0b5cb0 msgr2=0x7f745c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.002+0000 7f7460a00640 1 -- 172.21.15.3:0/2073677146 shutdown_connections 2024-09-10T07:49:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.002+0000 7f7460a00640 1 -- 172.21.15.3:0/2073677146 wait complete. 2024-09-10T07:49:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.003+0000 7f7460a00640 1 Processor -- start 2024-09-10T07:49:25.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.003+0000 7f7460a00640 1 -- start start 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c0cf270 0x7f745c140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f745c06df30 con 0x7f745c1451d0 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f745c0b38b0 con 0x7f745c145730 2024-09-10T07:49:25.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7460a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f745c06e0b0 con 0x7f745c0cf270 2024-09-10T07:49:25.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.004+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45956/0 (socket says 172.21.15.3:45956) 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 -- 172.21.15.3:0/1021673371 learned_addr learned my addr 172.21.15.3:0/1021673371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7459e00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c0cf270 0x7f745c140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f745a800640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 -- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c0cf270 msgr2=0x7f745c140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c0cf270 0x7f745c140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 -- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 msgr2=0x7f745c072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 -- 172.21.15.3:0/1021673371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c0b2f20 con 0x7f745c1451d0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7451400640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f74440097a0 tx=0x7f74440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.005+0000 7f7452a00640 1 -- 172.21.15.3:0/1021673371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444037040 con 0x7f745c1451d0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.006+0000 7f7452a00640 1 -- 172.21.15.3:0/1021673371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7444004030 con 0x7f745c1451d0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.006+0000 7f7452a00640 1 -- 172.21.15.3:0/1021673371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444004420 con 0x7f745c1451d0 2024-09-10T07:49:25.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.006+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c0b1e80 con 0x7f745c1451d0 2024-09-10T07:49:25.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.006+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c0b2440 con 0x7f745c1451d0 2024-09-10T07:49:25.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.008+0000 7f7452a00640 1 -- 172.21.15.3:0/1021673371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7444003610 con 0x7f745c1451d0 2024-09-10T07:49:25.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.009+0000 7f7452a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 0x7f743806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.009+0000 7f7452a00640 1 -- 172.21.15.3:0/1021673371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f74440ab920 con 0x7f745c1451d0 2024-09-10T07:49:25.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.009+0000 7f7459e00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 0x7f743806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.010+0000 7f7463a0b740 1 Processor -- start 2024-09-10T07:49:25.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.010+0000 7f7459e00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 0x7f743806e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f744c009b40 tx=0x7f744c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.010+0000 7f7463a0b740 1 -- start start 2024-09-10T07:49:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bdd63070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5622bdd2fcf0 con 0x5622bdd66240 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5622bdccb7f0 con 0x5622bdd6a210 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5622bdc73800 con 0x5622bdd68a80 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45964/0 (socket says 172.21.15.3:45964) 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 -- 172.21.15.3:0/524265112 learned_addr learned my addr 172.21.15.3:0/524265112 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f745a800640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bdd63070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7459e00640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 -- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 msgr2=0x5622bddae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 -- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 msgr2=0x5622bdd63070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bdd63070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7451400640 1 -- 172.21.15.3:0/524265112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bdaa87d0 con 0x5622bdd66240 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f745a800640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bdd63070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.011+0000 7f7459e00640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7451400640 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f74440a9df0 tx=0x7f7444079880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4faeb7d5740b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f744a600640 1 -- 172.21.15.3:0/524265112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440827b0 con 0x5622bdd66240 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f744a600640 1 -- 172.21.15.3:0/524265112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440a8070 con 0x5622bdd66240 2024-09-10T07:49:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f744a600640 1 -- 172.21.15.3:0/524265112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444077f80 con 0x5622bdd66240 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 -- 172.21.15.3:0/524265112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 msgr2=0x5622bdd68e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f74440a9df0 tx=0x7f7444079880 comp rx=0 tx=0).stop 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 -- 172.21.15.3:0/524265112 shutdown_connections 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bdd63070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd66240 0x5622bdd68e60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 --2- 172.21.15.3:0/524265112 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.012+0000 7f7463a0b740 1 -- 172.21.15.3:0/524265112 >> 172.21.15.3:0/524265112 conn(0x5622bdd65240 msgr2=0x5622bddad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.013+0000 7f7463a0b740 1 -- 172.21.15.3:0/524265112 shutdown_connections 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.013+0000 7f7463a0b740 1 -- 172.21.15.3:0/524265112 wait complete. 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.013+0000 7f7463a0b740 1 -- start start 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.013+0000 7f7463a0b740 1 Processor -- start 2024-09-10T07:49:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bddaad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd7bb10 0x5622bdd63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5622bdc54750 con 0x5622bdd68a80 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5622bdcc77b0 con 0x5622bdd6a210 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5622bdccb7f0 con 0x5622bdd7bb10 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.014+0000 7f7463a0b740 10 client.0 _ll_register_callbacks cb 0x5622bd9ddbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45976/0 (socket says 172.21.15.3:45976) 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 -- 172.21.15.3:0/3992288607 learned_addr learned my addr 172.21.15.3:0/3992288607 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7451400640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bddaad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f745a800640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd7bb10 0x5622bdd63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 -- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd7bb10 msgr2=0x5622bdd63310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd7bb10 0x5622bdd63310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 -- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 msgr2=0x5622bddaad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bddaad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 -- 172.21.15.3:0/3992288607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bd99e0f0 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f7459e00640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f744c0045c0 tx=0x7f744c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff45755a552513b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.015+0000 7f743ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c01e050 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74440a8070 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f744c023440 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f744c030050 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 10 client.5823 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5622bdb08f60 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440a8070 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f744c033020 con 0x5622bdd68a80 2024-09-10T07:49:25.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 10 client.5823 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 10 client.5823 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 20 client.5823 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd92bd0 0x5622bdd92fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 4 client.5823 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.016+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622bdcd8f20 con 0x5622bdd92bd0 2024-09-10T07:49:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.017+0000 7f7451400640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd92bd0 0x5622bdd92fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.017+0000 7f7451400640 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd92bd0 0x5622bdd92fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6e0b10877b1045 server_cookie=e9a86d6ebab2ef40 in_seq=0 out_seq=0 2024-09-10T07:49:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.017+0000 7f743ea00640 10 client.5823 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f743ea00640 1 -- 172.21.15.3:0/3992288607 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622bdcd8f20 con 0x5622bdd92bd0 2024-09-10T07:49:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f743ea00640 10 client.5823 handle_command_reply: tid=0 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 client.5823 shutdown 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 msgr2=0x5622bdd7dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f744c0045c0 tx=0x7f744c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 shutdown_connections 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd92bd0 0x5622bdd92fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd7bb10 0x5622bdd63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd6a210 0x5622bddaad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 --2- 172.21.15.3:0/3992288607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bdd7dba0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f743ea00640 0 client.5823 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 >> 172.21.15.3:0/3992288607 conn(0x5622bdd66240 msgr2=0x5622bddab770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 shutdown_connections 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 1 -- 172.21.15.3:0/3992288607 wait complete. 2024-09-10T07:49:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.018+0000 7f7463a0b740 20 client.5823 trim_cache size 0 max 16384 2024-09-10T07:49:25.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.036+0000 7f7463a0b740 1 Processor -- start 2024-09-10T07:49:25.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.036+0000 7f7463a0b740 1 -- start start 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bdd7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f745c200c70 con 0x5622bdd6a210 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ffe70 con 0x5622bdd7bb10 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f745c200540 con 0x5622bdd68a80 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45978/0 (socket says 172.21.15.3:45978) 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 -- 172.21.15.3:0/2454472474 learned_addr learned my addr 172.21.15.3:0/2454472474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 -- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 msgr2=0x5622bddab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 -- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 msgr2=0x5622bdd7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bdd7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 -- 172.21.15.3:0/2454472474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bd9771b0 con 0x5622bdd6a210 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.037+0000 7f7451400640 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f74440a9dc0 tx=0x7f7444090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80202290fafc3e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7449c00640 1 -- 172.21.15.3:0/2454472474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744409d070 con 0x5622bdd6a210 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7449c00640 1 -- 172.21.15.3:0/2454472474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440a8040 con 0x5622bdd6a210 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7449c00640 1 -- 172.21.15.3:0/2454472474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444077f80 con 0x5622bdd6a210 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 -- 172.21.15.3:0/2454472474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 msgr2=0x5622bddabcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f74440a9dc0 tx=0x7f7444090000 comp rx=0 tx=0).stop 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 -- 172.21.15.3:0/2454472474 shutdown_connections 2024-09-10T07:49:25.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bdd7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd6a210 0x5622bddabcb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2454472474 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd68a80 0x5622bddab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 -- 172.21.15.3:0/2454472474 >> 172.21.15.3:0/2454472474 conn(0x5622bdd66240 msgr2=0x5622bddae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 -- 172.21.15.3:0/2454472474 shutdown_connections 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.038+0000 7f7463a0b740 1 -- 172.21.15.3:0/2454472474 wait complete. 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.039+0000 7f7463a0b740 1 -- start start 2024-09-10T07:49:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.039+0000 7f7463a0b740 1 Processor -- start 2024-09-10T07:49:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd6a210 0x5622bddbc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bddbc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5622bdd2fcf0 con 0x5622bdd68a80 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1f4a70 con 0x5622bdd7bb10 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5622bdd8dde0 con 0x5622bdd6a210 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7463a0b740 10 client.0 _ll_register_callbacks cb 0x7f744c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7459e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7459e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45992/0 (socket says 172.21.15.3:45992) 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.040+0000 7f7459e00640 1 -- 172.21.15.3:0/2628416416 learned_addr learned my addr 172.21.15.3:0/2628416416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7451400640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd6a210 0x5622bddbc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f745a800640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bddbc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 -- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd6a210 msgr2=0x5622bddbc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd6a210 0x5622bddbc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 -- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 msgr2=0x5622bddbc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bddbc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 -- 172.21.15.3:0/2628416416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bdd7de00 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7459e00640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f744c006f30 tx=0x7f744c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c5c4ffecfbfffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f743ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c038070 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74440a8040 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.041+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f744c014650 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f744c043050 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 10 client.5829 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5622bddaea70 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440a8040 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f744c046020 con 0x5622bdd68a80 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 10 client.5829 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:25.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 10 client.5829 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 20 client.5829 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd63250 0x5622bdd7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 4 client.5829 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x5622bdcd3270 con 0x5622bdd63250 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.042+0000 7f7451400640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd63250 0x5622bdd7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7451400640 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd63250 0x5622bdd7a670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b14b4554c45f9c6 server_cookie=4d75bc82238059a1 in_seq=0 out_seq=0 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f743ea00640 10 client.5829 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f743ea00640 1 -- 172.21.15.3:0/2628416416 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622bdcd3270 con 0x5622bdd63250 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f743ea00640 10 client.5829 handle_command_reply: tid=0 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 client.5829 shutdown 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 msgr2=0x5622bde2da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f744c006f30 tx=0x7f744c016000 comp rx=0 tx=0).stop 2024-09-10T07:49:25.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 shutdown_connections 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x5622bdd63250 0x5622bdd7a670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5622bdd7bb10 0x5622bddbc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5622bdd6a210 0x5622bddbc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 --2- 172.21.15.3:0/2628416416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5622bdd68a80 0x5622bde2da30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f743ea00640 0 client.5829 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 >> 172.21.15.3:0/2628416416 conn(0x5622bdd66240 msgr2=0x5622bddab770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.043+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 shutdown_connections 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7463a0b740 1 -- 172.21.15.3:0/2628416416 wait complete. 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7463a0b740 20 client.5829 trim_cache size 0 max 16384 2024-09-10T07:49:25.047 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:25.048 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:49:25.048 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:49:25.048 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 msgr2=0x7f743806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 0x7f743806e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f744c009b40 tx=0x7f744c009190 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 msgr2=0x7f745c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f74440097a0 tx=0x7f74440046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 shutdown_connections 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f743806be30 0x7f743806e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f745c145730 0x7f745c072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f745c1451d0 0x7f745c072230 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 --2- 172.21.15.3:0/1021673371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f745c0cf270 0x7f745c140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.044+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 >> 172.21.15.3:0/1021673371 conn(0x7f745c0b5cb0 msgr2=0x7f745c0b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.045+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 shutdown_connections 2024-09-10T07:49:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.045+0000 7f7460a00640 1 -- 172.21.15.3:0/1021673371 wait complete. 2024-09-10T07:49:25.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:49:25.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-10T07:49:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.218+0000 7f9aea400640 1 Processor -- start 2024-09-10T07:49:25.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.218+0000 7f9aea400640 1 -- start start 2024-09-10T07:49:25.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.219+0000 7f9aea400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ae41451d0 0x7f9ae406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.219+0000 7f9aea400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ae40cf270 0x7f9ae40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.219+0000 7f9aea400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae40cd900 con 0x7f9ae4145730 2024-09-10T07:49:25.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.219+0000 7f9aea400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae406d7c0 con 0x7f9ae40cf270 2024-09-10T07:49:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.219+0000 7f9aea400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae40cdbe0 con 0x7f9ae41451d0 2024-09-10T07:49:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ae40cf270 0x7f9ae40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ae40cf270 0x7f9ae40cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45994/0 (socket says 172.21.15.3:45994) 2024-09-10T07:49:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 -- 172.21.15.3:0/1427011402 learned_addr learned my addr 172.21.15.3:0/1427011402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3400640 1 --2- 172.21.15.3:0/1427011402 >> v2:172.21.15.70:3300/0 conn(0x7f9ae41451d0 0x7f9ae406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 -- 172.21.15.3:0/1427011402 >> v2:172.21.15.70:3300/0 conn(0x7f9ae41451d0 msgr2=0x7f9ae406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/1427011402 >> v2:172.21.15.70:3300/0 conn(0x7f9ae41451d0 0x7f9ae406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 -- 172.21.15.3:0/1427011402 >> v1:172.21.15.70:6789/0 conn(0x7f9ae4145730 legacy=0x7f9ae40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.220+0000 7f9ae3e00640 1 -- 172.21.15.3:0/1427011402 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae406da10 con 0x7f9ae40cf270 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.221+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/1427011402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae40cf270 0x7f9ae40cf6d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40732d0 tx=0x7f9adc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fa742863d759f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.221+0000 7f9ae2000640 1 -- 172.21.15.3:0/1427011402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc036070 con 0x7f9ae40cf270 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.221+0000 7f9ae2000640 1 -- 172.21.15.3:0/1427011402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc031600 con 0x7f9ae40cf270 2024-09-10T07:49:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.221+0000 7f9ae2000640 1 -- 172.21.15.3:0/1427011402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0319d0 con 0x7f9ae40cf270 2024-09-10T07:49:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 -- 172.21.15.3:0/1427011402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae40cf270 msgr2=0x7f9ae40cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 --2- 172.21.15.3:0/1427011402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae40cf270 0x7f9ae40cf6d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40732d0 tx=0x7f9adc02a6a0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 -- 172.21.15.3:0/1427011402 shutdown_connections 2024-09-10T07:49:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 --2- 172.21.15.3:0/1427011402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae40cf270 0x7f9ae40cf6d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 --2- 172.21.15.3:0/1427011402 >> v2:172.21.15.70:3300/0 conn(0x7f9ae41451d0 0x7f9ae406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 -- 172.21.15.3:0/1427011402 >> 172.21.15.3:0/1427011402 conn(0x7f9ae40b5cb0 msgr2=0x7f9ae406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 -- 172.21.15.3:0/1427011402 shutdown_connections 2024-09-10T07:49:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.222+0000 7f9aea400640 1 -- 172.21.15.3:0/1427011402 wait complete. 2024-09-10T07:49:25.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.224+0000 7f9aea400640 1 Processor -- start 2024-09-10T07:49:25.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.224+0000 7f9aea400640 1 -- start start 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 0x7f9ae415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 0x7f9ae40b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae406df30 con 0x7f9ae41451d0 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40cfc10 con 0x7f9ae40cf270 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9aea400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae406e0b0 con 0x7f9ae4145730 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45998/0 (socket says 172.21.15.3:45998) 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9ae2a00640 1 -- 172.21.15.3:0/775943598 learned_addr learned my addr 172.21.15.3:0/775943598 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.225+0000 7f9ae3400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 0x7f9ae415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 0x7f9ae40b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 -- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 msgr2=0x7f9ae40b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 0x7f9ae40b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 -- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 msgr2=0x7f9ae415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 0x7f9ae415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 -- 172.21.15.3:0/775943598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40b1000 con 0x7f9ae41451d0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae3400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 0x7f9ae415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 0x7f9ae40b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c3f0 tx=0x7f9ad000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9adb400640 1 -- 172.21.15.3:0/775943598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00052d0 con 0x7f9ae41451d0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9adb400640 1 -- 172.21.15.3:0/775943598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad0005490 con 0x7f9ae41451d0 2024-09-10T07:49:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae40b12e0 con 0x7f9ae41451d0 2024-09-10T07:49:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9adb400640 1 -- 172.21.15.3:0/775943598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0007260 con 0x7f9ae41451d0 2024-09-10T07:49:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.226+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae40b18a0 con 0x7f9ae41451d0 2024-09-10T07:49:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.228+0000 7f9adb400640 1 -- 172.21.15.3:0/775943598 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ad0007400 con 0x7f9ae41451d0 2024-09-10T07:49:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.229+0000 7f9adb400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 0x7f9ac406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.229+0000 7f9adb400640 1 -- 172.21.15.3:0/775943598 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9ad008a470 con 0x7f9ae41451d0 2024-09-10T07:49:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.230+0000 7f9ae3400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 0x7f9ac406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.230+0000 7f9aed204740 1 Processor -- start 2024-09-10T07:49:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.230+0000 7f9ae3400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 0x7f9ac406e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009b90 tx=0x7f9ad4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.230+0000 7f9aed204740 1 -- start start 2024-09-10T07:49:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c92200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 0x5616d3c92740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5616d3baf660 con 0x5616d3c61420 2024-09-10T07:49:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5616d3b57680 con 0x5616d3c61800 2024-09-10T07:49:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5616d3bab620 con 0x5616d3c47480 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46004/0 (socket says 172.21.15.3:46004) 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.231+0000 7f9ae3400640 1 -- 172.21.15.3:0/2995911911 learned_addr learned my addr 172.21.15.3:0/2995911911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c92200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 0x5616d3c92740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 -- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 msgr2=0x5616d3c92740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 0x5616d3c92740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 -- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 msgr2=0x5616d3c92200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c92200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 -- 172.21.15.3:0/2995911911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d398c7d0 con 0x5616d3c61420 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 0x5616d3c92740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c92200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009510 tx=0x7f9ad4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e312765d15eb88f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9acfe00640 1 -- 172.21.15.3:0/2995911911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011e60 con 0x5616d3c61420 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9acfe00640 1 -- 172.21.15.3:0/2995911911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4026070 con 0x5616d3c61420 2024-09-10T07:49:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.232+0000 7f9acfe00640 1 -- 172.21.15.3:0/2995911911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4021640 con 0x5616d3c61420 2024-09-10T07:49:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 -- 172.21.15.3:0/2995911911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 msgr2=0x5616d3c91c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009510 tx=0x7f9ad4014000 comp rx=0 tx=0).stop 2024-09-10T07:49:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 -- 172.21.15.3:0/2995911911 shutdown_connections 2024-09-10T07:49:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c47480 0x5616d3c92740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c92200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 --2- 172.21.15.3:0/2995911911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c91c40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 -- 172.21.15.3:0/2995911911 >> 172.21.15.3:0/2995911911 conn(0x5616d3c48220 msgr2=0x5616d3c90840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 -- 172.21.15.3:0/2995911911 shutdown_connections 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.233+0000 7f9aed204740 1 -- 172.21.15.3:0/2995911911 wait complete. 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.234+0000 7f9aed204740 1 -- start start 2024-09-10T07:49:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.234+0000 7f9aed204740 1 Processor -- start 2024-09-10T07:49:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c4e2b0 0x5616d3c8af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5616d3c463e0 con 0x5616d3c61420 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5616d3b384c0 con 0x5616d3c61800 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5616d3c466c0 con 0x5616d3c4e2b0 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9aed204740 10 client.0 _ll_register_callbacks cb 0x5616d3ba7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46014/0 (socket says 172.21.15.3:46014) 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.235+0000 7f9ae3400640 1 -- 172.21.15.3:0/2626457303 learned_addr learned my addr 172.21.15.3:0/2626457303 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c4e2b0 0x5616d3c8af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 -- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c4e2b0 msgr2=0x5616d3c8af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c4e2b0 0x5616d3c8af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 -- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 msgr2=0x5616d3c8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 -- 172.21.15.3:0/2626457303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d38c23f0 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9ae3400640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4023040 tx=0x7f9ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4390d308a53c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9accc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011e60 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9ad4021d00 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.236+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad4026070 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ad403e050 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 10 client.5841 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5616d38820f0 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4026070 con 0x5616d3c61420 2024-09-10T07:49:25.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f9ad4041020 con 0x5616d3c61420 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 10 client.5841 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 10 client.5841 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 20 client.5841 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3ca0bb0 0x5616d3ca0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 4 client.5841 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d3bbcd90 con 0x5616d3ca0bb0 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.237+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3ca0bb0 0x5616d3ca0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.238+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3ca0bb0 0x5616d3ca0fd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=652ca23b95ed07b9 server_cookie=33258c267f92ee64 in_seq=0 out_seq=0 2024-09-10T07:49:25.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.238+0000 7f9accc00640 10 client.5841 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.238+0000 7f9accc00640 1 -- 172.21.15.3:0/2626457303 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616d3bbcd90 con 0x5616d3ca0bb0 2024-09-10T07:49:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.238+0000 7f9accc00640 10 client.5841 handle_command_reply: tid=0 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 client.5841 shutdown 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 msgr2=0x5616d3c75a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4023040 tx=0x7f9ad40046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 shutdown_connections 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3ca0bb0 0x5616d3ca0fd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c4e2b0 0x5616d3c8af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61800 0x5616d3c8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 --2- 172.21.15.3:0/2626457303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c61420 0x5616d3c75a20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 >> 172.21.15.3:0/2626457303 conn(0x5616d3c47480 msgr2=0x5616d3c907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9accc00640 0 client.5841 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 shutdown_connections 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 1 -- 172.21.15.3:0/2626457303 wait complete. 2024-09-10T07:49:25.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.239+0000 7f9aed204740 20 client.5841 trim_cache size 0 max 16384 2024-09-10T07:49:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.257+0000 7f9aed204740 1 Processor -- start 2024-09-10T07:49:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.257+0000 7f9aed204740 1 -- start start 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3cb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3cb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4200ec0 con 0x5616d3c4e2b0 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae42000c0 con 0x5616d3c61420 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.258+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae4200790 con 0x5616d3c61800 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46016/0 (socket says 172.21.15.3:46016) 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 -- 172.21.15.3:0/329858569 learned_addr learned my addr 172.21.15.3:0/329858569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3cb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3cb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 -- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 msgr2=0x5616d3cb38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3cb38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 -- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 msgr2=0x5616d3cb3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3cb3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 -- 172.21.15.3:0/329858569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d3c47990 con 0x5616d3c4e2b0 2024-09-10T07:49:25.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3cb3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3cb38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.259+0000 7f9ae3400640 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4015ca0 tx=0x7f9ad402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb884ea2d5a7e7da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9acf400640 1 -- 172.21.15.3:0/329858569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad403b070 con 0x5616d3c4e2b0 2024-09-10T07:49:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9acf400640 1 -- 172.21.15.3:0/329858569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4026040 con 0x5616d3c4e2b0 2024-09-10T07:49:25.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9acf400640 1 -- 172.21.15.3:0/329858569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4021750 con 0x5616d3c4e2b0 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 -- 172.21.15.3:0/329858569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 msgr2=0x5616d3c94040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4015ca0 tx=0x7f9ad402d070 comp rx=0 tx=0).stop 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 -- 172.21.15.3:0/329858569 shutdown_connections 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3cb38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3cb3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 --2- 172.21.15.3:0/329858569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3c94040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.260+0000 7f9aed204740 1 -- 172.21.15.3:0/329858569 >> 172.21.15.3:0/329858569 conn(0x5616d3c47480 msgr2=0x5616d3c4d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.261+0000 7f9aed204740 1 -- 172.21.15.3:0/329858569 shutdown_connections 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.261+0000 7f9aed204740 1 -- 172.21.15.3:0/329858569 wait complete. 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.261+0000 7f9aed204740 1 -- start start 2024-09-10T07:49:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.261+0000 7f9aed204740 1 Processor -- start 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3c4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3c4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5616d3c50250 con 0x5616d3c4e2b0 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae413f580 con 0x5616d3c61420 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5616d3c8fa90 con 0x5616d3c61800 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.262+0000 7f9aed204740 10 client.0 _ll_register_callbacks cb 0x7f9ad4011d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46024/0 (socket says 172.21.15.3:46024) 2024-09-10T07:49:25.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 -- 172.21.15.3:0/1186339286 learned_addr learned my addr 172.21.15.3:0/1186339286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3c4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3c4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 -- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 msgr2=0x5616d3c4dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3c4dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 -- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 msgr2=0x5616d3c4d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3c4d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 -- 172.21.15.3:0/1186339286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d3d0f000 con 0x5616d3c4e2b0 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3e00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3c4dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3c4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9ae3400640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009770 tx=0x7f9ad4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efe5ece800c6ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.263+0000 7f9accc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:25.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4036690 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad4026040 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9ad400f040 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4056050 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 10 client.5847 fetch_fsmap learned FSMap version 40 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5616d3cb2310 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4026040 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 40) ==== 3550+0+0 (secure 0 0 0) 0x7f9ad4059020 con 0x5616d3c4e2b0 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 10 client.5847 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 10 client.5847 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 20 client.5847 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3c76260 0x5616d3c8d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 4 client.5847 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.264+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616d3bb70e0 con 0x5616d3c76260 2024-09-10T07:49:25.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3c76260 0x5616d3c8d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9ae2a00640 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3c76260 0x5616d3c8d4f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c69bd6a627e65a32 server_cookie=7be0655522b9d287 in_seq=0 out_seq=0 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9accc00640 10 client.5847 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9accc00640 1 -- 172.21.15.3:0/1186339286 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5616d3bb70e0 con 0x5616d3c76260 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9accc00640 10 client.5847 handle_command_reply: tid=0 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9aed204740 1 client.5847 shutdown 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 msgr2=0x5616d3d0dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.265+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009770 tx=0x7f9ad4009f90 comp rx=0 tx=0).stop 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 shutdown_connections 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x5616d3c76260 0x5616d3c8d4f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5616d3c61800 0x5616d3c4dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5616d3c61420 0x5616d3c4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 --2- 172.21.15.3:0/1186339286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5616d3c4e2b0 0x5616d3d0dc10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9accc00640 0 client.5847 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 >> 172.21.15.3:0/1186339286 conn(0x5616d3c47480 msgr2=0x5616d3c8f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 shutdown_connections 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 1 -- 172.21.15.3:0/1186339286 wait complete. 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aed204740 20 client.5847 trim_cache size 0 max 16384 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stdout: "7158f455-bbda-49e6-8d65-b9654e3e4ce8": { 2024-09-10T07:49:25.270 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:49:25.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7158f455-bbda-49e6-8d65-b9654e3e4ce8", 2024-09-10T07:49:25.271 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:49:25.271 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:25.271 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:25.271 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 msgr2=0x7f9ac406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 0x7f9ac406e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009b90 tx=0x7f9ad4008040 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 msgr2=0x7f9ae4140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.266+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c3f0 tx=0x7f9ad000c8b0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 shutdown_connections 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ac406be30 0x7f9ac406e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ae4145730 0x7f9ae40b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ae41451d0 0x7f9ae4140630 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 --2- 172.21.15.3:0/775943598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ae40cf270 0x7f9ae415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 >> 172.21.15.3:0/775943598 conn(0x7f9ae40b5cb0 msgr2=0x7f9ae415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 shutdown_connections 2024-09-10T07:49:25.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:25.267+0000 7f9aea400640 1 -- 172.21.15.3:0/775943598 wait complete. 2024-09-10T07:49:26.595 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:49:26.596+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:49:30.282 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:49:30.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-10T07:49:30.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.450+0000 7f62ebe00640 1 Processor -- start 2024-09-10T07:49:30.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.450+0000 7f62ebe00640 1 -- start start 2024-09-10T07:49:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.451+0000 7f62ebe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.451+0000 7f62ebe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f62ec0d3860 0x7f62ec0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.451+0000 7f62ebe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ec0d3de0 con 0x7f62ec0d4ae0 2024-09-10T07:49:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.451+0000 7f62ebe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec161f50 con 0x7f62ec0d3860 2024-09-10T07:49:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.451+0000 7f62ebe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec0d40c0 con 0x7f62ec161b70 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62eb400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62ec0d4ae0 0x7f62ec0d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54906/0 (socket says 172.21.15.3:54906) 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62eb400640 1 -- 172.21.15.3:0/2229806185 learned_addr learned my addr 172.21.15.3:0/2229806185 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62ea000640 1 --2- 172.21.15.3:0/2229806185 >> v2:172.21.15.70:3300/0 conn(0x7f62ec0d3860 0x7f62ec0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62eaa00640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510983432 0 0) 0x7f62ec0d3de0 con 0x7f62ec0d4ae0 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d0003520 con 0x7f62ec0d4ae0 2024-09-10T07:49:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 378391428 0 0) 0x7f62d0003520 con 0x7f62ec0d4ae0 2024-09-10T07:49:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62ea000640 1 -- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 msgr2=0x7f62ec0d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62ea000640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec0d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62ea000640 1 -- 172.21.15.3:0/2229806185 >> v1:172.21.15.3:6789/0 conn(0x7f62ec0d4ae0 legacy=0x7f62ec0d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.452+0000 7f62ea000640 1 -- 172.21.15.3:0/2229806185 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec0cf690 con 0x7f62ec0d3860 2024-09-10T07:49:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.453+0000 7f62ea000640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d3860 0x7f62ec0d45a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097a0 tx=0x7f62d802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16175d3257ad1b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8037070 con 0x7f62ec0d3860 2024-09-10T07:49:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d802aba0 con 0x7f62ec0d3860 2024-09-10T07:49:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62e9600640 1 -- 172.21.15.3:0/2229806185 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40019f0 con 0x7f62ec0d3860 2024-09-10T07:49:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 -- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d3860 msgr2=0x7f62ec0d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d3860 0x7f62ec0d45a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097a0 tx=0x7f62d802a110 comp rx=0 tx=0).stop 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 -- 172.21.15.3:0/2229806185 shutdown_connections 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d3860 0x7f62ec0d45a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2229806185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec0d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.454+0000 7f62ebe00640 1 -- 172.21.15.3:0/2229806185 >> 172.21.15.3:0/2229806185 conn(0x7f62ec062630 msgr2=0x7f62ec062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.455+0000 7f62ebe00640 1 -- 172.21.15.3:0/2229806185 shutdown_connections 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.455+0000 7f62ebe00640 1 -- 172.21.15.3:0/2229806185 wait complete. 2024-09-10T07:49:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.456+0000 7f62ebe00640 1 Processor -- start 2024-09-10T07:49:30.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.456+0000 7f62ebe00640 1 -- start start 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d4ae0 0x7f62ec156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0cf2d0 con 0x7f62ec0d3860 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0c9fc0 con 0x7f62ec0d4ae0 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62ebe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec0cf450 con 0x7f62ec161b70 2024-09-10T07:49:30.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62eaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62eaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46038/0 (socket says 172.21.15.3:46038) 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.457+0000 7f62eaa00640 1 -- 172.21.15.3:0/2729297867 learned_addr learned my addr 172.21.15.3:0/2729297867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62ea000640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d4ae0 0x7f62ec156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eb400640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 -- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 msgr2=0x7f62ec152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 -- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d4ae0 msgr2=0x7f62ec156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d4ae0 0x7f62ec156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 -- 172.21.15.3:0/2729297867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec0cd8f0 con 0x7f62ec0d3860 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eb400640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62eaa00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00e8a0 tx=0x7f62dc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62e2a00640 1 -- 172.21.15.3:0/2729297867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc017070 con 0x7f62ec0d3860 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62e2a00640 1 -- 172.21.15.3:0/2729297867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc00c940 con 0x7f62ec0d3860 2024-09-10T07:49:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.458+0000 7f62e2a00640 1 -- 172.21.15.3:0/2729297867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00cd30 con 0x7f62ec0d3860 2024-09-10T07:49:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.459+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec0cdbd0 con 0x7f62ec0d3860 2024-09-10T07:49:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.459+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec0b4530 con 0x7f62ec0d3860 2024-09-10T07:49:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.461+0000 7f62e2a00640 1 -- 172.21.15.3:0/2729297867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62dc01a070 con 0x7f62ec0d3860 2024-09-10T07:49:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.461+0000 7f62e2a00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 0x7f62cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.461+0000 7f62e2a00640 1 -- 172.21.15.3:0/2729297867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f62dc08b1c0 con 0x7f62ec0d3860 2024-09-10T07:49:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.462+0000 7f62ea000640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 0x7f62cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.462+0000 7f62f49e1740 1 Processor -- start 2024-09-10T07:49:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.462+0000 7f62f49e1740 1 -- start start 2024-09-10T07:49:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.462+0000 7f62ea000640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 0x7f62cc06e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d8fd9e0 0x560a1d8fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a1d865660 con 0x560a1d947ea0 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560a1d80d680 con 0x560a1d9174f0 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560a1d861620 con 0x560a1d8fd9e0 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62eb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62eb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46050/0 (socket says 172.21.15.3:46050) 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.463+0000 7f62eb400640 1 -- 172.21.15.3:0/3175231568 learned_addr learned my addr 172.21.15.3:0/3175231568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eaa00640 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62ea000640 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d8fd9e0 0x560a1d8fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 -- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d8fd9e0 msgr2=0x560a1d8fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d8fd9e0 0x560a1d8fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 -- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 msgr2=0x560a1d9178d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9178d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 -- 172.21.15.3:0/3175231568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1d6427d0 con 0x560a1d947ea0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62eb400640 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e240 tx=0x7f62e4001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435e2085a3db02ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62d7400640 1 -- 172.21.15.3:0/3175231568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4019070 con 0x560a1d947ea0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62d7400640 1 -- 172.21.15.3:0/3175231568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e40072e0 con 0x560a1d947ea0 2024-09-10T07:49:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.464+0000 7f62d7400640 1 -- 172.21.15.3:0/3175231568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4007690 con 0x560a1d947ea0 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 -- 172.21.15.3:0/3175231568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 msgr2=0x560a1d9483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e240 tx=0x7f62e4001b20 comp rx=0 tx=0).stop 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 -- 172.21.15.3:0/3175231568 shutdown_connections 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d9483f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d8fd9e0 0x560a1d8fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3175231568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9178d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 -- 172.21.15.3:0/3175231568 >> 172.21.15.3:0/3175231568 conn(0x560a1d8fe410 msgr2=0x560a1d946a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 -- 172.21.15.3:0/3175231568 shutdown_connections 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.465+0000 7f62f49e1740 1 -- 172.21.15.3:0/3175231568 wait complete. 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.466+0000 7f62f49e1740 1 -- start start 2024-09-10T07:49:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.466+0000 7f62f49e1740 1 Processor -- start 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d93daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d957490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a1d8fc940 con 0x560a1d947ea0 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560a1d7ee4c0 con 0x560a1d957010 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560a1d8fcc20 con 0x560a1d9174f0 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62f49e1740 10 client.0 _ll_register_callbacks cb 0x560a1d85de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46066/0 (socket says 172.21.15.3:46066) 2024-09-10T07:49:30.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.467+0000 7f62ea000640 1 -- 172.21.15.3:0/3794240093 learned_addr learned my addr 172.21.15.3:0/3794240093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62eaa00640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d93daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62eb400640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d957490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 -- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 msgr2=0x560a1d93daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d93daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 -- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 msgr2=0x560a1d957490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d957490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 -- 172.21.15.3:0/3794240093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1d5783f0 con 0x560a1d947ea0 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62eaa00640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d93daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62eb400640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d957490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62ea000640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098c0 tx=0x7f62d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b36f7d43ef3ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d804c030 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e40072e0 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.468+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f62d8004380 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f62d8058050 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 10 client.5859 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x560a1d5380f0 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e40072e0 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f62d805b020 con 0x560a1d947ea0 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 10 client.5859 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 10 client.5859 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 20 client.5859 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d940ed0 0x560a1d941330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 4 client.5859 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:30.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a1d872d90 con 0x560a1d940ed0 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.469+0000 7f62eaa00640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d940ed0 0x560a1d941330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.470+0000 7f62eaa00640 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d940ed0 0x560a1d941330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a1ac5e13f00fab5 server_cookie=d83cb20010a4e24b in_seq=0 out_seq=0 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.470+0000 7f62cbe00640 10 client.5859 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.470+0000 7f62cbe00640 1 -- 172.21.15.3:0/3794240093 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a1d872d90 con 0x560a1d940ed0 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.470+0000 7f62cbe00640 10 client.5859 handle_command_reply: tid=0 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 client.5859 shutdown 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 msgr2=0x560a1d93dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098c0 tx=0x7f62d8004730 comp rx=0 tx=0).stop 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 shutdown_connections 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d940ed0 0x560a1d941330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d957490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d93dfe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62cbe00640 0 client.5859 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 --2- 172.21.15.3:0/3794240093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d93daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 >> 172.21.15.3:0/3794240093 conn(0x560a1d8fd9e0 msgr2=0x560a1d944cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 shutdown_connections 2024-09-10T07:49:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 1 -- 172.21.15.3:0/3794240093 wait complete. 2024-09-10T07:49:30.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.471+0000 7f62f49e1740 20 client.5859 trim_cache size 0 max 16384 2024-09-10T07:49:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.489+0000 7f62f49e1740 1 Processor -- start 2024-09-10T07:49:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 -- start start 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d947ea0 0x560a1d954600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec200b90 con 0x560a1d957010 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0cc4a0 con 0x560a1d9174f0 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.490+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec200460 con 0x560a1d947ea0 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46082/0 (socket says 172.21.15.3:46082) 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 -- 172.21.15.3:0/4153569571 learned_addr learned my addr 172.21.15.3:0/4153569571 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eaa00640 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62ea000640 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d947ea0 0x560a1d954600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 -- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d947ea0 msgr2=0x560a1d954600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d947ea0 0x560a1d954600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 -- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 msgr2=0x560a1d9540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 -- 172.21.15.3:0/4153569571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1d92bdd0 con 0x560a1d957010 2024-09-10T07:49:30.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.491+0000 7f62eb400640 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f62e4013000 tx=0x7f62e400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad29ff838be6e553 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62d6a00640 1 -- 172.21.15.3:0/4153569571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4024070 con 0x560a1d957010 2024-09-10T07:49:30.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62d6a00640 1 -- 172.21.15.3:0/4153569571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e401a040 con 0x560a1d957010 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62d6a00640 1 -- 172.21.15.3:0/4153569571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4019070 con 0x560a1d957010 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 -- 172.21.15.3:0/4153569571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 msgr2=0x560a1d93d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f62e4013000 tx=0x7f62e400c000 comp rx=0 tx=0).stop 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 -- 172.21.15.3:0/4153569571 shutdown_connections 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d957010 0x560a1d93d4c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d947ea0 0x560a1d954600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 --2- 172.21.15.3:0/4153569571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d9174f0 0x560a1d9540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 -- 172.21.15.3:0/4153569571 >> 172.21.15.3:0/4153569571 conn(0x560a1d8fd9e0 msgr2=0x560a1d945b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 -- 172.21.15.3:0/4153569571 shutdown_connections 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.492+0000 7f62f49e1740 1 -- 172.21.15.3:0/4153569571 wait complete. 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.493+0000 7f62f49e1740 1 -- start start 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.493+0000 7f62f49e1740 1 Processor -- start 2024-09-10T07:49:30.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d928c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d901bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560a1d943bc0 con 0x560a1d947ea0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec154170 con 0x560a1d957010 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560a1d941580 con 0x560a1d9174f0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62f49e1740 10 client.0 _ll_register_callbacks cb 0x7f62d00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62ea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46088/0 (socket says 172.21.15.3:46088) 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62ea000640 1 -- 172.21.15.3:0/1333289076 learned_addr learned my addr 172.21.15.3:0/1333289076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62eaa00640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d928c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.494+0000 7f62eb400640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d901bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 -- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 msgr2=0x560a1d928c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d928c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 -- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 msgr2=0x560a1d901bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d901bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 -- 172.21.15.3:0/1333289076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1d945000 con 0x560a1d947ea0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62eaa00640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d928c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62eb400640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d901bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62ea000640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f62d8006a70 tx=0x7f62d803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d792f9aa064196c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d804c070 con 0x560a1d947ea0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f62d8052b50 con 0x560a1d947ea0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e401a040 con 0x560a1d947ea0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f62d8041040 con 0x560a1d947ea0 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62f49e1740 10 client.5865 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:30.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.495+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x560a1d929270 con 0x560a1d947ea0 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e401a040 con 0x560a1d947ea0 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f62d805b040 con 0x560a1d947ea0 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 10 client.5865 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 10 client.5865 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 20 client.5865 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d9020f0 0x560a1d9c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 4 client.5865 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a1d86d0e0 con 0x560a1d9020f0 2024-09-10T07:49:30.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.496+0000 7f62eaa00640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d9020f0 0x560a1d9c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:30.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62eaa00640 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d9020f0 0x560a1d9c7cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a7e0345041b9a8c server_cookie=5805128609f34d9e in_seq=0 out_seq=0 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62cbe00640 10 client.5865 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62cbe00640 1 -- 172.21.15.3:0/1333289076 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560a1d86d0e0 con 0x560a1d9020f0 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62cbe00640 10 client.5865 handle_command_reply: tid=0 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 client.5865 shutdown 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 msgr2=0x560a1d928260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f62d8006a70 tx=0x7f62d803d000 comp rx=0 tx=0).stop 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 shutdown_connections 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x560a1d9020f0 0x560a1d9c7cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560a1d957010 0x560a1d901bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a1d947ea0 0x560a1d928260 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 --2- 172.21.15.3:0/1333289076 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560a1d9174f0 0x560a1d928c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62cbe00640 0 client.5865 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 >> 172.21.15.3:0/1333289076 conn(0x560a1d8fd9e0 msgr2=0x560a1d9459f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 shutdown_connections 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 1 -- 172.21.15.3:0/1333289076 wait complete. 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.497+0000 7f62f49e1740 20 client.5865 trim_cache size 0 max 16384 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: "7158f455-bbda-49e6-8d65-b9654e3e4ce8": { 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:49:30.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7158f455-bbda-49e6-8d65-b9654e3e4ce8", 2024-09-10T07:49:30.503 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:49:30.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:30.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:30.503 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 msgr2=0x7f62cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 0x7f62cc06e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 msgr2=0x7f62ec0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00e8a0 tx=0x7f62dc00ed60 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 shutdown_connections 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62cc06be30 0x7f62cc06e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62ec161b70 0x7f62ec152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec0d4ae0 0x7f62ec156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 --2- 172.21.15.3:0/2729297867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62ec0d3860 0x7f62ec0cd210 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:30.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 >> 172.21.15.3:0/2729297867 conn(0x7f62ec062630 msgr2=0x7f62ec0cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 shutdown_connections 2024-09-10T07:49:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:30.498+0000 7f62ebe00640 1 -- 172.21.15.3:0/2729297867 wait complete. 2024-09-10T07:49:35.514 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:49:35.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-10T07:49:35.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.760+0000 7f4ed7e00640 1 Processor -- start 2024-09-10T07:49:35.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.760+0000 7f4ed7e00640 1 -- start start 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed7e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ed8161b70 0x7f4ed80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed80d3860 0x7f4ed80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed7e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed80d3de0 con 0x7f4ed80d4ae0 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed7e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed80d4f50 con 0x7f4ed80d3860 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed7e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed8161f50 con 0x7f4ed8161b70 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed6000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed80d3860 0x7f4ed80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.761+0000 7f4ed6000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ed80d3860 0x7f4ed80d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52932/0 (socket says 172.21.15.3:52932) 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6000640 1 -- 172.21.15.3:0/1297780894 learned_addr learned my addr 172.21.15.3:0/1297780894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/1297780894 >> v2:172.21.15.70:3300/0 conn(0x7f4ed8161b70 0x7f4ed80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6000640 1 -- 172.21.15.3:0/1297780894 >> v2:172.21.15.70:3300/0 conn(0x7f4ed8161b70 msgr2=0x7f4ed80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6000640 1 --2- 172.21.15.3:0/1297780894 >> v2:172.21.15.70:3300/0 conn(0x7f4ed8161b70 0x7f4ed80d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6000640 1 -- 172.21.15.3:0/1297780894 >> v1:172.21.15.3:6789/0 conn(0x7f4ed80d4ae0 legacy=0x7f4ed80d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed6000640 1 -- 172.21.15.3:0/1297780894 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed80d40c0 con 0x7f4ed80d3860 2024-09-10T07:49:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.762+0000 7f4ed5600640 1 -- 172.21.15.3:0/1297780894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687254530 0 0) 0x7f4ed80d3de0 con 0x7f4ed80d4ae0 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.763+0000 7f4ed6000640 1 --2- 172.21.15.3:0/1297780894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed80d45a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80d55c0 tx=0x7f4ec402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573c843d2f4c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.763+0000 7f4ed5600640 1 -- 172.21.15.3:0/1297780894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037070 con 0x7f4ed80d3860 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.763+0000 7f4ed5600640 1 -- 172.21.15.3:0/1297780894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec4032440 con 0x7f4ed80d3860 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.763+0000 7f4ed5600640 1 -- 172.21.15.3:0/1297780894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4032810 con 0x7f4ed80d3860 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 -- 172.21.15.3:0/1297780894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 msgr2=0x7f4ed80d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/1297780894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed80d45a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80d55c0 tx=0x7f4ec402a2a0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 -- 172.21.15.3:0/1297780894 shutdown_connections 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/1297780894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed80d45a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/1297780894 >> v2:172.21.15.70:3300/0 conn(0x7f4ed8161b70 0x7f4ed80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 -- 172.21.15.3:0/1297780894 >> 172.21.15.3:0/1297780894 conn(0x7f4ed8062630 msgr2=0x7f4ed8062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 -- 172.21.15.3:0/1297780894 shutdown_connections 2024-09-10T07:49:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.764+0000 7f4ed7e00640 1 -- 172.21.15.3:0/1297780894 wait complete. 2024-09-10T07:49:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.766+0000 7f4ed7e00640 1 Processor -- start 2024-09-10T07:49:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.766+0000 7f4ed7e00640 1 -- start start 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed80d4ae0 0x7f4ed815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4ed8161b70 0x7f4ed80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed80c9fc0 con 0x7f4ed80d3860 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed80cf970 con 0x7f4ed80d4ae0 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed80cf2d0 con 0x7f4ed8161b70 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52944/0 (socket says 172.21.15.3:52944) 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6a00640 1 -- 172.21.15.3:0/2004256285 learned_addr learned my addr 172.21.15.3:0/2004256285 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed80d4ae0 0x7f4ed815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed7400640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4ed8161b70 0x7f4ed80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6a00640 1 -- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4ed8161b70 msgr2=0x7f4ed80cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.767+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4ed8161b70 0x7f4ed80cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed6a00640 1 -- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed80d4ae0 msgr2=0x7f4ed815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed80d4ae0 0x7f4ed815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed6a00640 1 -- 172.21.15.3:0/2004256285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed815b340 con 0x7f4ed80d3860 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d620 tx=0x7f4ec800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ecf400640 1 -- 172.21.15.3:0/2004256285 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800b620 con 0x7f4ed80d3860 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ecf400640 1 -- 172.21.15.3:0/2004256285 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8019070 con 0x7f4ed80d3860 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ecf400640 1 -- 172.21.15.3:0/2004256285 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800ba00 con 0x7f4ed80d3860 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed815b620 con 0x7f4ed80d3860 2024-09-10T07:49:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.768+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed815bbe0 con 0x7f4ed80d3860 2024-09-10T07:49:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.771+0000 7f4ecf400640 1 -- 172.21.15.3:0/2004256285 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ec800bba0 con 0x7f4ed80d3860 2024-09-10T07:49:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.772+0000 7f4ecf400640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 0x7f4eb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.772+0000 7f4ecf400640 1 -- 172.21.15.3:0/2004256285 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f4ec808b380 con 0x7f4ed80d3860 2024-09-10T07:49:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.772+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 0x7f4eb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.773+0000 7f4ee0cb5740 1 Processor -- start 2024-09-10T07:49:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.773+0000 7f4ee0cb5740 1 -- start start 2024-09-10T07:49:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.773+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 0x7f4eb806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4034000 tx=0x7f4ec402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf7cea0 0x55b7aaf7d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b7aae9a660 con 0x55b7aaf329e0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b7aae42680 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b7aae96620 con 0x55b7aaf7cea0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:49660/0 (socket says 172.21.15.3:49660) 2024-09-10T07:49:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52946/0 (socket says 172.21.15.3:52946) 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed6a00640 1 -- 172.21.15.3:0/3146872165 learned_addr learned my addr 172.21.15.3:0/3146872165 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.774+0000 7f4ed7400640 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf7cea0 0x55b7aaf7d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 -- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf7cea0 msgr2=0x55b7aaf7d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf7cea0 0x55b7aaf7d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 -- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 msgr2=0x55b7aaf4c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 -- 172.21.15.3:0/3146872165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7aac777d0 con 0x55b7aaf329e0 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ed6000640 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80d28e0 tx=0x7f4ec402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296934934e511944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ec3400640 1 -- 172.21.15.3:0/3146872165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037040 con 0x55b7aaf329e0 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ec3400640 1 -- 172.21.15.3:0/3146872165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec404e070 con 0x55b7aaf329e0 2024-09-10T07:49:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.775+0000 7f4ec3400640 1 -- 172.21.15.3:0/3146872165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4049480 con 0x55b7aaf329e0 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/3146872165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 msgr2=0x55b7aaf31240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80d28e0 tx=0x7f4ec402c000 comp rx=0 tx=0).stop 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/3146872165 shutdown_connections 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf7cea0 0x55b7aaf7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf329e0 0x55b7aaf31240 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/3146872165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/3146872165 >> 172.21.15.3:0/3146872165 conn(0x55b7aaf33410 msgr2=0x55b7aaf7ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/3146872165 shutdown_connections 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.776+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/3146872165 wait complete. 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.777+0000 7f4ee0cb5740 1 -- start start 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.777+0000 7f4ee0cb5740 1 Processor -- start 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf4c4f0 0x55b7aaf8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaf73f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b7aaf31940 con 0x55b7aaf7cea0 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b7aae234c0 con 0x55b7aaf8c210 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b7aaf31c20 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ee0cb5740 10 client.0 _ll_register_callbacks cb 0x55b7aae92e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52956/0 (socket says 172.21.15.3:52956) 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.778+0000 7f4ed6000640 1 -- 172.21.15.3:0/4025653094 learned_addr learned my addr 172.21.15.3:0/4025653094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf4c4f0 0x55b7aaf8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed7400640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaf73f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 -- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf4c4f0 msgr2=0x55b7aaf8bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf4c4f0 0x55b7aaf8bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 -- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 msgr2=0x55b7aaf73f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaf73f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 -- 172.21.15.3:0/4025653094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7aabad3f0 con 0x55b7aaf7cea0 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed7400640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaf73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ed6000640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000c00 tx=0x7f4ec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58056cf83890407 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4eb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037040 con 0x55b7aaf7cea0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f4ec4049890 con 0x55b7aaf7cea0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.779+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec404e070 con 0x55b7aaf7cea0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4ec4062050 con 0x55b7aaf7cea0 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 10 client.5880 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55b7aab6d0f0 con 0x55b7aaf7cea0 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec404e070 con 0x55b7aaf7cea0 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f4ec4065020 con 0x55b7aaf7cea0 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 10 client.5880 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 10 client.5880 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 20 client.5880 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf77f60 0x55b7aaf78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 4 client.5880 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.780+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7aaea7d90 con 0x55b7aaf77f60 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.781+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf77f60 0x55b7aaf78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.781+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf77f60 0x55b7aaf78370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dca28b22f438bb8 server_cookie=67a314a668ba755 in_seq=0 out_seq=0 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.781+0000 7f4eb7e00640 10 client.5880 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4eb7e00640 1 -- 172.21.15.3:0/4025653094 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7aaea7d90 con 0x55b7aaf77f60 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4eb7e00640 10 client.5880 handle_command_reply: tid=0 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 client.5880 shutdown 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 msgr2=0x55b7aaf73a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000c00 tx=0x7f4ec4004730 comp rx=0 tx=0).stop 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 shutdown_connections 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf77f60 0x55b7aaf78370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaf73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf73a40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/4025653094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf4c4f0 0x55b7aaf8bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4eb7e00640 0 client.5880 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 >> 172.21.15.3:0/4025653094 conn(0x55b7aaf329e0 msgr2=0x55b7aaf7d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 shutdown_connections 2024-09-10T07:49:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/4025653094 wait complete. 2024-09-10T07:49:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.782+0000 7f4ee0cb5740 20 client.5880 trim_cache size 0 max 16384 2024-09-10T07:49:35.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.800+0000 7f4ee0cb5740 1 Processor -- start 2024-09-10T07:49:35.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.800+0000 7f4ee0cb5740 1 -- start start 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf8e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 0x55b7aaf79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed8200b10 con 0x55b7aaf7cea0 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81ffd10 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed82003e0 con 0x55b7aaf8c210 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.801+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52968/0 (socket says 172.21.15.3:52968) 2024-09-10T07:49:35.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 -- 172.21.15.3:0/2297890230 learned_addr learned my addr 172.21.15.3:0/2297890230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf8e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 -- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 msgr2=0x55b7aaf79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed7400640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 0x55b7aaf79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 0x55b7aaf79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 -- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 msgr2=0x55b7aaf8e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf8e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 -- 172.21.15.3:0/2297890230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7aaf331b0 con 0x55b7aaf7cea0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed7400640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 0x55b7aaf79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ed6000640 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009210 tx=0x7f4ec4052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb98831b2540a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ec2a00640 1 -- 172.21.15.3:0/2297890230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4037040 con 0x55b7aaf7cea0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ec2a00640 1 -- 172.21.15.3:0/2297890230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec404e040 con 0x55b7aaf7cea0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.802+0000 7f4ec2a00640 1 -- 172.21.15.3:0/2297890230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4032620 con 0x55b7aaf7cea0 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/2297890230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 msgr2=0x55b7aaf8eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009210 tx=0x7f4ec4052000 comp rx=0 tx=0).stop 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/2297890230 shutdown_connections 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf8c210 0x55b7aaf79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf7cea0 0x55b7aaf8eb60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/2297890230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf8e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/2297890230 >> 172.21.15.3:0/2297890230 conn(0x55b7aaf329e0 msgr2=0x55b7aaf73a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/2297890230 shutdown_connections 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.803+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/2297890230 wait complete. 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.804+0000 7f4ee0cb5740 1 -- start start 2024-09-10T07:49:35.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.804+0000 7f4ee0cb5740 1 Processor -- start 2024-09-10T07:49:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaffdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf369c0 0x55b7aaf36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b7aaed0bf0 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81f4cd0 con 0x55b7aaf8c210 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b7aaf78ee0 con 0x55b7aaf369c0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ee0cb5740 10 client.0 _ll_register_callbacks cb 0x7f4ec800bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52982/0 (socket says 172.21.15.3:52982) 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 -- 172.21.15.3:0/588172537 learned_addr learned my addr 172.21.15.3:0/588172537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6000640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaffdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed7400640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf369c0 0x55b7aaf36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 -- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf369c0 msgr2=0x55b7aaf36e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf369c0 0x55b7aaf36e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 -- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 msgr2=0x55b7aaffdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaffdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6a00640 1 -- 172.21.15.3:0/588172537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7aaf61990 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.805+0000 7f4ed6000640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaffdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ed6a00640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d5f0 tx=0x7f4ec8055790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed0e6750e5495fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80893e0 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f4ec8057770 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec404e040 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4ec8055ae0 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 10 client.5886 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55b7aaf7d5f0 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec404e040 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f4ec8088070 con 0x55b7aaf4c4f0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 10 client.5886 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 10 client.5886 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.806+0000 7f4ee0cb5740 20 client.5886 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf9d5d0 0x55b7aaf9d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4ee0cb5740 4 client.5886 mds_command: new command op to 5430 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 --> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7aaea20e0 con 0x55b7aaf9d5d0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4ed6000640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf9d5d0 0x55b7aaf9d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4ed6000640 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf9d5d0 0x55b7aaf9d9e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b13ab5dc460ae8a server_cookie=f8341cf569e5d032 in_seq=0 out_seq=0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4eb7e00640 10 client.5886 ms_handle_connect on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4eb7e00640 1 -- 172.21.15.3:0/588172537 <== mds.1 v2:172.21.15.3:6835/2353998589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7aaea20e0 con 0x55b7aaf9d5d0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.807+0000 7f4eb7e00640 10 client.5886 handle_command_reply: tid=0 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 client.5886 shutdown 2024-09-10T07:49:35.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 msgr2=0x55b7aaf7a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d5f0 tx=0x7f4ec8055790 comp rx=0 tx=0).stop 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 shutdown_connections 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:6835/2353998589,v1:172.21.15.3:6837/2353998589] conn(0x55b7aaf9d5d0 0x55b7aaf9d9e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b7aaf369c0 0x55b7aaf36e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b7aaf8c210 0x55b7aaffdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 --2- 172.21.15.3:0/588172537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b7aaf4c4f0 0x55b7aaf7a670 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4eb7e00640 0 client.5886 ms_handle_reset on v2:172.21.15.3:6835/2353998589 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 >> 172.21.15.3:0/588172537 conn(0x55b7aaf329e0 msgr2=0x55b7aaf73940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 shutdown_connections 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 1 -- 172.21.15.3:0/588172537 wait complete. 2024-09-10T07:49:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.808+0000 7f4ee0cb5740 20 client.5886 trim_cache size 0 max 16384 2024-09-10T07:49:35.813 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:35.813 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-10T07:49:35.813 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-10T07:49:35.813 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 msgr2=0x7f4eb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 0x7f4eb806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4034000 tx=0x7f4ec402e000 comp rx=0 tx=0).stop 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 msgr2=0x7f4ed8149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d620 tx=0x7f4ec800dae0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 shutdown_connections 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4eb806be30 0x7f4eb806e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4ed8161b70 0x7f4ed80cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ed80d4ae0 0x7f4ed815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 --2- 172.21.15.3:0/2004256285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ed80d3860 0x7f4ed8149080 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 >> 172.21.15.3:0/2004256285 conn(0x7f4ed8062630 msgr2=0x7f4ed815f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 shutdown_connections 2024-09-10T07:49:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.809+0000 7f4ed7e00640 1 -- 172.21.15.3:0/2004256285 wait complete. 2024-09-10T07:49:35.823 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:49:35.824 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:49:35.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:2 scrub status 2024-09-10T07:49:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.984+0000 7f0a6d600640 1 Processor -- start 2024-09-10T07:49:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.984+0000 7f0a6d600640 1 -- start start 2024-09-10T07:49:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.985+0000 7f0a6d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.985+0000 7f0a6d600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 0x7f0a680d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.985+0000 7f0a6d600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a680d4f00 con 0x7f0a68161b70 2024-09-10T07:49:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.985+0000 7f0a6d600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0a68161f50 con 0x7f0a680cfe90 2024-09-10T07:49:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.985+0000 7f0a6d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680d3de0 con 0x7f0a680d45a0 2024-09-10T07:49:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.986+0000 7f0a67400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.986+0000 7f0a67400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:32774/0 (socket says 172.21.15.3:32774) 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.986+0000 7f0a67400640 1 -- 172.21.15.3:0/1634295187 learned_addr learned my addr 172.21.15.3:0/1634295187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.986+0000 7f0a67e00640 1 --1- 172.21.15.3:0/1634295187 >> v1:172.21.15.70:6789/0 conn(0x7f0a68161b70 0x7f0a680d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46578/0 (socket says 172.21.15.3:46578) 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.986+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/1634295187 >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 0x7f0a680d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a67400640 1 -- 172.21.15.3:0/1634295187 >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 msgr2=0x7f0a680d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a67400640 1 --2- 172.21.15.3:0/1634295187 >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 0x7f0a680d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a67400640 1 -- 172.21.15.3:0/1634295187 >> v1:172.21.15.70:6789/0 conn(0x7f0a68161b70 legacy=0x7f0a680d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a67400640 1 -- 172.21.15.3:0/1634295187 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680d40c0 con 0x7f0a680d45a0 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a66a00640 1 -- 172.21.15.3:0/1634295187 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646314186 0 0) 0x7f0a680d4f00 con 0x7f0a68161b70 2024-09-10T07:49:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.987+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/1634295187 >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 0x7f0a680d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.988+0000 7f0a67400640 1 --2- 172.21.15.3:0/1634295187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd28230237883c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.988+0000 7f0a66a00640 1 -- 172.21.15.3:0/1634295187 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54037070 con 0x7f0a680d45a0 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.988+0000 7f0a66a00640 1 -- 172.21.15.3:0/1634295187 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a54032440 con 0x7f0a680d45a0 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.988+0000 7f0a66a00640 1 -- 172.21.15.3:0/1634295187 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54032810 con 0x7f0a680d45a0 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 -- 172.21.15.3:0/1634295187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 msgr2=0x7f0a680d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 --2- 172.21.15.3:0/1634295187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a250 comp rx=0 tx=0).stop 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 -- 172.21.15.3:0/1634295187 shutdown_connections 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 --2- 172.21.15.3:0/1634295187 >> v2:172.21.15.70:3300/0 conn(0x7f0a680cfe90 0x7f0a680d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 --2- 172.21.15.3:0/1634295187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680d49c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 -- 172.21.15.3:0/1634295187 >> 172.21.15.3:0/1634295187 conn(0x7f0a68062630 msgr2=0x7f0a68062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 -- 172.21.15.3:0/1634295187 shutdown_connections 2024-09-10T07:49:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.989+0000 7f0a6d600640 1 -- 172.21.15.3:0/1634295187 wait complete. 2024-09-10T07:49:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.990+0000 7f0a6d600640 1 Processor -- start 2024-09-10T07:49:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.990+0000 7f0a6d600640 1 -- start start 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a68161b70 0x7f0a680b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680c9fc0 con 0x7f0a680cfe90 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680cf970 con 0x7f0a68161b70 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.991+0000 7f0a6d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680d3a30 con 0x7f0a680d45a0 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52996/0 (socket says 172.21.15.3:52996) 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 -- 172.21.15.3:0/146267512 learned_addr learned my addr 172.21.15.3:0/146267512 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67400640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a68161b70 0x7f0a680b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 -- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 msgr2=0x7f0a680b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 -- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a68161b70 msgr2=0x7f0a680b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a68161b70 0x7f0a680b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 -- 172.21.15.3:0/146267512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6815e970 con 0x7f0a680cfe90 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67400640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.992+0000 7f0a67e00640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c4d0 tx=0x7f0a5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.993+0000 7f0a5fe00640 1 -- 172.21.15.3:0/146267512 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58018070 con 0x7f0a680cfe90 2024-09-10T07:49:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.993+0000 7f0a5fe00640 1 -- 172.21.15.3:0/146267512 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a58007500 con 0x7f0a680cfe90 2024-09-10T07:49:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.993+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a68144ef0 con 0x7f0a680cfe90 2024-09-10T07:49:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.993+0000 7f0a5fe00640 1 -- 172.21.15.3:0/146267512 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004a20 con 0x7f0a680cfe90 2024-09-10T07:49:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.993+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6815ef10 con 0x7f0a680cfe90 2024-09-10T07:49:35.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.995+0000 7f0a5fe00640 1 -- 172.21.15.3:0/146267512 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a5801b070 con 0x7f0a680cfe90 2024-09-10T07:49:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.996+0000 7f0a5fe00640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 0x7f0a4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.996+0000 7f0a5fe00640 1 -- 172.21.15.3:0/146267512 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f0a5808a350 con 0x7f0a680cfe90 2024-09-10T07:49:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.997+0000 7f0a67400640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 0x7f0a4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.997+0000 7f0a71ab6740 1 Processor -- start 2024-09-10T07:49:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.997+0000 7f0a71ab6740 1 -- start start 2024-09-10T07:49:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.997+0000 7f0a67400640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 0x7f0a4806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a54034000 tx=0x7f0a5402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563f92a54660 con 0x563f92aec9e0 2024-09-10T07:49:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563f929fc680 con 0x563f92b36ea0 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563f92a50620 con 0x563f92b064f0 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53010/0 (socket says 172.21.15.3:53010) 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a67400640 1 -- 172.21.15.3:0/466668975 learned_addr learned my addr 172.21.15.3:0/466668975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a67e00640 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.998+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 -- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 msgr2=0x563f92b068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 -- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 msgr2=0x563f92b373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 -- 172.21.15.3:0/466668975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f928317d0 con 0x563f92aec9e0 2024-09-10T07:49:36.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a67400640 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0a540090b0 tx=0x7f0a5402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ece847a957faaff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a53e00640 1 -- 172.21.15.3:0/466668975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54037040 con 0x563f92aec9e0 2024-09-10T07:49:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a53e00640 1 -- 172.21.15.3:0/466668975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5404d070 con 0x563f92aec9e0 2024-09-10T07:49:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:35.999+0000 7f0a53e00640 1 -- 172.21.15.3:0/466668975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5403d470 con 0x563f92aec9e0 2024-09-10T07:49:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 -- 172.21.15.3:0/466668975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 msgr2=0x563f92aeb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0a540090b0 tx=0x7f0a5402c000 comp rx=0 tx=0).stop 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 -- 172.21.15.3:0/466668975 shutdown_connections 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92aec9e0 0x563f92aeb240 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/466668975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 -- 172.21.15.3:0/466668975 >> 172.21.15.3:0/466668975 conn(0x563f92aed410 msgr2=0x563f92b35a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 -- 172.21.15.3:0/466668975 shutdown_connections 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.000+0000 7f0a71ab6740 1 -- 172.21.15.3:0/466668975 wait complete. 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.001+0000 7f0a71ab6740 1 -- start start 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.001+0000 7f0a71ab6740 1 Processor -- start 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b0be60 0x563f92b0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563f92aeb940 con 0x563f92b064f0 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563f929dd4c0 con 0x563f92b0be60 2024-09-10T07:49:36.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563f92aebc20 con 0x563f92b36ea0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a71ab6740 10 client.0 _ll_register_callbacks cb 0x563f92a4ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53014/0 (socket says 172.21.15.3:53014) 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67e00640 1 -- 172.21.15.3:0/1888112888 learned_addr learned my addr 172.21.15.3:0/1888112888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67400640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b0be60 0x563f92b0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67e00640 1 -- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 msgr2=0x563f92b2ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.002+0000 7f0a67e00640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b2ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a67e00640 1 -- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b0be60 msgr2=0x563f92b0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a67e00640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b0be60 0x563f92b0c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a67e00640 1 -- 172.21.15.3:0/1888112888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f927673f0 con 0x563f92b064f0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a67400640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a67e00640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800ce90 tx=0x7f0a58058440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f629ab6d7ac864ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580610b0 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f0a58004bc0 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a5404d070 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a58004dc0 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a71ab6740 10 client.5895 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x563f927270f0 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.003+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5404d070 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f0a58088200 con 0x563f92b064f0 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 10 client.5895 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 10 client.5895 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-10T07:49:36.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 20 client.5895 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b0e210 0x563f92b0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 4 client.5895 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f92a61d90 con 0x563f92b0e210 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.004+0000 7f0a67400640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b0e210 0x563f92b0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.005+0000 7f0a67400640 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b0e210 0x563f92b0cd40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c7f924cc4be729d server_cookie=46b13e432219977a in_seq=0 out_seq=0 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.005+0000 7f0a50c00640 10 client.5895 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a50c00640 1 -- 172.21.15.3:0/1888112888 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f92a61d90 con 0x563f92b0e210 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a50c00640 10 client.5895 handle_command_reply: tid=0 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 client.5895 shutdown 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 msgr2=0x563f92b0dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800ce90 tx=0x7f0a58058440 comp rx=0 tx=0).stop 2024-09-10T07:49:36.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 shutdown_connections 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b0e210 0x563f92b0cd40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b0be60 0x563f92b0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1888112888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b064f0 0x563f92b0dcd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a50c00640 0 client.5895 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 >> 172.21.15.3:0/1888112888 conn(0x563f92aec9e0 msgr2=0x563f92b35a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 shutdown_connections 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.006+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1888112888 wait complete. 2024-09-10T07:49:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.007+0000 7f0a71ab6740 20 client.5895 trim_cache size 0 max 16384 2024-09-10T07:49:36.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 Processor -- start 2024-09-10T07:49:36.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 -- start start 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b064f0 0x563f92b59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68200ac0 con 0x563f92b0be60 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681ffce0 con 0x563f92b064f0 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.025+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0a68200390 con 0x563f92b36ea0 2024-09-10T07:49:36.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53026/0 (socket says 172.21.15.3:53026) 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 -- 172.21.15.3:0/1645350746 learned_addr learned my addr 172.21.15.3:0/1645350746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67e00640 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b064f0 0x563f92b59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 -- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 msgr2=0x563f92b36210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b36210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 -- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b064f0 msgr2=0x563f92b59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b064f0 0x563f92b59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 -- 172.21.15.3:0/1645350746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f92b058b0 con 0x563f92b0be60 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.026+0000 7f0a67400640 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009960 tx=0x7f0a54009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2953de525d03290a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a53400640 1 -- 172.21.15.3:0/1645350746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5405e070 con 0x563f92b0be60 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a53400640 1 -- 172.21.15.3:0/1645350746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5404d040 con 0x563f92b0be60 2024-09-10T07:49:36.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a53400640 1 -- 172.21.15.3:0/1645350746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540584d0 con 0x563f92b0be60 2024-09-10T07:49:36.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1645350746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 msgr2=0x563f92b35cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009960 tx=0x7f0a54009240 comp rx=0 tx=0).stop 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1645350746 shutdown_connections 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b36ea0 0x563f92b36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92b35cd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/1645350746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b064f0 0x563f92b59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1645350746 >> 172.21.15.3:0/1645350746 conn(0x563f92aec9e0 msgr2=0x563f92b30920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1645350746 shutdown_connections 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.027+0000 7f0a71ab6740 1 -- 172.21.15.3:0/1645350746 wait complete. 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.028+0000 7f0a71ab6740 1 -- start start 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.028+0000 7f0a71ab6740 1 Processor -- start 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563f92b06fc0 con 0x563f92b0be60 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680b63d0 con 0x563f92b36ea0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563f92b332e0 con 0x563f92b064f0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a71ab6740 10 client.0 _ll_register_callbacks cb 0x7f0a58058070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a67400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53038/0 (socket says 172.21.15.3:53038) 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.029+0000 7f0a67400640 1 -- 172.21.15.3:0/522933185 learned_addr learned my addr 172.21.15.3:0/522933185 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67e00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 -- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 msgr2=0x563f92b363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 -- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 msgr2=0x563f92b37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 -- 172.21.15.3:0/522933185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f92b1e340 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a6cc00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67e00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a67400640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009990 tx=0x7f0a54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc752f75b16787e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5405e020 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f0a540584d0 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a5404d040 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a54068050 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a71ab6740 10 client.5901 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.030+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x563f92b59d90 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5404d040 con 0x563f92b0be60 2024-09-10T07:49:36.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f0a5406b020 con 0x563f92b0be60 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 10 client.5901 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 10 client.5901 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 20 client.5901 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b377c0 0x563f92b38ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 4 client.5901 mds_command: new command op to 5266 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:36.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 --> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f92a5c0e0 con 0x563f92b377c0 2024-09-10T07:49:36.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.031+0000 7f0a67e00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b377c0 0x563f92b38ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a67e00640 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b377c0 0x563f92b38ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51b9150c5546d58a server_cookie=9ad1840977237198 in_seq=0 out_seq=0 2024-09-10T07:49:36.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a50c00640 10 client.5901 ms_handle_connect on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:36.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a50c00640 1 -- 172.21.15.3:0/522933185 <== mds.2 v2:172.21.15.70:6832/1001818622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f92a5c0e0 con 0x563f92b377c0 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a50c00640 10 client.5901 handle_command_reply: tid=0 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 client.5901 shutdown 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 msgr2=0x563f92bb3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009990 tx=0x7f0a54004730 comp rx=0 tx=0).stop 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 shutdown_connections 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:6832/1001818622,v1:172.21.15.70:6833/1001818622] conn(0x563f92b377c0 0x563f92b38ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563f92b36ea0 0x563f92b37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563f92b0be60 0x563f92bb3160 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a50c00640 0 client.5901 ms_handle_reset on v2:172.21.15.70:6832/1001818622 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 --2- 172.21.15.3:0/522933185 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563f92b064f0 0x563f92b363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.032+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 >> 172.21.15.3:0/522933185 conn(0x563f92aec9e0 msgr2=0x563f92b35cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 shutdown_connections 2024-09-10T07:49:36.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a71ab6740 1 -- 172.21.15.3:0/522933185 wait complete. 2024-09-10T07:49:36.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a71ab6740 20 client.5901 trim_cache size 0 max 16384 2024-09-10T07:49:36.037 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:36.037 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-10T07:49:36.037 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-10T07:49:36.037 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 msgr2=0x7f0a4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 0x7f0a4806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a54034000 tx=0x7f0a5402e000 comp rx=0 tx=0).stop 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 msgr2=0x7f0a68141720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c4d0 tx=0x7f0a5800c990 comp rx=0 tx=0).stop 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 shutdown_connections 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0a4806be30 0x7f0a4806e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a68161b70 0x7f0a680b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0a680d45a0 0x7f0a680b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 --2- 172.21.15.3:0/146267512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a680cfe90 0x7f0a68141720 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.033+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 >> 172.21.15.3:0/146267512 conn(0x7f0a68062630 msgr2=0x7f0a6815cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.034+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 shutdown_connections 2024-09-10T07:49:36.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.034+0000 7f0a6d600640 1 -- 172.21.15.3:0/146267512 wait complete. 2024-09-10T07:49:36.048 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:49:36.048 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:49:36.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-10T07:49:36.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.207+0000 7f9fbd200640 1 Processor -- start 2024-09-10T07:49:36.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.207+0000 7f9fbd200640 1 -- start start 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.208+0000 7f9fbd200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fb8161b70 0x7f9fb80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.208+0000 7f9fbd200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.208+0000 7f9fbd200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb80d3de0 con 0x7f9fb80d3860 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.208+0000 7f9fbd200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb8161f50 con 0x7f9fb8161b70 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.208+0000 7f9fbd200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80d40c0 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fb8161b70 0x7f9fb80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:32808/0 (socket says 172.21.15.3:32808) 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb6a00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9fb80d3860 0x7f9fb80d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46586/0 (socket says 172.21.15.3:46586) 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 -- 172.21.15.3:0/3945151119 learned_addr learned my addr 172.21.15.3:0/3945151119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 -- 172.21.15.3:0/3945151119 >> v2:172.21.15.70:3300/0 conn(0x7f9fb8161b70 msgr2=0x7f9fb80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/3945151119 >> v2:172.21.15.70:3300/0 conn(0x7f9fb8161b70 0x7f9fb80d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 -- 172.21.15.3:0/3945151119 >> v1:172.21.15.70:6789/0 conn(0x7f9fb80d3860 legacy=0x7f9fb80d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb7e00640 1 -- 172.21.15.3:0/3945151119 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80cf690 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.209+0000 7f9fb6000640 1 -- 172.21.15.3:0/3945151119 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890539138 0 0) 0x7f9fb80d3de0 con 0x7f9fb80d3860 2024-09-10T07:49:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.210+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/3945151119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28654be5831f0d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fb6000640 1 -- 172.21.15.3:0/3945151119 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0036070 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fb6000640 1 -- 172.21.15.3:0/3945151119 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0031600 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fb6000640 1 -- 172.21.15.3:0/3945151119 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00319d0 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 -- 172.21.15.3:0/3945151119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 msgr2=0x7f9fb80cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 --2- 172.21.15.3:0/3945151119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb002a730 comp rx=0 tx=0).stop 2024-09-10T07:49:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 -- 172.21.15.3:0/3945151119 shutdown_connections 2024-09-10T07:49:36.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 --2- 172.21.15.3:0/3945151119 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d4cb0 0x7f9fb80cfe90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 --2- 172.21.15.3:0/3945151119 >> v2:172.21.15.70:3300/0 conn(0x7f9fb8161b70 0x7f9fb80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 -- 172.21.15.3:0/3945151119 >> 172.21.15.3:0/3945151119 conn(0x7f9fb8062630 msgr2=0x7f9fb8062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.211+0000 7f9fbd200640 1 -- 172.21.15.3:0/3945151119 shutdown_connections 2024-09-10T07:49:36.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.212+0000 7f9fbd200640 1 -- 172.21.15.3:0/3945151119 wait complete. 2024-09-10T07:49:36.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.213+0000 7f9fbd200640 1 Processor -- start 2024-09-10T07:49:36.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.213+0000 7f9fbd200640 1 -- start start 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d3860 0x7f9fb81f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb80d4cb0 0x7f9fb80b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80cf2d0 con 0x7f9fb8161b70 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80c9fc0 con 0x7f9fb80d4cb0 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.214+0000 7f9fbd200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80cf450 con 0x7f9fb80d3860 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53054/0 (socket says 172.21.15.3:53054) 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 -- 172.21.15.3:0/623641540 learned_addr learned my addr 172.21.15.3:0/623641540 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb80d4cb0 0x7f9fb80b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7400640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d3860 0x7f9fb81f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 -- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d3860 msgr2=0x7f9fb81f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d3860 0x7f9fb81f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 -- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb80d4cb0 msgr2=0x7f9fb80b8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb80d4cb0 0x7f9fb80b8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 -- 172.21.15.3:0/623641540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81f4f20 con 0x7f9fb8161b70 2024-09-10T07:49:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.215+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0034040 tx=0x7f9fb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.216+0000 7f9faf400640 1 -- 172.21.15.3:0/623641540 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0036050 con 0x7f9fb8161b70 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.216+0000 7f9faf400640 1 -- 172.21.15.3:0/623641540 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0042070 con 0x7f9fb8161b70 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.216+0000 7f9faf400640 1 -- 172.21.15.3:0/623641540 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009510 con 0x7f9fb8161b70 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.216+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb80b51b0 con 0x7f9fb8161b70 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.216+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb80b5770 con 0x7f9fb8161b70 2024-09-10T07:49:36.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.218+0000 7f9faf400640 1 -- 172.21.15.3:0/623641540 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fb0007bf0 con 0x7f9fb8161b70 2024-09-10T07:49:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.219+0000 7f9faf400640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 0x7f9f9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.219+0000 7f9faf400640 1 -- 172.21.15.3:0/623641540 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9fb00abad0 con 0x7f9fb8161b70 2024-09-10T07:49:36.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.219+0000 7f9fb7400640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 0x7f9f9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.219+0000 7f9fc17fa740 1 Processor -- start 2024-09-10T07:49:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.220+0000 7f9fb7400640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 0x7f9f9806e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9fb815ca60 tx=0x7f9fa8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.220+0000 7f9fc17fa740 1 -- start start 2024-09-10T07:49:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.220+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d773718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.220+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773579e0 0x555d77356240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555d772bf660 con 0x555d773a1ea0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555d77267680 con 0x555d773579e0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d772bb620 con 0x555d773714f0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53058/0 (socket says 172.21.15.3:53058) 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 -- 172.21.15.3:0/702047905 learned_addr learned my addr 172.21.15.3:0/702047905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7400640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d773718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773579e0 0x555d77356240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 -- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 msgr2=0x555d773718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d773718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 -- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773579e0 msgr2=0x555d77356240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773579e0 0x555d77356240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 -- 172.21.15.3:0/702047905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7709c7d0 con 0x555d773a1ea0 2024-09-10T07:49:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7400640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d773718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.221+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0079e40 tx=0x7f9fb00799f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ae7d08c66cc7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fa3e00640 1 -- 172.21.15.3:0/702047905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00a8070 con 0x555d773a1ea0 2024-09-10T07:49:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fa3e00640 1 -- 172.21.15.3:0/702047905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0037040 con 0x555d773a1ea0 2024-09-10T07:49:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fa3e00640 1 -- 172.21.15.3:0/702047905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0079eb0 con 0x555d773a1ea0 2024-09-10T07:49:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 -- 172.21.15.3:0/702047905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 msgr2=0x555d773a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0079e40 tx=0x7f9fb00799f0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 -- 172.21.15.3:0/702047905 shutdown_connections 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d773a23f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773579e0 0x555d77356240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/702047905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d773718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 -- 172.21.15.3:0/702047905 >> 172.21.15.3:0/702047905 conn(0x555d77358410 msgr2=0x555d773a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 -- 172.21.15.3:0/702047905 shutdown_connections 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.222+0000 7f9fc17fa740 1 -- 172.21.15.3:0/702047905 wait complete. 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.223+0000 7f9fc17fa740 1 -- start start 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.223+0000 7f9fc17fa740 1 Processor -- start 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773714f0 0x555d773b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773a1ea0 0x555d7735fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555d772bf660 con 0x555d773860c0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555d772484c0 con 0x555d773714f0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d77356940 con 0x555d773a1ea0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.224+0000 7f9fc17fa740 10 client.0 _ll_register_callbacks cb 0x555d772b7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53062/0 (socket says 172.21.15.3:53062) 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 -- 172.21.15.3:0/4123686465 learned_addr learned my addr 172.21.15.3:0/4123686465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773a1ea0 0x555d7735fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7400640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773714f0 0x555d773b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 -- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773a1ea0 msgr2=0x555d7735fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773a1ea0 0x555d7735fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 -- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773714f0 msgr2=0x555d773b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773714f0 0x555d773b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 -- 172.21.15.3:0/4123686465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d76fd23f0 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0079bb0 tx=0x7f9fb0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96d50f9aa0c73b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.225+0000 7f9fa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0086020 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9fb0079500 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb0037040 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fb00a8070 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 10 client.5910 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x555d76f920f0 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0037040 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f9fb00803b0 con 0x555d773860c0 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 10 client.5910 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 10 client.5910 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 20 client.5910 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739b790 0x555d7735da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 4 client.5910 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:49:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.226+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d772ccd90 con 0x555d7739b790 2024-09-10T07:49:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.227+0000 7f9fb7400640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739b790 0x555d7735da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.227+0000 7f9fb7400640 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739b790 0x555d7735da30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53971bf67cfe9518 server_cookie=4e14b94b2cd0475e in_seq=0 out_seq=0 2024-09-10T07:49:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.227+0000 7f9fa0c00640 10 client.5910 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fa0c00640 1 -- 172.21.15.3:0/4123686465 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d772ccd90 con 0x555d7739b790 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fa0c00640 10 client.5910 handle_command_reply: tid=0 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 client.5910 shutdown 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 msgr2=0x555d77385710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0079bb0 tx=0x7f9fb0009810 comp rx=0 tx=0).stop 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 shutdown_connections 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739b790 0x555d7735da30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385710 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773a1ea0 0x555d7735fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/4123686465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773714f0 0x555d773b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fa0c00640 0 client.5910 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 >> 172.21.15.3:0/4123686465 conn(0x555d773579e0 msgr2=0x555d7739ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 shutdown_connections 2024-09-10T07:49:36.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 1 -- 172.21.15.3:0/4123686465 wait complete. 2024-09-10T07:49:36.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.228+0000 7f9fc17fa740 20 client.5910 trim_cache size 0 max 16384 2024-09-10T07:49:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.247+0000 7f9fc17fa740 1 Processor -- start 2024-09-10T07:49:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.247+0000 7f9fc17fa740 1 -- start start 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.247+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77357020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773a1ea0 0x555d773859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8200ac0 con 0x555d773860c0 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ffcc0 con 0x555d773a1ea0 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb8200390 con 0x555d773714f0 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53068/0 (socket says 172.21.15.3:53068) 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 -- 172.21.15.3:0/3711679299 learned_addr learned my addr 172.21.15.3:0/3711679299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 -- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 msgr2=0x555d77357020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77357020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 -- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773a1ea0 msgr2=0x555d773859b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773a1ea0 0x555d773859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 -- 172.21.15.3:0/3711679299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d77385ef0 con 0x555d773860c0 2024-09-10T07:49:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.248+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007640 tx=0x7f9fa4007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d5a2f4e228481f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fa3400640 1 -- 172.21.15.3:0/3711679299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa401a070 con 0x555d773860c0 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fa3400640 1 -- 172.21.15.3:0/3711679299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4001830 con 0x555d773860c0 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fa3400640 1 -- 172.21.15.3:0/3711679299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4001be0 con 0x555d773860c0 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 -- 172.21.15.3:0/3711679299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 msgr2=0x555d77385470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007640 tx=0x7f9fa4007d50 comp rx=0 tx=0).stop 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 -- 172.21.15.3:0/3711679299 shutdown_connections 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773a1ea0 0x555d773859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773860c0 0x555d77385470 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/3711679299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77357020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 -- 172.21.15.3:0/3711679299 >> 172.21.15.3:0/3711679299 conn(0x555d773579e0 msgr2=0x555d7739c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 -- 172.21.15.3:0/3711679299 shutdown_connections 2024-09-10T07:49:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.249+0000 7f9fc17fa740 1 -- 172.21.15.3:0/3711679299 wait complete. 2024-09-10T07:49:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.250+0000 7f9fc17fa740 1 -- start start 2024-09-10T07:49:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.250+0000 7f9fc17fa740 1 Processor -- start 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77399490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 0x555d7739c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555d77360bc0 con 0x555d773a1ea0 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81f7a00 con 0x555d773860c0 2024-09-10T07:49:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d7739b080 con 0x555d773714f0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fc17fa740 10 client.0 _ll_register_callbacks cb 0x7f9fb00a6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fb7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53074/0 (socket says 172.21.15.3:53074) 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.251+0000 7f9fb7e00640 1 -- 172.21.15.3:0/1202835000 learned_addr learned my addr 172.21.15.3:0/1202835000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 -- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 msgr2=0x555d77399490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77399490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 -- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 msgr2=0x555d7739c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 0x555d7739c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 0x555d7739c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 -- 172.21.15.3:0/1202835000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d77351b10 con 0x555d773a1ea0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb6a00640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 0x555d7739c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fb7e00640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb00a9e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb5aeb42eee61ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0086040 con 0x555d773a1ea0 2024-09-10T07:49:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9fb0075e10 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa4001830 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0079810 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fc17fa740 10 client.5916 fetch_fsmap learned FSMap version 43 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.252+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x555d77370a90 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4001830 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 43) ==== 3550+0+0 (secure 0 0 0) 0x7f9fb0078d70 con 0x555d773a1ea0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 10 client.5916 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 10 client.5916 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 20 client.5916 populate_metadata read hostname 'smithi003' 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739c600 0x555d7741fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 4 client.5916 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 --> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d772c70e0 con 0x555d7739c600 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.253+0000 7f9fb7400640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739c600 0x555d7741fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fb7400640 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739c600 0x555d7741fac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fae73854d8d1404 server_cookie=8c527a66c47c309b in_seq=0 out_seq=0 2024-09-10T07:49:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fa0c00640 10 client.5916 ms_handle_connect on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fa0c00640 1 -- 172.21.15.3:0/1202835000 <== mds.0 v2:172.21.15.3:6834/29592196 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d772c70e0 con 0x555d7739c600 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fa0c00640 10 client.5916 handle_command_reply: tid=0 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 client.5916 shutdown 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 msgr2=0x555d7741f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb00a9e30 comp rx=0 tx=0).stop 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 shutdown_connections 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:6834/29592196,v1:172.21.15.3:6836/29592196] conn(0x555d7739c600 0x555d7741fac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d773a1ea0 0x555d7741f580 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d773860c0 0x555d7739c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 --2- 172.21.15.3:0/1202835000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d773714f0 0x555d77399490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 >> 172.21.15.3:0/1202835000 conn(0x555d773579e0 msgr2=0x555d773c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fa0c00640 0 client.5916 ms_handle_reset on v2:172.21.15.3:6834/29592196 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 shutdown_connections 2024-09-10T07:49:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 1 -- 172.21.15.3:0/1202835000 wait complete. 2024-09-10T07:49:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.254+0000 7f9fc17fa740 20 client.5916 trim_cache size 0 max 16384 2024-09-10T07:49:36.258 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:36.258 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-10T07:49:36.258 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-10T07:49:36.258 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 msgr2=0x7f9f9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 0x7f9f9806e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9fb815ca60 tx=0x7f9fa8009190 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 msgr2=0x7f9fb80cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0034040 tx=0x7f9fb0004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 shutdown_connections 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f9806be30 0x7f9f9806e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb8161b70 0x7f9fb80cbe50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb80d4cb0 0x7f9fb80b8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 --2- 172.21.15.3:0/623641540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fb80d3860 0x7f9fb81f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 >> 172.21.15.3:0/623641540 conn(0x7f9fb8062630 msgr2=0x7f9fb80b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 shutdown_connections 2024-09-10T07:49:36.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.255+0000 7f9fbd200640 1 -- 172.21.15.3:0/623641540 wait complete. 2024-09-10T07:49:36.270 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:49:36.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.427+0000 7fed66800640 1 Processor -- start 2024-09-10T07:49:36.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.427+0000 7fed66800640 1 -- start start 2024-09-10T07:49:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.428+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed60162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.428+0000 7fed66800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed600b7920 con 0x7fed600cf1f0 2024-09-10T07:49:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.428+0000 7fed66800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed600cf6a0 con 0x7fed6015e8e0 2024-09-10T07:49:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.428+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed600cf940 con 0x7fed600d1ba0 2024-09-10T07:49:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed600cf1f0 0x7fed600d32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39866/0 (socket says 172.21.15.3:39866) 2024-09-10T07:49:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5fe00640 1 -- 172.21.15.3:0/3483887685 learned_addr learned my addr 172.21.15.3:0/3483887685 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5ea00640 1 --2- 172.21.15.3:0/3483887685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed60162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180757713 0 0) 0x7fed58001490 con 0x7fed600cf1f0 2024-09-10T07:49:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed44003520 con 0x7fed600cf1f0 2024-09-10T07:49:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655775652 0 0) 0x7fed600cf6a0 con 0x7fed6015e8e0 2024-09-10T07:49:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed58001490 con 0x7fed6015e8e0 2024-09-10T07:49:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5ea00640 1 -- 172.21.15.3:0/3483887685 >> v1:172.21.15.70:6789/0 conn(0x7fed6015e8e0 legacy=0x7fed600d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5ea00640 1 -- 172.21.15.3:0/3483887685 >> v1:172.21.15.3:6789/0 conn(0x7fed600cf1f0 legacy=0x7fed600d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.429+0000 7fed5ea00640 1 -- 172.21.15.3:0/3483887685 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600cfc20 con 0x7fed600d1ba0 2024-09-10T07:49:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.430+0000 7fed5ea00640 1 --2- 172.21.15.3:0/3483887685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed60162cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097d0 tx=0x7fed4c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324ab97b46848716 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.431+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c037070 con 0x7fed600d1ba0 2024-09-10T07:49:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.431+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4c032440 con 0x7fed600d1ba0 2024-09-10T07:49:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.431+0000 7fed5e000640 1 -- 172.21.15.3:0/3483887685 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c032810 con 0x7fed600d1ba0 2024-09-10T07:49:36.431 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:49:36.431+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 -- 172.21.15.3:0/3483887685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 msgr2=0x7fed60162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 --2- 172.21.15.3:0/3483887685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed60162cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097d0 tx=0x7fed4c02a180 comp rx=0 tx=0).stop 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 -- 172.21.15.3:0/3483887685 shutdown_connections 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 --2- 172.21.15.3:0/3483887685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed60162cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 -- 172.21.15.3:0/3483887685 >> 172.21.15.3:0/3483887685 conn(0x7fed6006bd00 msgr2=0x7fed6006c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 -- 172.21.15.3:0/3483887685 shutdown_connections 2024-09-10T07:49:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.432+0000 7fed66800640 1 -- 172.21.15.3:0/3483887685 wait complete. 2024-09-10T07:49:36.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.434+0000 7fed66800640 1 Processor -- start 2024-09-10T07:49:36.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.434+0000 7fed66800640 1 -- start start 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.434+0000 7fed66800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.434+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed600c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 0x7fed600b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed66800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed600d39f0 con 0x7fed600cf1f0 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed600d2dc0 con 0x7fed6015e8e0 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed600d3b70 con 0x7fed600d1ba0 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53086/0 (socket says 172.21.15.3:53086) 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 -- 172.21.15.3:0/1488876483 learned_addr learned my addr 172.21.15.3:0/1488876483 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5ea00640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed600c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5fe00640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 0x7fed600b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 -- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 msgr2=0x7fed600c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed600c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 -- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 msgr2=0x7fed600b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 0x7fed600b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 -- 172.21.15.3:0/1488876483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600cab30 con 0x7fed600cf1f0 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5fe00640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 0x7fed600b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.435+0000 7fed5f400640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c460 tx=0x7fed5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.436+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50017070 con 0x7fed600cf1f0 2024-09-10T07:49:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.436+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50005310 con 0x7fed600cf1f0 2024-09-10T07:49:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.436+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed600cae10 con 0x7fed600cf1f0 2024-09-10T07:49:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.436+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50012870 con 0x7fed600cf1f0 2024-09-10T07:49:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.436+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed600cb3d0 con 0x7fed600cf1f0 2024-09-10T07:49:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.437+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed60162cf0 con 0x7fed600cf1f0 2024-09-10T07:49:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.438+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed5001a070 con 0x7fed600cf1f0 2024-09-10T07:49:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.439+0000 7fed57400640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.439+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fed5008a170 con 0x7fed600cf1f0 2024-09-10T07:49:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.439+0000 7fed5ea00640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.440+0000 7fed5ea00640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.444+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed500583a0 con 0x7fed600cf1f0 2024-09-10T07:49:36.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.664+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed600b9fa0 con 0x7fed600cf1f0 2024-09-10T07:49:36.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.665+0000 7fed57400640 1 -- 172.21.15.3:0/1488876483 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7fed5005c6a0 con 0x7fed600cf1f0 2024-09-10T07:49:36.665 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:36.665 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:36.665 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 msgr2=0x7fed4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).stop 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 msgr2=0x7fed6015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c460 tx=0x7fed5000c920 comp rx=0 tx=0).stop 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 shutdown_connections 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed6015e8e0 0x7fed600b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed600d1ba0 0x7fed600c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 --2- 172.21.15.3:0/1488876483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed600cf1f0 0x7fed6015c590 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 >> 172.21.15.3:0/1488876483 conn(0x7fed6006bd00 msgr2=0x7fed600b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 shutdown_connections 2024-09-10T07:49:36.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.667+0000 7fed66800640 1 -- 172.21.15.3:0/1488876483 wait complete. 2024-09-10T07:49:36.682 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-10T07:49:36.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:49:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.838+0000 7f1dc0400640 1 Processor -- start 2024-09-10T07:49:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.838+0000 7f1dc0400640 1 -- start start 2024-09-10T07:49:36.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dc0400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1db815c720 0x7f1db8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dc0400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1db8070f30 0x7f1db8073610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dc0400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80cd7f0 con 0x7f1db80cfdc0 2024-09-10T07:49:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dc0400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1db8072210 con 0x7f1db8070f30 2024-09-10T07:49:36.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dc0400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1db80cf880 con 0x7f1db815c720 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dbd000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1db8070f30 0x7f1db8073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dbe400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1db80cfdc0 0x7f1db80ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39870/0 (socket says 172.21.15.3:39870) 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.839+0000 7f1dbe400640 1 -- 172.21.15.3:0/2780259355 learned_addr learned my addr 172.21.15.3:0/2780259355 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.840+0000 7f1dbda00640 1 --2- 172.21.15.3:0/2780259355 >> v2:172.21.15.70:3300/0 conn(0x7f1db815c720 0x7f1db8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.840+0000 7f1dbd000640 1 -- 172.21.15.3:0/2780259355 >> v2:172.21.15.70:3300/0 conn(0x7f1db815c720 msgr2=0x7f1db8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.840+0000 7f1dbd000640 1 --2- 172.21.15.3:0/2780259355 >> v2:172.21.15.70:3300/0 conn(0x7f1db815c720 0x7f1db8071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.840+0000 7f1dbd000640 1 -- 172.21.15.3:0/2780259355 >> v1:172.21.15.3:6789/0 conn(0x7f1db80cfdc0 legacy=0x7f1db80ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.840+0000 7f1dbd000640 1 -- 172.21.15.3:0/2780259355 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80cda90 con 0x7f1db8070f30 2024-09-10T07:49:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.841+0000 7f1dbd000640 1 --2- 172.21.15.3:0/2780259355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db8070f30 0x7f1db8073610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1db8075bd0 tx=0x7f1da402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3008665faed12aac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.841+0000 7f1db7e00640 1 -- 172.21.15.3:0/2780259355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4037070 con 0x7f1db8070f30 2024-09-10T07:49:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.841+0000 7f1db7e00640 1 -- 172.21.15.3:0/2780259355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4032440 con 0x7f1db8070f30 2024-09-10T07:49:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.841+0000 7f1db7e00640 1 -- 172.21.15.3:0/2780259355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da402ae60 con 0x7f1db8070f30 2024-09-10T07:49:36.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 -- 172.21.15.3:0/2780259355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db8070f30 msgr2=0x7f1db8073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 --2- 172.21.15.3:0/2780259355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db8070f30 0x7f1db8073610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1db8075bd0 tx=0x7f1da402a110 comp rx=0 tx=0).stop 2024-09-10T07:49:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 -- 172.21.15.3:0/2780259355 shutdown_connections 2024-09-10T07:49:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 --2- 172.21.15.3:0/2780259355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db8070f30 0x7f1db8073610 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 --2- 172.21.15.3:0/2780259355 >> v2:172.21.15.70:3300/0 conn(0x7f1db815c720 0x7f1db8071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 -- 172.21.15.3:0/2780259355 >> 172.21.15.3:0/2780259355 conn(0x7f1db806cfc0 msgr2=0x7f1db806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 -- 172.21.15.3:0/2780259355 shutdown_connections 2024-09-10T07:49:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.842+0000 7f1dc0400640 1 -- 172.21.15.3:0/2780259355 wait complete. 2024-09-10T07:49:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.843+0000 7f1dc0400640 1 Processor -- start 2024-09-10T07:49:36.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 -- start start 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 0x7f1db80b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80cfdc0 0x7f1db80b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8073b50 con 0x7f1db815c720 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1db806ed10 con 0x7f1db80cfdc0 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.844+0000 7f1dc0400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8073cd0 con 0x7f1db8070f30 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53102/0 (socket says 172.21.15.3:53102) 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 -- 172.21.15.3:0/3073678338 learned_addr learned my addr 172.21.15.3:0/3073678338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbd000640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80cfdc0 0x7f1db80b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbda00640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 0x7f1db80b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 -- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 msgr2=0x7f1db80b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 0x7f1db80b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 -- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80cfdc0 msgr2=0x7f1db80b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80cfdc0 0x7f1db80b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 -- 172.21.15.3:0/3073678338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80c56d0 con 0x7f1db815c720 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbda00640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 0x7f1db80b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:36.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.845+0000 7f1dbe400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1db000cbc0 tx=0x7f1db000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.846+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0017070 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.846+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db000ad40 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.846+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0012600 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.846+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db80b1c30 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.846+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db80b24d0 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.847+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db8073610 con 0x7f1db815c720 2024-09-10T07:49:36.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.848+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1db001a070 con 0x7f1db815c720 2024-09-10T07:49:36.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.848+0000 7f1db5600640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 0x7f1d9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:36.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.849+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f1db008ac20 con 0x7f1db815c720 2024-09-10T07:49:36.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.849+0000 7f1dbda00640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 0x7f1d9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:36.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.849+0000 7f1dbda00640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 0x7f1d9806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009af0 tx=0x7f1dac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:36.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:36.853+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1db0058e50 con 0x7f1db815c720 2024-09-10T07:49:37.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.104+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1db80cf2b0 con 0x7f1db815c720 2024-09-10T07:49:37.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.105+0000 7f1db5600640 1 -- 172.21.15.3:0/3073678338 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7f1db005d150 con 0x7f1db815c720 2024-09-10T07:49:37.105 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:37.105 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706301,"bytes_used":253321216,"bytes_avail":772840792064,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":4094,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":5266},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5430},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5433}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T07:49:31.529875+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:49:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 msgr2=0x7f1d9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 0x7f1d9806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009af0 tx=0x7f1dac008040 comp rx=0 tx=0).stop 2024-09-10T07:49:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 msgr2=0x7f1db814bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1db000cbc0 tx=0x7f1db000a430 comp rx=0 tx=0).stop 2024-09-10T07:49:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 shutdown_connections 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1d9806be30 0x7f1d9806e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db815c720 0x7f1db814bc10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80cfdc0 0x7f1db80b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 --2- 172.21.15.3:0/3073678338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1db8070f30 0x7f1db80b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 >> 172.21.15.3:0/3073678338 conn(0x7f1db806cfc0 msgr2=0x7f1db815ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 shutdown_connections 2024-09-10T07:49:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:37.107+0000 7f1dc0400640 1 -- 172.21.15.3:0/3073678338 wait complete. 2024-09-10T07:49:37.122 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:49:38.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:38.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.293+0000 7fd326400640 1 Processor -- start 2024-09-10T07:49:38.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.293+0000 7fd326400640 1 -- start start 2024-09-10T07:49:38.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd326400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3200d3c10 0x7fd320073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd326400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3200d4050 0x7fd320073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd326400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3200cfdc0 con 0x7fd32015e860 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd326400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd3200d3780 con 0x7fd3200d4050 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd326400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd3200cee40 con 0x7fd3200d3c10 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd32015e860 0x7fd3200cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39876/0 (socket says 172.21.15.3:39876) 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325000640 1 -- 172.21.15.3:0/3517660755 learned_addr learned my addr 172.21.15.3:0/3517660755 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325a00640 1 --2- 172.21.15.3:0/3517660755 >> v2:172.21.15.3:3300/0 conn(0x7fd3200d4050 0x7fd320073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd31fe00640 1 --2- 172.21.15.3:0/3517660755 >> v2:172.21.15.70:3300/0 conn(0x7fd3200d3c10 0x7fd320073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325a00640 1 -- 172.21.15.3:0/3517660755 >> v2:172.21.15.70:3300/0 conn(0x7fd3200d3c10 msgr2=0x7fd320073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325a00640 1 --2- 172.21.15.3:0/3517660755 >> v2:172.21.15.70:3300/0 conn(0x7fd3200d3c10 0x7fd320073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325a00640 1 -- 172.21.15.3:0/3517660755 >> v1:172.21.15.3:6789/0 conn(0x7fd32015e860 legacy=0x7fd3200cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd325a00640 1 -- 172.21.15.3:0/3517660755 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3200cf120 con 0x7fd3200d4050 2024-09-10T07:49:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.294+0000 7fd31f400640 1 -- 172.21.15.3:0/3517660755 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992162275 0 0) 0x7fd3200cfdc0 con 0x7fd32015e860 2024-09-10T07:49:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.295+0000 7fd325a00640 1 --2- 172.21.15.3:0/3517660755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d4050 0x7fd320073690 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd3200d0ad0 tx=0x7fd31802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc069c1e3bea35d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.295+0000 7fd31f400640 1 -- 172.21.15.3:0/3517660755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318007740 con 0x7fd3200d4050 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.295+0000 7fd31f400640 1 -- 172.21.15.3:0/3517660755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3180078e0 con 0x7fd3200d4050 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.295+0000 7fd31f400640 1 -- 172.21.15.3:0/3517660755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318007cb0 con 0x7fd3200d4050 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 -- 172.21.15.3:0/3517660755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d4050 msgr2=0x7fd320073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 --2- 172.21.15.3:0/3517660755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d4050 0x7fd320073690 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd3200d0ad0 tx=0x7fd31802a680 comp rx=0 tx=0).stop 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 -- 172.21.15.3:0/3517660755 shutdown_connections 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 --2- 172.21.15.3:0/3517660755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d4050 0x7fd320073690 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 --2- 172.21.15.3:0/3517660755 >> v2:172.21.15.70:3300/0 conn(0x7fd3200d3c10 0x7fd320073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 -- 172.21.15.3:0/3517660755 >> 172.21.15.3:0/3517660755 conn(0x7fd3200d4650 msgr2=0x7fd3200d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 -- 172.21.15.3:0/3517660755 shutdown_connections 2024-09-10T07:49:38.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.296+0000 7fd326400640 1 -- 172.21.15.3:0/3517660755 wait complete. 2024-09-10T07:49:38.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.298+0000 7fd326400640 1 Processor -- start 2024-09-10T07:49:38.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.298+0000 7fd326400640 1 -- start start 2024-09-10T07:49:38.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3200d4050 0x7fd32015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd32015e860 0x7fd3201610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3200d2460 con 0x7fd3200d3c10 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd3200cf400 con 0x7fd32015e860 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd326400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3200b7880 con 0x7fd3200d4050 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53114/0 (socket says 172.21.15.3:53114) 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 -- 172.21.15.3:0/1858349859 learned_addr learned my addr 172.21.15.3:0/1858349859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325a00640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd32015e860 0x7fd3201610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd31fe00640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3200d4050 0x7fd32015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 -- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3200d4050 msgr2=0x7fd32015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3200d4050 0x7fd32015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 -- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd32015e860 msgr2=0x7fd3201610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd32015e860 0x7fd3201610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.299+0000 7fd325000640 1 -- 172.21.15.3:0/1858349859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3200aafd0 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd325000640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c430 tx=0x7fd31400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140074e0 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3140047e0 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3200ab290 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314004bd0 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.300+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3200ab850 con 0x7fd3200d3c10 2024-09-10T07:49:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.301+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3200b7c40 con 0x7fd3200d3c10 2024-09-10T07:49:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.302+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd314011070 con 0x7fd3200d3c10 2024-09-10T07:49:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.303+0000 7fd31cc00640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.303+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fd31408ab10 con 0x7fd3200d3c10 2024-09-10T07:49:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.303+0000 7fd31fe00640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.304+0000 7fd31fe00640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 0x7fd30006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3200b3f50 tx=0x7fd30c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.307+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd314059390 con 0x7fd3200d3c10 2024-09-10T07:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.529+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3200b5d20 con 0x7fd3200d3c10 2024-09-10T07:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.531+0000 7fd31cc00640 1 -- 172.21.15.3:0/1858349859 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7fd314058c90 con 0x7fd3200d3c10 2024-09-10T07:49:38.530 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:38.530 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 msgr2=0x7fd30006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 0x7fd30006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3200b3f50 tx=0x7fd30c008040 comp rx=0 tx=0).stop 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 msgr2=0x7fd3200ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c430 tx=0x7fd31400c8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 shutdown_connections 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd30006be30 0x7fd30006e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd32015e860 0x7fd3201610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3200d4050 0x7fd32015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 --2- 172.21.15.3:0/1858349859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3200d3c10 0x7fd3200ac050 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 >> 172.21.15.3:0/1858349859 conn(0x7fd3200d4650 msgr2=0x7fd3200b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:38.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 shutdown_connections 2024-09-10T07:49:38.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.533+0000 7fd326400640 1 -- 172.21.15.3:0/1858349859 wait complete. 2024-09-10T07:49:38.548 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-10T07:49:38.548 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:49:38.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.707+0000 7fa606a00640 1 Processor -- start 2024-09-10T07:49:38.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.707+0000 7fa606a00640 1 -- start start 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa606a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6000714b0 0x7fa600071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa606a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa600074200 con 0x7fa6000cfbe0 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa606a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa600073c40 con 0x7fa600145d00 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa606a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa600160e40 con 0x7fa6000714b0 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa604c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa6000cfbe0 0x7fa600160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39878/0 (socket says 172.21.15.3:39878) 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa604c00640 1 -- 172.21.15.3:0/3110070071 learned_addr learned my addr 172.21.15.3:0/3110070071 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:38.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.708+0000 7fa606000640 1 --2- 172.21.15.3:0/3110070071 >> v2:172.21.15.3:3300/0 conn(0x7fa6000714b0 0x7fa600071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.709+0000 7fa606000640 1 -- 172.21.15.3:0/3110070071 >> v1:172.21.15.70:6789/0 conn(0x7fa600145d00 legacy=0x7fa600073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.709+0000 7fa606000640 1 -- 172.21.15.3:0/3110070071 >> v1:172.21.15.3:6789/0 conn(0x7fa6000cfbe0 legacy=0x7fa600160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.709+0000 7fa606000640 1 -- 172.21.15.3:0/3110070071 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa600073f20 con 0x7fa6000714b0 2024-09-10T07:49:38.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa606000640 1 --2- 172.21.15.3:0/3110070071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600071900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a6ea0b9806b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa5ffe00640 1 -- 172.21.15.3:0/3110070071 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8007740 con 0x7fa6000714b0 2024-09-10T07:49:38.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa5ffe00640 1 -- 172.21.15.3:0/3110070071 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f80078e0 con 0x7fa6000714b0 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa5ffe00640 1 -- 172.21.15.3:0/3110070071 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8007cb0 con 0x7fa6000714b0 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa606a00640 1 -- 172.21.15.3:0/3110070071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 msgr2=0x7fa600071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.710+0000 7fa606a00640 1 --2- 172.21.15.3:0/3110070071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600071900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f802a6f0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.711+0000 7fa606a00640 1 -- 172.21.15.3:0/3110070071 shutdown_connections 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.711+0000 7fa606a00640 1 --2- 172.21.15.3:0/3110070071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600071900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.711+0000 7fa606a00640 1 -- 172.21.15.3:0/3110070071 >> 172.21.15.3:0/3110070071 conn(0x7fa60006bb20 msgr2=0x7fa60006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.711+0000 7fa606a00640 1 -- 172.21.15.3:0/3110070071 shutdown_connections 2024-09-10T07:49:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.711+0000 7fa606a00640 1 -- 172.21.15.3:0/3110070071 wait complete. 2024-09-10T07:49:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.712+0000 7fa606a00640 1 Processor -- start 2024-09-10T07:49:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.713+0000 7fa606a00640 1 -- start start 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.713+0000 7fa606a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.713+0000 7fa606a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 0x7fa600143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.713+0000 7fa606a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 0x7fa60013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa606a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa600160210 con 0x7fa6000714b0 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa606a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa600071e40 con 0x7fa600145d00 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa606a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa6000ce200 con 0x7fa6000cfbe0 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53126/0 (socket says 172.21.15.3:53126) 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 -- 172.21.15.3:0/3774948998 learned_addr learned my addr 172.21.15.3:0/3774948998 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa606000640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 0x7fa60013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 -- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 msgr2=0x7fa600143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa604c00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 0x7fa600143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 0x7fa600143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 -- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 msgr2=0x7fa60013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 0x7fa60013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa605600640 1 -- 172.21.15.3:0/3774948998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60015dd80 con 0x7fa6000714b0 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa606000640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 0x7fa60013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.714+0000 7fa604c00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 0x7fa600143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa605600640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c3c0 tx=0x7fa5f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40052d0 con 0x7fa6000714b0 2024-09-10T07:49:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4005490 con 0x7fa6000714b0 2024-09-10T07:49:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4007260 con 0x7fa6000714b0 2024-09-10T07:49:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60015e060 con 0x7fa6000714b0 2024-09-10T07:49:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.715+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60015e620 con 0x7fa6000714b0 2024-09-10T07:49:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.716+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa600071900 con 0x7fa6000714b0 2024-09-10T07:49:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.717+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5f4007400 con 0x7fa6000714b0 2024-09-10T07:49:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.718+0000 7fa5fd600640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 0x7fa5e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.718+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fa5f408a340 con 0x7fa6000714b0 2024-09-10T07:49:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.718+0000 7fa604c00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 0x7fa5e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.719+0000 7fa604c00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 0x7fa5e006e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6000cd950 tx=0x7fa5ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:38.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.722+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5f4058cf0 con 0x7fa6000714b0 2024-09-10T07:49:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.976+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa6000cdd10 con 0x7fa6000714b0 2024-09-10T07:49:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.977+0000 7fa5fd600640 1 -- 172.21.15.3:0/3774948998 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1932 (secure 0 0 0) 0x7fa5f40585f0 con 0x7fa6000714b0 2024-09-10T07:49:38.976 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:38.976 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706301,"bytes_used":253321216,"bytes_avail":772840792064,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":4094,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":5266},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5430},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5433}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T07:49:37.531609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 msgr2=0x7fa5e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 0x7fa5e006e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6000cd950 tx=0x7fa5ec009040 comp rx=0 tx=0).stop 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 msgr2=0x7fa600144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c3c0 tx=0x7fa5f400c880 comp rx=0 tx=0).stop 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 shutdown_connections 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa5e006be30 0x7fa5e006e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa600145d00 0x7fa60013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6000cfbe0 0x7fa600143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 --2- 172.21.15.3:0/3774948998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6000714b0 0x7fa600144a80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 >> 172.21.15.3:0/3774948998 conn(0x7fa60006bb20 msgr2=0x7fa6000affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 shutdown_connections 2024-09-10T07:49:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:38.979+0000 7fa606a00640 1 -- 172.21.15.3:0/3774948998 wait complete. 2024-09-10T07:49:38.994 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:49:38.994 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:49:38.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:49:39.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:49:39.184 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:49:39.184 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:49:39.184 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:39.184 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:39.220 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:39.220 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:39.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:49:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.494+0000 7f4065a00640 1 Processor -- start 2024-09-10T07:49:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.494+0000 7f4065a00640 1 -- start start 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f4065a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 0x7f40600d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f4065a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40600cfdd0 con 0x7f4060160710 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f4065a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406015cc90 con 0x7f40600d05f0 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f4065a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f406015cf70 con 0x7f4060160af0 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f405ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4060160710 0x7f40600cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39884/0 (socket says 172.21.15.3:39884) 2024-09-10T07:49:39.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.495+0000 7f405ea00640 1 -- 172.21.15.3:0/2179854298 learned_addr learned my addr 172.21.15.3:0/2179854298 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405e000640 1 --2- 172.21.15.3:0/2179854298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 0x7f40600d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4089893791 0 0) 0x7f40600cfdd0 con 0x7f4060160710 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4044003520 con 0x7f4060160710 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 51105123 0 0) 0x7f406015cc90 con 0x7f40600d05f0 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40600cfdd0 con 0x7f40600d05f0 2024-09-10T07:49:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404771638 0 0) 0x7f4044003520 con 0x7f4060160710 2024-09-10T07:49:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405e000640 1 -- 172.21.15.3:0/2179854298 >> v1:172.21.15.70:6789/0 conn(0x7f40600d05f0 legacy=0x7f4060072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405e000640 1 -- 172.21.15.3:0/2179854298 >> v1:172.21.15.3:6789/0 conn(0x7f4060160710 legacy=0x7f40600cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.496+0000 7f405e000640 1 -- 172.21.15.3:0/2179854298 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406015d250 con 0x7f4060160af0 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.497+0000 7f405e000640 1 --2- 172.21.15.3:0/2179854298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 0x7f40600d00b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f404c0097d0 tx=0x7f404c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee37255994115edb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c037070 con 0x7f4060160af0 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c02ac30 con 0x7f4060160af0 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f405d600640 1 -- 172.21.15.3:0/2179854298 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050001670 con 0x7f4060160af0 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 -- 172.21.15.3:0/2179854298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 msgr2=0x7f40600d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 --2- 172.21.15.3:0/2179854298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 0x7f40600d00b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f404c0097d0 tx=0x7f404c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 -- 172.21.15.3:0/2179854298 shutdown_connections 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 --2- 172.21.15.3:0/2179854298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4060160af0 0x7f40600d00b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 -- 172.21.15.3:0/2179854298 >> 172.21.15.3:0/2179854298 conn(0x7f406006d000 msgr2=0x7f406006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 -- 172.21.15.3:0/2179854298 shutdown_connections 2024-09-10T07:49:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.498+0000 7f4065a00640 1 -- 172.21.15.3:0/2179854298 wait complete. 2024-09-10T07:49:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.500+0000 7f4065a00640 1 Processor -- start 2024-09-10T07:49:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.500+0000 7f4065a00640 1 -- start start 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 0x7f4060152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 0x7f40601589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f406015d770 con 0x7f4060160710 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40600d0a10 con 0x7f4060160af0 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f4065a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4060152650 con 0x7f40600d05f0 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f405e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f405f400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 0x7f40601589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:39.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f405ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 0x7f4060152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:39.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f405e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53128/0 (socket says 172.21.15.3:53128) 2024-09-10T07:49:39.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.501+0000 7f405e000640 1 -- 172.21.15.3:0/1861566453 learned_addr learned my addr 172.21.15.3:0/1861566453 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 -- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 msgr2=0x7f4060152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 0x7f4060152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 -- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 msgr2=0x7f40601589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 0x7f40601589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 -- 172.21.15.3:0/1861566453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4060156b50 con 0x7f4060160710 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405ea00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 0x7f4060152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405f400640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 0x7f40601589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f405e000640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f404c0097d0 tx=0x7f404c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:39.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c037050 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c038040 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.502+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c006e10 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.503+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406015b240 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.503+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4060156dc0 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.504+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40600cfa60 con 0x7f4060160710 2024-09-10T07:49:39.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.504+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f404c045020 con 0x7f4060160710 2024-09-10T07:49:39.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.506+0000 7f4056a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 0x7f404006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:39.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.506+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f404c0aaed0 con 0x7f4060160710 2024-09-10T07:49:39.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.506+0000 7f405ea00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 0x7f404006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:39.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.507+0000 7f405ea00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 0x7f404006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4050001eb0 tx=0x7f4050001610 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:39.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.510+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f404c04c020 con 0x7f4060160710 2024-09-10T07:49:39.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.712+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f406015d8f0 con 0x7f4060160710 2024-09-10T07:49:39.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.712+0000 7f4056a00640 1 -- 172.21.15.3:0/1861566453 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f404c079100 con 0x7f4060160710 2024-09-10T07:49:39.711 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 msgr2=0x7f404006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 0x7f404006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4050001eb0 tx=0x7f4050001610 comp rx=0 tx=0).stop 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 msgr2=0x7f4060158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f404c0097d0 tx=0x7f404c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 shutdown_connections 2024-09-10T07:49:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f404006be30 0x7f404006e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4060160af0 0x7f40601589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4060160710 0x7f4060158460 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 --2- 172.21.15.3:0/1861566453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40600d05f0 0x7f4060152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 >> 172.21.15.3:0/1861566453 conn(0x7f406006d000 msgr2=0x7f40600aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 shutdown_connections 2024-09-10T07:49:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:39.714+0000 7f4065a00640 1 -- 172.21.15.3:0/1861566453 wait complete. 2024-09-10T07:49:39.728 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:49:39.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:39.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:49:39.795 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:49:39.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:49:39.795 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:49:39.831 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:49:39.831 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:49:39.832 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:39.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:49:39.871 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:49:39.871 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:39.871 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:39.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:49:39.906 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:49:39.907 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:49:39.907 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:39.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:49:39.950 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.108+0000 7f3210800640 1 Processor -- start 2024-09-10T07:49:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.108+0000 7f3210800640 1 -- start start 2024-09-10T07:49:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.109+0000 7f3210800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3208160af0 0x7f32080d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.109+0000 7f3210800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f32080d05f0 0x7f3208072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.109+0000 7f3210800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32080cfdd0 con 0x7f3208160710 2024-09-10T07:49:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.109+0000 7f3210800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f320815cc90 con 0x7f32080d05f0 2024-09-10T07:49:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.109+0000 7f3210800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f320815cf70 con 0x7f3208160af0 2024-09-10T07:49:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320d400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3208160af0 0x7f32080d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320e800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f32080d05f0 0x7f3208072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320de00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3208160710 0x7f32080cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46626/0 (socket says 172.21.15.3:46626) 2024-09-10T07:49:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320de00640 1 -- 172.21.15.3:0/2314032452 learned_addr learned my addr 172.21.15.3:0/2314032452 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320e800640 1 -- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3208160af0 msgr2=0x7f32080d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320e800640 1 --2- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3208160af0 0x7f32080d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320e800640 1 -- 172.21.15.3:0/2314032452 >> v1:172.21.15.70:6789/0 conn(0x7f3208160710 legacy=0x7f32080cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320e800640 1 -- 172.21.15.3:0/2314032452 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320815d250 con 0x7f32080d05f0 2024-09-10T07:49:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.110+0000 7f320ca00640 1 -- 172.21.15.3:0/2314032452 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630178329 0 0) 0x7f32080cfdd0 con 0x7f3208160710 2024-09-10T07:49:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.111+0000 7f320e800640 1 --2- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32080d05f0 0x7f3208072070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a08f2049a706ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f320ca00640 1 -- 172.21.15.3:0/2314032452 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204007740 con 0x7f32080d05f0 2024-09-10T07:49:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f320ca00640 1 -- 172.21.15.3:0/2314032452 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32040078e0 con 0x7f32080d05f0 2024-09-10T07:49:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f320ca00640 1 -- 172.21.15.3:0/2314032452 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204007cb0 con 0x7f32080d05f0 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 -- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32080d05f0 msgr2=0x7f3208072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 --2- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32080d05f0 0x7f3208072070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320402a660 comp rx=0 tx=0).stop 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 -- 172.21.15.3:0/2314032452 shutdown_connections 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 --2- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32080d05f0 0x7f3208072070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 --2- 172.21.15.3:0/2314032452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3208160af0 0x7f32080d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 -- 172.21.15.3:0/2314032452 >> 172.21.15.3:0/2314032452 conn(0x7f320806d000 msgr2=0x7f320806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 -- 172.21.15.3:0/2314032452 shutdown_connections 2024-09-10T07:49:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.112+0000 7f3210800640 1 -- 172.21.15.3:0/2314032452 wait complete. 2024-09-10T07:49:40.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.114+0000 7f3210800640 1 Processor -- start 2024-09-10T07:49:40.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.114+0000 7f3210800640 1 -- start start 2024-09-10T07:49:40.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32080d05f0 0x7f3208146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 0x7f3208155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f320815d770 con 0x7f3208160af0 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32080d0a10 con 0x7f3208160710 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f3210800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3208147410 con 0x7f32080d05f0 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.115+0000 7f320e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53142/0 (socket says 172.21.15.3:53142) 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 -- 172.21.15.3:0/4095615413 learned_addr learned my addr 172.21.15.3:0/4095615413 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:40.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320d400640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 0x7f3208155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320de00640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32080d05f0 0x7f3208146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 -- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32080d05f0 msgr2=0x7f3208146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32080d05f0 0x7f3208146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 -- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 msgr2=0x7f3208155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 0x7f3208155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 -- 172.21.15.3:0/4095615413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320815b9f0 con 0x7f3208160af0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320d400640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 0x7f3208155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.116+0000 7f320e800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f3204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.117+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204040070 con 0x7f3208160af0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.117+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204031c40 con 0x7f3208160af0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.117+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204009450 con 0x7f3208160af0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.117+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320815bcd0 con 0x7f3208160af0 2024-09-10T07:49:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.117+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081538e0 con 0x7f3208160af0 2024-09-10T07:49:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.118+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32080cfa60 con 0x7f3208160af0 2024-09-10T07:49:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.119+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3204039040 con 0x7f3208160af0 2024-09-10T07:49:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.120+0000 7f3202000640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 0x7f31ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.120+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f32040abd80 con 0x7f3208160af0 2024-09-10T07:49:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.120+0000 7f320de00640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 0x7f31ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.121+0000 7f320de00640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 0x7f31ec06e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f60 tx=0x7f31fc007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.124+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f320407a690 con 0x7f3208160af0 2024-09-10T07:49:40.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.350+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32080afd50 con 0x7f3208160af0 2024-09-10T07:49:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.350+0000 7f3202000640 1 -- 172.21.15.3:0/4095615413 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7f3204079f90 con 0x7f3208160af0 2024-09-10T07:49:40.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:40.349 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:40.349 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 msgr2=0x7f31ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 0x7f31ec06e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f60 tx=0x7f31fc007b50 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 msgr2=0x7f32081563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f3204004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 shutdown_connections 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f31ec06be30 0x7f31ec06e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3208160af0 0x7f32081563e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3208160710 0x7f3208155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 --2- 172.21.15.3:0/4095615413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f32080d05f0 0x7f3208146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 >> 172.21.15.3:0/4095615413 conn(0x7f320806d000 msgr2=0x7f32080cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 shutdown_connections 2024-09-10T07:49:40.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.353+0000 7f3210800640 1 -- 172.21.15.3:0/4095615413 wait complete. 2024-09-10T07:49:40.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.524+0000 7f2ceee00640 1 Processor -- start 2024-09-10T07:49:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.524+0000 7f2ceee00640 1 -- start start 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.525+0000 7f2ceee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.525+0000 7f2ceee00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce8073d10 0x7f2ce80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.525+0000 7f2ceee00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce80ccd80 con 0x7f2ce815b110 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.525+0000 7f2ceee00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce80b44f0 con 0x7f2ce8073d10 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.525+0000 7f2ceee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce80cd060 con 0x7f2ce815b4f0 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2ce7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ce815b110 0x7f2ce80b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39900/0 (socket says 172.21.15.3:39900) 2024-09-10T07:49:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2ce7e00640 1 -- 172.21.15.3:0/664506461 learned_addr learned my addr 172.21.15.3:0/664506461 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:40.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2cece00640 1 --2- 172.21.15.3:0/664506461 >> v2:172.21.15.3:3300/0 conn(0x7f2ce8073d10 0x7f2ce80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2ce7400640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2ce6a00640 1 -- 172.21.15.3:0/664506461 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977514327 0 0) 0x7f2ce80ccd80 con 0x7f2ce815b110 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2ce6a00640 1 -- 172.21.15.3:0/664506461 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ccc003520 con 0x7f2ce815b110 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2cece00640 1 -- 172.21.15.3:0/664506461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 msgr2=0x7f2ce80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2cece00640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce80d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2cece00640 1 -- 172.21.15.3:0/664506461 >> v1:172.21.15.3:6789/0 conn(0x7f2ce815b110 legacy=0x7f2ce80b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.526+0000 7f2cece00640 1 -- 172.21.15.3:0/664506461 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce80cd340 con 0x7f2ce8073d10 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.527+0000 7f2cece00640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce80d0720 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009d90 tx=0x7f2ce002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f9bb8e992c79b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.527+0000 7f2ce6a00640 1 -- 172.21.15.3:0/664506461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0007740 con 0x7f2ce8073d10 2024-09-10T07:49:40.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.527+0000 7f2ce6a00640 1 -- 172.21.15.3:0/664506461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce00078e0 con 0x7f2ce8073d10 2024-09-10T07:49:40.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.527+0000 7f2ce6a00640 1 -- 172.21.15.3:0/664506461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0007c90 con 0x7f2ce8073d10 2024-09-10T07:49:40.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 -- 172.21.15.3:0/664506461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 msgr2=0x7f2ce80d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce80d0720 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009d90 tx=0x7f2ce002a660 comp rx=0 tx=0).stop 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 -- 172.21.15.3:0/664506461 shutdown_connections 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce80d0720 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 --2- 172.21.15.3:0/664506461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce80d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 -- 172.21.15.3:0/664506461 >> 172.21.15.3:0/664506461 conn(0x7f2ce806d000 msgr2=0x7f2ce806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 -- 172.21.15.3:0/664506461 shutdown_connections 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.528+0000 7f2ceee00640 1 -- 172.21.15.3:0/664506461 wait complete. 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.530+0000 7f2ceee00640 1 Processor -- start 2024-09-10T07:49:40.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.530+0000 7f2ceee00640 1 -- start start 2024-09-10T07:49:40.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce815b110 0x7f2ce814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce80cdcc0 con 0x7f2ce8073d10 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce80cdb40 con 0x7f2ce815b110 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ceee00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce80cde40 con 0x7f2ce815b4f0 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53160/0 (socket says 172.21.15.3:53160) 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 -- 172.21.15.3:0/1870899908 learned_addr learned my addr 172.21.15.3:0/1870899908 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:40.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7400640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce815b110 0x7f2ce814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2cece00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 -- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 msgr2=0x7f2ce814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce814fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 -- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce815b110 msgr2=0x7f2ce814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce815b110 0x7f2ce814f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.531+0000 7f2ce7e00640 1 -- 172.21.15.3:0/1870899908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce80b1840 con 0x7f2ce8073d10 2024-09-10T07:49:40.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2ce7e00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c430 tx=0x7f2cd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8017070 con 0x7f2ce8073d10 2024-09-10T07:49:40.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8005ce0 con 0x7f2ce8073d10 2024-09-10T07:49:40.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80072f0 con 0x7f2ce8073d10 2024-09-10T07:49:40.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce815b8d0 con 0x7f2ce8073d10 2024-09-10T07:49:40.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.532+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce815be90 con 0x7f2ce8073d10 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.533+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce80b3d10 con 0x7f2ce8073d10 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.534+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cd801a070 con 0x7f2ce8073d10 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.535+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 0x7f2cc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.535+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f2cd808a3d0 con 0x7f2ce8073d10 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.535+0000 7f2ce7400640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 0x7f2cc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.536+0000 7f2ce7400640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 0x7f2cc806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80af0d0 tx=0x7f2cd4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.539+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cd8058610 con 0x7f2ce8073d10 2024-09-10T07:49:40.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.764+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ce80aeb40 con 0x7f2ce8073d10 2024-09-10T07:49:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.765+0000 7f2cdfe00640 1 -- 172.21.15.3:0/1870899908 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7f2cd805c910 con 0x7f2ce8073d10 2024-09-10T07:49:40.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:40.764 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 msgr2=0x7f2cc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 0x7f2cc806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80af0d0 tx=0x7f2cd4009190 comp rx=0 tx=0).stop 2024-09-10T07:49:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 msgr2=0x7f2ce814e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c430 tx=0x7f2cd800c8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 shutdown_connections 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc806be30 0x7f2cc806e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce815b4f0 0x7f2ce814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce815b110 0x7f2ce814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 --2- 172.21.15.3:0/1870899908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce8073d10 0x7f2ce814e2f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 >> 172.21.15.3:0/1870899908 conn(0x7f2ce806d000 msgr2=0x7f2ce80d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.767+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 shutdown_connections 2024-09-10T07:49:40.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.768+0000 7f2ceee00640 1 -- 172.21.15.3:0/1870899908 wait complete. 2024-09-10T07:49:40.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.939+0000 7f51aca00640 1 Processor -- start 2024-09-10T07:49:40.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.939+0000 7f51aca00640 1 -- start start 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51aca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 0x7f51a80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51aca00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a80cfdc0 con 0x7f51a815e860 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51aca00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a80cee40 con 0x7f51a80d3c10 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51aca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80d3780 con 0x7f51a8073860 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51a7400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f51a815e860 0x7f51a80cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39902/0 (socket says 172.21.15.3:39902) 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.940+0000 7f51a7400640 1 -- 172.21.15.3:0/38254840 learned_addr learned my addr 172.21.15.3:0/38254840 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:40.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a7e00640 1 --2- 172.21.15.3:0/38254840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 0x7f51a80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152827153 0 0) 0x7f51a80cfdc0 con 0x7f51a815e860 2024-09-10T07:49:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f518c003520 con 0x7f51a815e860 2024-09-10T07:49:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859065856 0 0) 0x7f51a80cee40 con 0x7f51a80d3c10 2024-09-10T07:49:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51a80cfdc0 con 0x7f51a80d3c10 2024-09-10T07:49:40.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a7e00640 1 -- 172.21.15.3:0/38254840 >> v1:172.21.15.70:6789/0 conn(0x7f51a80d3c10 legacy=0x7f51a8073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a7e00640 1 -- 172.21.15.3:0/38254840 >> v1:172.21.15.3:6789/0 conn(0x7f51a815e860 legacy=0x7f51a80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.941+0000 7f51a7e00640 1 -- 172.21.15.3:0/38254840 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a80cf120 con 0x7f51a8073860 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.942+0000 7f51a7e00640 1 --2- 172.21.15.3:0/38254840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 0x7f51a80b7880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009dc0 tx=0x7f51a002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3f9057e53d742d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.942+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007740 con 0x7f51a8073860 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a00078e0 con 0x7f51a8073860 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51a6a00640 1 -- 172.21.15.3:0/38254840 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007c90 con 0x7f51a8073860 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 -- 172.21.15.3:0/38254840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 msgr2=0x7f51a80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 --2- 172.21.15.3:0/38254840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 0x7f51a80b7880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009dc0 tx=0x7f51a002a6d0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 -- 172.21.15.3:0/38254840 shutdown_connections 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 --2- 172.21.15.3:0/38254840 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a8073860 0x7f51a80b7880 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 -- 172.21.15.3:0/38254840 >> 172.21.15.3:0/38254840 conn(0x7f51a80d4650 msgr2=0x7f51a80d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 -- 172.21.15.3:0/38254840 shutdown_connections 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.943+0000 7f51aca00640 1 -- 172.21.15.3:0/38254840 wait complete. 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.945+0000 7f51aca00640 1 Processor -- start 2024-09-10T07:49:40.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.945+0000 7f51aca00640 1 -- start start 2024-09-10T07:49:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a80d3c10 0x7f51a8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 0x7f51a80cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80d2460 con 0x7f51a8073860 2024-09-10T07:49:40.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80cf400 con 0x7f51a815e860 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51aca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80d4050 con 0x7f51a80d3c10 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53174/0 (socket says 172.21.15.3:53174) 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51a7400640 1 -- 172.21.15.3:0/2512743799 learned_addr learned my addr 172.21.15.3:0/2512743799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51a7e00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 0x7f51a80cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f519fe00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a80d3c10 0x7f51a8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.946+0000 7f51a7400640 1 -- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a80d3c10 msgr2=0x7f51a8155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7400640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a80d3c10 0x7f51a8155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7400640 1 -- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 msgr2=0x7f51a80cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7400640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 0x7f51a80cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7400640 1 -- 172.21.15.3:0/2512743799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a80cc610 con 0x7f51a8073860 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7e00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 0x7f51a80cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:40.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51a7400640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f519800c410 tx=0x7f519800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198017070 con 0x7f51a8073860 2024-09-10T07:49:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5198005ce0 con 0x7f51a8073860 2024-09-10T07:49:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a80cc8f0 con 0x7f51a8073860 2024-09-10T07:49:40.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51980072d0 con 0x7f51a8073860 2024-09-10T07:49:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.947+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a80cd850 con 0x7f51a8073860 2024-09-10T07:49:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.948+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a80cf6b0 con 0x7f51a8073860 2024-09-10T07:49:40.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.949+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f519801a070 con 0x7f51a8073860 2024-09-10T07:49:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.950+0000 7f519f400640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 0x7f518406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.950+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f519808a290 con 0x7f51a8073860 2024-09-10T07:49:40.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.951+0000 7f519fe00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 0x7f518406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:40.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.951+0000 7f519fe00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 0x7f518406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f51a8160be0 tx=0x7f5190009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:40.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:40.954+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51980584c0 con 0x7f51a8073860 2024-09-10T07:49:41.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.139+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51a80cce10 con 0x7f51a8073860 2024-09-10T07:49:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.140+0000 7f519f400640 1 -- 172.21.15.3:0/2512743799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16889 (secure 0 0 0) 0x7f519805c7c0 con 0x7f51a8073860 2024-09-10T07:49:41.139 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:41.139 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":52,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:13.626960+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:48:48.829038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T07:48:50.029907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":0}]},{"pgid":"7.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 msgr2=0x7f518406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 0x7f518406e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f51a8160be0 tx=0x7f5190009040 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 msgr2=0x7f51a8155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f519800c410 tx=0x7f519800c8d0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 shutdown_connections 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f518406be30 0x7f518406e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a815e860 0x7f51a80cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51a80d3c10 0x7f51a8155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 --2- 172.21.15.3:0/2512743799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51a8073860 0x7f51a8155360 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 >> 172.21.15.3:0/2512743799 conn(0x7f51a80d4650 msgr2=0x7f51a80d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 shutdown_connections 2024-09-10T07:49:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.143+0000 7f51aca00640 1 -- 172.21.15.3:0/2512743799 wait complete. 2024-09-10T07:49:41.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:49:41.158 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:41.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.315+0000 7fb38be00640 1 Processor -- start 2024-09-10T07:49:41.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.315+0000 7fb38be00640 1 -- start start 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3840b6a70 0x7fb3840d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3840d1510 0x7fb3840d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb3840cf540 con 0x7fb3840b6a70 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb3840cff10 con 0x7fb3840d1510 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.316+0000 7fb38be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb3840d1d90 con 0x7fb3840b6690 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3840b6a70 0x7fb3840d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3840b6a70 0x7fb3840d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53188/0 (socket says 172.21.15.3:53188) 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 -- 172.21.15.3:0/1239869933 learned_addr learned my addr 172.21.15.3:0/1239869933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb389400640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb389e00640 1 --2- 172.21.15.3:0/1239869933 >> v2:172.21.15.70:3300/0 conn(0x7fb3840d1510 0x7fb3840d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 -- 172.21.15.3:0/1239869933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 msgr2=0x7fb3840cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 -- 172.21.15.3:0/1239869933 >> v2:172.21.15.70:3300/0 conn(0x7fb3840d1510 msgr2=0x7fb3840d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 --2- 172.21.15.3:0/1239869933 >> v2:172.21.15.70:3300/0 conn(0x7fb3840d1510 0x7fb3840d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.317+0000 7fb388a00640 1 -- 172.21.15.3:0/1239869933 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3840ceef0 con 0x7fb3840b6a70 2024-09-10T07:49:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.318+0000 7fb388a00640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840d0fd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb38408d960 tx=0x7fb37002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2d4bfc511d3d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb383e00640 1 -- 172.21.15.3:0/1239869933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370037070 con 0x7fb3840b6a70 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb383e00640 1 -- 172.21.15.3:0/1239869933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370032440 con 0x7fb3840b6a70 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb383e00640 1 -- 172.21.15.3:0/1239869933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370032810 con 0x7fb3840b6a70 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 -- 172.21.15.3:0/1239869933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 msgr2=0x7fb3840d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840d0fd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb38408d960 tx=0x7fb37002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 -- 172.21.15.3:0/1239869933 shutdown_connections 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 --2- 172.21.15.3:0/1239869933 >> v2:172.21.15.70:3300/0 conn(0x7fb3840d1510 0x7fb3840d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840d0fd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 --2- 172.21.15.3:0/1239869933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 -- 172.21.15.3:0/1239869933 >> 172.21.15.3:0/1239869933 conn(0x7fb384072560 msgr2=0x7fb384072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 -- 172.21.15.3:0/1239869933 shutdown_connections 2024-09-10T07:49:41.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.319+0000 7fb38be00640 1 -- 172.21.15.3:0/1239869933 wait complete. 2024-09-10T07:49:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.321+0000 7fb38be00640 1 Processor -- start 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.321+0000 7fb38be00640 1 -- start start 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 0x7fb3840cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840d19a0 con 0x7fb3840b6a70 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840b6ed0 con 0x7fb3840d1510 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb38be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb3840d1b20 con 0x7fb3840b6690 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb388a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb388a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53202/0 (socket says 172.21.15.3:53202) 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.322+0000 7fb388a00640 1 -- 172.21.15.3:0/4057212674 learned_addr learned my addr 172.21.15.3:0/4057212674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb389400640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 -- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 msgr2=0x7fb3840caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 -- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 msgr2=0x7fb3840cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb389e00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 0x7fb3840cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 0x7fb3840cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 -- 172.21.15.3:0/4057212674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3840cc1e0 con 0x7fb3840b6a70 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb389e00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 0x7fb3840cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb388a00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a220 tx=0x7fb3700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.323+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370041020 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.324+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37003d620 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.324+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3840cc4c0 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.324+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370037040 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.324+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3840cca80 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.325+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38415c530 con 0x7fb3840b6a70 2024-09-10T07:49:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.326+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb370044020 con 0x7fb3840b6a70 2024-09-10T07:49:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.326+0000 7fb381600640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 0x7fb36406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.326+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7fb3700ab5a0 con 0x7fb3840b6a70 2024-09-10T07:49:41.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.327+0000 7fb389400640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 0x7fb36406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.327+0000 7fb389400640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 0x7fb36406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb3780098f0 tx=0x7fb378008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.331+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb37004b030 con 0x7fb3840b6a70 2024-09-10T07:49:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.553+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb38415da00 con 0x7fb3840b6a70 2024-09-10T07:49:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.553+0000 7fb381600640 1 -- 172.21.15.3:0/4057212674 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7fb370079e30 con 0x7fb3840b6a70 2024-09-10T07:49:41.552 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:41.552 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 msgr2=0x7fb36406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 0x7fb36406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb3780098f0 tx=0x7fb378008040 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 msgr2=0x7fb3840cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a220 tx=0x7fb3700046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 shutdown_connections 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb36406be30 0x7fb36406e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3840d1510 0x7fb3840cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3840b6a70 0x7fb3840cb760 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 --2- 172.21.15.3:0/4057212674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb3840b6690 0x7fb3840caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 >> 172.21.15.3:0/4057212674 conn(0x7fb384072560 msgr2=0x7fb38415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 shutdown_connections 2024-09-10T07:49:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.556+0000 7fb38be00640 1 -- 172.21.15.3:0/4057212674 wait complete. 2024-09-10T07:49:41.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.731+0000 7f34c7e00640 1 Processor -- start 2024-09-10T07:49:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.731+0000 7f34c7e00640 1 -- start start 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.732+0000 7f34c7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34c80b2000 0x7f34c80b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.732+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c815ebf0 0x7f34c815f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.732+0000 7f34c7e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c80aea40 con 0x7f34c8141540 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.732+0000 7f34c7e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34c80aed20 con 0x7f34c80b2000 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.732+0000 7f34c7e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80af000 con 0x7f34c815ebf0 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34c80b2000 0x7f34c80b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34c80b2000 0x7f34c80b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53210/0 (socket says 172.21.15.3:53210) 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 -- 172.21.15.3:0/1139524891 learned_addr learned my addr 172.21.15.3:0/1139524891 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34c7400640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c815ebf0 0x7f34c815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 -- 172.21.15.3:0/1139524891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c815ebf0 msgr2=0x7f34c815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:41.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c815ebf0 0x7f34c815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 -- 172.21.15.3:0/1139524891 >> v1:172.21.15.70:6789/0 conn(0x7f34c8141540 legacy=0x7f34c80b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.733+0000 7f34bfe00640 1 -- 172.21.15.3:0/1139524891 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c80af2e0 con 0x7f34c80b2000 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.734+0000 7f34bfe00640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c80b2000 0x7f34c80b2490 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009870 tx=0x7f34b002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47254d87a7760f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.734+0000 7f34c6000640 1 -- 172.21.15.3:0/1139524891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0037070 con 0x7f34c80b2000 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c6000640 1 -- 172.21.15.3:0/1139524891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b0032440 con 0x7f34c80b2000 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c6000640 1 -- 172.21.15.3:0/1139524891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0032810 con 0x7f34c80b2000 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 -- 172.21.15.3:0/1139524891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c80b2000 msgr2=0x7f34c80b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c80b2000 0x7f34c80b2490 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009870 tx=0x7f34b002a250 comp rx=0 tx=0).stop 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 -- 172.21.15.3:0/1139524891 shutdown_connections 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c815ebf0 0x7f34c815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 --2- 172.21.15.3:0/1139524891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c80b2000 0x7f34c80b2490 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 -- 172.21.15.3:0/1139524891 >> 172.21.15.3:0/1139524891 conn(0x7f34c806bb20 msgr2=0x7f34c806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:41.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 -- 172.21.15.3:0/1139524891 shutdown_connections 2024-09-10T07:49:41.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.735+0000 7f34c7e00640 1 -- 172.21.15.3:0/1139524891 wait complete. 2024-09-10T07:49:41.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.737+0000 7f34c7e00640 1 Processor -- start 2024-09-10T07:49:41.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.737+0000 7f34c7e00640 1 -- start start 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34c80b2000 0x7f34c8144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c8141540 0x7f34c8141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8141aa0 con 0x7f34c815ebf0 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8141920 con 0x7f34c80b2000 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34c815f5c0 con 0x7f34c8141540 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53212/0 (socket says 172.21.15.3:53212) 2024-09-10T07:49:41.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 -- 172.21.15.3:0/2462088184 learned_addr learned my addr 172.21.15.3:0/2462088184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34bfe00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c8141540 0x7f34c8141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c6a00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34c80b2000 0x7f34c8144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 -- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c8141540 msgr2=0x7f34c8141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c8141540 0x7f34c8141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 -- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34c80b2000 msgr2=0x7f34c8144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34c80b2000 0x7f34c8144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.738+0000 7f34c7400640 1 -- 172.21.15.3:0/2462088184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c815a8a0 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34c7400640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34c0007e60 tx=0x7f34c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0014420 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0019070 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c815ab80 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0014820 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.739+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c815b140 con 0x7f34c815ebf0 2024-09-10T07:49:41.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.740+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c8063610 con 0x7f34c815ebf0 2024-09-10T07:49:41.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.741+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34c00149c0 con 0x7f34c815ebf0 2024-09-10T07:49:41.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.742+0000 7f34bea00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 0x7f34a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:41.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.742+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f34c008b6a0 con 0x7f34c815ebf0 2024-09-10T07:49:41.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.742+0000 7f34c6a00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 0x7f34a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:41.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.743+0000 7f34c6a00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 0x7f34a406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34c81f7460 tx=0x7f34b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:41.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.747+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34c00598d0 con 0x7f34c815ebf0 2024-09-10T07:49:41.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.971+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34c8144a50 con 0x7f34c815ebf0 2024-09-10T07:49:41.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.972+0000 7f34bea00640 1 -- 172.21.15.3:0/2462088184 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5255 (secure 0 0 0) 0x7f34c005dbd0 con 0x7f34c815ebf0 2024-09-10T07:49:41.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:41.971 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"btime":"2024-09-10T07:49:29:611328+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:48:51.031977+0000","modified":"2024-09-10T07:49:28.612926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5433,"mds_1":5430,"mds_2":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"b","rank":2,"incarnation":38,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1001818622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1001818622},{"type":"v1","addr":"172.21.15.70:6833","nonce":1001818622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5430":{"gid":5430,"name":"c","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.3:6837/2353998589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2353998589},{"type":"v1","addr":"172.21.15.3:6837","nonce":2353998589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.3:6836/29592196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":29592196},{"type":"v1","addr":"172.21.15.3:6836","nonce":29592196}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5266,5430,5433]},"id":3}]} 2024-09-10T07:49:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 43 2024-09-10T07:49:41.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 msgr2=0x7f34a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 0x7f34a406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34c81f7460 tx=0x7f34b8009040 comp rx=0 tx=0).stop 2024-09-10T07:49:41.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 msgr2=0x7f34c815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:41.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34c0007e60 tx=0x7f34c000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 shutdown_connections 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34a406be30 0x7f34a406e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34c815ebf0 0x7f34c815b660 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34c8141540 0x7f34c8141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 --2- 172.21.15.3:0/2462088184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34c80b2000 0x7f34c8144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 >> 172.21.15.3:0/2462088184 conn(0x7f34c806bb20 msgr2=0x7f34c80b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.974+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 shutdown_connections 2024-09-10T07:49:41.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:41.975+0000 7f34c7e00640 1 -- 172.21.15.3:0/2462088184 wait complete. 2024-09-10T07:49:41.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:42.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.146+0000 7f9daba00640 1 Processor -- start 2024-09-10T07:49:42.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.147+0000 7f9daba00640 1 -- start start 2024-09-10T07:49:42.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9daba00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9da4160710 0x7f9da40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9daba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9daba00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da40cfdd0 con 0x7f9da4160af0 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9daba00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9da415cc90 con 0x7f9da4160710 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9daba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9da415cf70 con 0x7f9da40d07c0 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9da9000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9da4160710 0x7f9da40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9da9000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9da4160710 0x7f9da40cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53226/0 (socket says 172.21.15.3:53226) 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9da3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9da4160af0 0x7f9da40d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39916/0 (socket says 172.21.15.3:39916) 2024-09-10T07:49:42.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9da9000640 1 -- 172.21.15.3:0/3435094394 learned_addr learned my addr 172.21.15.3:0/3435094394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.148+0000 7f9da9a00640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.149+0000 7f9da9000640 1 -- 172.21.15.3:0/3435094394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 msgr2=0x7f9da4072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.149+0000 7f9da9000640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da4072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.149+0000 7f9da9000640 1 -- 172.21.15.3:0/3435094394 >> v1:172.21.15.3:6789/0 conn(0x7f9da4160af0 legacy=0x7f9da40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.149+0000 7f9da3400640 1 -- 172.21.15.3:0/3435094394 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009763208 0 0) 0x7f9da40cfdd0 con 0x7f9da4160af0 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.149+0000 7f9da9000640 1 -- 172.21.15.3:0/3435094394 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da415d250 con 0x7f9da4160710 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9da9000640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40cf2c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009870 tx=0x7f9d9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5012ccbac46c1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9da3400640 1 -- 172.21.15.3:0/3435094394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98037070 con 0x7f9da4160710 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9da3400640 1 -- 172.21.15.3:0/3435094394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98032440 con 0x7f9da4160710 2024-09-10T07:49:42.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9da3400640 1 -- 172.21.15.3:0/3435094394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98032810 con 0x7f9da4160710 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9daba00640 1 -- 172.21.15.3:0/3435094394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 msgr2=0x7f9da40cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.150+0000 7f9daba00640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40cf2c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009870 tx=0x7f9d9802a270 comp rx=0 tx=0).stop 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 -- 172.21.15.3:0/3435094394 shutdown_connections 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da4072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 --2- 172.21.15.3:0/3435094394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40cf2c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 -- 172.21.15.3:0/3435094394 >> 172.21.15.3:0/3435094394 conn(0x7f9da406d000 msgr2=0x7f9da406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 -- 172.21.15.3:0/3435094394 shutdown_connections 2024-09-10T07:49:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.151+0000 7f9daba00640 1 -- 172.21.15.3:0/3435094394 wait complete. 2024-09-10T07:49:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.152+0000 7f9daba00640 1 Processor -- start 2024-09-10T07:49:42.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.152+0000 7f9daba00640 1 -- start start 2024-09-10T07:49:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da406bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 0x7f9da4148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9da415d770 con 0x7f9da4160710 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9da40cf800 con 0x7f9da4160af0 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.153+0000 7f9daba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9da40c7200 con 0x7f9da40d07c0 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53228/0 (socket says 172.21.15.3:53228) 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 -- 172.21.15.3:0/604427641 learned_addr learned my addr 172.21.15.3:0/604427641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da9000640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da406bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da9a00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 0x7f9da4148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 -- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 msgr2=0x7f9da406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da406bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 -- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 msgr2=0x7f9da4148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 0x7f9da4148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 -- 172.21.15.3:0/604427641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da40c65f0 con 0x7f9da4160710 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da9a00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 0x7f9da4148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:42.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.154+0000 7f9da3e00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c430 tx=0x7f9d9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.155+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d900074e0 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.155+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d900047e0 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.155+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da40c68d0 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.155+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004bd0 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.155+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da40c6e90 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.156+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da40afd50 con 0x7f9da4160710 2024-09-10T07:49:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.157+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d90011070 con 0x7f9da4160710 2024-09-10T07:49:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.158+0000 7f9da0c00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 0x7f9d8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.158+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f9d9008abf0 con 0x7f9da4160710 2024-09-10T07:49:42.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.158+0000 7f9da9000640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 0x7f9d8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.159+0000 7f9da9000640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 0x7f9d8406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9d98034000 tx=0x7f9d9802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.162+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d900593e0 con 0x7f9da4160710 2024-09-10T07:49:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.346+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9da40cda80 con 0x7f9da4160710 2024-09-10T07:49:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.348+0000 7f9da0c00640 1 -- 172.21.15.3:0/604427641 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16889 (secure 0 0 0) 0x7f9d90058ce0 con 0x7f9da4160710 2024-09-10T07:49:42.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:42.347 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":52,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:13.626960+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:48:48.829038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T07:48:50.029907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":0}]},{"pgid":"7.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 msgr2=0x7f9d8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 0x7f9d8406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9d98034000 tx=0x7f9d9802e000 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 msgr2=0x7f9da40c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c430 tx=0x7f9d9000c8f0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 shutdown_connections 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9d8406be60 0x7f9d8406e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9da4160af0 0x7f9da4148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9da4160710 0x7f9da40c5c80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 --2- 172.21.15.3:0/604427641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9da40d07c0 0x7f9da406bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 >> 172.21.15.3:0/604427641 conn(0x7f9da406d000 msgr2=0x7f9da40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:42.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 shutdown_connections 2024-09-10T07:49:42.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.350+0000 7f9daba00640 1 -- 172.21.15.3:0/604427641 wait complete. 2024-09-10T07:49:42.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:49:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.525+0000 7f8f7a200640 1 Processor -- start 2024-09-10T07:49:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.525+0000 7f8f7a200640 1 -- start start 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f7a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f740d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f7a200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f740d10f0 0x7f8f740d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f7a200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f740d3060 con 0x7f8f74161a00 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f7a200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8f740d1ae0 con 0x7f8f740d10f0 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f7a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f740d3340 con 0x7f8f74161de0 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f73e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f740d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f79800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f740d10f0 0x7f8f740d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f78e00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8f74161a00 0x7f8f740cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:46638/0 (socket says 172.21.15.3:46638) 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.526+0000 7f8f78e00640 1 -- 172.21.15.3:0/2212280314 learned_addr learned my addr 172.21.15.3:0/2212280314 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f73400640 1 -- 172.21.15.3:0/2212280314 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721483009 0 0) 0x7f8f740d3060 con 0x7f8f74161a00 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f73400640 1 -- 172.21.15.3:0/2212280314 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f5c003520 con 0x7f8f74161a00 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f79800640 1 -- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 msgr2=0x7f8f740d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f79800640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f740d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f79800640 1 -- 172.21.15.3:0/2212280314 >> v1:172.21.15.70:6789/0 conn(0x7f8f74161a00 legacy=0x7f8f740cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f79800640 1 -- 172.21.15.3:0/2212280314 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f740d3620 con 0x7f8f740d10f0 2024-09-10T07:49:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.527+0000 7f8f73e00640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f740d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.528+0000 7f8f79800640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740d15a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a884dc0132fed556 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.528+0000 7f8f73400640 1 -- 172.21.15.3:0/2212280314 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c007740 con 0x7f8f740d10f0 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.528+0000 7f8f73400640 1 -- 172.21.15.3:0/2212280314 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0078e0 con 0x7f8f740d10f0 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.528+0000 7f8f73400640 1 -- 172.21.15.3:0/2212280314 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c007cb0 con 0x7f8f740d10f0 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 -- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 msgr2=0x7f8f740d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740d15a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c02a660 comp rx=0 tx=0).stop 2024-09-10T07:49:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 -- 172.21.15.3:0/2212280314 shutdown_connections 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740d15a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2212280314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f740d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 -- 172.21.15.3:0/2212280314 >> 172.21.15.3:0/2212280314 conn(0x7f8f74072770 msgr2=0x7f8f7406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 -- 172.21.15.3:0/2212280314 shutdown_connections 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.529+0000 7f8f7a200640 1 -- 172.21.15.3:0/2212280314 wait complete. 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.531+0000 7f8f7a200640 1 Processor -- start 2024-09-10T07:49:42.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.531+0000 7f8f7a200640 1 -- start start 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f741f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740b79f0 con 0x7f8f74161a00 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740d3be0 con 0x7f8f740d10f0 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f7a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f740b7b70 con 0x7f8f74161de0 2024-09-10T07:49:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53240/0 (socket says 172.21.15.3:53240) 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 -- 172.21.15.3:0/2006799068 learned_addr learned my addr 172.21.15.3:0/2006799068 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f78e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f79800640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f741f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 -- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 msgr2=0x7f8f741f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f741f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 -- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 msgr2=0x7f8f740b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.532+0000 7f8f73e00640 1 -- 172.21.15.3:0/2006799068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f740caaf0 con 0x7f8f74161a00 2024-09-10T07:49:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f73e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000e990 tx=0x7f8f6000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60017070 con 0x7f8f74161a00 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6000c980 con 0x7f8f74161a00 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f740cadd0 con 0x7f8f74161a00 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000cd50 con 0x7f8f74161a00 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.533+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f740b4270 con 0x7f8f74161a00 2024-09-10T07:49:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.534+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f740d15a0 con 0x7f8f74161a00 2024-09-10T07:49:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.535+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f6001a070 con 0x7f8f74161a00 2024-09-10T07:49:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.536+0000 7f8f70c00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 0x7f8f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.536+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5854+0+0 (secure 0 0 0) 0x7f8f6008b040 con 0x7f8f74161a00 2024-09-10T07:49:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.536+0000 7f8f78e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 0x7f8f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.537+0000 7f8f78e00640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 0x7f8f5406e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005880 tx=0x7f8f68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.540+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f60059300 con 0x7f8f74161a00 2024-09-10T07:49:42.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.762+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8f740d0990 con 0x7f8f74161a00 2024-09-10T07:49:42.761 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:49:42.763+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:49:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.765+0000 7f8f70c00640 1 -- 172.21.15.3:0/2006799068 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f8f6005d600 con 0x7f8f74161a00 2024-09-10T07:49:42.763 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:49:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 msgr2=0x7f8f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 0x7f8f5406e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005880 tx=0x7f8f68009040 comp rx=0 tx=0).stop 2024-09-10T07:49:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 msgr2=0x7f8f740ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000e990 tx=0x7f8f6000ee50 comp rx=0 tx=0).stop 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 shutdown_connections 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.768+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5406be30 0x7f8f5406e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f74161de0 0x7f8f741f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f74161a00 0x7f8f740ca370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 --2- 172.21.15.3:0/2006799068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f740d10f0 0x7f8f740b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 >> 172.21.15.3:0/2006799068 conn(0x7f8f74072770 msgr2=0x7f8f74160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 shutdown_connections 2024-09-10T07:49:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.769+0000 7f8f7a200640 1 -- 172.21.15.3:0/2006799068 wait complete. 2024-09-10T07:49:42.777 INFO:tasks.ceph.mds.b.smithi070.stderr: -989> 2024-09-10T07:49:24.823+0000 7f9200e00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1) n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55784161f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.386499+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.777 INFO:tasks.ceph.mds.b.smithi070.stderr: -975> 2024-09-10T07:49:24.823+0000 7f9200e00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/AsLsXsFsx@3},l=5138 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55784161eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:49:42.788 INFO:tasks.ceph.mds.c.smithi003.stderr: -1150> 2024-09-10T07:49:24.822+0000 7f48cf200640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1) n(v2 rc2024-09-10T07:49:08.659495+0000 4=0+4) (inest lock) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5639f92b4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.384499+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:49:08.659495+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.789 INFO:tasks.ceph.mds.c.smithi003.stderr: -1011> 2024-09-10T07:49:24.824+0000 7f48d7400640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1) n(v1 rc2024-09-10T07:49:08.659495+0000 3=0+3) (inest sync->mix g=0,2) caps={5138=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x5639f92b4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.386499+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.385499+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:49:08.659495+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.799 INFO:tasks.ceph.mds.a.smithi003.stderr: -2317> 2024-09-10T07:49:24.821+0000 7f685dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1) n(v3 rc2024-09-10T07:49:08.659495+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5138=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5138 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x557178f19b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.382499+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:49:08.659495+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.799 INFO:tasks.ceph.mds.a.smithi003.stderr: -2195> 2024-09-10T07:49:24.823+0000 7f6865e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1) n(v4 rc2024-09-10T07:49:08.659495+0000 5=0+5) (inest sync->mix g=1,2) caps={5138=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x557178f1a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:49:08.659495+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:49:08.383499+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:49:08.659495+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:49:42.821 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:49:42.835 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:49:42.847 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:49:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.952+0000 7f37e1e00640 1 Processor -- start 2024-09-10T07:49:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.952+0000 7f37e1e00640 1 -- start start 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37e1e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37dc15b230 0x7f37dc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37e1e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37dc0b4c90 0x7f37dc071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37e1e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37dc15eff0 con 0x7f37dc072c50 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37e1e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc15fbe0 con 0x7f37dc0b4c90 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37e1e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc15c8d0 con 0x7f37dc15b230 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37db400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37dc0b4c90 0x7f37dc071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37d3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f37dc072c50 0x7f37dc0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39934/0 (socket says 172.21.15.3:39934) 2024-09-10T07:49:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.953+0000 7f37d3e00640 1 -- 172.21.15.3:0/1807371342 learned_addr learned my addr 172.21.15.3:0/1807371342 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:42.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.954+0000 7f37daa00640 1 --2- 172.21.15.3:0/1807371342 >> v2:172.21.15.70:3300/0 conn(0x7f37dc15b230 0x7f37dc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.954+0000 7f37db400640 1 -- 172.21.15.3:0/1807371342 >> v2:172.21.15.70:3300/0 conn(0x7f37dc15b230 msgr2=0x7f37dc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.954+0000 7f37db400640 1 --2- 172.21.15.3:0/1807371342 >> v2:172.21.15.70:3300/0 conn(0x7f37dc15b230 0x7f37dc15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.954+0000 7f37db400640 1 -- 172.21.15.3:0/1807371342 >> v1:172.21.15.3:6789/0 conn(0x7f37dc072c50 legacy=0x7f37dc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.954+0000 7f37db400640 1 -- 172.21.15.3:0/1807371342 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc15f2b0 con 0x7f37dc0b4c90 2024-09-10T07:49:42.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.955+0000 7f37db400640 1 --2- 172.21.15.3:0/1807371342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc0b4c90 0x7f37dc071010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0b65e0 tx=0x7f37d402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0ea10e18a61d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.955+0000 7f37da000640 1 -- 172.21.15.3:0/1807371342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4007740 con 0x7f37dc0b4c90 2024-09-10T07:49:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37da000640 1 -- 172.21.15.3:0/1807371342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d40078e0 con 0x7f37dc0b4c90 2024-09-10T07:49:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37da000640 1 -- 172.21.15.3:0/1807371342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4007cb0 con 0x7f37dc0b4c90 2024-09-10T07:49:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 -- 172.21.15.3:0/1807371342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc0b4c90 msgr2=0x7f37dc071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1807371342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc0b4c90 0x7f37dc071010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0b65e0 tx=0x7f37d402a680 comp rx=0 tx=0).stop 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 -- 172.21.15.3:0/1807371342 shutdown_connections 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1807371342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc0b4c90 0x7f37dc071010 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1807371342 >> v2:172.21.15.70:3300/0 conn(0x7f37dc15b230 0x7f37dc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 -- 172.21.15.3:0/1807371342 >> 172.21.15.3:0/1807371342 conn(0x7f37dc0b5cb0 msgr2=0x7f37dc06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 -- 172.21.15.3:0/1807371342 shutdown_connections 2024-09-10T07:49:42.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.956+0000 7f37e1e00640 1 -- 172.21.15.3:0/1807371342 wait complete. 2024-09-10T07:49:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.958+0000 7f37e1e00640 1 Processor -- start 2024-09-10T07:49:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.958+0000 7f37e1e00640 1 -- start start 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0b4c90 0x7f37dc14f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc15b230 0x7f37dc14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc15b610 con 0x7f37dc072c50 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc15f870 con 0x7f37dc0b4c90 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37e1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc15b790 con 0x7f37dc15b230 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37daa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37daa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53286/0 (socket says 172.21.15.3:53286) 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37daa00640 1 -- 172.21.15.3:0/1474825094 learned_addr learned my addr 172.21.15.3:0/1474825094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37d3e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0b4c90 0x7f37dc14f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.959+0000 7f37db400640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc15b230 0x7f37dc14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 -- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc15b230 msgr2=0x7f37dc14f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc15b230 0x7f37dc14f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 -- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0b4c90 msgr2=0x7f37dc14f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0b4c90 0x7f37dc14f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 -- 172.21.15.3:0/1474825094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc0b1470 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37daa00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d830 tx=0x7f37cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00b6c0 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37cc019070 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00ba80 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc0b1750 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.960+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc0b1d10 con 0x7f37dc072c50 2024-09-10T07:49:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.961+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37dc06e1e0 con 0x7f37dc072c50 2024-09-10T07:49:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.962+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37cc01e460 con 0x7f37dc072c50 2024-09-10T07:49:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.963+0000 7f37d3400640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 0x7f37b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.963+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6112+0+0 (secure 0 0 0) 0x7f37cc08b780 con 0x7f37dc072c50 2024-09-10T07:49:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.963+0000 7f37d3e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 0x7f37b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.964+0000 7f37d3e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 0x7f37b806e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0aefd0 tx=0x7f37c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:42.967+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37cc059f20 con 0x7f37dc072c50 2024-09-10T07:49:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.188+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f37dc0cf570 con 0x7f37dc072c50 2024-09-10T07:49:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.769+0000 7f37d3400640 1 -- 172.21.15.3:0/1474825094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f37cc0600f0 con 0x7f37dc072c50 2024-09-10T07:49:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 msgr2=0x7f37b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 0x7f37b806e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0aefd0 tx=0x7f37c4009040 comp rx=0 tx=0).stop 2024-09-10T07:49:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 msgr2=0x7f37dc1463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d830 tx=0x7f37cc00dcf0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 shutdown_connections 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37b806be30 0x7f37b806e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc15b230 0x7f37dc14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0b4c90 0x7f37dc14f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 --2- 172.21.15.3:0/1474825094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37dc072c50 0x7f37dc1463c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.772+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 >> 172.21.15.3:0/1474825094 conn(0x7f37dc0b5cb0 msgr2=0x7f37dc06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.773+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 shutdown_connections 2024-09-10T07:49:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.773+0000 7f37e1e00640 1 -- 172.21.15.3:0/1474825094 wait complete. 2024-09-10T07:49:43.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:49:43.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.995+0000 7f2ce7000640 1 Processor -- start 2024-09-10T07:49:43.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.995+0000 7f2ce7000640 1 -- start start 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 0x7f2ce00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce00cd740 0x7f2ce00b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b24d0 0x7f2ce006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce00cdbb0 con 0x7f2ce00cd740 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce00d00c0 con 0x7f2ce015c530 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.996+0000 7f2ce7000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00cf1f0 con 0x7f2ce00b24d0 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce00cd740 0x7f2ce00b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce00cd740 0x7f2ce00b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40518/0 (socket says 172.21.15.3:40518) 2024-09-10T07:49:43.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 -- 172.21.15.3:0/588693262 learned_addr learned my addr 172.21.15.3:0/588693262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2ce5000640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b24d0 0x7f2ce006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/588693262 >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 0x7f2ce00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 -- 172.21.15.3:0/588693262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b24d0 msgr2=0x7f2ce006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b24d0 0x7f2ce006bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 -- 172.21.15.3:0/588693262 >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 msgr2=0x7f2ce00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 --2- 172.21.15.3:0/588693262 >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 0x7f2ce00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdf400640 1 -- 172.21.15.3:0/588693262 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce00b2910 con 0x7f2ce00cd740 2024-09-10T07:49:43.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.997+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/588693262 >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 0x7f2ce00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:43.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.998+0000 7f2cdf400640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce00cd740 0x7f2ce00b1f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0063170 tx=0x7f2ccc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b368cf85fe8053f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:43.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.998+0000 7f2cdea00640 1 -- 172.21.15.3:0/588693262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc037070 con 0x7f2ce00cd740 2024-09-10T07:49:43.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2cdea00640 1 -- 172.21.15.3:0/588693262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc032440 con 0x7f2ce00cd740 2024-09-10T07:49:43.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2cdea00640 1 -- 172.21.15.3:0/588693262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc032810 con 0x7f2ce00cd740 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 -- 172.21.15.3:0/588693262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce00cd740 msgr2=0x7f2ce00b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce00cd740 0x7f2ce00b1f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0063170 tx=0x7f2ccc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 -- 172.21.15.3:0/588693262 shutdown_connections 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00b24d0 0x7f2ce006bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 --2- 172.21.15.3:0/588693262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce00cd740 0x7f2ce00b1f90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 --2- 172.21.15.3:0/588693262 >> v2:172.21.15.70:3300/0 conn(0x7f2ce015c530 0x7f2ce00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 -- 172.21.15.3:0/588693262 >> 172.21.15.3:0/588693262 conn(0x7f2ce0062cc0 msgr2=0x7f2ce006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 -- 172.21.15.3:0/588693262 shutdown_connections 2024-09-10T07:49:43.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:43.999+0000 7f2ce7000640 1 -- 172.21.15.3:0/588693262 wait complete. 2024-09-10T07:49:44.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.001+0000 7f2ce7000640 1 Processor -- start 2024-09-10T07:49:44.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.001+0000 7f2ce7000640 1 -- start start 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce00b24d0 0x7f2ce0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00cd740 0x7f2ce00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00ce710 con 0x7f2ce015c530 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00ce430 con 0x7f2ce00b24d0 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce7000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00ce9f0 con 0x7f2ce00cd740 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40532/0 (socket says 172.21.15.3:40532) 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2ce5000640 1 -- 172.21.15.3:0/706695220 learned_addr learned my addr 172.21.15.3:0/706695220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce00b24d0 0x7f2ce0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.002+0000 7f2cdf400640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00cd740 0x7f2ce00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 -- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00cd740 msgr2=0x7f2ce00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00cd740 0x7f2ce00b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 -- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce00b24d0 msgr2=0x7f2ce0155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce00b24d0 0x7f2ce0155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 -- 172.21.15.3:0/706695220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce0156300 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce5000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800ef00 tx=0x7f2cd800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8018020 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8014400 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce00aec80 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80147f0 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.003+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce00af240 con 0x7f2ce015c530 2024-09-10T07:49:44.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.004+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce00af480 con 0x7f2ce015c530 2024-09-10T07:49:44.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.005+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cd8005d10 con 0x7f2ce015c530 2024-09-10T07:49:44.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.006+0000 7f2cd7e00640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.006+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6112+0+0 (secure 0 0 0) 0x7f2cd808b5f0 con 0x7f2ce015c530 2024-09-10T07:49:44.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.006+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.007+0000 7f2cdfe00640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 0x7f2cc006e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.010+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cd8022070 con 0x7f2ce015c530 2024-09-10T07:49:44.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.197+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2ce01e5b40 con 0x7f2ce015c530 2024-09-10T07:49:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.771+0000 7f2cd7e00640 1 -- 172.21.15.3:0/706695220 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) ==== 181+0+0 (secure 0 0 0) 0x7f2cd8059740 con 0x7f2ce015c530 2024-09-10T07:49:44.770 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:49:44.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 msgr2=0x7f2cc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 0x7f2cc006e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0008040 comp rx=0 tx=0).stop 2024-09-10T07:49:44.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 msgr2=0x7f2ce01e6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800ef00 tx=0x7f2cd800b480 comp rx=0 tx=0).stop 2024-09-10T07:49:44.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 shutdown_connections 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce015c530 0x7f2ce01e6200 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce00cd740 0x7f2ce00b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 --2- 172.21.15.3:0/706695220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce00b24d0 0x7f2ce0155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.775+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 >> 172.21.15.3:0/706695220 conn(0x7f2ce0062cc0 msgr2=0x7f2ce015ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.776+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 shutdown_connections 2024-09-10T07:49:44.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.776+0000 7f2ce7000640 1 -- 172.21.15.3:0/706695220 wait complete. 2024-09-10T07:49:44.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:49:44.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.956+0000 7fae0a000640 1 Processor -- start 2024-09-10T07:49:44.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.956+0000 7fae0a000640 1 -- start start 2024-09-10T07:49:44.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae0a000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fae0406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae0a000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0406c770 con 0x7fae040b3050 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae0a000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0415f3e0 con 0x7fae040b3430 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae0a000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae0415f1a0 con 0x7fae0406bdc0 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae08c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fae040b3050 0x7fae0415ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35292/0 (socket says 172.21.15.3:35292) 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae08c00640 1 -- 172.21.15.3:0/3574557293 learned_addr learned my addr 172.21.15.3:0/3574557293 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae09600640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fae0406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735776886 0 0) 0x7fae0406c770 con 0x7fae040b3050 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.958+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadec003520 con 0x7fae040b3050 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102953539 0 0) 0x7fae0415f3e0 con 0x7fae040b3430 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae0406c770 con 0x7fae040b3430 2024-09-10T07:49:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1237855110 0 0) 0x7fadec003520 con 0x7fae040b3050 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 msgr2=0x7fae0406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fae0406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 >> v1:172.21.15.70:6789/0 conn(0x7fae040b3430 legacy=0x7fae0406f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0415f6c0 con 0x7fae040b3050 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fadf8001670 con 0x7fae040b3050 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 >> v1:172.21.15.3:6789/0 conn(0x7fae040b3050 legacy=0x7fae0415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec003b70 0x7fadec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec004800 0x7fadec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fadec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae0415f1a0 con 0x7fadec003b70 2024-09-10T07:49:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fadec00ba70 con 0x7fadec004800 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fadec00bc70 con 0x7fae0406bdc0 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae08c00640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec003b70 0x7fadec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae03e00640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec004800 0x7fadec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.959+0000 7fae09600640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fadec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.960+0000 7fae08c00640 1 -- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 msgr2=0x7fadec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.960+0000 7fae08c00640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae0406bdc0 0x7fadec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.960+0000 7fae08c00640 1 -- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec004800 msgr2=0x7fadec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.960+0000 7fae08c00640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec004800 0x7fadec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.960+0000 7fae08c00640 1 -- 172.21.15.3:0/3574557293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec003520 con 0x7fadec003b70 2024-09-10T07:49:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae08c00640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec003b70 0x7fadec003f70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007fb0 tx=0x7fadec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18dae1ef3114757 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8007c00 con 0x7fadec003b70 2024-09-10T07:49:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8005af0 con 0x7fadec003b70 2024-09-10T07:49:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae03400640 1 -- 172.21.15.3:0/3574557293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8007070 con 0x7fadec003b70 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae0a000640 1 -- 172.21.15.3:0/3574557293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec003b70 msgr2=0x7fadec003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae0a000640 1 --2- 172.21.15.3:0/3574557293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadec003b70 0x7fadec003f70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007fb0 tx=0x7fadec005830 comp rx=0 tx=0).stop 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae09600640 1 -- 172.21.15.3:0/3574557293 reap_dead start 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae0a000640 1 -- 172.21.15.3:0/3574557293 shutdown_connections 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae0a000640 1 -- 172.21.15.3:0/3574557293 >> 172.21.15.3:0/3574557293 conn(0x7fae040b5fb0 msgr2=0x7fae040b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.961+0000 7fae0a000640 1 -- 172.21.15.3:0/3574557293 shutdown_connections 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.962+0000 7fae0a000640 1 -- 172.21.15.3:0/3574557293 wait complete. 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.963+0000 7fae0a000640 1 Processor -- start 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.963+0000 7fae0a000640 1 -- start start 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec003b70 0x7fae041565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae04147f80 0x7fae040ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae0406efe0 con 0x7fae041f6fd0 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae0406fef0 con 0x7fadec003b70 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae0a000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae0406f2c0 con 0x7fae04147f80 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.964+0000 7fae09600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40554/0 (socket says 172.21.15.3:40554) 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 -- 172.21.15.3:0/127666838 learned_addr learned my addr 172.21.15.3:0/127666838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae03e00640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae04147f80 0x7fae040ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae08c00640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec003b70 0x7fae041565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 -- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae04147f80 msgr2=0x7fae040ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae04147f80 0x7fae040ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 -- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec003b70 msgr2=0x7fae041565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec003b70 0x7fae041565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 -- 172.21.15.3:0/127666838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0415aa30 con 0x7fae041f6fd0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae09600640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fadfc011c80 tx=0x7fadfc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc015430 con 0x7fae041f6fd0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc01d070 con 0x7fae041f6fd0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc015810 con 0x7fae041f6fd0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.965+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae041e8a40 con 0x7fae041f6fd0 2024-09-10T07:49:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.966+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae041e9000 con 0x7fae041f6fd0 2024-09-10T07:49:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.967+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae04069010 con 0x7fae041f6fd0 2024-09-10T07:49:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.968+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadfc0159b0 con 0x7fae041f6fd0 2024-09-10T07:49:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.968+0000 7fae00c00640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 0x7fade406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.968+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5674+0+0 (secure 0 0 0) 0x7fadfc08b8d0 con 0x7fae041f6fd0 2024-09-10T07:49:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.969+0000 7fae08c00640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 0x7fade406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.969+0000 7fae08c00640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 0x7fade406e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fadf8032000 tx=0x7fadf802d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:44.972+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae04069010 con 0x7fae041f6fd0 2024-09-10T07:49:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.159+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fae0406c230 con 0x7fae041f6fd0 2024-09-10T07:49:45.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.779+0000 7fae00c00640 1 -- 172.21.15.3:0/127666838 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) ==== 169+0+0 (secure 0 0 0) 0x7fadfc010040 con 0x7fae041f6fd0 2024-09-10T07:49:45.778 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:49:45.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.784+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 msgr2=0x7fade406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.785+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 0x7fade406e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fadf8032000 tx=0x7fadf802d000 comp rx=0 tx=0).stop 2024-09-10T07:49:45.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.785+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 msgr2=0x7fae041f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.785+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fadfc011c80 tx=0x7fadfc00f580 comp rx=0 tx=0).stop 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 shutdown_connections 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fade406be30 0x7fade406e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae041f6fd0 0x7fae041f7450 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae04147f80 0x7fae040ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 --2- 172.21.15.3:0/127666838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadec003b70 0x7fae041565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 >> 172.21.15.3:0/127666838 conn(0x7fae040b5fb0 msgr2=0x7fae04160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 shutdown_connections 2024-09-10T07:49:45.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.786+0000 7fae0a000640 1 -- 172.21.15.3:0/127666838 wait complete. 2024-09-10T07:49:45.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:49:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.969+0000 7f683de00640 1 Processor -- start 2024-09-10T07:49:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.969+0000 7f683de00640 1 -- start start 2024-09-10T07:49:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f683de00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68380cf980 0x7f683806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f683de00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68380cd900 con 0x7f68381451d0 2024-09-10T07:49:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f683de00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68380cdbe0 con 0x7f6838145730 2024-09-10T07:49:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f683de00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f683806d7c0 con 0x7f68380cf980 2024-09-10T07:49:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f6836a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f68381451d0 0x7f68380b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35308/0 (socket says 172.21.15.3:35308) 2024-09-10T07:49:45.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f6836a00640 1 -- 172.21.15.3:0/1641746200 learned_addr learned my addr 172.21.15.3:0/1641746200 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.970+0000 7f6837400640 1 --2- 172.21.15.3:0/1641746200 >> v2:172.21.15.70:3300/0 conn(0x7f68380cf980 0x7f683806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918391774 0 0) 0x7f68380cd900 con 0x7f68381451d0 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f681c003520 con 0x7f68381451d0 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598800930 0 0) 0x7f68380cdbe0 con 0x7f6838145730 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68380cd900 con 0x7f6838145730 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6837400640 1 -- 172.21.15.3:0/1641746200 >> v1:172.21.15.70:6789/0 conn(0x7f6838145730 legacy=0x7f68380cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6837400640 1 -- 172.21.15.3:0/1641746200 >> v1:172.21.15.3:6789/0 conn(0x7f68381451d0 legacy=0x7f68380b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.971+0000 7f6837400640 1 -- 172.21.15.3:0/1641746200 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683806da10 con 0x7f68380cf980 2024-09-10T07:49:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.972+0000 7f6837400640 1 --2- 172.21.15.3:0/1641746200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380cf980 0x7f683806cfc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f683806d940 tx=0x7f683002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4bcffced59ed0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.972+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830007740 con 0x7f68380cf980 2024-09-10T07:49:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.972+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68300078e0 con 0x7f68380cf980 2024-09-10T07:49:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.972+0000 7f6836000640 1 -- 172.21.15.3:0/1641746200 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830007cb0 con 0x7f68380cf980 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 -- 172.21.15.3:0/1641746200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380cf980 msgr2=0x7f683806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 --2- 172.21.15.3:0/1641746200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380cf980 0x7f683806cfc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f683806d940 tx=0x7f683002a660 comp rx=0 tx=0).stop 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 -- 172.21.15.3:0/1641746200 shutdown_connections 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 --2- 172.21.15.3:0/1641746200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380cf980 0x7f683806cfc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 -- 172.21.15.3:0/1641746200 >> 172.21.15.3:0/1641746200 conn(0x7f68380b5cb0 msgr2=0x7f683806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 -- 172.21.15.3:0/1641746200 shutdown_connections 2024-09-10T07:49:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.973+0000 7f683de00640 1 -- 172.21.15.3:0/1641746200 wait complete. 2024-09-10T07:49:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.975+0000 7f683de00640 1 Processor -- start 2024-09-10T07:49:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.975+0000 7f683de00640 1 -- start start 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68381451d0 0x7f68380aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838145730 0x7f68380ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f683806e0b0 con 0x7f68380cf980 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f683806df30 con 0x7f6838145730 2024-09-10T07:49:45.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f683de00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f683806e230 con 0x7f68381451d0 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f6836a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f6836a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40556/0 (socket says 172.21.15.3:40556) 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f6836a00640 1 -- 172.21.15.3:0/2462423928 learned_addr learned my addr 172.21.15.3:0/2462423928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f6837400640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838145730 0x7f68380ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.976+0000 7f682fe00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68381451d0 0x7f68380aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 -- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68381451d0 msgr2=0x7f68380aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68381451d0 0x7f68380aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 -- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838145730 msgr2=0x7f68380ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838145730 0x7f68380ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 -- 172.21.15.3:0/2462423928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68380ac190 con 0x7f68380cf980 2024-09-10T07:49:45.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f6836a00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f682800c500 tx=0x7f682800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828017070 con 0x7f68380cf980 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828005d00 con 0x7f68380cf980 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68380ac470 con 0x7f68380cf980 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828007330 con 0x7f68380cf980 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.977+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68380aca30 con 0x7f68380cf980 2024-09-10T07:49:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.978+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68380ccb80 con 0x7f68380cf980 2024-09-10T07:49:45.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.979+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f682801a070 con 0x7f68380cf980 2024-09-10T07:49:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.980+0000 7f682f400640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.980+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7f682808a000 con 0x7f68380cf980 2024-09-10T07:49:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.980+0000 7f682fe00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.981+0000 7f682fe00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 0x7f681406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6820007fd0 tx=0x7f6820009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:45.985+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6828058660 con 0x7f68380cf980 2024-09-10T07:49:46.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.171+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f683806cfc0 con 0x7f68380cf980 2024-09-10T07:49:46.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.174+0000 7f682f400640 1 -- 172.21.15.3:0/2462423928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f682805c960 con 0x7f68380cf980 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 msgr2=0x7f681406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 0x7f681406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6820007fd0 tx=0x7f6820009040 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 msgr2=0x7f6838136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f682800c500 tx=0x7f682800c9c0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 shutdown_connections 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681406be30 0x7f681406e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838145730 0x7f68380ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68381451d0 0x7f68380aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 --2- 172.21.15.3:0/2462423928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68380cf980 0x7f6838136aa0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 >> 172.21.15.3:0/2462423928 conn(0x7f68380b5cb0 msgr2=0x7f683806e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 shutdown_connections 2024-09-10T07:49:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.177+0000 7f683de00640 1 -- 172.21.15.3:0/2462423928 wait complete. 2024-09-10T07:49:46.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-10T07:49:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.350+0000 7ff1a3a00640 1 Processor -- start 2024-09-10T07:49:46.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.351+0000 7ff1a3a00640 1 -- start start 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a3a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff19c0d07c0 0x7ff19c167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a3a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c0cfdd0 con 0x7ff19c160af0 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a3a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c15cc90 con 0x7ff19c160710 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a3a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff19c15cf70 con 0x7ff19c0d07c0 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff193e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff19c160af0 0x7ff19c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35320/0 (socket says 172.21.15.3:35320) 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff193e00640 1 -- 172.21.15.3:0/494193651 learned_addr learned my addr 172.21.15.3:0/494193651 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:46.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a1a00640 1 --2- 172.21.15.3:0/494193651 >> v2:172.21.15.3:3300/0 conn(0x7ff19c0d07c0 0x7ff19c167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.352+0000 7ff1a1a00640 1 -- 172.21.15.3:0/494193651 >> v1:172.21.15.70:6789/0 conn(0x7ff19c160710 legacy=0x7ff19c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.353+0000 7ff1a1a00640 1 -- 172.21.15.3:0/494193651 >> v1:172.21.15.3:6789/0 conn(0x7ff19c160af0 legacy=0x7ff19c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.353+0000 7ff1a1a00640 1 -- 172.21.15.3:0/494193651 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c15d250 con 0x7ff19c0d07c0 2024-09-10T07:49:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.353+0000 7ff19be00640 1 -- 172.21.15.3:0/494193651 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560989002 0 0) 0x7ff19c0cfdd0 con 0x7ff19c160af0 2024-09-10T07:49:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a1a00640 1 --2- 172.21.15.3:0/494193651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c0d07c0 0x7ff19c167090 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff194009d90 tx=0x7ff19402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b16271baa6d0cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff19be00640 1 -- 172.21.15.3:0/494193651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194007740 con 0x7ff19c0d07c0 2024-09-10T07:49:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff19be00640 1 -- 172.21.15.3:0/494193651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1940078e0 con 0x7ff19c0d07c0 2024-09-10T07:49:46.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff19be00640 1 -- 172.21.15.3:0/494193651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194007cb0 con 0x7ff19c0d07c0 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 -- 172.21.15.3:0/494193651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c0d07c0 msgr2=0x7ff19c167090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/494193651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c0d07c0 0x7ff19c167090 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff194009d90 tx=0x7ff19402a6f0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 -- 172.21.15.3:0/494193651 shutdown_connections 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/494193651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c0d07c0 0x7ff19c167090 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 -- 172.21.15.3:0/494193651 >> 172.21.15.3:0/494193651 conn(0x7ff19c06d000 msgr2=0x7ff19c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.354+0000 7ff1a3a00640 1 -- 172.21.15.3:0/494193651 shutdown_connections 2024-09-10T07:49:46.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.355+0000 7ff1a3a00640 1 -- 172.21.15.3:0/494193651 wait complete. 2024-09-10T07:49:46.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.356+0000 7ff1a3a00640 1 Processor -- start 2024-09-10T07:49:46.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.356+0000 7ff1a3a00640 1 -- start start 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff19c0d07c0 0x7ff19c14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff19c160710 0x7ff19c0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c15d770 con 0x7ff19c160af0 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c073080 con 0x7ff19c160710 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a3a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c0abd70 con 0x7ff19c0d07c0 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.357+0000 7ff1a1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40566/0 (socket says 172.21.15.3:40566) 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 -- 172.21.15.3:0/145439666 learned_addr learned my addr 172.21.15.3:0/145439666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff193e00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff19c160710 0x7ff19c0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1000640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff19c0d07c0 0x7ff19c14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 -- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff19c0d07c0 msgr2=0x7ff19c14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff19c0d07c0 0x7ff19c14ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 -- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff19c160710 msgr2=0x7ff19c0ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff19c160710 0x7ff19c0ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 -- 172.21.15.3:0/145439666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c0aba90 con 0x7ff19c160af0 2024-09-10T07:49:46.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.358+0000 7ff1a1a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff194009d90 tx=0x7ff194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.359+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194040070 con 0x7ff19c160af0 2024-09-10T07:49:46.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.359+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff194031c40 con 0x7ff19c160af0 2024-09-10T07:49:46.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.359+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194003990 con 0x7ff19c160af0 2024-09-10T07:49:46.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.359+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1516f0 con 0x7ff19c160af0 2024-09-10T07:49:46.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.359+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c151cb0 con 0x7ff19c160af0 2024-09-10T07:49:46.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.360+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c0ccd00 con 0x7ff19c160af0 2024-09-10T07:49:46.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.361+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff194039040 con 0x7ff19c160af0 2024-09-10T07:49:46.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.361+0000 7ff199600640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 0x7ff17806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.362+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7ff1940abef0 con 0x7ff19c160af0 2024-09-10T07:49:46.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.362+0000 7ff1a1000640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 0x7ff17806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.362+0000 7ff1a1000640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 0x7ff17806e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000f60 tx=0x7ff18c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.366+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff19c0ccd00 con 0x7ff19c160af0 2024-09-10T07:49:46.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.615+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7ff19c0cfa60 con 0x7ff19c160af0 2024-09-10T07:49:46.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.780+0000 7ff199600640 1 -- 172.21.15.3:0/145439666 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v182) ==== 143+0+0 (secure 0 0 0) 0x7ff19407a490 con 0x7ff19c160af0 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 msgr2=0x7ff17806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 0x7ff17806e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000f60 tx=0x7ff18c009040 comp rx=0 tx=0).stop 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 msgr2=0x7ff19c0ab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff194009d90 tx=0x7ff194004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 shutdown_connections 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff17806be30 0x7ff17806e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff19c160af0 0x7ff19c0ab260 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff19c160710 0x7ff19c0ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 --2- 172.21.15.3:0/145439666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff19c0d07c0 0x7ff19c14ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 >> 172.21.15.3:0/145439666 conn(0x7ff19c06d000 msgr2=0x7ff19c0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 shutdown_connections 2024-09-10T07:49:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.782+0000 7ff1a3a00640 1 -- 172.21.15.3:0/145439666 wait complete. 2024-09-10T07:49:46.798 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:49:46.798 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:49:46.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:46.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.954+0000 7f73f3000640 1 Processor -- start 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.954+0000 7f73f3000640 1 -- start start 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f3000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73ec160af0 0x7f73ec0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f3000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f3000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73ec0d0a80 con 0x7f73ec160710 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f3000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f73ec0cfdd0 con 0x7f73ec160af0 2024-09-10T07:49:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f3000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec15cc90 con 0x7f73ec0d05f0 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73f1000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73ebe00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f73ec160710 0x7f73ec0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:38864/0 (socket says 172.21.15.3:38864) 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73e3e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73ec160af0 0x7f73ec0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.956+0000 7f73ebe00640 1 -- 172.21.15.3:0/3382786677 learned_addr learned my addr 172.21.15.3:0/3382786677 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73eb400640 1 -- 172.21.15.3:0/3382786677 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811441272 0 0) 0x7f73ec0d0a80 con 0x7f73ec160710 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73eb400640 1 -- 172.21.15.3:0/3382786677 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73d0003520 con 0x7f73ec160710 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73e3e00640 1 -- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 msgr2=0x7f73ec072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73e3e00640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73e3e00640 1 -- 172.21.15.3:0/3382786677 >> v1:172.21.15.70:6789/0 conn(0x7f73ec160710 legacy=0x7f73ec0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73e3e00640 1 -- 172.21.15.3:0/3382786677 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec15cf70 con 0x7f73ec160af0 2024-09-10T07:49:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.957+0000 7f73f1000640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.958+0000 7f73e3e00640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec0d00b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097a0 tx=0x7f73d402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87d351fc6643a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.958+0000 7f73eb400640 1 -- 172.21.15.3:0/3382786677 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4037070 con 0x7f73ec160af0 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.958+0000 7f73eb400640 1 -- 172.21.15.3:0/3382786677 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d4032440 con 0x7f73ec160af0 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.958+0000 7f73eb400640 1 -- 172.21.15.3:0/3382786677 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4032810 con 0x7f73ec160af0 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 -- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 msgr2=0x7f73ec0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec0d00b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097a0 tx=0x7f73d402a180 comp rx=0 tx=0).stop 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 -- 172.21.15.3:0/3382786677 shutdown_connections 2024-09-10T07:49:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 --2- 172.21.15.3:0/3382786677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec0d00b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 -- 172.21.15.3:0/3382786677 >> 172.21.15.3:0/3382786677 conn(0x7f73ec06d000 msgr2=0x7f73ec06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 -- 172.21.15.3:0/3382786677 shutdown_connections 2024-09-10T07:49:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.959+0000 7f73f3000640 1 -- 172.21.15.3:0/3382786677 wait complete. 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.961+0000 7f73f3000640 1 Processor -- start 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.961+0000 7f73f3000640 1 -- start start 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec15d770 con 0x7f73ec160710 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec15c960 con 0x7f73ec160af0 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f3000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec142c60 con 0x7f73ec0d05f0 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40576/0 (socket says 172.21.15.3:40576) 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 -- 172.21.15.3:0/1460272248 learned_addr learned my addr 172.21.15.3:0/1460272248 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73ebe00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73f1000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 -- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 msgr2=0x7f73ec15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 -- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 msgr2=0x7f73ec155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.962+0000 7f73e3e00640 1 -- 172.21.15.3:0/1460272248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec142eb0 con 0x7f73ec160710 2024-09-10T07:49:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73e3e00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f73d40098c0 tx=0x7f73d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4041030 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d403d420 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4037040 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec143190 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.963+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec143750 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.964+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec143950 con 0x7f73ec160710 2024-09-10T07:49:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.965+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73d4044070 con 0x7f73ec160710 2024-09-10T07:49:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.966+0000 7f73e8c00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 0x7f73c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.966+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7f73d40ab170 con 0x7f73ec160710 2024-09-10T07:49:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.966+0000 7f73ebe00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 0x7f73c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.967+0000 7f73ebe00640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 0x7f73c806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005880 tx=0x7f73dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:46.971+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f73ec143950 con 0x7f73ec160710 2024-09-10T07:49:47.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.160+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ec0cfaf0 con 0x7f73ec160710 2024-09-10T07:49:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.161+0000 7f73e8c00640 1 -- 172.21.15.3:0/1460272248 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+13212 (secure 0 0 0) 0x7f73d4079ce0 con 0x7f73ec160710 2024-09-10T07:49:47.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:47.161 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":55,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:45.772200+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6833/1001818622":"2024-09-11T07:49:42.763185+0000","172.21.15.70:6832/1001818622":"2024-09-11T07:49:42.763185+0000","172.21.15.3:6837/2353998589":"2024-09-11T07:49:42.763204+0000","172.21.15.3:6835/2353998589":"2024-09-11T07:49:42.763204+0000","172.21.15.3:6836/29592196":"2024-09-11T07:49:42.763209+0000","172.21.15.3:6834/29592196":"2024-09-11T07:49:42.763209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 msgr2=0x7f73c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 0x7f73c806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005880 tx=0x7f73dc009040 comp rx=0 tx=0).stop 2024-09-10T07:49:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 msgr2=0x7f73ec1579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f73d40098c0 tx=0x7f73d40046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 shutdown_connections 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f73c806be30 0x7f73c806e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73ec160af0 0x7f73ec155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73ec160710 0x7f73ec1579d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 --2- 172.21.15.3:0/1460272248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73ec0d05f0 0x7f73ec15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.163+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 >> 172.21.15.3:0/1460272248 conn(0x7f73ec06d000 msgr2=0x7f73ec0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.164+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 shutdown_connections 2024-09-10T07:49:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.164+0000 7f73f3000640 1 -- 172.21.15.3:0/1460272248 wait complete. 2024-09-10T07:49:47.179 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:49:47.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:49:47.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.339+0000 7f3315a00640 1 Processor -- start 2024-09-10T07:49:47.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.339+0000 7f3315a00640 1 -- start start 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33100d45a0 0x7f33100d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33100d4f00 con 0x7f33100cfe90 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33100d3de0 con 0x7f3310161b70 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3310161f50 con 0x7f33100d45a0 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f33100cfe90 0x7f33100d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35322/0 (socket says 172.21.15.3:35322) 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.340+0000 7f3315000640 1 -- 172.21.15.3:0/709655470 learned_addr learned my addr 172.21.15.3:0/709655470 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:47.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330f400640 1 --2- 172.21.15.3:0/709655470 >> v2:172.21.15.70:3300/0 conn(0x7f33100d45a0 0x7f33100d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330ea00640 1 -- 172.21.15.3:0/709655470 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027308604 0 0) 0x7f33100d4f00 con 0x7f33100cfe90 2024-09-10T07:49:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330ea00640 1 -- 172.21.15.3:0/709655470 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f4003520 con 0x7f33100cfe90 2024-09-10T07:49:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330f400640 1 -- 172.21.15.3:0/709655470 >> v1:172.21.15.70:6789/0 conn(0x7f3310161b70 legacy=0x7f33100d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330f400640 1 -- 172.21.15.3:0/709655470 >> v1:172.21.15.3:6789/0 conn(0x7f33100cfe90 legacy=0x7f33100d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.341+0000 7f330f400640 1 -- 172.21.15.3:0/709655470 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33100d40c0 con 0x7f33100d45a0 2024-09-10T07:49:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.342+0000 7f330f400640 1 --2- 172.21.15.3:0/709655470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33100d45a0 0x7f33100d49c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d0f46a6f79d26d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.342+0000 7f330ea00640 1 -- 172.21.15.3:0/709655470 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc037070 con 0x7f33100d45a0 2024-09-10T07:49:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.342+0000 7f330ea00640 1 -- 172.21.15.3:0/709655470 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc032440 con 0x7f33100d45a0 2024-09-10T07:49:47.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.342+0000 7f330ea00640 1 -- 172.21.15.3:0/709655470 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc032810 con 0x7f33100d45a0 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 -- 172.21.15.3:0/709655470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33100d45a0 msgr2=0x7f33100d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 --2- 172.21.15.3:0/709655470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33100d45a0 0x7f33100d49c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02a230 comp rx=0 tx=0).stop 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 -- 172.21.15.3:0/709655470 shutdown_connections 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 --2- 172.21.15.3:0/709655470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33100d45a0 0x7f33100d49c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 -- 172.21.15.3:0/709655470 >> 172.21.15.3:0/709655470 conn(0x7f3310062630 msgr2=0x7f3310062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 -- 172.21.15.3:0/709655470 shutdown_connections 2024-09-10T07:49:47.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.343+0000 7f3315a00640 1 -- 172.21.15.3:0/709655470 wait complete. 2024-09-10T07:49:47.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.345+0000 7f3315a00640 1 Processor -- start 2024-09-10T07:49:47.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.345+0000 7f3315a00640 1 -- start start 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33100d45a0 0x7f331014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3310161b70 0x7f33100aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33100c9fc0 con 0x7f33100cfe90 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f33100cf970 con 0x7f3310161b70 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33100d3a30 con 0x7f33100d45a0 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40580/0 (socket says 172.21.15.3:40580) 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 -- 172.21.15.3:0/1075383609 learned_addr learned my addr 172.21.15.3:0/1075383609 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f3315000640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3310161b70 0x7f33100aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330f400640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33100d45a0 0x7f331014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 -- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33100d45a0 msgr2=0x7f331014f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33100d45a0 0x7f331014f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 -- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3310161b70 msgr2=0x7f33100aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3310161b70 0x7f33100aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.346+0000 7f330fe00640 1 -- 172.21.15.3:0/1075383609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101f76d0 con 0x7f33100cfe90 2024-09-10T07:49:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f330fe00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f330000c3f0 tx=0x7f330000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000052d0 con 0x7f33100cfe90 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3300005490 con 0x7f33100cfe90 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300007260 con 0x7f33100cfe90 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331014f690 con 0x7f33100cfe90 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.347+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101f4690 con 0x7f33100cfe90 2024-09-10T07:49:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.348+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33100cf2d0 con 0x7f33100cfe90 2024-09-10T07:49:47.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.349+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3300007400 con 0x7f33100cfe90 2024-09-10T07:49:47.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.350+0000 7f3307e00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 0x7f32f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.350+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7f330008a0f0 con 0x7f33100cfe90 2024-09-10T07:49:47.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.350+0000 7f330f400640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 0x7f32f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.351+0000 7f330f400640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 0x7f32f006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f32fc034000 tx=0x7f32fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.354+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3300058600 con 0x7f33100cfe90 2024-09-10T07:49:47.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.539+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f331015f020 con 0x7f33100cfe90 2024-09-10T07:49:47.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.540+0000 7f3307e00640 1 -- 172.21.15.3:0/1075383609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7f330005c900 con 0x7f33100cfe90 2024-09-10T07:49:47.539 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:49:47.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 msgr2=0x7f32f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 0x7f32f006e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f32fc034000 tx=0x7f32fc02e000 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 msgr2=0x7f33101f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f330000c3f0 tx=0x7f330000c8b0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 shutdown_connections 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f32f006be30 0x7f32f006e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3310161b70 0x7f33100aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33100d45a0 0x7f331014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 --2- 172.21.15.3:0/1075383609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33100cfe90 0x7f33101f4060 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 >> 172.21.15.3:0/1075383609 conn(0x7f3310062630 msgr2=0x7f33100ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 shutdown_connections 2024-09-10T07:49:47.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.542+0000 7f3315a00640 1 -- 172.21.15.3:0/1075383609 wait complete. 2024-09-10T07:49:47.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-10T07:49:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.714+0000 7f3d4cc00640 1 Processor -- start 2024-09-10T07:49:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.714+0000 7f3d4cc00640 1 -- start start 2024-09-10T07:49:47.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.715+0000 7f3d4cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 0x7f3d480cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.715+0000 7f3d4cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d480d07c0 0x7f3d48072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d4cc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d480cfdd0 con 0x7f3d48160af0 2024-09-10T07:49:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d4cc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3d4815cc90 con 0x7f3d480d07c0 2024-09-10T07:49:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d4cc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4815cf70 con 0x7f3d48160710 2024-09-10T07:49:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d46800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d480d07c0 0x7f3d48072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d3fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3d48160af0 0x7f3d480d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35338/0 (socket says 172.21.15.3:35338) 2024-09-10T07:49:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d3fe00640 1 -- 172.21.15.3:0/3292016710 learned_addr learned my addr 172.21.15.3:0/3292016710 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d45e00640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 0x7f3d480cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d46800640 1 -- 172.21.15.3:0/3292016710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 msgr2=0x7f3d480cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d46800640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 0x7f3d480cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d46800640 1 -- 172.21.15.3:0/3292016710 >> v1:172.21.15.3:6789/0 conn(0x7f3d48160af0 legacy=0x7f3d480d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d46800640 1 -- 172.21.15.3:0/3292016710 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4815d250 con 0x7f3d480d07c0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d45400640 1 -- 172.21.15.3:0/3292016710 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757225565 0 0) 0x7f3d480cfdd0 con 0x7f3d48160af0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.716+0000 7f3d45e00640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 0x7f3d480cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.717+0000 7f3d46800640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d48072070 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009d90 tx=0x7f3d4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3b8518e851aa75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d45400640 1 -- 172.21.15.3:0/3292016710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40007740 con 0x7f3d480d07c0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d45400640 1 -- 172.21.15.3:0/3292016710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d400078e0 con 0x7f3d480d07c0 2024-09-10T07:49:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d45400640 1 -- 172.21.15.3:0/3292016710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40007cb0 con 0x7f3d480d07c0 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3292016710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 msgr2=0x7f3d48072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d48072070 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009d90 tx=0x7f3d4002a660 comp rx=0 tx=0).stop 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3292016710 shutdown_connections 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d48072070 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3292016710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160710 0x7f3d480cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3292016710 >> 172.21.15.3:0/3292016710 conn(0x7f3d4806d000 msgr2=0x7f3d4806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3292016710 shutdown_connections 2024-09-10T07:49:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.718+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3292016710 wait complete. 2024-09-10T07:49:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.720+0000 7f3d4cc00640 1 Processor -- start 2024-09-10T07:49:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.720+0000 7f3d4cc00640 1 -- start start 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 0x7f3d481407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160af0 0x7f3d480729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4815d770 con 0x7f3d480d07c0 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480cf800 con 0x7f3d48160710 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d4cc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3d48140d30 con 0x7f3d48160af0 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40602/0 (socket says 172.21.15.3:40602) 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 -- 172.21.15.3:0/3386844637 learned_addr learned my addr 172.21.15.3:0/3386844637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d46800640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160af0 0x7f3d480729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d3fe00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 0x7f3d481407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 -- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160af0 msgr2=0x7f3d480729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160af0 0x7f3d480729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 -- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 msgr2=0x7f3d481407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 0x7f3d481407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d45e00640 1 -- 172.21.15.3:0/3386844637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d48151790 con 0x7f3d480d07c0 2024-09-10T07:49:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.721+0000 7f3d3fe00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 0x7f3d481407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d45e00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800c7f0 tx=0x7f3d3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38015070 con 0x7f3d480d07c0 2024-09-10T07:49:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d38007a30 con 0x7f3d480d07c0 2024-09-10T07:49:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d48151a30 con 0x7f3d480d07c0 2024-09-10T07:49:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38007e00 con 0x7f3d480d07c0 2024-09-10T07:49:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.722+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d48151ff0 con 0x7f3d480d07c0 2024-09-10T07:49:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.723+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d480afd50 con 0x7f3d480d07c0 2024-09-10T07:49:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.724+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d38018070 con 0x7f3d480d07c0 2024-09-10T07:49:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.725+0000 7f3d3e000640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 0x7f3d2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.725+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7f3d3808ab80 con 0x7f3d480d07c0 2024-09-10T07:49:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.725+0000 7f3d3fe00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 0x7f3d2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.726+0000 7f3d3fe00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 0x7f3d2406e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d4814f8d0 tx=0x7f3d30007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:47.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.729+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d38059090 con 0x7f3d480d07c0 2024-09-10T07:49:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:47.984+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7f3d4815bbe0 con 0x7f3d480d07c0 2024-09-10T07:49:48.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.788+0000 7f3d3e000640 1 -- 172.21.15.3:0/3386844637 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v184) ==== 145+0+0 (secure 0 0 0) 0x7f3d3805d390 con 0x7f3d480d07c0 2024-09-10T07:49:48.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 msgr2=0x7f3d2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 0x7f3d2406e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d4814f8d0 tx=0x7f3d30007ca0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 msgr2=0x7f3d481527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800c7f0 tx=0x7f3d3800cbf0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 shutdown_connections 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3d2406be30 0x7f3d2406e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d48160af0 0x7f3d480729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d48160710 0x7f3d481407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 --2- 172.21.15.3:0/3386844637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480d07c0 0x7f3d481527f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 >> 172.21.15.3:0/3386844637 conn(0x7f3d4806d000 msgr2=0x7f3d4815c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 shutdown_connections 2024-09-10T07:49:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.791+0000 7f3d4cc00640 1 -- 172.21.15.3:0/3386844637 wait complete. 2024-09-10T07:49:48.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:49:48.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.974+0000 7f2cff200640 1 Processor -- start 2024-09-10T07:49:48.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.974+0000 7f2cff200640 1 -- start start 2024-09-10T07:49:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cff200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 0x7f2d000ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cff200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d0015feb0 0x7f2d0015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cff200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000ce580 con 0x7f2d000cfcf0 2024-09-10T07:49:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cff200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2d000ce850 con 0x7f2d0015feb0 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cff200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d000ceb30 con 0x7f2d0015f940 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d0015feb0 0x7f2d0015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cfe800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2d000cfcf0 0x7f2d000726f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35344/0 (socket says 172.21.15.3:35344) 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cfe800640 1 -- 172.21.15.3:0/26158860 learned_addr learned my addr 172.21.15.3:0/26158860 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 --2- 172.21.15.3:0/26158860 >> v2:172.21.15.3:3300/0 conn(0x7f2d0015feb0 0x7f2d0015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40616/0 (socket says 172.21.15.3:40616) 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cfde00640 1 --2- 172.21.15.3:0/26158860 >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 0x7f2d000ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 -- 172.21.15.3:0/26158860 >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 msgr2=0x7f2d000ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 --2- 172.21.15.3:0/26158860 >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 0x7f2d000ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 -- 172.21.15.3:0/26158860 >> v1:172.21.15.3:6789/0 conn(0x7f2d000cfcf0 legacy=0x7f2d000726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cf7e00640 1 -- 172.21.15.3:0/26158860 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000cee10 con 0x7f2d0015feb0 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cfde00640 1 --2- 172.21.15.3:0/26158860 >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 0x7f2d000ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.976+0000 7f2cfd400640 1 -- 172.21.15.3:0/26158860 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463411426 0 0) 0x7f2d000ce580 con 0x7f2d000cfcf0 2024-09-10T07:49:48.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.977+0000 7f2cf7e00640 1 --2- 172.21.15.3:0/26158860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015d9d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006ddc0 tx=0x7f2ce802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f6a45e6f692d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cfd400640 1 -- 172.21.15.3:0/26158860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8037070 con 0x7f2d0015feb0 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cfd400640 1 -- 172.21.15.3:0/26158860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce802add0 con 0x7f2d0015feb0 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cfd400640 1 -- 172.21.15.3:0/26158860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8032660 con 0x7f2d0015feb0 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 -- 172.21.15.3:0/26158860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 msgr2=0x7f2d0015d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 --2- 172.21.15.3:0/26158860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015d9d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2d0006ddc0 tx=0x7f2ce802a080 comp rx=0 tx=0).stop 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 -- 172.21.15.3:0/26158860 shutdown_connections 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 --2- 172.21.15.3:0/26158860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015d9d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 --2- 172.21.15.3:0/26158860 >> v2:172.21.15.70:3300/0 conn(0x7f2d0015f940 0x7f2d000ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 -- 172.21.15.3:0/26158860 >> 172.21.15.3:0/26158860 conn(0x7f2d000b1e90 msgr2=0x7f2d000b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 -- 172.21.15.3:0/26158860 shutdown_connections 2024-09-10T07:49:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.978+0000 7f2cff200640 1 -- 172.21.15.3:0/26158860 wait complete. 2024-09-10T07:49:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.980+0000 7f2cff200640 1 Processor -- start 2024-09-10T07:49:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.980+0000 7f2cff200640 1 -- start start 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d000cfcf0 0x7f2d001f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d0015f940 0x7f2d0015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00071ea0 con 0x7f2d0015feb0 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0015df10 con 0x7f2d000cfcf0 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cff200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d00072e00 con 0x7f2d0015f940 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cfe800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cfe800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40624/0 (socket says 172.21.15.3:40624) 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.981+0000 7f2cfe800640 1 -- 172.21.15.3:0/558518098 learned_addr learned my addr 172.21.15.3:0/558518098 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cf7e00640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d0015f940 0x7f2d0015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 -- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d0015f940 msgr2=0x7f2d0015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d0015f940 0x7f2d0015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 -- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d000cfcf0 msgr2=0x7f2d001f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d000cfcf0 0x7f2d001f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 -- 172.21.15.3:0/558518098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000aea50 con 0x7f2d0015feb0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cfe800640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7a0 tx=0x7f2cf800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8018070 con 0x7f2d0015feb0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf800a640 con 0x7f2d0015feb0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d001f2a00 con 0x7f2d0015feb0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800aab0 con 0x7f2d0015feb0 2024-09-10T07:49:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.982+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d00149b60 con 0x7f2d0015feb0 2024-09-10T07:49:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.983+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d000731c0 con 0x7f2d0015feb0 2024-09-10T07:49:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.984+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cf801b070 con 0x7f2d0015feb0 2024-09-10T07:49:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.985+0000 7f2cf6000640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 0x7f2cdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.985+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7f2cf808a280 con 0x7f2d0015feb0 2024-09-10T07:49:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.986+0000 7f2cfde00640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 0x7f2cdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.986+0000 7f2cfde00640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 0x7f2cdc06e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006fd0 tx=0x7f2cf0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:48.990+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d000aea50 con 0x7f2d0015feb0 2024-09-10T07:49:49.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.175+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2d00069c20 con 0x7f2d0015feb0 2024-09-10T07:49:49.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.179+0000 7f2cf6000640 1 -- 172.21.15.3:0/558518098 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2cf8058de0 con 0x7f2d0015feb0 2024-09-10T07:49:49.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 msgr2=0x7f2cdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 0x7f2cdc06e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006fd0 tx=0x7f2cf0008040 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 msgr2=0x7f2d0015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7a0 tx=0x7f2cf800cc60 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 shutdown_connections 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cdc06be60 0x7f2cdc06e320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.181+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d0015feb0 0x7f2d0015cb10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.182+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d0015f940 0x7f2d0015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.182+0000 7f2cff200640 1 --2- 172.21.15.3:0/558518098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d000cfcf0 0x7f2d001f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.182+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 >> 172.21.15.3:0/558518098 conn(0x7f2d000b1e90 msgr2=0x7f2d0015d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.182+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 shutdown_connections 2024-09-10T07:49:49.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.182+0000 7f2cff200640 1 -- 172.21.15.3:0/558518098 wait complete. 2024-09-10T07:49:49.198 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:49:49.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:49:49.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:49:49.370 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:49:49.370 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:49.370 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:49.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:49:49.406 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:49:49.406 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:49.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:49:49.443 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:49:49.443 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:49.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:49:49.481 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:49:49.481 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:49:49.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:49:49.519 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:49:49.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.677+0000 7fea5f200640 1 Processor -- start 2024-09-10T07:49:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.678+0000 7fea5f200640 1 -- start start 2024-09-10T07:49:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea600afdc0 0x7fea6006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5f200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea600b6290 con 0x7fea6006d070 2024-09-10T07:49:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fea6006e4c0 con 0x7fea600afdc0 2024-09-10T07:49:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea600b6060 con 0x7fea60148000 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5de00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5de00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:41538/0 (socket says 172.21.15.3:41538) 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea55400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea600afdc0 0x7fea6006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5de00640 1 -- 172.21.15.3:0/1206147153 learned_addr learned my addr 172.21.15.3:0/1206147153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.679+0000 7fea5e800640 1 --1- 172.21.15.3:0/1206147153 >> v1:172.21.15.70:6789/0 conn(0x7fea6006d070 0x7fea60062630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:38880/0 (socket says 172.21.15.3:38880) 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.680+0000 7fea5de00640 1 -- 172.21.15.3:0/1206147153 >> v2:172.21.15.70:3300/0 conn(0x7fea600afdc0 msgr2=0x7fea6006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.680+0000 7fea5de00640 1 --2- 172.21.15.3:0/1206147153 >> v2:172.21.15.70:3300/0 conn(0x7fea600afdc0 0x7fea6006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.680+0000 7fea5de00640 1 -- 172.21.15.3:0/1206147153 >> v1:172.21.15.70:6789/0 conn(0x7fea6006d070 legacy=0x7fea60062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.680+0000 7fea5de00640 1 -- 172.21.15.3:0/1206147153 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea600b6570 con 0x7fea60148000 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.681+0000 7fea5de00640 1 --2- 172.21.15.3:0/1206147153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea50009870 tx=0x7fea5002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c96a49299c38cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.681+0000 7fea5d400640 1 -- 172.21.15.3:0/1206147153 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50037070 con 0x7fea60148000 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.681+0000 7fea5d400640 1 -- 172.21.15.3:0/1206147153 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea50032440 con 0x7fea60148000 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.681+0000 7fea5d400640 1 -- 172.21.15.3:0/1206147153 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50032810 con 0x7fea60148000 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 -- 172.21.15.3:0/1206147153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 msgr2=0x7fea600b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 --2- 172.21.15.3:0/1206147153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea50009870 tx=0x7fea5002a250 comp rx=0 tx=0).stop 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 -- 172.21.15.3:0/1206147153 shutdown_connections 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 --2- 172.21.15.3:0/1206147153 >> v2:172.21.15.70:3300/0 conn(0x7fea600afdc0 0x7fea6006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 --2- 172.21.15.3:0/1206147153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea600b5b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 -- 172.21.15.3:0/1206147153 >> 172.21.15.3:0/1206147153 conn(0x7fea60071fb0 msgr2=0x7fea600723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 -- 172.21.15.3:0/1206147153 shutdown_connections 2024-09-10T07:49:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.682+0000 7fea5f200640 1 -- 172.21.15.3:0/1206147153 wait complete. 2024-09-10T07:49:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.684+0000 7fea5f200640 1 Processor -- start 2024-09-10T07:49:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.684+0000 7fea5f200640 1 -- start start 2024-09-10T07:49:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 0x7fea6015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea6015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea600b6b30 con 0x7fea6006d070 2024-09-10T07:49:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fea60148410 con 0x7fea600afdc0 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea6006d4a0 con 0x7fea60148000 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40626/0 (socket says 172.21.15.3:40626) 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 -- 172.21.15.3:0/2485973619 learned_addr learned my addr 172.21.15.3:0/2485973619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 -- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 msgr2=0x7fea6015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea6015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 -- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 msgr2=0x7fea6015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea55400640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 0x7fea6015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 0x7fea6015def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea5de00640 1 -- 172.21.15.3:0/2485973619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea600cb3b0 con 0x7fea6006d070 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.685+0000 7fea55400640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 0x7fea6015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea5de00640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a290 tx=0x7fea500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50041030 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea50030440 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50037040 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea600cb690 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.686+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea600cbc50 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.687+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6006cb30 con 0x7fea6006d070 2024-09-10T07:49:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.688+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea50044070 con 0x7fea6006d070 2024-09-10T07:49:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.689+0000 7fea56a00640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 0x7fea3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.689+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7fea500ab290 con 0x7fea6006d070 2024-09-10T07:49:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.689+0000 7fea55400640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 0x7fea3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.690+0000 7fea55400640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 0x7fea3c06e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea600d0ff0 tx=0x7fea48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.693+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fea50079fa0 con 0x7fea6006d070 2024-09-10T07:49:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.914+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea6006f450 con 0x7fea6006d070 2024-09-10T07:49:49.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.914+0000 7fea56a00640 1 -- 172.21.15.3:0/2485973619 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+3905 (secure 0 0 0) 0x7fea500798a0 con 0x7fea6006d070 2024-09-10T07:49:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:49.915 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-10T07:49:43:765765+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5015,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1062685533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1062685533},{"type":"v1","addr":"172.21.15.70:6835","nonce":1062685533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5767,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3829889049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3829889049},{"type":"v1","addr":"172.21.15.70:6833","nonce":3829889049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6012,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2110362417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2110362417},{"type":"v1","addr":"172.21.15.3:6835","nonce":2110362417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2083213508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2083213508},{"type":"v1","addr":"172.21.15.3:6837","nonce":2083213508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[]} 2024-09-10T07:49:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 msgr2=0x7fea3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 0x7fea3c06e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea600d0ff0 tx=0x7fea48008040 comp rx=0 tx=0).stop 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 msgr2=0x7fea600cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a290 tx=0x7fea500046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 shutdown_connections 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fea3c06be30 0x7fea3c06e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea60148000 0x7fea6015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea600afdc0 0x7fea6015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 --2- 172.21.15.3:0/2485973619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea6006d070 0x7fea600cdfa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 >> 172.21.15.3:0/2485973619 conn(0x7fea60071fb0 msgr2=0x7fea600b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 shutdown_connections 2024-09-10T07:49:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:49.917+0000 7fea5f200640 1 -- 172.21.15.3:0/2485973619 wait complete. 2024-09-10T07:49:49.933 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:49:49.933 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:49:49.933 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:49:49.933 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:49:49.934 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:49:49.934 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:49:49.934 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:49:49.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:49:49.934 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:49:49.934 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:49:49.935 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:49:49.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:49:49.935 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:49:49.935 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:49:49.935 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:49:49.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:49:49.948 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:49:49.948 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:49:49.949 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:49:49.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:49:49.950 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:49:49.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:49:49.951 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:49:49.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:49:49.993 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:49:49.993 DEBUG:teuthology.parallel:result is None 2024-09-10T07:49:49.994 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:49:49.995 DEBUG:teuthology.parallel:result is None 2024-09-10T07:49:49.995 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:49:49.995 DEBUG:teuthology.parallel:result is None 2024-09-10T07:49:49.997 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:49:49.997 DEBUG:teuthology.parallel:result is None 2024-09-10T07:49:49.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:49:50.123 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:49:50.124 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:49:50.127 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:49:50.127 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:49:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.164+0000 7fae1f000640 1 Processor -- start 2024-09-10T07:49:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.164+0000 7fae1f000640 1 -- start start 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.165+0000 7fae1f000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae180d1c10 0x7fae180b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.165+0000 7fae1f000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae180ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.165+0000 7fae1f000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae180b70b0 con 0x7fae1815e970 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.165+0000 7fae1f000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fae180b78b0 con 0x7fae180d1c10 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.165+0000 7fae1f000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae180b62f0 con 0x7fae18162cd0 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae180d1c10 0x7fae180b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae180d1c10 0x7fae180b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:38052/0 (socket says 172.21.15.3:38052) 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 -- 172.21.15.3:0/3720297726 learned_addr learned my addr 172.21.15.3:0/3720297726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae1d000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae180ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 -- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 msgr2=0x7fae180ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 --2- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae180ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 -- 172.21.15.3:0/3720297726 >> v1:172.21.15.70:6789/0 conn(0x7fae1815e970 legacy=0x7fae180d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.166+0000 7fae0fe00640 1 -- 172.21.15.3:0/3720297726 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae180b65d0 con 0x7fae180d1c10 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.167+0000 7fae0fe00640 1 --2- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae180d1c10 0x7fae180b37a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae00009870 tx=0x7fae0002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf745c4ee2e1ed69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae17400640 1 -- 172.21.15.3:0/3720297726 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00037070 con 0x7fae180d1c10 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae17400640 1 -- 172.21.15.3:0/3720297726 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae00032440 con 0x7fae180d1c10 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae17400640 1 -- 172.21.15.3:0/3720297726 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00032810 con 0x7fae180d1c10 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 -- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae180d1c10 msgr2=0x7fae180b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 --2- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae180d1c10 0x7fae180b37a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae00009870 tx=0x7fae0002a1c0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 -- 172.21.15.3:0/3720297726 shutdown_connections 2024-09-10T07:49:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 --2- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae180ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 --2- 172.21.15.3:0/3720297726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae180d1c10 0x7fae180b37a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 -- 172.21.15.3:0/3720297726 >> 172.21.15.3:0/3720297726 conn(0x7fae1806bd00 msgr2=0x7fae1806c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 -- 172.21.15.3:0/3720297726 shutdown_connections 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.168+0000 7fae1f000640 1 -- 172.21.15.3:0/3720297726 wait complete. 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.170+0000 7fae1f000640 1 Processor -- start 2024-09-10T07:49:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.170+0000 7fae1f000640 1 -- start start 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1815e970 0x7fae181f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae1815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae18163120 con 0x7fae180d1c10 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae180b6b90 con 0x7fae1815e970 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1f000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae180cf280 con 0x7fae18162cd0 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae17e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae17e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40676/0 (socket says 172.21.15.3:40676) 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae17e00640 1 -- 172.21.15.3:0/3498693744 learned_addr learned my addr 172.21.15.3:0/3498693744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae0fe00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1815e970 0x7fae181f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.171+0000 7fae1d000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae1815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 -- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 msgr2=0x7fae1815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae1815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 -- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1815e970 msgr2=0x7fae181f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1815e970 0x7fae181f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 -- 172.21.15.3:0/3498693744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181f5940 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae17e00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c400 tx=0x7fae0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080052d0 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae08005490 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08007260 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181605a0 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.172+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae18160b60 con 0x7fae180d1c10 2024-09-10T07:49:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.173+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae180b37a0 con 0x7fae180d1c10 2024-09-10T07:49:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.174+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae08007400 con 0x7fae180d1c10 2024-09-10T07:49:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.175+0000 7fae14c00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 0x7fadf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.175+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5115+0+0 (secure 0 0 0) 0x7fae0808a120 con 0x7fae180d1c10 2024-09-10T07:49:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.175+0000 7fae0fe00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 0x7fadf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.176+0000 7fae0fe00640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 0x7fadf406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae00034000 tx=0x7fae0002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.179+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae180b70b0 con 0x7fae180d1c10 2024-09-10T07:49:50.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:50.382+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fae181627a0 con 0x7fae180d1c10 2024-09-10T07:49:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.194+0000 7fae14c00640 1 -- 172.21.15.3:0/3498693744 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) ==== 97+0+0 (secure 0 0 0) 0x7fae08058630 con 0x7fae180d1c10 2024-09-10T07:49:51.194 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:49:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.197+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 msgr2=0x7fadf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.197+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 0x7fadf406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae00034000 tx=0x7fae0002e000 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.197+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 msgr2=0x7fae181559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.197+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c400 tx=0x7fae0800c8c0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 shutdown_connections 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf406be30 0x7fadf406e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae18162cd0 0x7fae1815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1815e970 0x7fae181f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 --2- 172.21.15.3:0/3498693744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae180d1c10 0x7fae181559e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 >> 172.21.15.3:0/3498693744 conn(0x7fae1806bd00 msgr2=0x7fae1806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 shutdown_connections 2024-09-10T07:49:51.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.198+0000 7fae1f000640 1 -- 172.21.15.3:0/3498693744 wait complete. 2024-09-10T07:49:51.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:49:51.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.379+0000 7f2849e00640 1 Processor -- start 2024-09-10T07:49:51.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.379+0000 7f2849e00640 1 -- start start 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f28440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2844071cd0 0x7f28440b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28440ce730 con 0x7f2844145010 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f28440ced10 con 0x7f2844071cd0 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2844073e60 con 0x7f284415ca80 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2844071cd0 0x7f28440b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2844071cd0 0x7f28440b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40688/0 (socket says 172.21.15.3:40688) 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2848a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2844145010 0x7f2844073410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35378/0 (socket says 172.21.15.3:35378) 2024-09-10T07:49:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2849400640 1 -- 172.21.15.3:0/3052060042 learned_addr learned my addr 172.21.15.3:0/3052060042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.380+0000 7f2843e00640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f28440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.381+0000 7f2849400640 1 -- 172.21.15.3:0/3052060042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 msgr2=0x7f28440b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.381+0000 7f2849400640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f28440b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.381+0000 7f2849400640 1 -- 172.21.15.3:0/3052060042 >> v1:172.21.15.3:6789/0 conn(0x7f2844145010 legacy=0x7f2844073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.381+0000 7f2849400640 1 -- 172.21.15.3:0/3052060042 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28440ce9a0 con 0x7f2844071cd0 2024-09-10T07:49:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.382+0000 7f2849400640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844071cd0 0x7f28440b2540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f283c009e50 tx=0x7f283c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a531606e04ab226 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.382+0000 7f2843400640 1 -- 172.21.15.3:0/3052060042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c036070 con 0x7f2844071cd0 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.382+0000 7f2843400640 1 -- 172.21.15.3:0/3052060042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c031600 con 0x7f2844071cd0 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.382+0000 7f2843400640 1 -- 172.21.15.3:0/3052060042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0319d0 con 0x7f2844071cd0 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 -- 172.21.15.3:0/3052060042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844071cd0 msgr2=0x7f28440b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844071cd0 0x7f28440b2540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f283c009e50 tx=0x7f283c02a710 comp rx=0 tx=0).stop 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 -- 172.21.15.3:0/3052060042 shutdown_connections 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844071cd0 0x7f28440b2540 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 --2- 172.21.15.3:0/3052060042 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f28440b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 -- 172.21.15.3:0/3052060042 >> 172.21.15.3:0/3052060042 conn(0x7f284406bb20 msgr2=0x7f284406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 -- 172.21.15.3:0/3052060042 shutdown_connections 2024-09-10T07:49:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.383+0000 7f2849e00640 1 -- 172.21.15.3:0/3052060042 wait complete. 2024-09-10T07:49:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.384+0000 7f2849e00640 1 Processor -- start 2024-09-10T07:49:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.385+0000 7f2849e00640 1 -- start start 2024-09-10T07:49:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.385+0000 7f2849e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 0x7f28440ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.385+0000 7f2849e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2849e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f2844140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2849e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28441453f0 con 0x7f2844145010 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2849e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2844074140 con 0x7f2844071cd0 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2849e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2844145570 con 0x7f284415ca80 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40696/0 (socket says 172.21.15.3:40696) 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 -- 172.21.15.3:0/1799469798 learned_addr learned my addr 172.21.15.3:0/1799469798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 -- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 msgr2=0x7f2844140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f2844140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2848a00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 0x7f28440ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 -- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 msgr2=0x7f28440ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 0x7f28440ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2843e00640 1 -- 172.21.15.3:0/1799469798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28440ad500 con 0x7f2844145010 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.386+0000 7f2848a00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 0x7f28440ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2843e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f283000d5c0 tx=0x7f283000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300125f0 con 0x7f2844145010 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2830019070 con 0x7f2844145010 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300129d0 con 0x7f2844145010 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28440ad7e0 con 0x7f2844145010 2024-09-10T07:49:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.387+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28440adda0 con 0x7f2844145010 2024-09-10T07:49:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.388+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2844063610 con 0x7f2844145010 2024-09-10T07:49:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.389+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2830012b70 con 0x7f2844145010 2024-09-10T07:49:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.390+0000 7f2840c00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 0x7f282406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.390+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4857+0+0 (secure 0 0 0) 0x7f283008aff0 con 0x7f2844145010 2024-09-10T07:49:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.391+0000 7f2848a00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 0x7f282406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.391+0000 7f2848a00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 0x7f282406e320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f60 tx=0x7f2838009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.395+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2830059bc0 con 0x7f2844145010 2024-09-10T07:49:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.651+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f28440693c0 con 0x7f2844145010 2024-09-10T07:49:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.651+0000 7f2840c00640 1 -- 172.21.15.3:0/1799469798 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f28440693c0 con 0x7f2844145010 2024-09-10T07:49:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:51.651 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:49:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:49:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:49:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:49:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:49:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:49:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:49:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:49:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:49:51.666 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 msgr2=0x7f282406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 0x7f282406e320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f60 tx=0x7f2838009040 comp rx=0 tx=0).stop 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 msgr2=0x7f28440c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f283000d5c0 tx=0x7f283000da80 comp rx=0 tx=0).stop 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 shutdown_connections 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f282406be60 0x7f282406e320 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f284415ca80 0x7f2844140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2844145010 0x7f28440c52e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 --2- 172.21.15.3:0/1799469798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2844071cd0 0x7f28440ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.653+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 >> 172.21.15.3:0/1799469798 conn(0x7f284406bb20 msgr2=0x7f28441421b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.654+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 shutdown_connections 2024-09-10T07:49:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.654+0000 7f2849e00640 1 -- 172.21.15.3:0/1799469798 wait complete. 2024-09-10T07:49:51.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:49:51.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.827+0000 7fb356c00640 1 Processor -- start 2024-09-10T07:49:51.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.827+0000 7fb356c00640 1 -- start start 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.828+0000 7fb356c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb35015c720 0x7fb350071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.828+0000 7fb356c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3500cfdc0 0x7fb35015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.828+0000 7fb356c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3500cd7f0 con 0x7fb350070f30 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.828+0000 7fb356c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb350072210 con 0x7fb35015c720 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.828+0000 7fb356c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb3500cf880 con 0x7fb3500cfdc0 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb35015c720 0x7fb350071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb35015c720 0x7fb350071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40698/0 (socket says 172.21.15.3:40698) 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 -- 172.21.15.3:0/1599366587 learned_addr learned my addr 172.21.15.3:0/1599366587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb354c00640 1 --2- 172.21.15.3:0/1599366587 >> v2:172.21.15.70:3300/0 conn(0x7fb3500cfdc0 0x7fb35015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 -- 172.21.15.3:0/1599366587 >> v2:172.21.15.70:3300/0 conn(0x7fb3500cfdc0 msgr2=0x7fb35015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 --2- 172.21.15.3:0/1599366587 >> v2:172.21.15.70:3300/0 conn(0x7fb3500cfdc0 0x7fb35015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 -- 172.21.15.3:0/1599366587 >> v1:172.21.15.70:6789/0 conn(0x7fb350070f30 legacy=0x7fb350073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:51.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.829+0000 7fb34fe00640 1 -- 172.21.15.3:0/1599366587 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500cda90 con 0x7fb35015c720 2024-09-10T07:49:51.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.830+0000 7fb34fe00640 1 --2- 172.21.15.3:0/1599366587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb35015c720 0x7fb350071cd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb35006dd40 tx=0x7fb34002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f00b4a312a4a5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.830+0000 7fb34ea00640 1 -- 172.21.15.3:0/1599366587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340037070 con 0x7fb35015c720 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.830+0000 7fb34ea00640 1 -- 172.21.15.3:0/1599366587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb340032440 con 0x7fb35015c720 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.830+0000 7fb34ea00640 1 -- 172.21.15.3:0/1599366587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340032810 con 0x7fb35015c720 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 -- 172.21.15.3:0/1599366587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb35015c720 msgr2=0x7fb350071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 --2- 172.21.15.3:0/1599366587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb35015c720 0x7fb350071cd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb35006dd40 tx=0x7fb34002a230 comp rx=0 tx=0).stop 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 -- 172.21.15.3:0/1599366587 shutdown_connections 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 --2- 172.21.15.3:0/1599366587 >> v2:172.21.15.70:3300/0 conn(0x7fb3500cfdc0 0x7fb35015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 --2- 172.21.15.3:0/1599366587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb35015c720 0x7fb350071cd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 -- 172.21.15.3:0/1599366587 >> 172.21.15.3:0/1599366587 conn(0x7fb35006cfc0 msgr2=0x7fb35006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 -- 172.21.15.3:0/1599366587 shutdown_connections 2024-09-10T07:49:51.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.831+0000 7fb356c00640 1 -- 172.21.15.3:0/1599366587 wait complete. 2024-09-10T07:49:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.833+0000 7fb356c00640 1 Processor -- start 2024-09-10T07:49:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.833+0000 7fb356c00640 1 -- start start 2024-09-10T07:49:51.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb350070f30 0x7fb3500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb35015c720 0x7fb3501f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb350073d20 con 0x7fb3500cfdc0 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb35006ed10 con 0x7fb350070f30 2024-09-10T07:49:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb356c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb350073ea0 con 0x7fb35015c720 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40710/0 (socket says 172.21.15.3:40710) 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34f400640 1 -- 172.21.15.3:0/3698079412 learned_addr learned my addr 172.21.15.3:0/3698079412 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34fe00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb350070f30 0x7fb3500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb354c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb35015c720 0x7fb3501f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34f400640 1 -- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb35015c720 msgr2=0x7fb3501f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.834+0000 7fb34f400640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb35015c720 0x7fb3501f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb34f400640 1 -- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb350070f30 msgr2=0x7fb3500b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb34f400640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb350070f30 0x7fb3500b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb34f400640 1 -- 172.21.15.3:0/3698079412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500b0bf0 con 0x7fb3500cfdc0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb34f400640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b3f0 tx=0x7fb33c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c015020 con 0x7fb3500cfdc0 2024-09-10T07:49:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c011830 con 0x7fb3500cfdc0 2024-09-10T07:49:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011c20 con 0x7fb3500cfdc0 2024-09-10T07:49:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500b0ed0 con 0x7fb3500cfdc0 2024-09-10T07:49:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.835+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3500b0600 con 0x7fb3500cfdc0 2024-09-10T07:49:51.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:49:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.836+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3500b12c0 con 0x7fb3500cfdc0 2024-09-10T07:49:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.838+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb33c018020 con 0x7fb3500cfdc0 2024-09-10T07:49:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.839+0000 7fb347e00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 0x7fb33006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.839+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4857+0+0 (secure 0 0 0) 0x7fb33c08a640 con 0x7fb3500cfdc0 2024-09-10T07:49:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.839+0000 7fb34fe00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 0x7fb33006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.840+0000 7fb34fe00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 0x7fb33006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb340034000 tx=0x7fb34002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:51.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:51.844+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb33c01f030 con 0x7fb3500cfdc0 2024-09-10T07:49:52.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.030+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb3500cf2b0 con 0x7fb3500cfdc0 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.031+0000 7fb347e00640 1 -- 172.21.15.3:0/3698079412 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) ==== 84+0+65 (secure 0 0 0) 0x7fb33c058ce0 con 0x7fb3500cfdc0 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:49:52.031 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:49:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 msgr2=0x7fb33006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 0x7fb33006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb340034000 tx=0x7fb34002e000 comp rx=0 tx=0).stop 2024-09-10T07:49:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 msgr2=0x7fb3500b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b3f0 tx=0x7fb33c00b8b0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 shutdown_connections 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb33006be30 0x7fb33006e2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb35015c720 0x7fb3501f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3500cfdc0 0x7fb3500b47d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 --2- 172.21.15.3:0/3698079412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb350070f30 0x7fb3500b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 >> 172.21.15.3:0/3698079412 conn(0x7fb35006cfc0 msgr2=0x7fb350159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 shutdown_connections 2024-09-10T07:49:52.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.033+0000 7fb356c00640 1 -- 172.21.15.3:0/3698079412 wait complete. 2024-09-10T07:49:52.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:49:52.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:49:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.210+0000 7fb755a00640 1 Processor -- start 2024-09-10T07:49:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.210+0000 7fb755a00640 1 -- start start 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.211+0000 7fb755a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7500cd050 0x7fb75015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.211+0000 7fb755a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb7500b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.211+0000 7fb755a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7500ce9a0 con 0x7fb750158bf0 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.211+0000 7fb755a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb7500b2920 con 0x7fb7500cd050 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.211+0000 7fb755a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7500cec80 con 0x7fb7500b1f90 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7500cd050 0x7fb75015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb7500cd050 0x7fb75015f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40720/0 (socket says 172.21.15.3:40720) 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 -- 172.21.15.3:0/1242336067 learned_addr learned my addr 172.21.15.3:0/1242336067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb74f400640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb7500b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 -- 172.21.15.3:0/1242336067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 msgr2=0x7fb7500b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb7500b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 -- 172.21.15.3:0/1242336067 >> v1:172.21.15.70:6789/0 conn(0x7fb750158bf0 legacy=0x7fb7500ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.212+0000 7fb747e00640 1 -- 172.21.15.3:0/1242336067 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75015efb0 con 0x7fb7500cd050 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.213+0000 7fb747e00640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7500cd050 0x7fb75015f7b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb738009870 tx=0x7fb73802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d458985a3795c3f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.213+0000 7fb74e000640 1 -- 172.21.15.3:0/1242336067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738037070 con 0x7fb7500cd050 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb74e000640 1 -- 172.21.15.3:0/1242336067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73802ae50 con 0x7fb7500cd050 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb74e000640 1 -- 172.21.15.3:0/1242336067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738032860 con 0x7fb7500cd050 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 -- 172.21.15.3:0/1242336067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7500cd050 msgr2=0x7fb75015f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7500cd050 0x7fb75015f7b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb738009870 tx=0x7fb73802a300 comp rx=0 tx=0).stop 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 -- 172.21.15.3:0/1242336067 shutdown_connections 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb7500b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 --2- 172.21.15.3:0/1242336067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7500cd050 0x7fb75015f7b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 -- 172.21.15.3:0/1242336067 >> 172.21.15.3:0/1242336067 conn(0x7fb750062cc0 msgr2=0x7fb75006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 -- 172.21.15.3:0/1242336067 shutdown_connections 2024-09-10T07:49:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.214+0000 7fb755a00640 1 -- 172.21.15.3:0/1242336067 wait complete. 2024-09-10T07:49:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.216+0000 7fb755a00640 1 Processor -- start 2024-09-10T07:49:52.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.216+0000 7fb755a00640 1 -- start start 2024-09-10T07:49:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb750157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7500cd050 0x7fb75015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb750158fd0 con 0x7fb750158bf0 2024-09-10T07:49:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb75015eca0 con 0x7fb7500cd050 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb755a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb750159150 con 0x7fb7500b1f90 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb74f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb74f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40724/0 (socket says 172.21.15.3:40724) 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb74f400640 1 -- 172.21.15.3:0/3627644024 learned_addr learned my addr 172.21.15.3:0/3627644024 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb74ea00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb750157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.217+0000 7fb747e00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7500cd050 0x7fb75015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 -- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 msgr2=0x7fb750157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb750157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 -- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7500cd050 msgr2=0x7fb75015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7500cd050 0x7fb75015abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 -- 172.21.15.3:0/3627644024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb750158530 con 0x7fb750158bf0 2024-09-10T07:49:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb74f400640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb748007e40 tx=0x7fb74800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748014620 con 0x7fb750158bf0 2024-09-10T07:49:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb748019070 con 0x7fb750158bf0 2024-09-10T07:49:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748014a20 con 0x7fb750158bf0 2024-09-10T07:49:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.218+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75015b4b0 con 0x7fb750158bf0 2024-09-10T07:49:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.219+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75015ba70 con 0x7fb750158bf0 2024-09-10T07:49:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.219+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7500b23e0 con 0x7fb750158bf0 2024-09-10T07:49:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.220+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb748014bc0 con 0x7fb750158bf0 2024-09-10T07:49:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.221+0000 7fb746a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 0x7fb72c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.221+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4857+0+0 (secure 0 0 0) 0x7fb74808b2e0 con 0x7fb750158bf0 2024-09-10T07:49:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.222+0000 7fb74ea00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 0x7fb72c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.222+0000 7fb74ea00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 0x7fb72c06e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7400016e0 tx=0x7fb740009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.226+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb748059980 con 0x7fb750158bf0 2024-09-10T07:49:52.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:52.411+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb7500b44d0 con 0x7fb750158bf0 2024-09-10T07:49:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.200+0000 7fb746a00640 1 -- 172.21.15.3:0/3627644024 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) ==== 138+0+0 (secure 0 0 0) 0x7fb74808b0c0 con 0x7fb750158bf0 2024-09-10T07:49:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:49:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 msgr2=0x7fb72c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 0x7fb72c06e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7400016e0 tx=0x7fb740009040 comp rx=0 tx=0).stop 2024-09-10T07:49:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 msgr2=0x7fb750160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb748007e40 tx=0x7fb74800e4b0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 shutdown_connections 2024-09-10T07:49:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb72c06be30 0x7fb72c06e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb750158bf0 0x7fb750160d00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7500cd050 0x7fb75015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 --2- 172.21.15.3:0/3627644024 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb7500b1f90 0x7fb750157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.204+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 >> 172.21.15.3:0/3627644024 conn(0x7fb750062cc0 msgr2=0x7fb7500ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:53.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.205+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 shutdown_connections 2024-09-10T07:49:53.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.205+0000 7fb755a00640 1 -- 172.21.15.3:0/3627644024 wait complete. 2024-09-10T07:49:53.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:49:53.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.395+0000 7f75eb200640 1 Processor -- start 2024-09-10T07:49:53.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.395+0000 7f75eb200640 1 -- start start 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75eb200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 0x7f75ec071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75eb200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75eb200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75ec071bf0 con 0x7f75ec0aede0 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75eb200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec0719c0 con 0x7f75ec15f2b0 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75eb200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec071ed0 con 0x7f75ec15eed0 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75e3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75e3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40738/0 (socket says 172.21.15.3:40738) 2024-09-10T07:49:53.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.396+0000 7f75e3e00640 1 -- 172.21.15.3:0/3229090682 learned_addr learned my addr 172.21.15.3:0/3229090682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:53.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e9e00640 1 --2- 172.21.15.3:0/3229090682 >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 0x7f75ec071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e3e00640 1 -- 172.21.15.3:0/3229090682 >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 msgr2=0x7f75ec071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:53.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e3e00640 1 --2- 172.21.15.3:0/3229090682 >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 0x7f75ec071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e3e00640 1 -- 172.21.15.3:0/3229090682 >> v1:172.21.15.70:6789/0 conn(0x7f75ec0aede0 legacy=0x7f75ec161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:53.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e3e00640 1 -- 172.21.15.3:0/3229090682 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec0721b0 con 0x7f75ec15f2b0 2024-09-10T07:49:53.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.397+0000 7f75e9e00640 1 --2- 172.21.15.3:0/3229090682 >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 0x7f75ec071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:53.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.398+0000 7f75e3e00640 1 --2- 172.21.15.3:0/3229090682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0d1980 tx=0x7f75d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2551156e4768255f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:53.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.398+0000 7f75e9400640 1 -- 172.21.15.3:0/3229090682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4037070 con 0x7f75ec15f2b0 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.398+0000 7f75e9400640 1 -- 172.21.15.3:0/3229090682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d4032440 con 0x7f75ec15f2b0 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.398+0000 7f75e9400640 1 -- 172.21.15.3:0/3229090682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4032810 con 0x7f75ec15f2b0 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 -- 172.21.15.3:0/3229090682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec15f2b0 msgr2=0x7f75ec0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 --2- 172.21.15.3:0/3229090682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0d1980 tx=0x7f75d402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 -- 172.21.15.3:0/3229090682 shutdown_connections 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 --2- 172.21.15.3:0/3229090682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec15f2b0 0x7f75ec0ae8a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 --2- 172.21.15.3:0/3229090682 >> v2:172.21.15.70:3300/0 conn(0x7f75ec15eed0 0x7f75ec071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 -- 172.21.15.3:0/3229090682 >> 172.21.15.3:0/3229090682 conn(0x7f75ec0b2540 msgr2=0x7f75ec0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 -- 172.21.15.3:0/3229090682 shutdown_connections 2024-09-10T07:49:53.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.399+0000 7f75eb200640 1 -- 172.21.15.3:0/3229090682 wait complete. 2024-09-10T07:49:53.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.400+0000 7f75eb200640 1 Processor -- start 2024-09-10T07:49:53.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 -- start start 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75ec15eed0 0x7f75ec1e9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75ec15f2b0 0x7f75ec158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec15f6f0 con 0x7f75ec0aede0 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec072490 con 0x7f75ec15eed0 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.401+0000 7f75eb200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec15f870 con 0x7f75ec15f2b0 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40742/0 (socket says 172.21.15.3:40742) 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 -- 172.21.15.3:0/1365708658 learned_addr learned my addr 172.21.15.3:0/1365708658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e3e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75ec15eed0 0x7f75ec1e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75ea800640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75ec15f2b0 0x7f75ec158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 -- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75ec15f2b0 msgr2=0x7f75ec158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75ec15f2b0 0x7f75ec158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 -- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75ec15eed0 msgr2=0x7f75ec1e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75ec15eed0 0x7f75ec1e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:53.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 -- 172.21.15.3:0/1365708658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec1594c0 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.402+0000 7f75e9e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7f0 tx=0x7f75dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.403+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc015070 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.403+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc007820 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.403+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc007bd0 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.403+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec159740 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.403+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec159d00 con 0x7f75ec0aede0 2024-09-10T07:49:53.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.404+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec071480 con 0x7f75ec0aede0 2024-09-10T07:49:53.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.406+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75dc018070 con 0x7f75ec0aede0 2024-09-10T07:49:53.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.407+0000 7f75e2000640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 0x7f75c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:53.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.407+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5227+0+0 (secure 0 0 0) 0x7f75dc08b050 con 0x7f75ec0aede0 2024-09-10T07:49:53.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.407+0000 7f75e3e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 0x7f75c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:53.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.408+0000 7f75e3e00640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 0x7f75c806e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75d4034000 tx=0x7f75d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:53.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.411+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f75ec1594c0 con 0x7f75ec0aede0 2024-09-10T07:49:53.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:53.599+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f75ec0afed0 con 0x7f75ec0aede0 2024-09-10T07:49:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.207+0000 7f75e2000640 1 -- 172.21.15.3:0/1365708658 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) ==== 170+0+0 (secure 0 0 0) 0x7f75dc059d50 con 0x7f75ec0aede0 2024-09-10T07:49:54.207 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 msgr2=0x7f75c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 0x7f75c806e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75d4034000 tx=0x7f75d402e000 comp rx=0 tx=0).stop 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 msgr2=0x7f75ec146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7f0 tx=0x7f75dc00ccb0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 shutdown_connections 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f75c806be30 0x7f75c806e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75ec15f2b0 0x7f75ec158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75ec15eed0 0x7f75ec1e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 --2- 172.21.15.3:0/1365708658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75ec0aede0 0x7f75ec146560 secure :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7f0 tx=0x7f75dc00ccb0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.211+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 >> 172.21.15.3:0/1365708658 conn(0x7f75ec0b2540 msgr2=0x7f75ec15c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.212+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 shutdown_connections 2024-09-10T07:49:54.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.212+0000 7f75eb200640 1 -- 172.21.15.3:0/1365708658 wait complete. 2024-09-10T07:49:54.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:49:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.408+0000 7fced1e00640 1 Processor -- start 2024-09-10T07:49:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.408+0000 7fced1e00640 1 -- start start 2024-09-10T07:49:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.409+0000 7fced1e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcecc145730 0x7fcecc06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.409+0000 7fced1e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcecc0cd900 con 0x7fcecc0cf270 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.409+0000 7fced1e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcecc0cdbe0 con 0x7fcecc1451d0 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.409+0000 7fced1e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcecc06d7c0 con 0x7fcecc145730 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcecb400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcecc0cf270 0x7fcecc0cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43042/0 (socket says 172.21.15.3:43042) 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcecb400640 1 -- 172.21.15.3:0/737703312 learned_addr learned my addr 172.21.15.3:0/737703312 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fceca000640 1 --2- 172.21.15.3:0/737703312 >> v2:172.21.15.70:3300/0 conn(0x7fcecc145730 0x7fcecc06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740330171 0 0) 0x7fcecc0cd900 con 0x7fcecc0cf270 2024-09-10T07:49:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fcecc0cf270 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838175990 0 0) 0x7fcecc0cdbe0 con 0x7fcecc1451d0 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcecc0cd900 con 0x7fcecc1451d0 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2306258685 0 0) 0x7fceb0003520 con 0x7fcecc0cf270 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fceca000640 1 -- 172.21.15.3:0/737703312 >> v1:172.21.15.70:6789/0 conn(0x7fcecc1451d0 legacy=0x7fcecc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fceca000640 1 -- 172.21.15.3:0/737703312 >> v1:172.21.15.3:6789/0 conn(0x7fcecc0cf270 legacy=0x7fcecc0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.410+0000 7fceca000640 1 -- 172.21.15.3:0/737703312 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc06da10 con 0x7fcecc145730 2024-09-10T07:49:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fceca000640 1 --2- 172.21.15.3:0/737703312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc145730 0x7fcecc06cfc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcecc06d940 tx=0x7fceb802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3878eb04360229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8037070 con 0x7fcecc145730 2024-09-10T07:49:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb802aba0 con 0x7fcecc145730 2024-09-10T07:49:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fcec9600640 1 -- 172.21.15.3:0/737703312 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40019f0 con 0x7fcecc145730 2024-09-10T07:49:54.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fced1e00640 1 -- 172.21.15.3:0/737703312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc145730 msgr2=0x7fcecc06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fced1e00640 1 --2- 172.21.15.3:0/737703312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc145730 0x7fcecc06cfc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcecc06d940 tx=0x7fceb802a110 comp rx=0 tx=0).stop 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fced1e00640 1 -- 172.21.15.3:0/737703312 shutdown_connections 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.412+0000 7fced1e00640 1 --2- 172.21.15.3:0/737703312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc145730 0x7fcecc06cfc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.413+0000 7fced1e00640 1 -- 172.21.15.3:0/737703312 >> 172.21.15.3:0/737703312 conn(0x7fcecc0b5cb0 msgr2=0x7fcecc06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.413+0000 7fced1e00640 1 -- 172.21.15.3:0/737703312 shutdown_connections 2024-09-10T07:49:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.413+0000 7fced1e00640 1 -- 172.21.15.3:0/737703312 wait complete. 2024-09-10T07:49:54.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.414+0000 7fced1e00640 1 Processor -- start 2024-09-10T07:49:54.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.414+0000 7fced1e00640 1 -- start start 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc0cf270 0x7fcecc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcecc145730 0x7fcecc15e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc06e0b0 con 0x7fcecc1451d0 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc06df30 con 0x7fcecc0cf270 2024-09-10T07:49:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fced1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcecc06e230 con 0x7fcecc145730 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fceca000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fceca000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40092/0 (socket says 172.21.15.3:40092) 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.415+0000 7fceca000640 1 -- 172.21.15.3:0/4278856448 learned_addr learned my addr 172.21.15.3:0/4278856448 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fcecaa00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc0cf270 0x7fcecc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fcecb400640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcecc145730 0x7fcecc15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 -- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcecc145730 msgr2=0x7fcecc15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcecc145730 0x7fcecc15e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 -- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc0cf270 msgr2=0x7fcecc0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc0cf270 0x7fcecc0b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 -- 172.21.15.3:0/4278856448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc15c8a0 con 0x7fcecc1451d0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fceca000640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009890 tx=0x7fceb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8037050 con 0x7fcecc1451d0 2024-09-10T07:49:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb8043070 con 0x7fcecc1451d0 2024-09-10T07:49:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.416+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8006e10 con 0x7fcecc1451d0 2024-09-10T07:49:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.417+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc15cb80 con 0x7fcecc1451d0 2024-09-10T07:49:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.417+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc15d140 con 0x7fcecc1451d0 2024-09-10T07:49:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.418+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc06cfc0 con 0x7fcecc1451d0 2024-09-10T07:49:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.419+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fceb8048460 con 0x7fcecc1451d0 2024-09-10T07:49:54.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.420+0000 7fcec3400640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 0x7fceac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:54.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.420+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5609+0+0 (secure 0 0 0) 0x7fceb80abff0 con 0x7fcecc1451d0 2024-09-10T07:49:54.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.420+0000 7fcecaa00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 0x7fceac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:54.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.421+0000 7fcecaa00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 0x7fceac06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0011e0 tx=0x7fcebc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.424+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fceb804d020 con 0x7fcecc1451d0 2024-09-10T07:49:54.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:54.646+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcecc0b38a0 con 0x7fcecc1451d0 2024-09-10T07:49:55.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.210+0000 7fcec3400640 1 -- 172.21.15.3:0/4278856448 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v47) ==== 172+0+0 (secure 0 0 0) 0x7fceb807a970 con 0x7fcecc1451d0 2024-09-10T07:49:55.210 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T07:49:55.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 msgr2=0x7fceac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 0x7fceac06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0011e0 tx=0x7fcebc009040 comp rx=0 tx=0).stop 2024-09-10T07:49:55.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 msgr2=0x7fcecc15dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009890 tx=0x7fceb80046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 shutdown_connections 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fceac06be30 0x7fceac06e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcecc145730 0x7fcecc15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcecc1451d0 0x7fcecc15dc20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 --2- 172.21.15.3:0/4278856448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcecc0cf270 0x7fcecc0b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 >> 172.21.15.3:0/4278856448 conn(0x7fcecc0b5cb0 msgr2=0x7fcecc144010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.214+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 shutdown_connections 2024-09-10T07:49:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.215+0000 7fced1e00640 1 -- 172.21.15.3:0/4278856448 wait complete. 2024-09-10T07:49:55.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:55.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.414+0000 7f2c52200640 1 Processor -- start 2024-09-10T07:49:55.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.414+0000 7f2c52200640 1 -- start start 2024-09-10T07:49:55.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c52200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 0x7f2c4c073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c52200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c52200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c0cfdc0 con 0x7f2c4c15e860 2024-09-10T07:49:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c52200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c0d3780 con 0x7f2c4c0d4050 2024-09-10T07:49:55.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c52200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cee40 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 0x7f2c4c073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.415+0000 7f2c51800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c51800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:59098/0 (socket says 172.21.15.3:59098) 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c51800640 1 -- 172.21.15.3:0/1593215558 learned_addr learned my addr 172.21.15.3:0/1593215558 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c4be00640 1 -- 172.21.15.3:0/1593215558 >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 msgr2=0x7f2c4c073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c4be00640 1 --2- 172.21.15.3:0/1593215558 >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c4be00640 1 -- 172.21.15.3:0/1593215558 >> v1:172.21.15.70:6789/0 conn(0x7f2c4c15e860 legacy=0x7f2c4c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c4be00640 1 -- 172.21.15.3:0/1593215558 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c0cf120 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.416+0000 7f2c51800640 1 --2- 172.21.15.3:0/1593215558 >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.417+0000 7f2c4be00640 1 --2- 172.21.15.3:0/1593215558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 0x7f2c4c073150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097a0 tx=0x7f2c3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ce316b3ce6aebb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.417+0000 7f2c4b400640 1 -- 172.21.15.3:0/1593215558 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38037070 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c4b400640 1 -- 172.21.15.3:0/1593215558 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c38032440 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c4b400640 1 -- 172.21.15.3:0/1593215558 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38032810 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 -- 172.21.15.3:0/1593215558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 msgr2=0x7f2c4c073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 --2- 172.21.15.3:0/1593215558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 0x7f2c4c073150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097a0 tx=0x7f2c3802a180 comp rx=0 tx=0).stop 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 -- 172.21.15.3:0/1593215558 shutdown_connections 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 --2- 172.21.15.3:0/1593215558 >> v2:172.21.15.70:3300/0 conn(0x7f2c4c0d4050 0x7f2c4c073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 --2- 172.21.15.3:0/1593215558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d3c10 0x7f2c4c073150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 -- 172.21.15.3:0/1593215558 >> 172.21.15.3:0/1593215558 conn(0x7f2c4c0d4650 msgr2=0x7f2c4c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 -- 172.21.15.3:0/1593215558 shutdown_connections 2024-09-10T07:49:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.418+0000 7f2c52200640 1 -- 172.21.15.3:0/1593215558 wait complete. 2024-09-10T07:49:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.420+0000 7f2c52200640 1 Processor -- start 2024-09-10T07:49:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.420+0000 7f2c52200640 1 -- start start 2024-09-10T07:49:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d4050 0x7f2c4c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c15e860 0x7f2c4c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0d2460 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cf400 con 0x7f2c4c15e860 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c52200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0b7880 con 0x7f2c4c0d4050 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c50e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c50e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40094/0 (socket says 172.21.15.3:40094) 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c50e00640 1 -- 172.21.15.3:0/1491749128 learned_addr learned my addr 172.21.15.3:0/1491749128 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c4be00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d4050 0x7f2c4c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.421+0000 7f2c51800640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c15e860 0x7f2c4c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 -- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d4050 msgr2=0x7f2c4c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d4050 0x7f2c4c15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 -- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c15e860 msgr2=0x7f2c4c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c15e860 0x7f2c4c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 -- 172.21.15.3:0/1491749128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c15fce0 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c50e00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c3f0 tx=0x7f2c4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400052d0 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40005490 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40007260 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c15ec40 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.422+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c15f200 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.423+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0b5f60 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.424+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c40007400 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.425+0000 7f2c48c00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.425+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2c4008a290 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.425+0000 7f2c4be00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.426+0000 7f2c4be00640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c38034000 tx=0x7f2c3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.430+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c4c0b5f60 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.618+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c4c0cf6b0 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.619+0000 7f2c48c00640 1 -- 172.21.15.3:0/1491749128 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16666 (secure 0 0 0) 0x7f2c4c15fce0 con 0x7f2c4c0d3c10 2024-09-10T07:49:55.619 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:55.619 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":63,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:54.647591+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:49:52.454520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T07:49:53.642181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 msgr2=0x7f2c2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c38034000 tx=0x7f2c3802e000 comp rx=0 tx=0).stop 2024-09-10T07:49:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 msgr2=0x7f2c4c072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c3f0 tx=0x7f2c4000c8b0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 shutdown_connections 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c2c06be30 0x7f2c2c06e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c15e860 0x7f2c4c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c4c0d4050 0x7f2c4c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 --2- 172.21.15.3:0/1491749128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c4c0d3c10 0x7f2c4c072640 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.621+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 >> 172.21.15.3:0/1491749128 conn(0x7f2c4c0d4650 msgr2=0x7f2c4c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.622+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 shutdown_connections 2024-09-10T07:49:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.622+0000 7f2c52200640 1 -- 172.21.15.3:0/1491749128 wait complete. 2024-09-10T07:49:55.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.794+0000 7f9608000640 1 Processor -- start 2024-09-10T07:49:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.794+0000 7f9608000640 1 -- start start 2024-09-10T07:49:55.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9608000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 0x7f9600167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9608000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96000cfdd0 con 0x7f9600160af0 2024-09-10T07:49:55.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9608000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960015cc90 con 0x7f9600160710 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9608000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f960015cf70 con 0x7f96000d07c0 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9604c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9600160af0 0x7f96000d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43056/0 (socket says 172.21.15.3:43056) 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9604c00640 1 -- 172.21.15.3:0/4115784454 learned_addr learned my addr 172.21.15.3:0/4115784454 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f9606000640 1 --2- 172.21.15.3:0/4115784454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 0x7f9600167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.796+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062046996 0 0) 0x7f96000cfdd0 con 0x7f9600160af0 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95e8003520 con 0x7f9600160af0 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820058738 0 0) 0x7f960015cc90 con 0x7f9600160710 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96000cfdd0 con 0x7f9600160710 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f9606000640 1 -- 172.21.15.3:0/4115784454 >> v1:172.21.15.70:6789/0 conn(0x7f9600160710 legacy=0x7f96000cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f9606000640 1 -- 172.21.15.3:0/4115784454 >> v1:172.21.15.3:6789/0 conn(0x7f9600160af0 legacy=0x7f96000d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.797+0000 7f9606000640 1 -- 172.21.15.3:0/4115784454 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960015d250 con 0x7f96000d07c0 2024-09-10T07:49:55.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.798+0000 7f9606000640 1 --2- 172.21.15.3:0/4115784454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 0x7f9600167090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009dc0 tx=0x7f95f802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9462e74f8131ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.798+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007740 con 0x7f96000d07c0 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.798+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f80078e0 con 0x7f96000d07c0 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.798+0000 7f95ffe00640 1 -- 172.21.15.3:0/4115784454 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007cb0 con 0x7f96000d07c0 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 -- 172.21.15.3:0/4115784454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 msgr2=0x7f9600167090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 --2- 172.21.15.3:0/4115784454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 0x7f9600167090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009dc0 tx=0x7f95f802a6d0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 -- 172.21.15.3:0/4115784454 shutdown_connections 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 --2- 172.21.15.3:0/4115784454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96000d07c0 0x7f9600167090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 -- 172.21.15.3:0/4115784454 >> 172.21.15.3:0/4115784454 conn(0x7f960006d000 msgr2=0x7f960006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 -- 172.21.15.3:0/4115784454 shutdown_connections 2024-09-10T07:49:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.799+0000 7f9608000640 1 -- 172.21.15.3:0/4115784454 wait complete. 2024-09-10T07:49:55.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.801+0000 7f9608000640 1 Processor -- start 2024-09-10T07:49:55.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.801+0000 7f9608000640 1 -- start start 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000d07c0 0x7f9600143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9600160af0 0x7f96000b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f960015d770 con 0x7f9600160710 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9600073080 con 0x7f96000d07c0 2024-09-10T07:49:55.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9608000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9600155b90 con 0x7f9600160af0 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40108/0 (socket says 172.21.15.3:40108) 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 -- 172.21.15.3:0/2969357965 learned_addr learned my addr 172.21.15.3:0/2969357965 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9605600640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000d07c0 0x7f9600143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9606000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9600160af0 0x7f96000b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 -- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9600160af0 msgr2=0x7f96000b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9600160af0 0x7f96000b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 -- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000d07c0 msgr2=0x7f9600143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000d07c0 0x7f9600143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.802+0000 7f9604c00640 1 -- 172.21.15.3:0/2969357965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9600154790 con 0x7f9600160710 2024-09-10T07:49:55.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f9604c00640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c3c0 tx=0x7f95ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec017070 con 0x7f9600160710 2024-09-10T07:49:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec005ce0 con 0x7f9600160710 2024-09-10T07:49:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0072d0 con 0x7f9600160710 2024-09-10T07:49:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9600154f80 con 0x7f9600160710 2024-09-10T07:49:55.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.803+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9600155540 con 0x7f9600160710 2024-09-10T07:49:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.804+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96000cfa60 con 0x7f9600160710 2024-09-10T07:49:55.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.805+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95ec01a070 con 0x7f9600160710 2024-09-10T07:49:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.806+0000 7f95fd600640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 0x7f95e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.806+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f95ec08a140 con 0x7f9600160710 2024-09-10T07:49:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.806+0000 7f9605600640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 0x7f95e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:55.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.807+0000 7f9605600640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 0x7f95e006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007fd0 tx=0x7f95f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:55.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.810+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95ec058490 con 0x7f9600160710 2024-09-10T07:49:55.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.995+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f960015d8f0 con 0x7f9600160710 2024-09-10T07:49:55.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.996+0000 7f95fd600640 1 -- 172.21.15.3:0/2969357965 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16666 (secure 0 0 0) 0x7f95ec05c790 con 0x7f9600160710 2024-09-10T07:49:55.996 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:55.996 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":63,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:54.647591+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:49:52.454520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T07:49:53.642181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:55.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 msgr2=0x7f95e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 0x7f95e006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95f4007fd0 tx=0x7f95f4009040 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 msgr2=0x7f9600154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c3c0 tx=0x7f95ec00c880 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 shutdown_connections 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f95e006be30 0x7f95e006e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9600160af0 0x7f96000b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9600160710 0x7f9600154250 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 --2- 172.21.15.3:0/2969357965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000d07c0 0x7f9600143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.998+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 >> 172.21.15.3:0/2969357965 conn(0x7f960006d000 msgr2=0x7f960015bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.999+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 shutdown_connections 2024-09-10T07:49:55.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:55.999+0000 7f9608000640 1 -- 172.21.15.3:0/2969357965 wait complete. 2024-09-10T07:49:56.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:49:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.173+0000 7f017ba00640 1 Processor -- start 2024-09-10T07:49:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.174+0000 7f017ba00640 1 -- start start 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f017ba00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01740cf980 0x7f017406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f017ba00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01740cd900 con 0x7f0174145730 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f017ba00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01740cdbe0 con 0x7f01741451d0 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f017ba00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f017406d7c0 con 0x7f01740cf980 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f0173e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0174145730 0x7f01740cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43058/0 (socket says 172.21.15.3:43058) 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f0173e00640 1 -- 172.21.15.3:0/1344662904 learned_addr learned my addr 172.21.15.3:0/1344662904 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:56.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f0179a00640 1 --2- 172.21.15.3:0/1344662904 >> v2:172.21.15.3:3300/0 conn(0x7f01740cf980 0x7f017406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:56.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f0179a00640 1 -- 172.21.15.3:0/1344662904 >> v1:172.21.15.70:6789/0 conn(0x7f01741451d0 legacy=0x7f01740b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:56.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.175+0000 7f0179a00640 1 -- 172.21.15.3:0/1344662904 >> v1:172.21.15.3:6789/0 conn(0x7f0174145730 legacy=0x7f01740cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:56.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.176+0000 7f0179a00640 1 -- 172.21.15.3:0/1344662904 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017406da10 con 0x7f01740cf980 2024-09-10T07:49:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.176+0000 7f0173400640 1 -- 172.21.15.3:0/1344662904 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 472413643 0 0) 0x7f01740cd900 con 0x7f0174145730 2024-09-10T07:49:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.176+0000 7f0179a00640 1 --2- 172.21.15.3:0/1344662904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01740cf980 0x7f017406cfc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f017406d940 tx=0x7f016c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73cbd334c6378a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f0173400640 1 -- 172.21.15.3:0/1344662904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c007740 con 0x7f01740cf980 2024-09-10T07:49:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f0173400640 1 -- 172.21.15.3:0/1344662904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016c0078e0 con 0x7f01740cf980 2024-09-10T07:49:56.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f0173400640 1 -- 172.21.15.3:0/1344662904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c007cb0 con 0x7f01740cf980 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f017ba00640 1 -- 172.21.15.3:0/1344662904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01740cf980 msgr2=0x7f017406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f017ba00640 1 --2- 172.21.15.3:0/1344662904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01740cf980 0x7f017406cfc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f017406d940 tx=0x7f016c02a660 comp rx=0 tx=0).stop 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f017ba00640 1 -- 172.21.15.3:0/1344662904 shutdown_connections 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f017ba00640 1 --2- 172.21.15.3:0/1344662904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01740cf980 0x7f017406cfc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.177+0000 7f017ba00640 1 -- 172.21.15.3:0/1344662904 >> 172.21.15.3:0/1344662904 conn(0x7f01740b5cb0 msgr2=0x7f017406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.178+0000 7f017ba00640 1 -- 172.21.15.3:0/1344662904 shutdown_connections 2024-09-10T07:49:56.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.178+0000 7f017ba00640 1 -- 172.21.15.3:0/1344662904 wait complete. 2024-09-10T07:49:56.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.179+0000 7f017ba00640 1 Processor -- start 2024-09-10T07:49:56.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.179+0000 7f017ba00640 1 -- start start 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01740cf980 0x7f017415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01741451d0 0x7f017415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f017406e0b0 con 0x7f0174145730 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f017406df30 con 0x7f01740cf980 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f017ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f017406e230 con 0x7f01741451d0 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f0179a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f0179a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40126/0 (socket says 172.21.15.3:40126) 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f0179a00640 1 -- 172.21.15.3:0/1141155156 learned_addr learned my addr 172.21.15.3:0/1141155156 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f0179000640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01740cf980 0x7f017415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.180+0000 7f0173e00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01741451d0 0x7f017415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 -- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01741451d0 msgr2=0x7f017415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01741451d0 0x7f017415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 -- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01740cf980 msgr2=0x7f017415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01740cf980 0x7f017415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 -- 172.21.15.3:0/1141155156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017415eba0 con 0x7f0174145730 2024-09-10T07:49:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0179a00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e50 tx=0x7f016c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.181+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c040070 con 0x7f0174145730 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.182+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016c031bf0 con 0x7f0174145730 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.182+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c045460 con 0x7f0174145730 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.182+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017415ee50 con 0x7f0174145730 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.182+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017413b520 con 0x7f0174145730 2024-09-10T07:49:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.183+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01741609c0 con 0x7f0174145730 2024-09-10T07:49:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.184+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f016c039040 con 0x7f0174145730 2024-09-10T07:49:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.185+0000 7f0170c00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.185+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f016c0ac730 con 0x7f0174145730 2024-09-10T07:49:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.185+0000 7f0179000640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.185+0000 7f0179000640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01741f5700 tx=0x7f0168009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:56.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.189+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f016c07aa80 con 0x7f0174145730 2024-09-10T07:49:56.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:56.415+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f01740b2af0 con 0x7f0174145730 2024-09-10T07:49:57.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.245+0000 7f0170c00640 1 -- 172.21.15.3:0/1141155156 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) ==== 118+0+0 (secure 0 0 0) 0x7f016c07ed80 con 0x7f0174145730 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 msgr2=0x7f015406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 0x7f015406e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f01741f5700 tx=0x7f0168009040 comp rx=0 tx=0).stop 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 msgr2=0x7f017415e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f016c009e50 tx=0x7f016c004a40 comp rx=0 tx=0).stop 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 shutdown_connections 2024-09-10T07:49:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f015406be30 0x7f015406e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0174145730 0x7f017415e660 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f01741451d0 0x7f017415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 --2- 172.21.15.3:0/1141155156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01740cf980 0x7f017415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.247+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 >> 172.21.15.3:0/1141155156 conn(0x7f01740b5cb0 msgr2=0x7f017415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.248+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 shutdown_connections 2024-09-10T07:49:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.248+0000 7f017ba00640 1 -- 172.21.15.3:0/1141155156 wait complete. 2024-09-10T07:49:57.264 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:49:57.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:49:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.425+0000 7fe053600640 1 Processor -- start 2024-09-10T07:49:57.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.425+0000 7fe053600640 1 -- start start 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.426+0000 7fe053600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe04c15b230 0x7fe04c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe053600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c0b4c90 0x7fe04c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe053600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe04c15eff0 con 0x7fe04c072c50 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe053600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe04c15fbe0 con 0x7fe04c15b230 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe053600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe04c15c8d0 con 0x7fe04c0b4c90 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe04c15b230 0x7fe04c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe04c15b230 0x7fe04c15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40132/0 (socket says 172.21.15.3:40132) 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 -- 172.21.15.3:0/443804708 learned_addr learned my addr 172.21.15.3:0/443804708 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe051600640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c0b4c90 0x7fe04c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 -- 172.21.15.3:0/443804708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c0b4c90 msgr2=0x7fe04c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c0b4c90 0x7fe04c071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 -- 172.21.15.3:0/443804708 >> v1:172.21.15.70:6789/0 conn(0x7fe04c072c50 legacy=0x7fe04c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:57.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.427+0000 7fe050c00640 1 -- 172.21.15.3:0/443804708 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c15f2b0 con 0x7fe04c15b230 2024-09-10T07:49:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.428+0000 7fe050c00640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c15b230 0x7fe04c15c390 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe040009870 tx=0x7fe04002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb6ee0128ac5283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe04b400640 1 -- 172.21.15.3:0/443804708 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040037070 con 0x7fe04c15b230 2024-09-10T07:49:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe04b400640 1 -- 172.21.15.3:0/443804708 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe040032440 con 0x7fe04c15b230 2024-09-10T07:49:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe04b400640 1 -- 172.21.15.3:0/443804708 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040032810 con 0x7fe04c15b230 2024-09-10T07:49:57.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 -- 172.21.15.3:0/443804708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c15b230 msgr2=0x7fe04c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c15b230 0x7fe04c15c390 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe040009870 tx=0x7fe04002a270 comp rx=0 tx=0).stop 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 -- 172.21.15.3:0/443804708 shutdown_connections 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c0b4c90 0x7fe04c071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 --2- 172.21.15.3:0/443804708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c15b230 0x7fe04c15c390 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 -- 172.21.15.3:0/443804708 >> 172.21.15.3:0/443804708 conn(0x7fe04c0b5cb0 msgr2=0x7fe04c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 -- 172.21.15.3:0/443804708 shutdown_connections 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.429+0000 7fe053600640 1 -- 172.21.15.3:0/443804708 wait complete. 2024-09-10T07:49:57.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.431+0000 7fe053600640 1 Processor -- start 2024-09-10T07:49:57.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.431+0000 7fe053600640 1 -- start start 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 0x7fe04c0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c15b230 0x7fe04c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c15b610 con 0x7fe04c072c50 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c15f870 con 0x7fe04c0b4c90 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe053600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe04c15b790 con 0x7fe04c15b230 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe050c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe050c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40134/0 (socket says 172.21.15.3:40134) 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.432+0000 7fe050c00640 1 -- 172.21.15.3:0/4013221729 learned_addr learned my addr 172.21.15.3:0/4013221729 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:57.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 -- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c15b230 msgr2=0x7fe04c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c15b230 0x7fe04c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe04be00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 0x7fe04c0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 -- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 msgr2=0x7fe04c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 0x7fe04c0b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 -- 172.21.15.3:0/4013221729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c150390 con 0x7fe04c072c50 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe04be00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 0x7fe04c0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe050c00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe0400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040041020 con 0x7fe04c072c50 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0400306c0 con 0x7fe04c072c50 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040037040 con 0x7fe04c072c50 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c150670 con 0x7fe04c072c50 2024-09-10T07:49:57.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.433+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c150c30 con 0x7fe04c072c50 2024-09-10T07:49:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.434+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04c06e1e0 con 0x7fe04c072c50 2024-09-10T07:49:57.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.435+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe040044020 con 0x7fe04c072c50 2024-09-10T07:49:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.436+0000 7fe048c00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 0x7fe02c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.436+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fe0400ab700 con 0x7fe04c072c50 2024-09-10T07:49:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.436+0000 7fe04be00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 0x7fe02c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:57.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.437+0000 7fe04be00640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 0x7fe02c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe04c1f3c80 tx=0x7fe038009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.441+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe04004b030 con 0x7fe04c072c50 2024-09-10T07:49:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:57.664+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe04c06dc90 con 0x7fe04c072c50 2024-09-10T07:49:58.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.254+0000 7fe048c00640 1 -- 172.21.15.3:0/4013221729 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fe04007a010 con 0x7fe04c072c50 2024-09-10T07:49:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.256+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 msgr2=0x7fe02c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.256+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 0x7fe02c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe04c1f3c80 tx=0x7fe038009040 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.256+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 msgr2=0x7fe04c14b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.256+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe0400046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 shutdown_connections 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe02c06be30 0x7fe02c06e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe04c15b230 0x7fe04c155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c0b4c90 0x7fe04c0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 --2- 172.21.15.3:0/4013221729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe04c072c50 0x7fe04c14b5e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 >> 172.21.15.3:0/4013221729 conn(0x7fe04c0b5cb0 msgr2=0x7fe04c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 shutdown_connections 2024-09-10T07:49:58.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.257+0000 7fe053600640 1 -- 172.21.15.3:0/4013221729 wait complete. 2024-09-10T07:49:58.275 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:49:58.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:49:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.435+0000 7f4f98c00640 1 Processor -- start 2024-09-10T07:49:58.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.435+0000 7f4f98c00640 1 -- start start 2024-09-10T07:49:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f94145010 0x7f4f94073410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 0x7f4f940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94073950 0x7f4f940b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4f940ced10 con 0x7f4f94145010 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4f940ce730 con 0x7f4f9415ca80 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f98c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f94073e60 con 0x7f4f94073950 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f93400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f94145010 0x7f4f94073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f93400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f94145010 0x7f4f94073410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40138/0 (socket says 172.21.15.3:40138) 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f93400640 1 -- 172.21.15.3:0/1991155022 learned_addr learned my addr 172.21.15.3:0/1991155022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f93e00640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94073950 0x7f4f940b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.436+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1991155022 >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 0x7f4f940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f93400640 1 -- 172.21.15.3:0/1991155022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94073950 msgr2=0x7f4f940b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f93400640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94073950 0x7f4f940b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f93400640 1 -- 172.21.15.3:0/1991155022 >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 msgr2=0x7f4f940b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f93400640 1 --2- 172.21.15.3:0/1991155022 >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 0x7f4f940b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f93400640 1 -- 172.21.15.3:0/1991155022 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f940ce9a0 con 0x7f4f94145010 2024-09-10T07:49:58.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.437+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1991155022 >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 0x7f4f940b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.438+0000 7f4f93400640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94145010 0x7f4f94073410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4f940aff80 tx=0x7f4f8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6733887cb3bedb97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.438+0000 7f4f92000640 1 -- 172.21.15.3:0/1991155022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84037070 con 0x7f4f94145010 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.438+0000 7f4f92000640 1 -- 172.21.15.3:0/1991155022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f84032440 con 0x7f4f94145010 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.438+0000 7f4f92000640 1 -- 172.21.15.3:0/1991155022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84032810 con 0x7f4f94145010 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 -- 172.21.15.3:0/1991155022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94145010 msgr2=0x7f4f94073410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94145010 0x7f4f94073410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4f940aff80 tx=0x7f4f8402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 -- 172.21.15.3:0/1991155022 shutdown_connections 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94073950 0x7f4f940b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1991155022 >> v2:172.21.15.70:3300/0 conn(0x7f4f9415ca80 0x7f4f940b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1991155022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94145010 0x7f4f94073410 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 -- 172.21.15.3:0/1991155022 >> 172.21.15.3:0/1991155022 conn(0x7f4f9406bb20 msgr2=0x7f4f9406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 -- 172.21.15.3:0/1991155022 shutdown_connections 2024-09-10T07:49:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.439+0000 7f4f98c00640 1 -- 172.21.15.3:0/1991155022 wait complete. 2024-09-10T07:49:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.440+0000 7f4f98c00640 1 Processor -- start 2024-09-10T07:49:58.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 -- start start 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 0x7f4f941f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 0x7f4f941564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94145570 con 0x7f4f94073950 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941453f0 con 0x7f4f9415ca80 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.441+0000 7f4f98c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f94071cd0 con 0x7f4f94145010 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40148/0 (socket says 172.21.15.3:40148) 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 -- 172.21.15.3:0/1442577530 learned_addr learned my addr 172.21.15.3:0/1442577530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 0x7f4f941f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93e00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 0x7f4f941564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 -- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 msgr2=0x7f4f941f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 0x7f4f941f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 -- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 msgr2=0x7f4f941564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 0x7f4f941564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 -- 172.21.15.3:0/1442577530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9413d860 con 0x7f4f94073950 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 0x7f4f941f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93e00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 0x7f4f941564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:58.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.442+0000 7f4f93400640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402a220 tx=0x7f4f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.443+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84037040 con 0x7f4f94073950 2024-09-10T07:49:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.443+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f84042070 con 0x7f4f94073950 2024-09-10T07:49:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.443+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84030880 con 0x7f4f94073950 2024-09-10T07:49:58.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.443+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9413db40 con 0x7f4f94073950 2024-09-10T07:49:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.443+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f94154910 con 0x7f4f94073950 2024-09-10T07:49:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.444+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f94063610 con 0x7f4f94073950 2024-09-10T07:49:58.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.445+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f84030a20 con 0x7f4f94073950 2024-09-10T07:49:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.446+0000 7f4f8b400640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 0x7f4f7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.446+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f4f840ab790 con 0x7f4f94073950 2024-09-10T07:49:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.446+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 0x7f4f7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.447+0000 7f4f92a00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 0x7f4f7406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f94160e70 tx=0x7f4f80008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.451+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f84045360 con 0x7f4f94073950 2024-09-10T07:49:58.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.671+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f94073410 con 0x7f4f94073950 2024-09-10T07:49:58.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.672+0000 7f4f8b400640 1 -- 172.21.15.3:0/1442577530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5217 (secure 0 0 0) 0x7f4f8407a0b0 con 0x7f4f94073950 2024-09-10T07:49:58.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:58.672 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"btime":"2024-09-10T07:49:58:246472+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:49:57.665936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114]},"id":4}]} 2024-09-10T07:49:58.672 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2024-09-10T07:49:58.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 msgr2=0x7f4f7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 0x7f4f7406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f94160e70 tx=0x7f4f80008040 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 msgr2=0x7f4f9413b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402a220 tx=0x7f4f840046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 shutdown_connections 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f7406be30 0x7f4f7406e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9415ca80 0x7f4f941564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f94145010 0x7f4f941f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 --2- 172.21.15.3:0/1442577530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f94073950 0x7f4f9413b8a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 >> 172.21.15.3:0/1442577530 conn(0x7f4f9406bb20 msgr2=0x7f4f94141640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 shutdown_connections 2024-09-10T07:49:58.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.674+0000 7f4f98c00640 1 -- 172.21.15.3:0/1442577530 wait complete. 2024-09-10T07:49:58.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:49:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.846+0000 7fde4be00640 1 Processor -- start 2024-09-10T07:49:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.846+0000 7fde4be00640 1 -- start start 2024-09-10T07:49:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.847+0000 7fde4be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde4c145010 0x7fde4c073410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4be00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c0ce730 con 0x7fde4c15ca80 2024-09-10T07:49:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4be00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4c073e60 con 0x7fde4c0b2710 2024-09-10T07:49:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4be00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fde4c0ced10 con 0x7fde4c145010 2024-09-10T07:49:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde43e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fde4c15ca80 0x7fde4c0b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43062/0 (socket says 172.21.15.3:43062) 2024-09-10T07:49:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde43e00640 1 -- 172.21.15.3:0/3496635327 learned_addr learned my addr 172.21.15.3:0/3496635327 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4aa00640 1 --2- 172.21.15.3:0/3496635327 >> v2:172.21.15.70:3300/0 conn(0x7fde4c145010 0x7fde4c073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4a000640 1 -- 172.21.15.3:0/3496635327 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590269277 0 0) 0x7fde4c0ce730 con 0x7fde4c15ca80 2024-09-10T07:49:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.848+0000 7fde4a000640 1 -- 172.21.15.3:0/3496635327 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde30003520 con 0x7fde4c15ca80 2024-09-10T07:49:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.849+0000 7fde4aa00640 1 -- 172.21.15.3:0/3496635327 >> v1:172.21.15.70:6789/0 conn(0x7fde4c0b2710 legacy=0x7fde4c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.849+0000 7fde4aa00640 1 -- 172.21.15.3:0/3496635327 >> v1:172.21.15.3:6789/0 conn(0x7fde4c15ca80 legacy=0x7fde4c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.849+0000 7fde4aa00640 1 -- 172.21.15.3:0/3496635327 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c0ce9a0 con 0x7fde4c145010 2024-09-10T07:49:58.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4aa00640 1 --2- 172.21.15.3:0/3496635327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c073410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd4dc5c8c4cb0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4a000640 1 -- 172.21.15.3:0/3496635327 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c037070 con 0x7fde4c145010 2024-09-10T07:49:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4a000640 1 -- 172.21.15.3:0/3496635327 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c032440 con 0x7fde4c145010 2024-09-10T07:49:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4a000640 1 -- 172.21.15.3:0/3496635327 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c032810 con 0x7fde4c145010 2024-09-10T07:49:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4be00640 1 -- 172.21.15.3:0/3496635327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 msgr2=0x7fde4c073410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.850+0000 7fde4be00640 1 --2- 172.21.15.3:0/3496635327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c073410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.851+0000 7fde4be00640 1 -- 172.21.15.3:0/3496635327 shutdown_connections 2024-09-10T07:49:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.851+0000 7fde4be00640 1 --2- 172.21.15.3:0/3496635327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c073410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.851+0000 7fde4be00640 1 -- 172.21.15.3:0/3496635327 >> 172.21.15.3:0/3496635327 conn(0x7fde4c06bb20 msgr2=0x7fde4c06bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.851+0000 7fde4be00640 1 -- 172.21.15.3:0/3496635327 shutdown_connections 2024-09-10T07:49:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.851+0000 7fde4be00640 1 -- 172.21.15.3:0/3496635327 wait complete. 2024-09-10T07:49:58.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.852+0000 7fde4be00640 1 Processor -- start 2024-09-10T07:49:58.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.852+0000 7fde4be00640 1 -- start start 2024-09-10T07:49:58.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde4c15ca80 0x7fde4c15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c145570 con 0x7fde4c0b2710 2024-09-10T07:49:58.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c1453f0 con 0x7fde4c145010 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c073950 con 0x7fde4c15ca80 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.853+0000 7fde4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40160/0 (socket says 172.21.15.3:40160) 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 -- 172.21.15.3:0/1169759733 learned_addr learned my addr 172.21.15.3:0/1169759733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4b400640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde4c15ca80 0x7fde4c15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde43e00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 -- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde4c15ca80 msgr2=0x7fde4c15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde4c15ca80 0x7fde4c15ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 -- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 msgr2=0x7fde4c143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 -- 172.21.15.3:0/1169759733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c0c6350 con 0x7fde4c0b2710 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde43e00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:58.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.854+0000 7fde4aa00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02a790 tx=0x7fde3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.855+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c037050 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.855+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c043070 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.855+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c03cb30 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.855+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c0c6630 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.855+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c0c6bf0 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.856+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c063610 con 0x7fde4c0b2710 2024-09-10T07:49:58.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.857+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde3c03ccd0 con 0x7fde4c0b2710 2024-09-10T07:49:58.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.857+0000 7fde42a00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:58.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.858+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c0abb90 con 0x7fde4c0b2710 2024-09-10T07:49:58.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.858+0000 7fde43e00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:58.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.858+0000 7fde43e00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 0x7fde2806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0b1820 tx=0x7fde34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:58.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:58.862+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fde3c046360 con 0x7fde4c0b2710 2024-09-10T07:49:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.046+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde4c071cd0 con 0x7fde4c0b2710 2024-09-10T07:49:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.047+0000 7fde42a00640 1 -- 172.21.15.3:0/1169759733 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16653 (secure 0 0 0) 0x7fde3c07a530 con 0x7fde4c0b2710 2024-09-10T07:49:59.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:59.047 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":64,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:49:56.226976+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:49:52.454520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T07:49:53.642181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:49:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 msgr2=0x7fde2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 0x7fde2806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0b1820 tx=0x7fde34009040 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 msgr2=0x7fde4c13f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02a790 tx=0x7fde3c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 shutdown_connections 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde2806be30 0x7fde2806e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.049+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde4c15ca80 0x7fde4c15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.050+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde4c145010 0x7fde4c143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.050+0000 7fde4be00640 1 --2- 172.21.15.3:0/1169759733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c0b2710 0x7fde4c13f300 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.050+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 >> 172.21.15.3:0/1169759733 conn(0x7fde4c06bb20 msgr2=0x7fde4c0b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.050+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 shutdown_connections 2024-09-10T07:49:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.050+0000 7fde4be00640 1 -- 172.21.15.3:0/1169759733 wait complete. 2024-09-10T07:49:59.065 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:49:59.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:49:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.222+0000 7f7b7a400640 1 Processor -- start 2024-09-10T07:49:59.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.222+0000 7f7b7a400640 1 -- start start 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b74161dd0 0x7f7b7406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b740b5ea0 0x7f7b740b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 0x7f7b74062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7b7406fe10 con 0x7f7b74161dd0 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7b740b47d0 con 0x7f7b74062630 2024-09-10T07:49:59.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.223+0000 7f7b7a400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7406c870 con 0x7f7b740b5ea0 2024-09-10T07:49:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b74161dd0 0x7f7b7406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b74161dd0 0x7f7b7406f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40176/0 (socket says 172.21.15.3:40176) 2024-09-10T07:49:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 -- 172.21.15.3:0/3614393572 learned_addr learned my addr 172.21.15.3:0/3614393572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:59.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b72a00640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b740b5ea0 0x7f7b740b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73e00640 1 --2- 172.21.15.3:0/3614393572 >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 0x7f7b74062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 -- 172.21.15.3:0/3614393572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b740b5ea0 msgr2=0x7f7b740b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b740b5ea0 0x7f7b740b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 -- 172.21.15.3:0/3614393572 >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 msgr2=0x7f7b74062aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 --2- 172.21.15.3:0/3614393572 >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 0x7f7b74062aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73400640 1 -- 172.21.15.3:0/3614393572 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7406cb50 con 0x7f7b74161dd0 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.224+0000 7f7b73e00640 1 --2- 172.21.15.3:0/3614393572 >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 0x7f7b74062aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:49:59.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.225+0000 7f7b73400640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b74161dd0 0x7f7b7406f8d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b74071de0 tx=0x7f7b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2053308669e13d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.225+0000 7f7b72000640 1 -- 172.21.15.3:0/3614393572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037070 con 0x7f7b74161dd0 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.225+0000 7f7b72000640 1 -- 172.21.15.3:0/3614393572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b64032440 con 0x7f7b74161dd0 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b72000640 1 -- 172.21.15.3:0/3614393572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64032810 con 0x7f7b74161dd0 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 -- 172.21.15.3:0/3614393572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b74161dd0 msgr2=0x7f7b7406f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b74161dd0 0x7f7b7406f8d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b74071de0 tx=0x7f7b6402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 -- 172.21.15.3:0/3614393572 shutdown_connections 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 --2- 172.21.15.3:0/3614393572 >> v2:172.21.15.70:3300/0 conn(0x7f7b74062630 0x7f7b74062aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b740b5ea0 0x7f7b740b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 --2- 172.21.15.3:0/3614393572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b74161dd0 0x7f7b7406f8d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 -- 172.21.15.3:0/3614393572 >> 172.21.15.3:0/3614393572 conn(0x7f7b740b3540 msgr2=0x7f7b740b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 -- 172.21.15.3:0/3614393572 shutdown_connections 2024-09-10T07:49:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.226+0000 7f7b7a400640 1 -- 172.21.15.3:0/3614393572 wait complete. 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.228+0000 7f7b7a400640 1 Processor -- start 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.228+0000 7f7b7a400640 1 -- start start 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 0x7f7b74145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b74161dd0 0x7f7b7406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b6820 con 0x7f7b740b5ea0 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b06c0 con 0x7f7b74161dd0 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b7a400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7b740b43f0 con 0x7f7b74062630 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40190/0 (socket says 172.21.15.3:40190) 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 -- 172.21.15.3:0/1987184415 learned_addr learned my addr 172.21.15.3:0/1987184415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b73400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 0x7f7b74145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b73e00640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b74161dd0 0x7f7b7406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 -- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 msgr2=0x7f7b74145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 0x7f7b74145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 -- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b74161dd0 msgr2=0x7f7b7406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b74161dd0 0x7f7b7406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b72a00640 1 -- 172.21.15.3:0/1987184415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b74156bf0 con 0x7f7b740b5ea0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.229+0000 7f7b73400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 0x7f7b74145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b72a00640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d830 tx=0x7f7b6000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000b6c0 con 0x7f7b740b5ea0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60019070 con 0x7f7b740b5ea0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b74151de0 con 0x7f7b740b5ea0 2024-09-10T07:49:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000ba80 con 0x7f7b740b5ea0 2024-09-10T07:49:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.230+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b74152680 con 0x7f7b740b5ea0 2024-09-10T07:49:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.231+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b740b62e0 con 0x7f7b740b5ea0 2024-09-10T07:49:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.232+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b6001e460 con 0x7f7b740b5ea0 2024-09-10T07:49:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.233+0000 7f7b6b400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.233+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f7b6008b680 con 0x7f7b740b5ea0 2024-09-10T07:49:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.233+0000 7f7b73400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.234+0000 7f7b73400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 0x7f7b5406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b64034000 tx=0x7f7b6402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.237+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b60059f00 con 0x7f7b740b5ea0 2024-09-10T07:49:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.418+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b740af830 con 0x7f7b5406be30 2024-09-10T07:49:59.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.424+0000 7f7b6b400640 1 -- 172.21.15.3:0/1987184415 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250664 (secure 0 0 0) 0x7f7b740af830 con 0x7f7b5406be30 2024-09-10T07:49:59.427 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:59.428 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 msgr2=0x7f7b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 0x7f7b5406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b64034000 tx=0x7f7b6402e000 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 msgr2=0x7f7b741f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d830 tx=0x7f7b6000dcf0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 shutdown_connections 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b74161dd0 0x7f7b7406e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b740b5ea0 0x7f7b741f45b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 --2- 172.21.15.3:0/1987184415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7b74062630 0x7f7b74145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 >> 172.21.15.3:0/1987184415 conn(0x7f7b740b3540 msgr2=0x7f7b740b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 shutdown_connections 2024-09-10T07:49:59.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.428+0000 7f7b7a400640 1 -- 172.21.15.3:0/1987184415 wait complete. 2024-09-10T07:49:59.444 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-10T07:49:57.534944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":260724,"kb_used_data":6988,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713996,"statfs":{"total":773094113280,"available":772827131904,"internally_reserved":0,"allocated":7155712,"data_stored":3668468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.332862"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241325+0000","last_change":"2024-09-10T07:49:55.234160+0000","last_active":"2024-09-10T07:49:55.241325+0000","last_peered":"2024-09-10T07:49:55.241325+0000","last_clean":"2024-09-10T07:49:55.241325+0000","last_became_active":"2024-09-10T07:49:55.233435+0000","last_became_peered":"2024-09-10T07:49:55.233435+0000","last_unstale":"2024-09-10T07:49:55.241325+0000","last_undegraded":"2024-09-10T07:49:55.241325+0000","last_fullsized":"2024-09-10T07:49:55.241325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:22.901478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235402+0000","last_change":"2024-09-10T07:49:55.231309+0000","last_active":"2024-09-10T07:49:55.235402+0000","last_peered":"2024-09-10T07:49:55.235402+0000","last_clean":"2024-09-10T07:49:55.235402+0000","last_became_active":"2024-09-10T07:49:55.231022+0000","last_became_peered":"2024-09-10T07:49:55.231022+0000","last_unstale":"2024-09-10T07:49:55.235402+0000","last_undegraded":"2024-09-10T07:49:55.235402+0000","last_fullsized":"2024-09-10T07:49:55.235402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:29.658524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314264+0000","last_change":"2024-09-10T07:49:55.236559+0000","last_active":"2024-09-10T07:49:56.314264+0000","last_peered":"2024-09-10T07:49:56.314264+0000","last_clean":"2024-09-10T07:49:56.314264+0000","last_became_active":"2024-09-10T07:49:55.236291+0000","last_became_peered":"2024-09-10T07:49:55.236291+0000","last_unstale":"2024-09-10T07:49:56.314264+0000","last_undegraded":"2024-09-10T07:49:56.314264+0000","last_fullsized":"2024-09-10T07:49:56.314264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:17.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237862+0000","last_change":"2024-09-10T07:49:55.233233+0000","last_active":"2024-09-10T07:49:56.237862+0000","last_peered":"2024-09-10T07:49:56.237862+0000","last_clean":"2024-09-10T07:49:56.237862+0000","last_became_active":"2024-09-10T07:49:55.232946+0000","last_became_peered":"2024-09-10T07:49:55.232946+0000","last_unstale":"2024-09-10T07:49:56.237862+0000","last_undegraded":"2024-09-10T07:49:56.237862+0000","last_fullsized":"2024-09-10T07:49:56.237862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:13.029127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.236549+0000","last_change":"2024-09-10T07:49:55.231030+0000","last_active":"2024-09-10T07:49:55.236549+0000","last_peered":"2024-09-10T07:49:55.236549+0000","last_clean":"2024-09-10T07:49:55.236549+0000","last_became_active":"2024-09-10T07:49:55.230754+0000","last_became_peered":"2024-09-10T07:49:55.230754+0000","last_unstale":"2024-09-10T07:49:55.236549+0000","last_undegraded":"2024-09-10T07:49:55.236549+0000","last_fullsized":"2024-09-10T07:49:55.236549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241559+0000","last_change":"2024-09-10T07:49:55.234383+0000","last_active":"2024-09-10T07:49:55.241559+0000","last_peered":"2024-09-10T07:49:55.241559+0000","last_clean":"2024-09-10T07:49:55.241559+0000","last_became_active":"2024-09-10T07:49:55.233735+0000","last_became_peered":"2024-09-10T07:49:55.233735+0000","last_unstale":"2024-09-10T07:49:55.241559+0000","last_undegraded":"2024-09-10T07:49:55.241559+0000","last_fullsized":"2024-09-10T07:49:55.241559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:15.078487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237835+0000","last_change":"2024-09-10T07:49:55.245257+0000","last_active":"2024-09-10T07:49:56.237835+0000","last_peered":"2024-09-10T07:49:56.237835+0000","last_clean":"2024-09-10T07:49:56.237835+0000","last_became_active":"2024-09-10T07:49:55.244981+0000","last_became_peered":"2024-09-10T07:49:55.244981+0000","last_unstale":"2024-09-10T07:49:56.237835+0000","last_undegraded":"2024-09-10T07:49:56.237835+0000","last_fullsized":"2024-09-10T07:49:56.237835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:22.002166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.240335+0000","last_change":"2024-09-10T07:49:55.232117+0000","last_active":"2024-09-10T07:49:55.240335+0000","last_peered":"2024-09-10T07:49:55.240335+0000","last_clean":"2024-09-10T07:49:55.240335+0000","last_became_active":"2024-09-10T07:49:55.230521+0000","last_became_peered":"2024-09-10T07:49:55.230521+0000","last_unstale":"2024-09-10T07:49:55.240335+0000","last_undegraded":"2024-09-10T07:49:55.240335+0000","last_fullsized":"2024-09-10T07:49:55.240335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:36.497697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239231+0000","last_change":"2024-09-10T07:49:55.232810+0000","last_active":"2024-09-10T07:49:56.239231+0000","last_peered":"2024-09-10T07:49:56.239231+0000","last_clean":"2024-09-10T07:49:56.239231+0000","last_became_active":"2024-09-10T07:49:55.232552+0000","last_became_peered":"2024-09-10T07:49:55.232552+0000","last_unstale":"2024-09-10T07:49:56.239231+0000","last_undegraded":"2024-09-10T07:49:56.239231+0000","last_fullsized":"2024-09-10T07:49:56.239231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:43.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.247368+0000","last_change":"2024-09-10T07:49:55.244152+0000","last_active":"2024-09-10T07:49:55.247368+0000","last_peered":"2024-09-10T07:49:55.247368+0000","last_clean":"2024-09-10T07:49:55.247368+0000","last_became_active":"2024-09-10T07:49:55.243878+0000","last_became_peered":"2024-09-10T07:49:55.243878+0000","last_unstale":"2024-09-10T07:49:55.247368+0000","last_undegraded":"2024-09-10T07:49:55.247368+0000","last_fullsized":"2024-09-10T07:49:55.247368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:02.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:54.201757+0000","last_change":"2024-09-10T07:49:54.201757+0000","last_active":"2024-09-10T07:49:54.201757+0000","last_peered":"2024-09-10T07:49:54.201757+0000","last_clean":"2024-09-10T07:49:54.201757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:54.201757+0000","last_undegraded":"2024-09-10T07:49:54.201757+0000","last_fullsized":"2024-09-10T07:49:54.201757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239006+0000","last_change":"2024-09-10T07:49:55.237090+0000","last_active":"2024-09-10T07:49:56.239006+0000","last_peered":"2024-09-10T07:49:56.239006+0000","last_clean":"2024-09-10T07:49:56.239006+0000","last_became_active":"2024-09-10T07:49:55.236823+0000","last_became_peered":"2024-09-10T07:49:55.236823+0000","last_unstale":"2024-09-10T07:49:56.239006+0000","last_undegraded":"2024-09-10T07:49:56.239006+0000","last_fullsized":"2024-09-10T07:49:56.239006+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314906+0000","last_change":"2024-09-10T07:49:55.238531+0000","last_active":"2024-09-10T07:49:56.314906+0000","last_peered":"2024-09-10T07:49:56.314906+0000","last_clean":"2024-09-10T07:49:56.314906+0000","last_became_active":"2024-09-10T07:49:55.238223+0000","last_became_peered":"2024-09-10T07:49:55.238223+0000","last_unstale":"2024-09-10T07:49:56.314906+0000","last_undegraded":"2024-09-10T07:49:56.314906+0000","last_fullsized":"2024-09-10T07:49:56.314906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:22.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313990+0000","last_change":"2024-09-10T07:49:55.235828+0000","last_active":"2024-09-10T07:49:56.313990+0000","last_peered":"2024-09-10T07:49:56.313990+0000","last_clean":"2024-09-10T07:49:56.313990+0000","last_became_active":"2024-09-10T07:49:55.235526+0000","last_became_peered":"2024-09-10T07:49:55.235526+0000","last_unstale":"2024-09-10T07:49:56.313990+0000","last_undegraded":"2024-09-10T07:49:56.313990+0000","last_fullsized":"2024-09-10T07:49:56.313990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:56.169860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314773+0000","last_change":"2024-09-10T07:49:55.232930+0000","last_active":"2024-09-10T07:49:56.314773+0000","last_peered":"2024-09-10T07:49:56.314773+0000","last_clean":"2024-09-10T07:49:56.314773+0000","last_became_active":"2024-09-10T07:49:55.232364+0000","last_became_peered":"2024-09-10T07:49:55.232364+0000","last_unstale":"2024-09-10T07:49:56.314773+0000","last_undegraded":"2024-09-10T07:49:56.314773+0000","last_fullsized":"2024-09-10T07:49:56.314773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.877990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.236509+0000","last_change":"2024-09-10T07:49:55.229285+0000","last_active":"2024-09-10T07:49:55.236509+0000","last_peered":"2024-09-10T07:49:55.236509+0000","last_clean":"2024-09-10T07:49:55.236509+0000","last_became_active":"2024-09-10T07:49:55.228951+0000","last_became_peered":"2024-09-10T07:49:55.228951+0000","last_unstale":"2024-09-10T07:49:55.236509+0000","last_undegraded":"2024-09-10T07:49:55.236509+0000","last_fullsized":"2024-09-10T07:49:55.236509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245720+0000","last_change":"2024-09-10T07:49:55.236951+0000","last_active":"2024-09-10T07:49:55.245720+0000","last_peered":"2024-09-10T07:49:55.245720+0000","last_clean":"2024-09-10T07:49:55.245720+0000","last_became_active":"2024-09-10T07:49:55.234617+0000","last_became_peered":"2024-09-10T07:49:55.234617+0000","last_unstale":"2024-09-10T07:49:55.245720+0000","last_undegraded":"2024-09-10T07:49:55.245720+0000","last_fullsized":"2024-09-10T07:49:55.245720+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:29.015621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314023+0000","last_change":"2024-09-10T07:49:55.244009+0000","last_active":"2024-09-10T07:49:56.314023+0000","last_peered":"2024-09-10T07:49:56.314023+0000","last_clean":"2024-09-10T07:49:56.314023+0000","last_became_active":"2024-09-10T07:49:55.243720+0000","last_became_peered":"2024-09-10T07:49:55.243720+0000","last_unstale":"2024-09-10T07:49:56.314023+0000","last_undegraded":"2024-09-10T07:49:56.314023+0000","last_fullsized":"2024-09-10T07:49:56.314023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:31.165932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244607+0000","last_change":"2024-09-10T07:49:55.238643+0000","last_active":"2024-09-10T07:49:55.244607+0000","last_peered":"2024-09-10T07:49:55.244607+0000","last_clean":"2024-09-10T07:49:55.244607+0000","last_became_active":"2024-09-10T07:49:55.238331+0000","last_became_peered":"2024-09-10T07:49:55.238331+0000","last_unstale":"2024-09-10T07:49:55.244607+0000","last_undegraded":"2024-09-10T07:49:55.244607+0000","last_fullsized":"2024-09-10T07:49:55.244607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:54.201757+0000","last_change":"2024-09-10T07:49:54.201757+0000","last_active":"2024-09-10T07:49:54.201757+0000","last_peered":"2024-09-10T07:49:54.201757+0000","last_clean":"2024-09-10T07:49:54.201757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:54.201757+0000","last_undegraded":"2024-09-10T07:49:54.201757+0000","last_fullsized":"2024-09-10T07:49:54.201757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314953+0000","last_change":"2024-09-10T07:49:55.232221+0000","last_active":"2024-09-10T07:49:56.314953+0000","last_peered":"2024-09-10T07:49:56.314953+0000","last_clean":"2024-09-10T07:49:56.314953+0000","last_became_active":"2024-09-10T07:49:55.231825+0000","last_became_peered":"2024-09-10T07:49:55.231825+0000","last_unstale":"2024-09-10T07:49:56.314953+0000","last_undegraded":"2024-09-10T07:49:56.314953+0000","last_fullsized":"2024-09-10T07:49:56.314953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:34.662956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:54.201757+0000","last_change":"2024-09-10T07:49:54.201757+0000","last_active":"2024-09-10T07:49:54.201757+0000","last_peered":"2024-09-10T07:49:54.201757+0000","last_clean":"2024-09-10T07:49:54.201757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:54.201757+0000","last_undegraded":"2024-09-10T07:49:54.201757+0000","last_fullsized":"2024-09-10T07:49:54.201757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237987+0000","last_change":"2024-09-10T07:49:55.237209+0000","last_active":"2024-09-10T07:49:56.237987+0000","last_peered":"2024-09-10T07:49:56.237987+0000","last_clean":"2024-09-10T07:49:56.237987+0000","last_became_active":"2024-09-10T07:49:55.236601+0000","last_became_peered":"2024-09-10T07:49:55.236601+0000","last_unstale":"2024-09-10T07:49:56.237987+0000","last_undegraded":"2024-09-10T07:49:56.237987+0000","last_fullsized":"2024-09-10T07:49:56.237987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:20.174536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238746+0000","last_change":"2024-09-10T07:49:55.234622+0000","last_active":"2024-09-10T07:49:56.238746+0000","last_peered":"2024-09-10T07:49:56.238746+0000","last_clean":"2024-09-10T07:49:56.238746+0000","last_became_active":"2024-09-10T07:49:55.231542+0000","last_became_peered":"2024-09-10T07:49:55.231542+0000","last_unstale":"2024-09-10T07:49:56.238746+0000","last_undegraded":"2024-09-10T07:49:56.238746+0000","last_fullsized":"2024-09-10T07:49:56.238746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237693+0000","last_change":"2024-09-10T07:49:55.232235+0000","last_active":"2024-09-10T07:49:55.237693+0000","last_peered":"2024-09-10T07:49:55.237693+0000","last_clean":"2024-09-10T07:49:55.237693+0000","last_became_active":"2024-09-10T07:49:55.231902+0000","last_became_peered":"2024-09-10T07:49:55.231902+0000","last_unstale":"2024-09-10T07:49:55.237693+0000","last_undegraded":"2024-09-10T07:49:55.237693+0000","last_fullsized":"2024-09-10T07:49:55.237693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:34.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315221+0000","last_change":"2024-09-10T07:49:55.244518+0000","last_active":"2024-09-10T07:49:56.315221+0000","last_peered":"2024-09-10T07:49:56.315221+0000","last_clean":"2024-09-10T07:49:56.315221+0000","last_became_active":"2024-09-10T07:49:55.244201+0000","last_became_peered":"2024-09-10T07:49:55.244201+0000","last_unstale":"2024-09-10T07:49:56.315221+0000","last_undegraded":"2024-09-10T07:49:56.315221+0000","last_fullsized":"2024-09-10T07:49:56.315221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:28.714769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.242590+0000","last_change":"2024-09-10T07:49:55.234762+0000","last_active":"2024-09-10T07:49:55.242590+0000","last_peered":"2024-09-10T07:49:55.242590+0000","last_clean":"2024-09-10T07:49:55.242590+0000","last_became_active":"2024-09-10T07:49:55.234491+0000","last_became_peered":"2024-09-10T07:49:55.234491+0000","last_unstale":"2024-09-10T07:49:55.242590+0000","last_undegraded":"2024-09-10T07:49:55.242590+0000","last_fullsized":"2024-09-10T07:49:55.242590+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:09.323053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245775+0000","last_change":"2024-09-10T07:49:55.237411+0000","last_active":"2024-09-10T07:49:55.245775+0000","last_peered":"2024-09-10T07:49:55.245775+0000","last_clean":"2024-09-10T07:49:55.245775+0000","last_became_active":"2024-09-10T07:49:55.237124+0000","last_became_peered":"2024-09-10T07:49:55.237124+0000","last_unstale":"2024-09-10T07:49:55.245775+0000","last_undegraded":"2024-09-10T07:49:55.245775+0000","last_fullsized":"2024-09-10T07:49:55.245775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:04.121735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237830+0000","last_change":"2024-09-10T07:49:55.233086+0000","last_active":"2024-09-10T07:49:55.237830+0000","last_peered":"2024-09-10T07:49:55.237830+0000","last_clean":"2024-09-10T07:49:55.237830+0000","last_became_active":"2024-09-10T07:49:55.230813+0000","last_became_peered":"2024-09-10T07:49:55.230813+0000","last_unstale":"2024-09-10T07:49:55.237830+0000","last_undegraded":"2024-09-10T07:49:55.237830+0000","last_fullsized":"2024-09-10T07:49:55.237830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:46.287208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237803+0000","last_change":"2024-09-10T07:49:55.235433+0000","last_active":"2024-09-10T07:49:56.237803+0000","last_peered":"2024-09-10T07:49:56.237803+0000","last_clean":"2024-09-10T07:49:56.237803+0000","last_became_active":"2024-09-10T07:49:55.235148+0000","last_became_peered":"2024-09-10T07:49:55.235148+0000","last_unstale":"2024-09-10T07:49:56.237803+0000","last_undegraded":"2024-09-10T07:49:56.237803+0000","last_fullsized":"2024-09-10T07:49:56.237803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:56.653577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244759+0000","last_change":"2024-09-10T07:49:55.238906+0000","last_active":"2024-09-10T07:49:55.244759+0000","last_peered":"2024-09-10T07:49:55.244759+0000","last_clean":"2024-09-10T07:49:55.244759+0000","last_became_active":"2024-09-10T07:49:55.238394+0000","last_became_peered":"2024-09-10T07:49:55.238394+0000","last_unstale":"2024-09-10T07:49:55.244759+0000","last_undegraded":"2024-09-10T07:49:55.244759+0000","last_fullsized":"2024-09-10T07:49:55.244759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:54.201757+0000","last_change":"2024-09-10T07:49:54.201757+0000","last_active":"2024-09-10T07:49:54.201757+0000","last_peered":"2024-09-10T07:49:54.201757+0000","last_clean":"2024-09-10T07:49:54.201757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:54.201757+0000","last_undegraded":"2024-09-10T07:49:54.201757+0000","last_fullsized":"2024-09-10T07:49:54.201757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237220+0000","last_change":"2024-09-10T07:49:55.232053+0000","last_active":"2024-09-10T07:49:55.237220+0000","last_peered":"2024-09-10T07:49:55.237220+0000","last_clean":"2024-09-10T07:49:55.237220+0000","last_became_active":"2024-09-10T07:49:55.231743+0000","last_became_peered":"2024-09-10T07:49:55.231743+0000","last_unstale":"2024-09-10T07:49:55.237220+0000","last_undegraded":"2024-09-10T07:49:55.237220+0000","last_fullsized":"2024-09-10T07:49:55.237220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:43.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:53.197418+0000","last_change":"2024-09-10T07:49:53.197418+0000","last_active":"2024-09-10T07:49:53.197418+0000","last_peered":"2024-09-10T07:49:53.197418+0000","last_clean":"2024-09-10T07:49:53.197418+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:53.197418+0000","last_undegraded":"2024-09-10T07:49:53.197418+0000","last_fullsized":"2024-09-10T07:49:53.197418+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241304+0000","last_change":"2024-09-10T07:49:55.233708+0000","last_active":"2024-09-10T07:49:55.241304+0000","last_peered":"2024-09-10T07:49:55.241304+0000","last_clean":"2024-09-10T07:49:55.241304+0000","last_became_active":"2024-09-10T07:49:55.233035+0000","last_became_peered":"2024-09-10T07:49:55.233035+0000","last_unstale":"2024-09-10T07:49:55.241304+0000","last_undegraded":"2024-09-10T07:49:55.241304+0000","last_fullsized":"2024-09-10T07:49:55.241304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:52.316375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.253565+0000","last_change":"2024-09-10T07:49:54.235741+0000","last_active":"2024-09-10T07:49:55.253565+0000","last_peered":"2024-09-10T07:49:55.253565+0000","last_clean":"2024-09-10T07:49:55.253565+0000","last_became_active":"2024-09-10T07:49:54.234631+0000","last_became_peered":"2024-09-10T07:49:54.234631+0000","last_unstale":"2024-09-10T07:49:55.253565+0000","last_undegraded":"2024-09-10T07:49:55.253565+0000","last_fullsized":"2024-09-10T07:49:55.253565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:09.498213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.247519+0000","last_change":"2024-09-10T07:49:55.245934+0000","last_active":"2024-09-10T07:49:55.247519+0000","last_peered":"2024-09-10T07:49:55.247519+0000","last_clean":"2024-09-10T07:49:55.247519+0000","last_became_active":"2024-09-10T07:49:55.245591+0000","last_became_peered":"2024-09-10T07:49:55.245591+0000","last_unstale":"2024-09-10T07:49:55.247519+0000","last_undegraded":"2024-09-10T07:49:55.247519+0000","last_fullsized":"2024-09-10T07:49:55.247519+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.114001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250107+0000","last_change":"2024-09-10T07:49:54.229770+0000","last_active":"2024-09-10T07:49:55.250107+0000","last_peered":"2024-09-10T07:49:55.250107+0000","last_clean":"2024-09-10T07:49:55.250107+0000","last_became_active":"2024-09-10T07:49:54.229476+0000","last_became_peered":"2024-09-10T07:49:54.229476+0000","last_unstale":"2024-09-10T07:49:55.250107+0000","last_undegraded":"2024-09-10T07:49:55.250107+0000","last_fullsized":"2024-09-10T07:49:55.250107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:07.917229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.238283+0000","last_change":"2024-09-10T07:49:55.226343+0000","last_active":"2024-09-10T07:49:55.238283+0000","last_peered":"2024-09-10T07:49:55.238283+0000","last_clean":"2024-09-10T07:49:55.238283+0000","last_became_active":"2024-09-10T07:49:55.222906+0000","last_became_peered":"2024-09-10T07:49:55.222906+0000","last_unstale":"2024-09-10T07:49:55.238283+0000","last_undegraded":"2024-09-10T07:49:55.238283+0000","last_fullsized":"2024-09-10T07:49:55.238283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238086+0000","last_change":"2024-09-10T07:49:54.224380+0000","last_active":"2024-09-10T07:49:56.238086+0000","last_peered":"2024-09-10T07:49:56.238086+0000","last_clean":"2024-09-10T07:49:56.238086+0000","last_became_active":"2024-09-10T07:49:54.224065+0000","last_became_peered":"2024-09-10T07:49:54.224065+0000","last_unstale":"2024-09-10T07:49:56.238086+0000","last_undegraded":"2024-09-10T07:49:56.238086+0000","last_fullsized":"2024-09-10T07:49:56.238086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:40.824351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244697+0000","last_change":"2024-09-10T07:49:55.239335+0000","last_active":"2024-09-10T07:49:55.244697+0000","last_peered":"2024-09-10T07:49:55.244697+0000","last_clean":"2024-09-10T07:49:55.244697+0000","last_became_active":"2024-09-10T07:49:55.234297+0000","last_became_peered":"2024-09-10T07:49:55.234297+0000","last_unstale":"2024-09-10T07:49:55.244697+0000","last_undegraded":"2024-09-10T07:49:55.244697+0000","last_fullsized":"2024-09-10T07:49:55.244697+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:18.237272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313881+0000","last_change":"2024-09-10T07:49:54.233192+0000","last_active":"2024-09-10T07:49:56.313881+0000","last_peered":"2024-09-10T07:49:56.313881+0000","last_clean":"2024-09-10T07:49:56.313881+0000","last_became_active":"2024-09-10T07:49:54.232011+0000","last_became_peered":"2024-09-10T07:49:54.232011+0000","last_unstale":"2024-09-10T07:49:56.313881+0000","last_undegraded":"2024-09-10T07:49:56.313881+0000","last_fullsized":"2024-09-10T07:49:56.313881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:37.679105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:54.201757+0000","last_change":"2024-09-10T07:49:54.201757+0000","last_active":"2024-09-10T07:49:54.201757+0000","last_peered":"2024-09-10T07:49:54.201757+0000","last_clean":"2024-09-10T07:49:54.201757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:54.201757+0000","last_undegraded":"2024-09-10T07:49:54.201757+0000","last_fullsized":"2024-09-10T07:49:54.201757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.253574+0000","last_change":"2024-09-10T07:49:54.236049+0000","last_active":"2024-09-10T07:49:55.253574+0000","last_peered":"2024-09-10T07:49:55.253574+0000","last_clean":"2024-09-10T07:49:55.253574+0000","last_became_active":"2024-09-10T07:49:54.235801+0000","last_became_peered":"2024-09-10T07:49:54.235801+0000","last_unstale":"2024-09-10T07:49:55.253574+0000","last_undegraded":"2024-09-10T07:49:55.253574+0000","last_fullsized":"2024-09-10T07:49:55.253574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:28.654184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313912+0000","last_change":"2024-09-10T07:49:55.233993+0000","last_active":"2024-09-10T07:49:56.313912+0000","last_peered":"2024-09-10T07:49:56.313912+0000","last_clean":"2024-09-10T07:49:56.313912+0000","last_became_active":"2024-09-10T07:49:55.233626+0000","last_became_peered":"2024-09-10T07:49:55.233626+0000","last_unstale":"2024-09-10T07:49:56.313912+0000","last_undegraded":"2024-09-10T07:49:56.313912+0000","last_fullsized":"2024-09-10T07:49:56.313912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:16.446101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238547+0000","last_change":"2024-09-10T07:49:54.234520+0000","last_active":"2024-09-10T07:49:56.238547+0000","last_peered":"2024-09-10T07:49:56.238547+0000","last_clean":"2024-09-10T07:49:56.238547+0000","last_became_active":"2024-09-10T07:49:54.234065+0000","last_became_peered":"2024-09-10T07:49:54.234065+0000","last_unstale":"2024-09-10T07:49:56.238547+0000","last_undegraded":"2024-09-10T07:49:56.238547+0000","last_fullsized":"2024-09-10T07:49:56.238547+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:34:47.319841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.246579+0000","last_change":"2024-09-10T07:49:55.237921+0000","last_active":"2024-09-10T07:49:55.246579+0000","last_peered":"2024-09-10T07:49:55.246579+0000","last_clean":"2024-09-10T07:49:55.246579+0000","last_became_active":"2024-09-10T07:49:55.237573+0000","last_became_peered":"2024-09-10T07:49:55.237573+0000","last_unstale":"2024-09-10T07:49:55.246579+0000","last_undegraded":"2024-09-10T07:49:55.246579+0000","last_fullsized":"2024-09-10T07:49:55.246579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:12.231876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314493+0000","last_change":"2024-09-10T07:49:54.232693+0000","last_active":"2024-09-10T07:49:56.314493+0000","last_peered":"2024-09-10T07:49:56.314493+0000","last_clean":"2024-09-10T07:49:56.314493+0000","last_became_active":"2024-09-10T07:49:54.232210+0000","last_became_peered":"2024-09-10T07:49:54.232210+0000","last_unstale":"2024-09-10T07:49:56.314493+0000","last_undegraded":"2024-09-10T07:49:56.314493+0000","last_fullsized":"2024-09-10T07:49:56.314493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:11.227536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315310+0000","last_change":"2024-09-10T07:49:55.246775+0000","last_active":"2024-09-10T07:49:56.315310+0000","last_peered":"2024-09-10T07:49:56.315310+0000","last_clean":"2024-09-10T07:49:56.315310+0000","last_became_active":"2024-09-10T07:49:55.241443+0000","last_became_peered":"2024-09-10T07:49:55.241443+0000","last_unstale":"2024-09-10T07:49:56.315310+0000","last_undegraded":"2024-09-10T07:49:56.315310+0000","last_fullsized":"2024-09-10T07:49:56.315310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:50.963732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239165+0000","last_change":"2024-09-10T07:49:54.235701+0000","last_active":"2024-09-10T07:49:56.239165+0000","last_peered":"2024-09-10T07:49:56.239165+0000","last_clean":"2024-09-10T07:49:56.239165+0000","last_became_active":"2024-09-10T07:49:54.235393+0000","last_became_peered":"2024-09-10T07:49:54.235393+0000","last_unstale":"2024-09-10T07:49:56.239165+0000","last_undegraded":"2024-09-10T07:49:56.239165+0000","last_fullsized":"2024-09-10T07:49:56.239165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:59.763343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314615+0000","last_change":"2024-09-10T07:49:55.242384+0000","last_active":"2024-09-10T07:49:56.314615+0000","last_peered":"2024-09-10T07:49:56.314615+0000","last_clean":"2024-09-10T07:49:56.314615+0000","last_became_active":"2024-09-10T07:49:55.241007+0000","last_became_peered":"2024-09-10T07:49:55.241007+0000","last_unstale":"2024-09-10T07:49:56.314615+0000","last_undegraded":"2024-09-10T07:49:56.314615+0000","last_fullsized":"2024-09-10T07:49:56.314615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:15.138119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.226141+0000","last_change":"2024-09-10T07:49:54.229298+0000","last_active":"2024-09-10T07:49:55.226141+0000","last_peered":"2024-09-10T07:49:55.226141+0000","last_clean":"2024-09-10T07:49:55.226141+0000","last_became_active":"2024-09-10T07:49:54.229007+0000","last_became_peered":"2024-09-10T07:49:54.229007+0000","last_unstale":"2024-09-10T07:49:55.226141+0000","last_undegraded":"2024-09-10T07:49:55.226141+0000","last_fullsized":"2024-09-10T07:49:55.226141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:55.165520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245825+0000","last_change":"2024-09-10T07:49:55.243077+0000","last_active":"2024-09-10T07:49:55.245825+0000","last_peered":"2024-09-10T07:49:55.245825+0000","last_clean":"2024-09-10T07:49:55.245825+0000","last_became_active":"2024-09-10T07:49:55.242816+0000","last_became_peered":"2024-09-10T07:49:55.242816+0000","last_unstale":"2024-09-10T07:49:55.245825+0000","last_undegraded":"2024-09-10T07:49:55.245825+0000","last_fullsized":"2024-09-10T07:49:55.245825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:32.821456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238400+0000","last_change":"2024-09-10T07:49:54.228989+0000","last_active":"2024-09-10T07:49:56.238400+0000","last_peered":"2024-09-10T07:49:56.238400+0000","last_clean":"2024-09-10T07:49:56.238400+0000","last_became_active":"2024-09-10T07:49:54.228715+0000","last_became_peered":"2024-09-10T07:49:54.228715+0000","last_unstale":"2024-09-10T07:49:56.238400+0000","last_undegraded":"2024-09-10T07:49:56.238400+0000","last_fullsized":"2024-09-10T07:49:56.238400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:31.300326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238308+0000","last_change":"2024-09-10T07:49:55.234796+0000","last_active":"2024-09-10T07:49:56.238308+0000","last_peered":"2024-09-10T07:49:56.238308+0000","last_clean":"2024-09-10T07:49:56.238308+0000","last_became_active":"2024-09-10T07:49:55.234519+0000","last_became_peered":"2024-09-10T07:49:55.234519+0000","last_unstale":"2024-09-10T07:49:56.238308+0000","last_undegraded":"2024-09-10T07:49:56.238308+0000","last_fullsized":"2024-09-10T07:49:56.238308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:05.114577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315069+0000","last_change":"2024-09-10T07:49:54.237406+0000","last_active":"2024-09-10T07:49:56.315069+0000","last_peered":"2024-09-10T07:49:56.315069+0000","last_clean":"2024-09-10T07:49:56.315069+0000","last_became_active":"2024-09-10T07:49:54.236989+0000","last_became_peered":"2024-09-10T07:49:54.236989+0000","last_unstale":"2024-09-10T07:49:56.315069+0000","last_undegraded":"2024-09-10T07:49:56.315069+0000","last_fullsized":"2024-09-10T07:49:56.315069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:55.896433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238601+0000","last_change":"2024-09-10T07:49:55.224183+0000","last_active":"2024-09-10T07:49:56.238601+0000","last_peered":"2024-09-10T07:49:56.238601+0000","last_clean":"2024-09-10T07:49:56.238601+0000","last_became_active":"2024-09-10T07:49:55.223835+0000","last_became_peered":"2024-09-10T07:49:55.223835+0000","last_unstale":"2024-09-10T07:49:56.238601+0000","last_undegraded":"2024-09-10T07:49:56.238601+0000","last_fullsized":"2024-09-10T07:49:56.238601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:34.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250794+0000","last_change":"2024-09-10T07:49:54.232415+0000","last_active":"2024-09-10T07:49:55.250794+0000","last_peered":"2024-09-10T07:49:55.250794+0000","last_clean":"2024-09-10T07:49:55.250794+0000","last_became_active":"2024-09-10T07:49:54.231725+0000","last_became_peered":"2024-09-10T07:49:54.231725+0000","last_unstale":"2024-09-10T07:49:55.250794+0000","last_undegraded":"2024-09-10T07:49:55.250794+0000","last_fullsized":"2024-09-10T07:49:55.250794+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:15.441761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313937+0000","last_change":"2024-09-10T07:49:55.237774+0000","last_active":"2024-09-10T07:49:56.313937+0000","last_peered":"2024-09-10T07:49:56.313937+0000","last_clean":"2024-09-10T07:49:56.313937+0000","last_became_active":"2024-09-10T07:49:55.237448+0000","last_became_peered":"2024-09-10T07:49:55.237448+0000","last_unstale":"2024-09-10T07:49:56.313937+0000","last_undegraded":"2024-09-10T07:49:56.313937+0000","last_fullsized":"2024-09-10T07:49:56.313937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:42.265970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237894+0000","last_change":"2024-09-10T07:49:54.223427+0000","last_active":"2024-09-10T07:49:56.237894+0000","last_peered":"2024-09-10T07:49:56.237894+0000","last_clean":"2024-09-10T07:49:56.237894+0000","last_became_active":"2024-09-10T07:49:54.223114+0000","last_became_peered":"2024-09-10T07:49:54.223114+0000","last_unstale":"2024-09-10T07:49:56.237894+0000","last_undegraded":"2024-09-10T07:49:56.237894+0000","last_fullsized":"2024-09-10T07:49:56.237894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:41.737915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238637+0000","last_change":"2024-09-10T07:49:55.234199+0000","last_active":"2024-09-10T07:49:56.238637+0000","last_peered":"2024-09-10T07:49:56.238637+0000","last_clean":"2024-09-10T07:49:56.238637+0000","last_became_active":"2024-09-10T07:49:55.233758+0000","last_became_peered":"2024-09-10T07:49:55.233758+0000","last_unstale":"2024-09-10T07:49:56.238637+0000","last_undegraded":"2024-09-10T07:49:56.238637+0000","last_fullsized":"2024-09-10T07:49:56.238637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:09.323053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238092+0000","last_change":"2024-09-10T07:49:54.230943+0000","last_active":"2024-09-10T07:49:56.238092+0000","last_peered":"2024-09-10T07:49:56.238092+0000","last_clean":"2024-09-10T07:49:56.238092+0000","last_became_active":"2024-09-10T07:49:54.230499+0000","last_became_peered":"2024-09-10T07:49:54.230499+0000","last_unstale":"2024-09-10T07:49:56.238092+0000","last_undegraded":"2024-09-10T07:49:56.238092+0000","last_fullsized":"2024-09-10T07:49:56.238092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:45.500563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238062+0000","last_change":"2024-09-10T07:49:55.237422+0000","last_active":"2024-09-10T07:49:56.238062+0000","last_peered":"2024-09-10T07:49:56.238062+0000","last_clean":"2024-09-10T07:49:56.238062+0000","last_became_active":"2024-09-10T07:49:55.236954+0000","last_became_peered":"2024-09-10T07:49:55.236954+0000","last_unstale":"2024-09-10T07:49:56.238062+0000","last_undegraded":"2024-09-10T07:49:56.238062+0000","last_fullsized":"2024-09-10T07:49:56.238062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.298086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237725+0000","last_change":"2024-09-10T07:49:54.228023+0000","last_active":"2024-09-10T07:49:56.237725+0000","last_peered":"2024-09-10T07:49:56.237725+0000","last_clean":"2024-09-10T07:49:56.237725+0000","last_became_active":"2024-09-10T07:49:54.227304+0000","last_became_peered":"2024-09-10T07:49:54.227304+0000","last_unstale":"2024-09-10T07:49:56.237725+0000","last_undegraded":"2024-09-10T07:49:56.237725+0000","last_fullsized":"2024-09-10T07:49:56.237725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:05.124949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314348+0000","last_change":"2024-09-10T07:49:55.234248+0000","last_active":"2024-09-10T07:49:56.314348+0000","last_peered":"2024-09-10T07:49:56.314348+0000","last_clean":"2024-09-10T07:49:56.314348+0000","last_became_active":"2024-09-10T07:49:55.233949+0000","last_became_peered":"2024-09-10T07:49:55.233949+0000","last_unstale":"2024-09-10T07:49:56.314348+0000","last_undegraded":"2024-09-10T07:49:56.314348+0000","last_fullsized":"2024-09-10T07:49:56.314348+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:36.497697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.249125+0000","last_change":"2024-09-10T07:49:54.235097+0000","last_active":"2024-09-10T07:49:55.249125+0000","last_peered":"2024-09-10T07:49:55.249125+0000","last_clean":"2024-09-10T07:49:55.249125+0000","last_became_active":"2024-09-10T07:49:54.234799+0000","last_became_peered":"2024-09-10T07:49:54.234799+0000","last_unstale":"2024-09-10T07:49:55.249125+0000","last_undegraded":"2024-09-10T07:49:55.249125+0000","last_fullsized":"2024-09-10T07:49:55.249125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:06.748639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314645+0000","last_change":"2024-09-10T07:49:55.242190+0000","last_active":"2024-09-10T07:49:56.314645+0000","last_peered":"2024-09-10T07:49:56.314645+0000","last_clean":"2024-09-10T07:49:56.314645+0000","last_became_active":"2024-09-10T07:49:55.241254+0000","last_became_peered":"2024-09-10T07:49:55.241254+0000","last_unstale":"2024-09-10T07:49:56.314645+0000","last_undegraded":"2024-09-10T07:49:56.314645+0000","last_fullsized":"2024-09-10T07:49:56.314645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:19.761738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250456+0000","last_change":"2024-09-10T07:49:54.230270+0000","last_active":"2024-09-10T07:49:55.250456+0000","last_peered":"2024-09-10T07:49:55.250456+0000","last_clean":"2024-09-10T07:49:55.250456+0000","last_became_active":"2024-09-10T07:49:54.230006+0000","last_became_peered":"2024-09-10T07:49:54.230006+0000","last_unstale":"2024-09-10T07:49:55.250456+0000","last_undegraded":"2024-09-10T07:49:55.250456+0000","last_fullsized":"2024-09-10T07:49:55.250456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:30.161593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244355+0000","last_change":"2024-09-10T07:49:55.236047+0000","last_active":"2024-09-10T07:49:55.244355+0000","last_peered":"2024-09-10T07:49:55.244355+0000","last_clean":"2024-09-10T07:49:55.244355+0000","last_became_active":"2024-09-10T07:49:55.235751+0000","last_became_peered":"2024-09-10T07:49:55.235751+0000","last_unstale":"2024-09-10T07:49:55.244355+0000","last_undegraded":"2024-09-10T07:49:55.244355+0000","last_fullsized":"2024-09-10T07:49:55.244355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314355+0000","last_change":"2024-09-10T07:49:54.229486+0000","last_active":"2024-09-10T07:49:56.314355+0000","last_peered":"2024-09-10T07:49:56.314355+0000","last_clean":"2024-09-10T07:49:56.314355+0000","last_became_active":"2024-09-10T07:49:54.229194+0000","last_became_peered":"2024-09-10T07:49:54.229194+0000","last_unstale":"2024-09-10T07:49:56.314355+0000","last_undegraded":"2024-09-10T07:49:56.314355+0000","last_fullsized":"2024-09-10T07:49:56.314355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:55.896433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238309+0000","last_change":"2024-09-10T07:49:55.228256+0000","last_active":"2024-09-10T07:49:56.238309+0000","last_peered":"2024-09-10T07:49:56.238309+0000","last_clean":"2024-09-10T07:49:56.238309+0000","last_became_active":"2024-09-10T07:49:55.227981+0000","last_became_peered":"2024-09-10T07:49:55.227981+0000","last_unstale":"2024-09-10T07:49:56.238309+0000","last_undegraded":"2024-09-10T07:49:56.238309+0000","last_fullsized":"2024-09-10T07:49:56.238309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:41.110878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.225851+0000","last_change":"2024-09-10T07:49:54.225953+0000","last_active":"2024-09-10T07:49:55.225851+0000","last_peered":"2024-09-10T07:49:55.225851+0000","last_clean":"2024-09-10T07:49:55.225851+0000","last_became_active":"2024-09-10T07:49:54.224988+0000","last_became_peered":"2024-09-10T07:49:54.224988+0000","last_unstale":"2024-09-10T07:49:55.225851+0000","last_undegraded":"2024-09-10T07:49:55.225851+0000","last_fullsized":"2024-09-10T07:49:55.225851+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.261630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238688+0000","last_change":"2024-09-10T07:49:55.232189+0000","last_active":"2024-09-10T07:49:56.238688+0000","last_peered":"2024-09-10T07:49:56.238688+0000","last_clean":"2024-09-10T07:49:56.238688+0000","last_became_active":"2024-09-10T07:49:55.231928+0000","last_became_peered":"2024-09-10T07:49:55.231928+0000","last_unstale":"2024-09-10T07:49:56.238688+0000","last_undegraded":"2024-09-10T07:49:56.238688+0000","last_fullsized":"2024-09-10T07:49:56.238688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:49:53.197418+0000","last_change":"2024-09-10T07:49:53.197418+0000","last_active":"2024-09-10T07:49:53.197418+0000","last_peered":"2024-09-10T07:49:53.197418+0000","last_clean":"2024-09-10T07:49:53.197418+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:49:53.197418+0000","last_undegraded":"2024-09-10T07:49:53.197418+0000","last_fullsized":"2024-09-10T07:49:53.197418+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.238812+0000","last_change":"2024-09-10T07:49:55.232739+0000","last_active":"2024-09-10T07:49:55.238812+0000","last_peered":"2024-09-10T07:49:55.238812+0000","last_clean":"2024-09-10T07:49:55.238812+0000","last_became_active":"2024-09-10T07:49:55.228224+0000","last_became_peered":"2024-09-10T07:49:55.228224+0000","last_unstale":"2024-09-10T07:49:55.238812+0000","last_undegraded":"2024-09-10T07:49:55.238812+0000","last_fullsized":"2024-09-10T07:49:55.238812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:15.138119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.224754+0000","last_change":"2024-09-10T07:49:54.231286+0000","last_active":"2024-09-10T07:49:55.224754+0000","last_peered":"2024-09-10T07:49:55.224754+0000","last_clean":"2024-09-10T07:49:55.224754+0000","last_became_active":"2024-09-10T07:49:54.229132+0000","last_became_peered":"2024-09-10T07:49:54.229132+0000","last_unstale":"2024-09-10T07:49:55.224754+0000","last_undegraded":"2024-09-10T07:49:55.224754+0000","last_fullsized":"2024-09-10T07:49:55.224754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:27.710429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244934+0000","last_change":"2024-09-10T07:49:55.236978+0000","last_active":"2024-09-10T07:49:55.244934+0000","last_peered":"2024-09-10T07:49:55.244934+0000","last_clean":"2024-09-10T07:49:55.244934+0000","last_became_active":"2024-09-10T07:49:55.236540+0000","last_became_peered":"2024-09-10T07:49:55.236540+0000","last_unstale":"2024-09-10T07:49:55.244934+0000","last_undegraded":"2024-09-10T07:49:55.244934+0000","last_fullsized":"2024-09-10T07:49:55.244934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:39.998603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238731+0000","last_change":"2024-09-10T07:49:54.233652+0000","last_active":"2024-09-10T07:49:56.238731+0000","last_peered":"2024-09-10T07:49:56.238731+0000","last_clean":"2024-09-10T07:49:56.238731+0000","last_became_active":"2024-09-10T07:49:54.232434+0000","last_became_peered":"2024-09-10T07:49:54.232434+0000","last_unstale":"2024-09-10T07:49:56.238731+0000","last_undegraded":"2024-09-10T07:49:56.238731+0000","last_fullsized":"2024-09-10T07:49:56.238731+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:33.589981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314431+0000","last_change":"2024-09-10T07:49:55.240967+0000","last_active":"2024-09-10T07:49:56.314431+0000","last_peered":"2024-09-10T07:49:56.314431+0000","last_clean":"2024-09-10T07:49:56.314431+0000","last_became_active":"2024-09-10T07:49:55.240651+0000","last_became_peered":"2024-09-10T07:49:55.240651+0000","last_unstale":"2024-09-10T07:49:56.314431+0000","last_undegraded":"2024-09-10T07:49:56.314431+0000","last_fullsized":"2024-09-10T07:49:56.314431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:46.287208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.248889+0000","last_change":"2024-09-10T07:49:54.232977+0000","last_active":"2024-09-10T07:49:55.248889+0000","last_peered":"2024-09-10T07:49:55.248889+0000","last_clean":"2024-09-10T07:49:55.248889+0000","last_became_active":"2024-09-10T07:49:54.232717+0000","last_became_peered":"2024-09-10T07:49:54.232717+0000","last_unstale":"2024-09-10T07:49:55.248889+0000","last_undegraded":"2024-09-10T07:49:55.248889+0000","last_fullsized":"2024-09-10T07:49:55.248889+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:48.126076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314759+0000","last_change":"2024-09-10T07:49:55.240941+0000","last_active":"2024-09-10T07:49:56.314759+0000","last_peered":"2024-09-10T07:49:56.314759+0000","last_clean":"2024-09-10T07:49:56.314759+0000","last_became_active":"2024-09-10T07:49:55.240613+0000","last_became_peered":"2024-09-10T07:49:55.240613+0000","last_unstale":"2024-09-10T07:49:56.314759+0000","last_undegraded":"2024-09-10T07:49:56.314759+0000","last_fullsized":"2024-09-10T07:49:56.314759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:39.998603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217709+0000","last_change":"2024-09-10T07:49:54.236283+0000","last_active":"2024-09-10T07:49:55.217709+0000","last_peered":"2024-09-10T07:49:55.217709+0000","last_clean":"2024-09-10T07:49:55.217709+0000","last_became_active":"2024-09-10T07:49:54.235705+0000","last_became_peered":"2024-09-10T07:49:54.235705+0000","last_unstale":"2024-09-10T07:49:55.217709+0000","last_undegraded":"2024-09-10T07:49:55.217709+0000","last_fullsized":"2024-09-10T07:49:55.217709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:08.318713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235941+0000","last_change":"2024-09-10T07:49:55.227769+0000","last_active":"2024-09-10T07:49:55.235941+0000","last_peered":"2024-09-10T07:49:55.235941+0000","last_clean":"2024-09-10T07:49:55.235941+0000","last_became_active":"2024-09-10T07:49:55.227479+0000","last_became_peered":"2024-09-10T07:49:55.227479+0000","last_unstale":"2024-09-10T07:49:55.235941+0000","last_undegraded":"2024-09-10T07:49:55.235941+0000","last_fullsized":"2024-09-10T07:49:55.235941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:59.407886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239036+0000","last_change":"2024-09-10T07:49:54.236202+0000","last_active":"2024-09-10T07:49:56.239036+0000","last_peered":"2024-09-10T07:49:56.239036+0000","last_clean":"2024-09-10T07:49:56.239036+0000","last_became_active":"2024-09-10T07:49:54.235905+0000","last_became_peered":"2024-09-10T07:49:54.235905+0000","last_unstale":"2024-09-10T07:49:56.239036+0000","last_undegraded":"2024-09-10T07:49:56.239036+0000","last_fullsized":"2024-09-10T07:49:56.239036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:15.836910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.243510+0000","last_change":"2024-09-10T07:49:55.236751+0000","last_active":"2024-09-10T07:49:55.243510+0000","last_peered":"2024-09-10T07:49:55.243510+0000","last_clean":"2024-09-10T07:49:55.243510+0000","last_became_active":"2024-09-10T07:49:55.236334+0000","last_became_peered":"2024-09-10T07:49:55.236334+0000","last_unstale":"2024-09-10T07:49:55.243510+0000","last_undegraded":"2024-09-10T07:49:55.243510+0000","last_fullsized":"2024-09-10T07:49:55.243510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:19.761738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314135+0000","last_change":"2024-09-10T07:49:54.226043+0000","last_active":"2024-09-10T07:49:56.314135+0000","last_peered":"2024-09-10T07:49:56.314135+0000","last_clean":"2024-09-10T07:49:56.314135+0000","last_became_active":"2024-09-10T07:49:54.225742+0000","last_became_peered":"2024-09-10T07:49:54.225742+0000","last_unstale":"2024-09-10T07:49:56.314135+0000","last_undegraded":"2024-09-10T07:49:56.314135+0000","last_fullsized":"2024-09-10T07:49:56.314135+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:17.208468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241538+0000","last_change":"2024-09-10T07:49:55.233403+0000","last_active":"2024-09-10T07:49:55.241538+0000","last_peered":"2024-09-10T07:49:55.241538+0000","last_clean":"2024-09-10T07:49:55.241538+0000","last_became_active":"2024-09-10T07:49:55.233063+0000","last_became_peered":"2024-09-10T07:49:55.233063+0000","last_unstale":"2024-09-10T07:49:55.241538+0000","last_undegraded":"2024-09-10T07:49:55.241538+0000","last_fullsized":"2024-09-10T07:49:55.241538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:52.677291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217758+0000","last_change":"2024-09-10T07:49:54.235515+0000","last_active":"2024-09-10T07:49:55.217758+0000","last_peered":"2024-09-10T07:49:55.217758+0000","last_clean":"2024-09-10T07:49:55.217758+0000","last_became_active":"2024-09-10T07:49:54.234632+0000","last_became_peered":"2024-09-10T07:49:54.234632+0000","last_unstale":"2024-09-10T07:49:55.217758+0000","last_undegraded":"2024-09-10T07:49:55.217758+0000","last_fullsized":"2024-09-10T07:49:55.217758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:59.763343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315055+0000","last_change":"2024-09-10T07:49:55.235236+0000","last_active":"2024-09-10T07:49:56.315055+0000","last_peered":"2024-09-10T07:49:56.315055+0000","last_clean":"2024-09-10T07:49:56.315055+0000","last_became_active":"2024-09-10T07:49:55.234856+0000","last_became_peered":"2024-09-10T07:49:55.234856+0000","last_unstale":"2024-09-10T07:49:56.315055+0000","last_undegraded":"2024-09-10T07:49:56.315055+0000","last_fullsized":"2024-09-10T07:49:56.315055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:21.517463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314079+0000","last_change":"2024-09-10T07:49:54.231251+0000","last_active":"2024-09-10T07:49:56.314079+0000","last_peered":"2024-09-10T07:49:56.314079+0000","last_clean":"2024-09-10T07:49:56.314079+0000","last_became_active":"2024-09-10T07:49:54.230977+0000","last_became_peered":"2024-09-10T07:49:54.230977+0000","last_unstale":"2024-09-10T07:49:56.314079+0000","last_undegraded":"2024-09-10T07:49:56.314079+0000","last_fullsized":"2024-09-10T07:49:56.314079+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:01.906396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244817+0000","last_change":"2024-09-10T07:49:55.239481+0000","last_active":"2024-09-10T07:49:55.244817+0000","last_peered":"2024-09-10T07:49:55.244817+0000","last_clean":"2024-09-10T07:49:55.244817+0000","last_became_active":"2024-09-10T07:49:55.235239+0000","last_became_peered":"2024-09-10T07:49:55.235239+0000","last_unstale":"2024-09-10T07:49:55.244817+0000","last_undegraded":"2024-09-10T07:49:55.244817+0000","last_fullsized":"2024-09-10T07:49:55.244817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:38.113241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217412+0000","last_change":"2024-09-10T07:49:54.230722+0000","last_active":"2024-09-10T07:49:55.217412+0000","last_peered":"2024-09-10T07:49:55.217412+0000","last_clean":"2024-09-10T07:49:55.217412+0000","last_became_active":"2024-09-10T07:49:54.230413+0000","last_became_peered":"2024-09-10T07:49:54.230413+0000","last_unstale":"2024-09-10T07:49:55.217412+0000","last_undegraded":"2024-09-10T07:49:55.217412+0000","last_fullsized":"2024-09-10T07:49:55.217412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:33.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235345+0000","last_change":"2024-09-10T07:49:55.230369+0000","last_active":"2024-09-10T07:49:55.235345+0000","last_peered":"2024-09-10T07:49:55.235345+0000","last_clean":"2024-09-10T07:49:55.235345+0000","last_became_active":"2024-09-10T07:49:55.230103+0000","last_became_peered":"2024-09-10T07:49:55.230103+0000","last_unstale":"2024-09-10T07:49:55.235345+0000","last_undegraded":"2024-09-10T07:49:55.235345+0000","last_fullsized":"2024-09-10T07:49:55.235345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:08.921569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.224788+0000","last_change":"2024-09-10T07:49:54.226293+0000","last_active":"2024-09-10T07:49:55.224788+0000","last_peered":"2024-09-10T07:49:55.224788+0000","last_clean":"2024-09-10T07:49:55.224788+0000","last_became_active":"2024-09-10T07:49:54.225955+0000","last_became_peered":"2024-09-10T07:49:54.225955+0000","last_unstale":"2024-09-10T07:49:55.224788+0000","last_undegraded":"2024-09-10T07:49:55.224788+0000","last_fullsized":"2024-09-10T07:49:55.224788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:49.959392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.239360+0000","last_change":"2024-09-10T07:49:55.229860+0000","last_active":"2024-09-10T07:49:55.239360+0000","last_peered":"2024-09-10T07:49:55.239360+0000","last_clean":"2024-09-10T07:49:55.239360+0000","last_became_active":"2024-09-10T07:49:55.229545+0000","last_became_peered":"2024-09-10T07:49:55.229545+0000","last_unstale":"2024-09-10T07:49:55.239360+0000","last_undegraded":"2024-09-10T07:49:55.239360+0000","last_fullsized":"2024-09-10T07:49:55.239360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:17.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314774+0000","last_change":"2024-09-10T07:49:54.238177+0000","last_active":"2024-09-10T07:49:56.314774+0000","last_peered":"2024-09-10T07:49:56.314774+0000","last_clean":"2024-09-10T07:49:56.314774+0000","last_became_active":"2024-09-10T07:49:54.237506+0000","last_became_peered":"2024-09-10T07:49:54.237506+0000","last_unstale":"2024-09-10T07:49:56.314774+0000","last_undegraded":"2024-09-10T07:49:56.314774+0000","last_fullsized":"2024-09-10T07:49:56.314774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:01.906396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":245,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238781+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:49:56.238781+0000","last_peered":"2024-09-10T07:49:56.238781+0000","last_clean":"2024-09-10T07:49:56.238781+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:49:56.238781+0000","last_undegraded":"2024-09-10T07:49:56.238781+0000","last_fullsized":"2024-09-10T07:49:56.238781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476783,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":749568,"data_stored":313739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":765952,"data_stored":315421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476781,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1343488,"data_stored":904599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":757760,"data_stored":315261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":753664,"data_stored":313741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":733184,"data_stored":306874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1343488,"data_stored":904051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36052,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":708608,"data_stored":294782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:49:59.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:49:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.641+0000 7f7d69400640 1 Processor -- start 2024-09-10T07:49:59.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.641+0000 7f7d69400640 1 -- start start 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.642+0000 7f7d69400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d64073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.642+0000 7f7d69400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d640cfdc0 con 0x7f7d6415e860 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.642+0000 7f7d69400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d640cee40 con 0x7f7d640d4050 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.642+0000 7f7d69400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7d640d3780 con 0x7f7d640d3c10 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d63e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7d6415e860 0x7f7d640cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43074/0 (socket says 172.21.15.3:43074) 2024-09-10T07:49:59.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d63e00640 1 -- 172.21.15.3:0/1619909196 learned_addr learned my addr 172.21.15.3:0/1619909196 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:49:59.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857649022 0 0) 0x7f7d640cfdc0 con 0x7f7d6415e860 2024-09-10T07:49:59.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d63400640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d64073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d48003520 con 0x7f7d6415e860 2024-09-10T07:49:59.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 268432709 0 0) 0x7f7d48003520 con 0x7f7d6415e860 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 msgr2=0x7f7d64073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d64073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 >> v1:172.21.15.70:6789/0 conn(0x7f7d640d4050 legacy=0x7f7d640b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d640cf120 con 0x7f7d6415e860 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f7d54001670 con 0x7f7d6415e860 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 >> v1:172.21.15.3:6789/0 conn(0x7f7d6415e860 legacy=0x7f7d640cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d48003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d48004800 0x7f7d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.643+0000 7f7d62a00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640d3780 con 0x7f7d48003a50 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4800bac0 con 0x7f7d48004800 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4800bcc0 con 0x7f7d640d3c10 2024-09-10T07:49:59.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d48003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63400640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d48004800 0x7f7d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d68a00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 -- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 msgr2=0x7f7d480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d640d3c10 0x7f7d480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 -- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d48004800 msgr2=0x7f7d48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d48004800 0x7f7d48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63e00640 1 -- 172.21.15.3:0/1619909196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48003520 con 0x7f7d48003a50 2024-09-10T07:49:59.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.644+0000 7f7d63400640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d48004800 0x7f7d48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:49:59.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.645+0000 7f7d63e00640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d48003e70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c210 tx=0x7f7d48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defea82b45ae544d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.645+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007d30 con 0x7f7d48003a50 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54005ad0 con 0x7f7d48003a50 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d62a00640 1 -- 172.21.15.3:0/1619909196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007070 con 0x7f7d48003a50 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 -- 172.21.15.3:0/1619909196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 msgr2=0x7f7d48003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 --2- 172.21.15.3:0/1619909196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d48003e70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c210 tx=0x7f7d48005830 comp rx=0 tx=0).stop 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d68a00640 1 -- 172.21.15.3:0/1619909196 reap_dead start 2024-09-10T07:49:59.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 -- 172.21.15.3:0/1619909196 shutdown_connections 2024-09-10T07:49:59.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 -- 172.21.15.3:0/1619909196 >> 172.21.15.3:0/1619909196 conn(0x7f7d640d4650 msgr2=0x7f7d640d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 -- 172.21.15.3:0/1619909196 shutdown_connections 2024-09-10T07:49:59.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.646+0000 7f7d69400640 1 -- 172.21.15.3:0/1619909196 wait complete. 2024-09-10T07:49:59.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.648+0000 7f7d69400640 1 Processor -- start 2024-09-10T07:49:59.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.648+0000 7f7d69400640 1 -- start start 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d6415be80 0x7f7d641f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d6415b8c0 0x7f7d640aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640d2460 con 0x7f7d48003a50 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640cf400 con 0x7f7d6415b8c0 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d69400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7d640b7f90 con 0x7f7d6415be80 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40214/0 (socket says 172.21.15.3:40214) 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d63e00640 1 -- 172.21.15.3:0/2988873587 learned_addr learned my addr 172.21.15.3:0/2988873587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d63400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d6415be80 0x7f7d641f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.649+0000 7f7d68a00640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d6415b8c0 0x7f7d640aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 -- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d6415be80 msgr2=0x7f7d641f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d6415be80 0x7f7d641f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 -- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d6415b8c0 msgr2=0x7f7d640aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d6415b8c0 0x7f7d640aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 -- 172.21.15.3:0/2988873587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d640ae240 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d63e00640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7d5402ae20 tx=0x7f7d54001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54042020 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54004bf0 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007070 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641f7830 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.650+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d640cbda0 con 0x7f7d48003a50 2024-09-10T07:49:59.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.651+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d640b7880 con 0x7f7d48003a50 2024-09-10T07:49:59.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.652+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d54045020 con 0x7f7d48003a50 2024-09-10T07:49:59.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.653+0000 7f7d5be00640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 0x7f7d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:49:59.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.653+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f7d540aaef0 con 0x7f7d48003a50 2024-09-10T07:49:59.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.653+0000 7f7d63400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 0x7f7d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:49:59.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.654+0000 7f7d63400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 0x7f7d4406e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b33c0 tx=0x7f7d50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:49:59.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.657+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d5404c020 con 0x7f7d48003a50 2024-09-10T07:49:59.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.838+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d640ba5b0 con 0x7f7d4406be30 2024-09-10T07:49:59.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.845+0000 7f7d5be00640 1 -- 172.21.15.3:0/2988873587 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251925 (secure 0 0 0) 0x7f7d640ba5b0 con 0x7f7d4406be30 2024-09-10T07:49:59.847 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:49:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.848+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 msgr2=0x7f7d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.848+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 0x7f7d4406e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d640b33c0 tx=0x7f7d50008040 comp rx=0 tx=0).stop 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.848+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 msgr2=0x7f7d641f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.848+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7d5402ae20 tx=0x7f7d54001a40 comp rx=0 tx=0).stop 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 shutdown_connections 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7d4406be30 0x7f7d4406e2f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d6415b8c0 0x7f7d640aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d6415be80 0x7f7d641f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 --2- 172.21.15.3:0/2988873587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d48003a50 0x7f7d641f4560 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:49:59.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 >> 172.21.15.3:0/2988873587 conn(0x7f7d640d4650 msgr2=0x7f7d640d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:49:59.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 shutdown_connections 2024-09-10T07:49:59.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:49:59.849+0000 7f7d69400640 1 -- 172.21.15.3:0/2988873587 wait complete. 2024-09-10T07:49:59.864 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-10T07:49:59.535334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260772,"kb_used_data":7044,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713948,"statfs":{"total":773094113280,"available":772827082752,"internally_reserved":0,"allocated":7213056,"data_stored":3694251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001006"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241325+0000","last_change":"2024-09-10T07:49:55.234160+0000","last_active":"2024-09-10T07:49:55.241325+0000","last_peered":"2024-09-10T07:49:55.241325+0000","last_clean":"2024-09-10T07:49:55.241325+0000","last_became_active":"2024-09-10T07:49:55.233435+0000","last_became_peered":"2024-09-10T07:49:55.233435+0000","last_unstale":"2024-09-10T07:49:55.241325+0000","last_undegraded":"2024-09-10T07:49:55.241325+0000","last_fullsized":"2024-09-10T07:49:55.241325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:22.901478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235402+0000","last_change":"2024-09-10T07:49:55.231309+0000","last_active":"2024-09-10T07:49:55.235402+0000","last_peered":"2024-09-10T07:49:55.235402+0000","last_clean":"2024-09-10T07:49:55.235402+0000","last_became_active":"2024-09-10T07:49:55.231022+0000","last_became_peered":"2024-09-10T07:49:55.231022+0000","last_unstale":"2024-09-10T07:49:55.235402+0000","last_undegraded":"2024-09-10T07:49:55.235402+0000","last_fullsized":"2024-09-10T07:49:55.235402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:29.658524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314264+0000","last_change":"2024-09-10T07:49:55.236559+0000","last_active":"2024-09-10T07:49:56.314264+0000","last_peered":"2024-09-10T07:49:56.314264+0000","last_clean":"2024-09-10T07:49:56.314264+0000","last_became_active":"2024-09-10T07:49:55.236291+0000","last_became_peered":"2024-09-10T07:49:55.236291+0000","last_unstale":"2024-09-10T07:49:56.314264+0000","last_undegraded":"2024-09-10T07:49:56.314264+0000","last_fullsized":"2024-09-10T07:49:56.314264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:17.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237862+0000","last_change":"2024-09-10T07:49:55.233233+0000","last_active":"2024-09-10T07:49:56.237862+0000","last_peered":"2024-09-10T07:49:56.237862+0000","last_clean":"2024-09-10T07:49:56.237862+0000","last_became_active":"2024-09-10T07:49:55.232946+0000","last_became_peered":"2024-09-10T07:49:55.232946+0000","last_unstale":"2024-09-10T07:49:56.237862+0000","last_undegraded":"2024-09-10T07:49:56.237862+0000","last_fullsized":"2024-09-10T07:49:56.237862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:13.029127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.236549+0000","last_change":"2024-09-10T07:49:55.231030+0000","last_active":"2024-09-10T07:49:55.236549+0000","last_peered":"2024-09-10T07:49:55.236549+0000","last_clean":"2024-09-10T07:49:55.236549+0000","last_became_active":"2024-09-10T07:49:55.230754+0000","last_became_peered":"2024-09-10T07:49:55.230754+0000","last_unstale":"2024-09-10T07:49:55.236549+0000","last_undegraded":"2024-09-10T07:49:55.236549+0000","last_fullsized":"2024-09-10T07:49:55.236549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241559+0000","last_change":"2024-09-10T07:49:55.234383+0000","last_active":"2024-09-10T07:49:55.241559+0000","last_peered":"2024-09-10T07:49:55.241559+0000","last_clean":"2024-09-10T07:49:55.241559+0000","last_became_active":"2024-09-10T07:49:55.233735+0000","last_became_peered":"2024-09-10T07:49:55.233735+0000","last_unstale":"2024-09-10T07:49:55.241559+0000","last_undegraded":"2024-09-10T07:49:55.241559+0000","last_fullsized":"2024-09-10T07:49:55.241559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:15.078487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237835+0000","last_change":"2024-09-10T07:49:55.245257+0000","last_active":"2024-09-10T07:49:56.237835+0000","last_peered":"2024-09-10T07:49:56.237835+0000","last_clean":"2024-09-10T07:49:56.237835+0000","last_became_active":"2024-09-10T07:49:55.244981+0000","last_became_peered":"2024-09-10T07:49:55.244981+0000","last_unstale":"2024-09-10T07:49:56.237835+0000","last_undegraded":"2024-09-10T07:49:56.237835+0000","last_fullsized":"2024-09-10T07:49:56.237835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:22.002166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.240335+0000","last_change":"2024-09-10T07:49:55.232117+0000","last_active":"2024-09-10T07:49:55.240335+0000","last_peered":"2024-09-10T07:49:55.240335+0000","last_clean":"2024-09-10T07:49:55.240335+0000","last_became_active":"2024-09-10T07:49:55.230521+0000","last_became_peered":"2024-09-10T07:49:55.230521+0000","last_unstale":"2024-09-10T07:49:55.240335+0000","last_undegraded":"2024-09-10T07:49:55.240335+0000","last_fullsized":"2024-09-10T07:49:55.240335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:36.497697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239231+0000","last_change":"2024-09-10T07:49:55.232810+0000","last_active":"2024-09-10T07:49:56.239231+0000","last_peered":"2024-09-10T07:49:56.239231+0000","last_clean":"2024-09-10T07:49:56.239231+0000","last_became_active":"2024-09-10T07:49:55.232552+0000","last_became_peered":"2024-09-10T07:49:55.232552+0000","last_unstale":"2024-09-10T07:49:56.239231+0000","last_undegraded":"2024-09-10T07:49:56.239231+0000","last_fullsized":"2024-09-10T07:49:56.239231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:43.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.247368+0000","last_change":"2024-09-10T07:49:55.244152+0000","last_active":"2024-09-10T07:49:55.247368+0000","last_peered":"2024-09-10T07:49:55.247368+0000","last_clean":"2024-09-10T07:49:55.247368+0000","last_became_active":"2024-09-10T07:49:55.243878+0000","last_became_peered":"2024-09-10T07:49:55.243878+0000","last_unstale":"2024-09-10T07:49:55.247368+0000","last_undegraded":"2024-09-10T07:49:55.247368+0000","last_fullsized":"2024-09-10T07:49:55.247368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:02.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238327+0000","last_change":"2024-09-10T07:49:55.242745+0000","last_active":"2024-09-10T07:49:56.238327+0000","last_peered":"2024-09-10T07:49:56.238327+0000","last_clean":"2024-09-10T07:49:56.238327+0000","last_became_active":"2024-09-10T07:49:55.242482+0000","last_became_peered":"2024-09-10T07:49:55.242482+0000","last_unstale":"2024-09-10T07:49:56.238327+0000","last_undegraded":"2024-09-10T07:49:56.238327+0000","last_fullsized":"2024-09-10T07:49:56.238327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:30.044942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239006+0000","last_change":"2024-09-10T07:49:55.237090+0000","last_active":"2024-09-10T07:49:56.239006+0000","last_peered":"2024-09-10T07:49:56.239006+0000","last_clean":"2024-09-10T07:49:56.239006+0000","last_became_active":"2024-09-10T07:49:55.236823+0000","last_became_peered":"2024-09-10T07:49:55.236823+0000","last_unstale":"2024-09-10T07:49:56.239006+0000","last_undegraded":"2024-09-10T07:49:56.239006+0000","last_fullsized":"2024-09-10T07:49:56.239006+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314906+0000","last_change":"2024-09-10T07:49:55.238531+0000","last_active":"2024-09-10T07:49:56.314906+0000","last_peered":"2024-09-10T07:49:56.314906+0000","last_clean":"2024-09-10T07:49:56.314906+0000","last_became_active":"2024-09-10T07:49:55.238223+0000","last_became_peered":"2024-09-10T07:49:55.238223+0000","last_unstale":"2024-09-10T07:49:56.314906+0000","last_undegraded":"2024-09-10T07:49:56.314906+0000","last_fullsized":"2024-09-10T07:49:56.314906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:22.148823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313990+0000","last_change":"2024-09-10T07:49:55.235828+0000","last_active":"2024-09-10T07:49:56.313990+0000","last_peered":"2024-09-10T07:49:56.313990+0000","last_clean":"2024-09-10T07:49:56.313990+0000","last_became_active":"2024-09-10T07:49:55.235526+0000","last_became_peered":"2024-09-10T07:49:55.235526+0000","last_unstale":"2024-09-10T07:49:56.313990+0000","last_undegraded":"2024-09-10T07:49:56.313990+0000","last_fullsized":"2024-09-10T07:49:56.313990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:56.169860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314773+0000","last_change":"2024-09-10T07:49:55.232930+0000","last_active":"2024-09-10T07:49:56.314773+0000","last_peered":"2024-09-10T07:49:56.314773+0000","last_clean":"2024-09-10T07:49:56.314773+0000","last_became_active":"2024-09-10T07:49:55.232364+0000","last_became_peered":"2024-09-10T07:49:55.232364+0000","last_unstale":"2024-09-10T07:49:56.314773+0000","last_undegraded":"2024-09-10T07:49:56.314773+0000","last_fullsized":"2024-09-10T07:49:56.314773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.877990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.236509+0000","last_change":"2024-09-10T07:49:55.229285+0000","last_active":"2024-09-10T07:49:55.236509+0000","last_peered":"2024-09-10T07:49:55.236509+0000","last_clean":"2024-09-10T07:49:55.236509+0000","last_became_active":"2024-09-10T07:49:55.228951+0000","last_became_peered":"2024-09-10T07:49:55.228951+0000","last_unstale":"2024-09-10T07:49:55.236509+0000","last_undegraded":"2024-09-10T07:49:55.236509+0000","last_fullsized":"2024-09-10T07:49:55.236509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245720+0000","last_change":"2024-09-10T07:49:55.236951+0000","last_active":"2024-09-10T07:49:55.245720+0000","last_peered":"2024-09-10T07:49:55.245720+0000","last_clean":"2024-09-10T07:49:55.245720+0000","last_became_active":"2024-09-10T07:49:55.234617+0000","last_became_peered":"2024-09-10T07:49:55.234617+0000","last_unstale":"2024-09-10T07:49:55.245720+0000","last_undegraded":"2024-09-10T07:49:55.245720+0000","last_fullsized":"2024-09-10T07:49:55.245720+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:29.015621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314023+0000","last_change":"2024-09-10T07:49:55.244009+0000","last_active":"2024-09-10T07:49:56.314023+0000","last_peered":"2024-09-10T07:49:56.314023+0000","last_clean":"2024-09-10T07:49:56.314023+0000","last_became_active":"2024-09-10T07:49:55.243720+0000","last_became_peered":"2024-09-10T07:49:55.243720+0000","last_unstale":"2024-09-10T07:49:56.314023+0000","last_undegraded":"2024-09-10T07:49:56.314023+0000","last_fullsized":"2024-09-10T07:49:56.314023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:31.165932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244607+0000","last_change":"2024-09-10T07:49:55.238643+0000","last_active":"2024-09-10T07:49:55.244607+0000","last_peered":"2024-09-10T07:49:55.244607+0000","last_clean":"2024-09-10T07:49:55.244607+0000","last_became_active":"2024-09-10T07:49:55.238331+0000","last_became_peered":"2024-09-10T07:49:55.238331+0000","last_unstale":"2024-09-10T07:49:55.244607+0000","last_undegraded":"2024-09-10T07:49:55.244607+0000","last_fullsized":"2024-09-10T07:49:55.244607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238518+0000","last_change":"2024-09-10T07:49:55.241871+0000","last_active":"2024-09-10T07:49:56.238518+0000","last_peered":"2024-09-10T07:49:56.238518+0000","last_clean":"2024-09-10T07:49:56.238518+0000","last_became_active":"2024-09-10T07:49:55.241547+0000","last_became_peered":"2024-09-10T07:49:55.241547+0000","last_unstale":"2024-09-10T07:49:56.238518+0000","last_undegraded":"2024-09-10T07:49:56.238518+0000","last_fullsized":"2024-09-10T07:49:56.238518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:51.243636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314953+0000","last_change":"2024-09-10T07:49:55.232221+0000","last_active":"2024-09-10T07:49:56.314953+0000","last_peered":"2024-09-10T07:49:56.314953+0000","last_clean":"2024-09-10T07:49:56.314953+0000","last_became_active":"2024-09-10T07:49:55.231825+0000","last_became_peered":"2024-09-10T07:49:55.231825+0000","last_unstale":"2024-09-10T07:49:56.314953+0000","last_undegraded":"2024-09-10T07:49:56.314953+0000","last_fullsized":"2024-09-10T07:49:56.314953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:34.662956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238198+0000","last_change":"2024-09-10T07:49:55.240055+0000","last_active":"2024-09-10T07:49:56.238198+0000","last_peered":"2024-09-10T07:49:56.238198+0000","last_clean":"2024-09-10T07:49:56.238198+0000","last_became_active":"2024-09-10T07:49:55.239757+0000","last_became_peered":"2024-09-10T07:49:55.239757+0000","last_unstale":"2024-09-10T07:49:56.238198+0000","last_undegraded":"2024-09-10T07:49:56.238198+0000","last_fullsized":"2024-09-10T07:49:56.238198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:45.710182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237987+0000","last_change":"2024-09-10T07:49:55.237209+0000","last_active":"2024-09-10T07:49:56.237987+0000","last_peered":"2024-09-10T07:49:56.237987+0000","last_clean":"2024-09-10T07:49:56.237987+0000","last_became_active":"2024-09-10T07:49:55.236601+0000","last_became_peered":"2024-09-10T07:49:55.236601+0000","last_unstale":"2024-09-10T07:49:56.237987+0000","last_undegraded":"2024-09-10T07:49:56.237987+0000","last_fullsized":"2024-09-10T07:49:56.237987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:20.174536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238746+0000","last_change":"2024-09-10T07:49:55.234622+0000","last_active":"2024-09-10T07:49:56.238746+0000","last_peered":"2024-09-10T07:49:56.238746+0000","last_clean":"2024-09-10T07:49:56.238746+0000","last_became_active":"2024-09-10T07:49:55.231542+0000","last_became_peered":"2024-09-10T07:49:55.231542+0000","last_unstale":"2024-09-10T07:49:56.238746+0000","last_undegraded":"2024-09-10T07:49:56.238746+0000","last_fullsized":"2024-09-10T07:49:56.238746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:37.465799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237693+0000","last_change":"2024-09-10T07:49:55.232235+0000","last_active":"2024-09-10T07:49:55.237693+0000","last_peered":"2024-09-10T07:49:55.237693+0000","last_clean":"2024-09-10T07:49:55.237693+0000","last_became_active":"2024-09-10T07:49:55.231902+0000","last_became_peered":"2024-09-10T07:49:55.231902+0000","last_unstale":"2024-09-10T07:49:55.237693+0000","last_undegraded":"2024-09-10T07:49:55.237693+0000","last_fullsized":"2024-09-10T07:49:55.237693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:34.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315221+0000","last_change":"2024-09-10T07:49:55.244518+0000","last_active":"2024-09-10T07:49:56.315221+0000","last_peered":"2024-09-10T07:49:56.315221+0000","last_clean":"2024-09-10T07:49:56.315221+0000","last_became_active":"2024-09-10T07:49:55.244201+0000","last_became_peered":"2024-09-10T07:49:55.244201+0000","last_unstale":"2024-09-10T07:49:56.315221+0000","last_undegraded":"2024-09-10T07:49:56.315221+0000","last_fullsized":"2024-09-10T07:49:56.315221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:28.714769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.242590+0000","last_change":"2024-09-10T07:49:55.234762+0000","last_active":"2024-09-10T07:49:55.242590+0000","last_peered":"2024-09-10T07:49:55.242590+0000","last_clean":"2024-09-10T07:49:55.242590+0000","last_became_active":"2024-09-10T07:49:55.234491+0000","last_became_peered":"2024-09-10T07:49:55.234491+0000","last_unstale":"2024-09-10T07:49:55.242590+0000","last_undegraded":"2024-09-10T07:49:55.242590+0000","last_fullsized":"2024-09-10T07:49:55.242590+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:09.323053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245775+0000","last_change":"2024-09-10T07:49:55.237411+0000","last_active":"2024-09-10T07:49:55.245775+0000","last_peered":"2024-09-10T07:49:55.245775+0000","last_clean":"2024-09-10T07:49:55.245775+0000","last_became_active":"2024-09-10T07:49:55.237124+0000","last_became_peered":"2024-09-10T07:49:55.237124+0000","last_unstale":"2024-09-10T07:49:55.245775+0000","last_undegraded":"2024-09-10T07:49:55.245775+0000","last_fullsized":"2024-09-10T07:49:55.245775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:04.121735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237830+0000","last_change":"2024-09-10T07:49:55.233086+0000","last_active":"2024-09-10T07:49:55.237830+0000","last_peered":"2024-09-10T07:49:55.237830+0000","last_clean":"2024-09-10T07:49:55.237830+0000","last_became_active":"2024-09-10T07:49:55.230813+0000","last_became_peered":"2024-09-10T07:49:55.230813+0000","last_unstale":"2024-09-10T07:49:55.237830+0000","last_undegraded":"2024-09-10T07:49:55.237830+0000","last_fullsized":"2024-09-10T07:49:55.237830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:46.287208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237803+0000","last_change":"2024-09-10T07:49:55.235433+0000","last_active":"2024-09-10T07:49:56.237803+0000","last_peered":"2024-09-10T07:49:56.237803+0000","last_clean":"2024-09-10T07:49:56.237803+0000","last_became_active":"2024-09-10T07:49:55.235148+0000","last_became_peered":"2024-09-10T07:49:55.235148+0000","last_unstale":"2024-09-10T07:49:56.237803+0000","last_undegraded":"2024-09-10T07:49:56.237803+0000","last_fullsized":"2024-09-10T07:49:56.237803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:56.653577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244759+0000","last_change":"2024-09-10T07:49:55.238906+0000","last_active":"2024-09-10T07:49:55.244759+0000","last_peered":"2024-09-10T07:49:55.244759+0000","last_clean":"2024-09-10T07:49:55.244759+0000","last_became_active":"2024-09-10T07:49:55.238394+0000","last_became_peered":"2024-09-10T07:49:55.238394+0000","last_unstale":"2024-09-10T07:49:55.244759+0000","last_undegraded":"2024-09-10T07:49:55.244759+0000","last_fullsized":"2024-09-10T07:49:55.244759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:47.913153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237893+0000","last_change":"2024-09-10T07:49:55.241104+0000","last_active":"2024-09-10T07:49:56.237893+0000","last_peered":"2024-09-10T07:49:56.237893+0000","last_clean":"2024-09-10T07:49:56.237893+0000","last_became_active":"2024-09-10T07:49:55.240650+0000","last_became_peered":"2024-09-10T07:49:55.240650+0000","last_unstale":"2024-09-10T07:49:56.237893+0000","last_undegraded":"2024-09-10T07:49:56.237893+0000","last_fullsized":"2024-09-10T07:49:56.237893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:53.820666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.237220+0000","last_change":"2024-09-10T07:49:55.232053+0000","last_active":"2024-09-10T07:49:55.237220+0000","last_peered":"2024-09-10T07:49:55.237220+0000","last_clean":"2024-09-10T07:49:55.237220+0000","last_became_active":"2024-09-10T07:49:55.231743+0000","last_became_peered":"2024-09-10T07:49:55.231743+0000","last_unstale":"2024-09-10T07:49:55.237220+0000","last_undegraded":"2024-09-10T07:49:55.237220+0000","last_fullsized":"2024-09-10T07:49:55.237220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:43.185394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238518+0000","last_change":"2024-09-10T07:49:54.230935+0000","last_active":"2024-09-10T07:49:56.238518+0000","last_peered":"2024-09-10T07:49:56.238518+0000","last_clean":"2024-09-10T07:49:56.238518+0000","last_became_active":"2024-09-10T07:49:54.230465+0000","last_became_peered":"2024-09-10T07:49:54.230465+0000","last_unstale":"2024-09-10T07:49:56.238518+0000","last_undegraded":"2024-09-10T07:49:56.238518+0000","last_fullsized":"2024-09-10T07:49:56.238518+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:22.334216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241304+0000","last_change":"2024-09-10T07:49:55.233708+0000","last_active":"2024-09-10T07:49:55.241304+0000","last_peered":"2024-09-10T07:49:55.241304+0000","last_clean":"2024-09-10T07:49:55.241304+0000","last_became_active":"2024-09-10T07:49:55.233035+0000","last_became_peered":"2024-09-10T07:49:55.233035+0000","last_unstale":"2024-09-10T07:49:55.241304+0000","last_undegraded":"2024-09-10T07:49:55.241304+0000","last_fullsized":"2024-09-10T07:49:55.241304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:52.316375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.253565+0000","last_change":"2024-09-10T07:49:54.235741+0000","last_active":"2024-09-10T07:49:55.253565+0000","last_peered":"2024-09-10T07:49:55.253565+0000","last_clean":"2024-09-10T07:49:55.253565+0000","last_became_active":"2024-09-10T07:49:54.234631+0000","last_became_peered":"2024-09-10T07:49:54.234631+0000","last_unstale":"2024-09-10T07:49:55.253565+0000","last_undegraded":"2024-09-10T07:49:55.253565+0000","last_fullsized":"2024-09-10T07:49:55.253565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:09.498213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.247519+0000","last_change":"2024-09-10T07:49:55.245934+0000","last_active":"2024-09-10T07:49:55.247519+0000","last_peered":"2024-09-10T07:49:55.247519+0000","last_clean":"2024-09-10T07:49:55.247519+0000","last_became_active":"2024-09-10T07:49:55.245591+0000","last_became_peered":"2024-09-10T07:49:55.245591+0000","last_unstale":"2024-09-10T07:49:55.247519+0000","last_undegraded":"2024-09-10T07:49:55.247519+0000","last_fullsized":"2024-09-10T07:49:55.247519+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.114001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250107+0000","last_change":"2024-09-10T07:49:54.229770+0000","last_active":"2024-09-10T07:49:55.250107+0000","last_peered":"2024-09-10T07:49:55.250107+0000","last_clean":"2024-09-10T07:49:55.250107+0000","last_became_active":"2024-09-10T07:49:54.229476+0000","last_became_peered":"2024-09-10T07:49:54.229476+0000","last_unstale":"2024-09-10T07:49:55.250107+0000","last_undegraded":"2024-09-10T07:49:55.250107+0000","last_fullsized":"2024-09-10T07:49:55.250107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:07.917229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.238283+0000","last_change":"2024-09-10T07:49:55.226343+0000","last_active":"2024-09-10T07:49:55.238283+0000","last_peered":"2024-09-10T07:49:55.238283+0000","last_clean":"2024-09-10T07:49:55.238283+0000","last_became_active":"2024-09-10T07:49:55.222906+0000","last_became_peered":"2024-09-10T07:49:55.222906+0000","last_unstale":"2024-09-10T07:49:55.238283+0000","last_undegraded":"2024-09-10T07:49:55.238283+0000","last_fullsized":"2024-09-10T07:49:55.238283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238086+0000","last_change":"2024-09-10T07:49:54.224380+0000","last_active":"2024-09-10T07:49:56.238086+0000","last_peered":"2024-09-10T07:49:56.238086+0000","last_clean":"2024-09-10T07:49:56.238086+0000","last_became_active":"2024-09-10T07:49:54.224065+0000","last_became_peered":"2024-09-10T07:49:54.224065+0000","last_unstale":"2024-09-10T07:49:56.238086+0000","last_undegraded":"2024-09-10T07:49:56.238086+0000","last_fullsized":"2024-09-10T07:49:56.238086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:40.824351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244697+0000","last_change":"2024-09-10T07:49:55.239335+0000","last_active":"2024-09-10T07:49:55.244697+0000","last_peered":"2024-09-10T07:49:55.244697+0000","last_clean":"2024-09-10T07:49:55.244697+0000","last_became_active":"2024-09-10T07:49:55.234297+0000","last_became_peered":"2024-09-10T07:49:55.234297+0000","last_unstale":"2024-09-10T07:49:55.244697+0000","last_undegraded":"2024-09-10T07:49:55.244697+0000","last_fullsized":"2024-09-10T07:49:55.244697+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:18.237272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313881+0000","last_change":"2024-09-10T07:49:54.233192+0000","last_active":"2024-09-10T07:49:56.313881+0000","last_peered":"2024-09-10T07:49:56.313881+0000","last_clean":"2024-09-10T07:49:56.313881+0000","last_became_active":"2024-09-10T07:49:54.232011+0000","last_became_peered":"2024-09-10T07:49:54.232011+0000","last_unstale":"2024-09-10T07:49:56.313881+0000","last_undegraded":"2024-09-10T07:49:56.313881+0000","last_fullsized":"2024-09-10T07:49:56.313881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:37.679105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238543+0000","last_change":"2024-09-10T07:49:55.241903+0000","last_active":"2024-09-10T07:49:56.238543+0000","last_peered":"2024-09-10T07:49:56.238543+0000","last_clean":"2024-09-10T07:49:56.238543+0000","last_became_active":"2024-09-10T07:49:55.241272+0000","last_became_peered":"2024-09-10T07:49:55.241272+0000","last_unstale":"2024-09-10T07:49:56.238543+0000","last_undegraded":"2024-09-10T07:49:56.238543+0000","last_fullsized":"2024-09-10T07:49:56.238543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:38.766025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.253574+0000","last_change":"2024-09-10T07:49:54.236049+0000","last_active":"2024-09-10T07:49:55.253574+0000","last_peered":"2024-09-10T07:49:55.253574+0000","last_clean":"2024-09-10T07:49:55.253574+0000","last_became_active":"2024-09-10T07:49:54.235801+0000","last_became_peered":"2024-09-10T07:49:54.235801+0000","last_unstale":"2024-09-10T07:49:55.253574+0000","last_undegraded":"2024-09-10T07:49:55.253574+0000","last_fullsized":"2024-09-10T07:49:55.253574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:28.654184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313912+0000","last_change":"2024-09-10T07:49:55.233993+0000","last_active":"2024-09-10T07:49:56.313912+0000","last_peered":"2024-09-10T07:49:56.313912+0000","last_clean":"2024-09-10T07:49:56.313912+0000","last_became_active":"2024-09-10T07:49:55.233626+0000","last_became_peered":"2024-09-10T07:49:55.233626+0000","last_unstale":"2024-09-10T07:49:56.313912+0000","last_undegraded":"2024-09-10T07:49:56.313912+0000","last_fullsized":"2024-09-10T07:49:56.313912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:16.446101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238547+0000","last_change":"2024-09-10T07:49:54.234520+0000","last_active":"2024-09-10T07:49:56.238547+0000","last_peered":"2024-09-10T07:49:56.238547+0000","last_clean":"2024-09-10T07:49:56.238547+0000","last_became_active":"2024-09-10T07:49:54.234065+0000","last_became_peered":"2024-09-10T07:49:54.234065+0000","last_unstale":"2024-09-10T07:49:56.238547+0000","last_undegraded":"2024-09-10T07:49:56.238547+0000","last_fullsized":"2024-09-10T07:49:56.238547+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:34:47.319841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.246579+0000","last_change":"2024-09-10T07:49:55.237921+0000","last_active":"2024-09-10T07:49:55.246579+0000","last_peered":"2024-09-10T07:49:55.246579+0000","last_clean":"2024-09-10T07:49:55.246579+0000","last_became_active":"2024-09-10T07:49:55.237573+0000","last_became_peered":"2024-09-10T07:49:55.237573+0000","last_unstale":"2024-09-10T07:49:55.246579+0000","last_undegraded":"2024-09-10T07:49:55.246579+0000","last_fullsized":"2024-09-10T07:49:55.246579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:12.231876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314493+0000","last_change":"2024-09-10T07:49:54.232693+0000","last_active":"2024-09-10T07:49:56.314493+0000","last_peered":"2024-09-10T07:49:56.314493+0000","last_clean":"2024-09-10T07:49:56.314493+0000","last_became_active":"2024-09-10T07:49:54.232210+0000","last_became_peered":"2024-09-10T07:49:54.232210+0000","last_unstale":"2024-09-10T07:49:56.314493+0000","last_undegraded":"2024-09-10T07:49:56.314493+0000","last_fullsized":"2024-09-10T07:49:56.314493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:11.227536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315310+0000","last_change":"2024-09-10T07:49:55.246775+0000","last_active":"2024-09-10T07:49:56.315310+0000","last_peered":"2024-09-10T07:49:56.315310+0000","last_clean":"2024-09-10T07:49:56.315310+0000","last_became_active":"2024-09-10T07:49:55.241443+0000","last_became_peered":"2024-09-10T07:49:55.241443+0000","last_unstale":"2024-09-10T07:49:56.315310+0000","last_undegraded":"2024-09-10T07:49:56.315310+0000","last_fullsized":"2024-09-10T07:49:56.315310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:50.963732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239165+0000","last_change":"2024-09-10T07:49:54.235701+0000","last_active":"2024-09-10T07:49:56.239165+0000","last_peered":"2024-09-10T07:49:56.239165+0000","last_clean":"2024-09-10T07:49:56.239165+0000","last_became_active":"2024-09-10T07:49:54.235393+0000","last_became_peered":"2024-09-10T07:49:54.235393+0000","last_unstale":"2024-09-10T07:49:56.239165+0000","last_undegraded":"2024-09-10T07:49:56.239165+0000","last_fullsized":"2024-09-10T07:49:56.239165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:59.763343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314615+0000","last_change":"2024-09-10T07:49:55.242384+0000","last_active":"2024-09-10T07:49:56.314615+0000","last_peered":"2024-09-10T07:49:56.314615+0000","last_clean":"2024-09-10T07:49:56.314615+0000","last_became_active":"2024-09-10T07:49:55.241007+0000","last_became_peered":"2024-09-10T07:49:55.241007+0000","last_unstale":"2024-09-10T07:49:56.314615+0000","last_undegraded":"2024-09-10T07:49:56.314615+0000","last_fullsized":"2024-09-10T07:49:56.314615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:15.138119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.226141+0000","last_change":"2024-09-10T07:49:54.229298+0000","last_active":"2024-09-10T07:49:55.226141+0000","last_peered":"2024-09-10T07:49:55.226141+0000","last_clean":"2024-09-10T07:49:55.226141+0000","last_became_active":"2024-09-10T07:49:54.229007+0000","last_became_peered":"2024-09-10T07:49:54.229007+0000","last_unstale":"2024-09-10T07:49:55.226141+0000","last_undegraded":"2024-09-10T07:49:55.226141+0000","last_fullsized":"2024-09-10T07:49:55.226141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:55.165520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.245825+0000","last_change":"2024-09-10T07:49:55.243077+0000","last_active":"2024-09-10T07:49:55.245825+0000","last_peered":"2024-09-10T07:49:55.245825+0000","last_clean":"2024-09-10T07:49:55.245825+0000","last_became_active":"2024-09-10T07:49:55.242816+0000","last_became_peered":"2024-09-10T07:49:55.242816+0000","last_unstale":"2024-09-10T07:49:55.245825+0000","last_undegraded":"2024-09-10T07:49:55.245825+0000","last_fullsized":"2024-09-10T07:49:55.245825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:32.821456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238400+0000","last_change":"2024-09-10T07:49:54.228989+0000","last_active":"2024-09-10T07:49:56.238400+0000","last_peered":"2024-09-10T07:49:56.238400+0000","last_clean":"2024-09-10T07:49:56.238400+0000","last_became_active":"2024-09-10T07:49:54.228715+0000","last_became_peered":"2024-09-10T07:49:54.228715+0000","last_unstale":"2024-09-10T07:49:56.238400+0000","last_undegraded":"2024-09-10T07:49:56.238400+0000","last_fullsized":"2024-09-10T07:49:56.238400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:31.300326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238308+0000","last_change":"2024-09-10T07:49:55.234796+0000","last_active":"2024-09-10T07:49:56.238308+0000","last_peered":"2024-09-10T07:49:56.238308+0000","last_clean":"2024-09-10T07:49:56.238308+0000","last_became_active":"2024-09-10T07:49:55.234519+0000","last_became_peered":"2024-09-10T07:49:55.234519+0000","last_unstale":"2024-09-10T07:49:56.238308+0000","last_undegraded":"2024-09-10T07:49:56.238308+0000","last_fullsized":"2024-09-10T07:49:56.238308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:05.114577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315069+0000","last_change":"2024-09-10T07:49:54.237406+0000","last_active":"2024-09-10T07:49:56.315069+0000","last_peered":"2024-09-10T07:49:56.315069+0000","last_clean":"2024-09-10T07:49:56.315069+0000","last_became_active":"2024-09-10T07:49:54.236989+0000","last_became_peered":"2024-09-10T07:49:54.236989+0000","last_unstale":"2024-09-10T07:49:56.315069+0000","last_undegraded":"2024-09-10T07:49:56.315069+0000","last_fullsized":"2024-09-10T07:49:56.315069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:55.896433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238601+0000","last_change":"2024-09-10T07:49:55.224183+0000","last_active":"2024-09-10T07:49:56.238601+0000","last_peered":"2024-09-10T07:49:56.238601+0000","last_clean":"2024-09-10T07:49:56.238601+0000","last_became_active":"2024-09-10T07:49:55.223835+0000","last_became_peered":"2024-09-10T07:49:55.223835+0000","last_unstale":"2024-09-10T07:49:56.238601+0000","last_undegraded":"2024-09-10T07:49:56.238601+0000","last_fullsized":"2024-09-10T07:49:56.238601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:34.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250794+0000","last_change":"2024-09-10T07:49:54.232415+0000","last_active":"2024-09-10T07:49:55.250794+0000","last_peered":"2024-09-10T07:49:55.250794+0000","last_clean":"2024-09-10T07:49:55.250794+0000","last_became_active":"2024-09-10T07:49:54.231725+0000","last_became_peered":"2024-09-10T07:49:54.231725+0000","last_unstale":"2024-09-10T07:49:55.250794+0000","last_undegraded":"2024-09-10T07:49:55.250794+0000","last_fullsized":"2024-09-10T07:49:55.250794+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:15.441761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.313937+0000","last_change":"2024-09-10T07:49:55.237774+0000","last_active":"2024-09-10T07:49:56.313937+0000","last_peered":"2024-09-10T07:49:56.313937+0000","last_clean":"2024-09-10T07:49:56.313937+0000","last_became_active":"2024-09-10T07:49:55.237448+0000","last_became_peered":"2024-09-10T07:49:55.237448+0000","last_unstale":"2024-09-10T07:49:56.313937+0000","last_undegraded":"2024-09-10T07:49:56.313937+0000","last_fullsized":"2024-09-10T07:49:56.313937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:42.265970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237894+0000","last_change":"2024-09-10T07:49:54.223427+0000","last_active":"2024-09-10T07:49:56.237894+0000","last_peered":"2024-09-10T07:49:56.237894+0000","last_clean":"2024-09-10T07:49:56.237894+0000","last_became_active":"2024-09-10T07:49:54.223114+0000","last_became_peered":"2024-09-10T07:49:54.223114+0000","last_unstale":"2024-09-10T07:49:56.237894+0000","last_undegraded":"2024-09-10T07:49:56.237894+0000","last_fullsized":"2024-09-10T07:49:56.237894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:41.737915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238637+0000","last_change":"2024-09-10T07:49:55.234199+0000","last_active":"2024-09-10T07:49:56.238637+0000","last_peered":"2024-09-10T07:49:56.238637+0000","last_clean":"2024-09-10T07:49:56.238637+0000","last_became_active":"2024-09-10T07:49:55.233758+0000","last_became_peered":"2024-09-10T07:49:55.233758+0000","last_unstale":"2024-09-10T07:49:56.238637+0000","last_undegraded":"2024-09-10T07:49:56.238637+0000","last_fullsized":"2024-09-10T07:49:56.238637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:09.323053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238092+0000","last_change":"2024-09-10T07:49:54.230943+0000","last_active":"2024-09-10T07:49:56.238092+0000","last_peered":"2024-09-10T07:49:56.238092+0000","last_clean":"2024-09-10T07:49:56.238092+0000","last_became_active":"2024-09-10T07:49:54.230499+0000","last_became_peered":"2024-09-10T07:49:54.230499+0000","last_unstale":"2024-09-10T07:49:56.238092+0000","last_undegraded":"2024-09-10T07:49:56.238092+0000","last_fullsized":"2024-09-10T07:49:56.238092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:45.500563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238062+0000","last_change":"2024-09-10T07:49:55.237422+0000","last_active":"2024-09-10T07:49:56.238062+0000","last_peered":"2024-09-10T07:49:56.238062+0000","last_clean":"2024-09-10T07:49:56.238062+0000","last_became_active":"2024-09-10T07:49:55.236954+0000","last_became_peered":"2024-09-10T07:49:55.236954+0000","last_unstale":"2024-09-10T07:49:56.238062+0000","last_undegraded":"2024-09-10T07:49:56.238062+0000","last_fullsized":"2024-09-10T07:49:56.238062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.298086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.237725+0000","last_change":"2024-09-10T07:49:54.228023+0000","last_active":"2024-09-10T07:49:56.237725+0000","last_peered":"2024-09-10T07:49:56.237725+0000","last_clean":"2024-09-10T07:49:56.237725+0000","last_became_active":"2024-09-10T07:49:54.227304+0000","last_became_peered":"2024-09-10T07:49:54.227304+0000","last_unstale":"2024-09-10T07:49:56.237725+0000","last_undegraded":"2024-09-10T07:49:56.237725+0000","last_fullsized":"2024-09-10T07:49:56.237725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:05.124949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314348+0000","last_change":"2024-09-10T07:49:55.234248+0000","last_active":"2024-09-10T07:49:56.314348+0000","last_peered":"2024-09-10T07:49:56.314348+0000","last_clean":"2024-09-10T07:49:56.314348+0000","last_became_active":"2024-09-10T07:49:55.233949+0000","last_became_peered":"2024-09-10T07:49:55.233949+0000","last_unstale":"2024-09-10T07:49:56.314348+0000","last_undegraded":"2024-09-10T07:49:56.314348+0000","last_fullsized":"2024-09-10T07:49:56.314348+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:36.497697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.249125+0000","last_change":"2024-09-10T07:49:54.235097+0000","last_active":"2024-09-10T07:49:55.249125+0000","last_peered":"2024-09-10T07:49:55.249125+0000","last_clean":"2024-09-10T07:49:55.249125+0000","last_became_active":"2024-09-10T07:49:54.234799+0000","last_became_peered":"2024-09-10T07:49:54.234799+0000","last_unstale":"2024-09-10T07:49:55.249125+0000","last_undegraded":"2024-09-10T07:49:55.249125+0000","last_fullsized":"2024-09-10T07:49:55.249125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:06.748639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314645+0000","last_change":"2024-09-10T07:49:55.242190+0000","last_active":"2024-09-10T07:49:56.314645+0000","last_peered":"2024-09-10T07:49:56.314645+0000","last_clean":"2024-09-10T07:49:56.314645+0000","last_became_active":"2024-09-10T07:49:55.241254+0000","last_became_peered":"2024-09-10T07:49:55.241254+0000","last_unstale":"2024-09-10T07:49:56.314645+0000","last_undegraded":"2024-09-10T07:49:56.314645+0000","last_fullsized":"2024-09-10T07:49:56.314645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:19.761738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.250456+0000","last_change":"2024-09-10T07:49:54.230270+0000","last_active":"2024-09-10T07:49:55.250456+0000","last_peered":"2024-09-10T07:49:55.250456+0000","last_clean":"2024-09-10T07:49:55.250456+0000","last_became_active":"2024-09-10T07:49:54.230006+0000","last_became_peered":"2024-09-10T07:49:54.230006+0000","last_unstale":"2024-09-10T07:49:55.250456+0000","last_undegraded":"2024-09-10T07:49:55.250456+0000","last_fullsized":"2024-09-10T07:49:55.250456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:30.161593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244355+0000","last_change":"2024-09-10T07:49:55.236047+0000","last_active":"2024-09-10T07:49:55.244355+0000","last_peered":"2024-09-10T07:49:55.244355+0000","last_clean":"2024-09-10T07:49:55.244355+0000","last_became_active":"2024-09-10T07:49:55.235751+0000","last_became_peered":"2024-09-10T07:49:55.235751+0000","last_unstale":"2024-09-10T07:49:55.244355+0000","last_undegraded":"2024-09-10T07:49:55.244355+0000","last_fullsized":"2024-09-10T07:49:55.244355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314355+0000","last_change":"2024-09-10T07:49:54.229486+0000","last_active":"2024-09-10T07:49:56.314355+0000","last_peered":"2024-09-10T07:49:56.314355+0000","last_clean":"2024-09-10T07:49:56.314355+0000","last_became_active":"2024-09-10T07:49:54.229194+0000","last_became_peered":"2024-09-10T07:49:54.229194+0000","last_unstale":"2024-09-10T07:49:56.314355+0000","last_undegraded":"2024-09-10T07:49:56.314355+0000","last_fullsized":"2024-09-10T07:49:56.314355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:55.896433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238309+0000","last_change":"2024-09-10T07:49:55.228256+0000","last_active":"2024-09-10T07:49:56.238309+0000","last_peered":"2024-09-10T07:49:56.238309+0000","last_clean":"2024-09-10T07:49:56.238309+0000","last_became_active":"2024-09-10T07:49:55.227981+0000","last_became_peered":"2024-09-10T07:49:55.227981+0000","last_unstale":"2024-09-10T07:49:56.238309+0000","last_undegraded":"2024-09-10T07:49:56.238309+0000","last_fullsized":"2024-09-10T07:49:56.238309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:41.110878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.225851+0000","last_change":"2024-09-10T07:49:54.225953+0000","last_active":"2024-09-10T07:49:55.225851+0000","last_peered":"2024-09-10T07:49:55.225851+0000","last_clean":"2024-09-10T07:49:55.225851+0000","last_became_active":"2024-09-10T07:49:54.224988+0000","last_became_peered":"2024-09-10T07:49:54.224988+0000","last_unstale":"2024-09-10T07:49:55.225851+0000","last_undegraded":"2024-09-10T07:49:55.225851+0000","last_fullsized":"2024-09-10T07:49:55.225851+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:41.261630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238688+0000","last_change":"2024-09-10T07:49:55.232189+0000","last_active":"2024-09-10T07:49:56.238688+0000","last_peered":"2024-09-10T07:49:56.238688+0000","last_clean":"2024-09-10T07:49:56.238688+0000","last_became_active":"2024-09-10T07:49:55.231928+0000","last_became_peered":"2024-09-10T07:49:55.231928+0000","last_unstale":"2024-09-10T07:49:56.238688+0000","last_undegraded":"2024-09-10T07:49:56.238688+0000","last_fullsized":"2024-09-10T07:49:56.238688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:51.617140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238329+0000","last_change":"2024-09-10T07:49:54.229635+0000","last_active":"2024-09-10T07:49:56.238329+0000","last_peered":"2024-09-10T07:49:56.238329+0000","last_clean":"2024-09-10T07:49:56.238329+0000","last_became_active":"2024-09-10T07:49:54.229374+0000","last_became_peered":"2024-09-10T07:49:54.229374+0000","last_unstale":"2024-09-10T07:49:56.238329+0000","last_undegraded":"2024-09-10T07:49:56.238329+0000","last_fullsized":"2024-09-10T07:49:56.238329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:13.082458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.238812+0000","last_change":"2024-09-10T07:49:55.232739+0000","last_active":"2024-09-10T07:49:55.238812+0000","last_peered":"2024-09-10T07:49:55.238812+0000","last_clean":"2024-09-10T07:49:55.238812+0000","last_became_active":"2024-09-10T07:49:55.228224+0000","last_became_peered":"2024-09-10T07:49:55.228224+0000","last_unstale":"2024-09-10T07:49:55.238812+0000","last_undegraded":"2024-09-10T07:49:55.238812+0000","last_fullsized":"2024-09-10T07:49:55.238812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:15.138119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.224754+0000","last_change":"2024-09-10T07:49:54.231286+0000","last_active":"2024-09-10T07:49:55.224754+0000","last_peered":"2024-09-10T07:49:55.224754+0000","last_clean":"2024-09-10T07:49:55.224754+0000","last_became_active":"2024-09-10T07:49:54.229132+0000","last_became_peered":"2024-09-10T07:49:54.229132+0000","last_unstale":"2024-09-10T07:49:55.224754+0000","last_undegraded":"2024-09-10T07:49:55.224754+0000","last_fullsized":"2024-09-10T07:49:55.224754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:27.710429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244934+0000","last_change":"2024-09-10T07:49:55.236978+0000","last_active":"2024-09-10T07:49:55.244934+0000","last_peered":"2024-09-10T07:49:55.244934+0000","last_clean":"2024-09-10T07:49:55.244934+0000","last_became_active":"2024-09-10T07:49:55.236540+0000","last_became_peered":"2024-09-10T07:49:55.236540+0000","last_unstale":"2024-09-10T07:49:55.244934+0000","last_undegraded":"2024-09-10T07:49:55.244934+0000","last_fullsized":"2024-09-10T07:49:55.244934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:39.998603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238731+0000","last_change":"2024-09-10T07:49:54.233652+0000","last_active":"2024-09-10T07:49:56.238731+0000","last_peered":"2024-09-10T07:49:56.238731+0000","last_clean":"2024-09-10T07:49:56.238731+0000","last_became_active":"2024-09-10T07:49:54.232434+0000","last_became_peered":"2024-09-10T07:49:54.232434+0000","last_unstale":"2024-09-10T07:49:56.238731+0000","last_undegraded":"2024-09-10T07:49:56.238731+0000","last_fullsized":"2024-09-10T07:49:56.238731+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:33.589981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314431+0000","last_change":"2024-09-10T07:49:55.240967+0000","last_active":"2024-09-10T07:49:56.314431+0000","last_peered":"2024-09-10T07:49:56.314431+0000","last_clean":"2024-09-10T07:49:56.314431+0000","last_became_active":"2024-09-10T07:49:55.240651+0000","last_became_peered":"2024-09-10T07:49:55.240651+0000","last_unstale":"2024-09-10T07:49:56.314431+0000","last_undegraded":"2024-09-10T07:49:56.314431+0000","last_fullsized":"2024-09-10T07:49:56.314431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:46.287208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"63'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.248889+0000","last_change":"2024-09-10T07:49:54.232977+0000","last_active":"2024-09-10T07:49:55.248889+0000","last_peered":"2024-09-10T07:49:55.248889+0000","last_clean":"2024-09-10T07:49:55.248889+0000","last_became_active":"2024-09-10T07:49:54.232717+0000","last_became_peered":"2024-09-10T07:49:54.232717+0000","last_unstale":"2024-09-10T07:49:55.248889+0000","last_undegraded":"2024-09-10T07:49:55.248889+0000","last_fullsized":"2024-09-10T07:49:55.248889+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:48.126076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314759+0000","last_change":"2024-09-10T07:49:55.240941+0000","last_active":"2024-09-10T07:49:56.314759+0000","last_peered":"2024-09-10T07:49:56.314759+0000","last_clean":"2024-09-10T07:49:56.314759+0000","last_became_active":"2024-09-10T07:49:55.240613+0000","last_became_peered":"2024-09-10T07:49:55.240613+0000","last_unstale":"2024-09-10T07:49:56.314759+0000","last_undegraded":"2024-09-10T07:49:56.314759+0000","last_fullsized":"2024-09-10T07:49:56.314759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:39.998603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217709+0000","last_change":"2024-09-10T07:49:54.236283+0000","last_active":"2024-09-10T07:49:55.217709+0000","last_peered":"2024-09-10T07:49:55.217709+0000","last_clean":"2024-09-10T07:49:55.217709+0000","last_became_active":"2024-09-10T07:49:54.235705+0000","last_became_peered":"2024-09-10T07:49:54.235705+0000","last_unstale":"2024-09-10T07:49:55.217709+0000","last_undegraded":"2024-09-10T07:49:55.217709+0000","last_fullsized":"2024-09-10T07:49:55.217709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:08.318713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235941+0000","last_change":"2024-09-10T07:49:55.227769+0000","last_active":"2024-09-10T07:49:55.235941+0000","last_peered":"2024-09-10T07:49:55.235941+0000","last_clean":"2024-09-10T07:49:55.235941+0000","last_became_active":"2024-09-10T07:49:55.227479+0000","last_became_peered":"2024-09-10T07:49:55.227479+0000","last_unstale":"2024-09-10T07:49:55.235941+0000","last_undegraded":"2024-09-10T07:49:55.235941+0000","last_fullsized":"2024-09-10T07:49:55.235941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:59.407886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.239036+0000","last_change":"2024-09-10T07:49:54.236202+0000","last_active":"2024-09-10T07:49:56.239036+0000","last_peered":"2024-09-10T07:49:56.239036+0000","last_clean":"2024-09-10T07:49:56.239036+0000","last_became_active":"2024-09-10T07:49:54.235905+0000","last_became_peered":"2024-09-10T07:49:54.235905+0000","last_unstale":"2024-09-10T07:49:56.239036+0000","last_undegraded":"2024-09-10T07:49:56.239036+0000","last_fullsized":"2024-09-10T07:49:56.239036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:15.836910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.243510+0000","last_change":"2024-09-10T07:49:55.236751+0000","last_active":"2024-09-10T07:49:55.243510+0000","last_peered":"2024-09-10T07:49:55.243510+0000","last_clean":"2024-09-10T07:49:55.243510+0000","last_became_active":"2024-09-10T07:49:55.236334+0000","last_became_peered":"2024-09-10T07:49:55.236334+0000","last_unstale":"2024-09-10T07:49:55.243510+0000","last_undegraded":"2024-09-10T07:49:55.243510+0000","last_fullsized":"2024-09-10T07:49:55.243510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:19.761738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314135+0000","last_change":"2024-09-10T07:49:54.226043+0000","last_active":"2024-09-10T07:49:56.314135+0000","last_peered":"2024-09-10T07:49:56.314135+0000","last_clean":"2024-09-10T07:49:56.314135+0000","last_became_active":"2024-09-10T07:49:54.225742+0000","last_became_peered":"2024-09-10T07:49:54.225742+0000","last_unstale":"2024-09-10T07:49:56.314135+0000","last_undegraded":"2024-09-10T07:49:56.314135+0000","last_fullsized":"2024-09-10T07:49:56.314135+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:17.208468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.241538+0000","last_change":"2024-09-10T07:49:55.233403+0000","last_active":"2024-09-10T07:49:55.241538+0000","last_peered":"2024-09-10T07:49:55.241538+0000","last_clean":"2024-09-10T07:49:55.241538+0000","last_became_active":"2024-09-10T07:49:55.233063+0000","last_became_peered":"2024-09-10T07:49:55.233063+0000","last_unstale":"2024-09-10T07:49:55.241538+0000","last_undegraded":"2024-09-10T07:49:55.241538+0000","last_fullsized":"2024-09-10T07:49:55.241538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:52.677291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217758+0000","last_change":"2024-09-10T07:49:54.235515+0000","last_active":"2024-09-10T07:49:55.217758+0000","last_peered":"2024-09-10T07:49:55.217758+0000","last_clean":"2024-09-10T07:49:55.217758+0000","last_became_active":"2024-09-10T07:49:54.234632+0000","last_became_peered":"2024-09-10T07:49:54.234632+0000","last_unstale":"2024-09-10T07:49:55.217758+0000","last_undegraded":"2024-09-10T07:49:55.217758+0000","last_fullsized":"2024-09-10T07:49:55.217758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:59.763343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.315055+0000","last_change":"2024-09-10T07:49:55.235236+0000","last_active":"2024-09-10T07:49:56.315055+0000","last_peered":"2024-09-10T07:49:56.315055+0000","last_clean":"2024-09-10T07:49:56.315055+0000","last_became_active":"2024-09-10T07:49:55.234856+0000","last_became_peered":"2024-09-10T07:49:55.234856+0000","last_unstale":"2024-09-10T07:49:56.315055+0000","last_undegraded":"2024-09-10T07:49:56.315055+0000","last_fullsized":"2024-09-10T07:49:56.315055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:21.517463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314079+0000","last_change":"2024-09-10T07:49:54.231251+0000","last_active":"2024-09-10T07:49:56.314079+0000","last_peered":"2024-09-10T07:49:56.314079+0000","last_clean":"2024-09-10T07:49:56.314079+0000","last_became_active":"2024-09-10T07:49:54.230977+0000","last_became_peered":"2024-09-10T07:49:54.230977+0000","last_unstale":"2024-09-10T07:49:56.314079+0000","last_undegraded":"2024-09-10T07:49:56.314079+0000","last_fullsized":"2024-09-10T07:49:56.314079+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:01.906396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.244817+0000","last_change":"2024-09-10T07:49:55.239481+0000","last_active":"2024-09-10T07:49:55.244817+0000","last_peered":"2024-09-10T07:49:55.244817+0000","last_clean":"2024-09-10T07:49:55.244817+0000","last_became_active":"2024-09-10T07:49:55.235239+0000","last_became_peered":"2024-09-10T07:49:55.235239+0000","last_unstale":"2024-09-10T07:49:55.244817+0000","last_undegraded":"2024-09-10T07:49:55.244817+0000","last_fullsized":"2024-09-10T07:49:55.244817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:38.113241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.217412+0000","last_change":"2024-09-10T07:49:54.230722+0000","last_active":"2024-09-10T07:49:55.217412+0000","last_peered":"2024-09-10T07:49:55.217412+0000","last_clean":"2024-09-10T07:49:55.217412+0000","last_became_active":"2024-09-10T07:49:54.230413+0000","last_became_peered":"2024-09-10T07:49:54.230413+0000","last_unstale":"2024-09-10T07:49:55.217412+0000","last_undegraded":"2024-09-10T07:49:55.217412+0000","last_fullsized":"2024-09-10T07:49:55.217412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:33.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.235345+0000","last_change":"2024-09-10T07:49:55.230369+0000","last_active":"2024-09-10T07:49:55.235345+0000","last_peered":"2024-09-10T07:49:55.235345+0000","last_clean":"2024-09-10T07:49:55.235345+0000","last_became_active":"2024-09-10T07:49:55.230103+0000","last_became_peered":"2024-09-10T07:49:55.230103+0000","last_unstale":"2024-09-10T07:49:55.235345+0000","last_undegraded":"2024-09-10T07:49:55.235345+0000","last_fullsized":"2024-09-10T07:49:55.235345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:08.921569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.224788+0000","last_change":"2024-09-10T07:49:54.226293+0000","last_active":"2024-09-10T07:49:55.224788+0000","last_peered":"2024-09-10T07:49:55.224788+0000","last_clean":"2024-09-10T07:49:55.224788+0000","last_became_active":"2024-09-10T07:49:54.225955+0000","last_became_peered":"2024-09-10T07:49:54.225955+0000","last_unstale":"2024-09-10T07:49:55.224788+0000","last_undegraded":"2024-09-10T07:49:55.224788+0000","last_fullsized":"2024-09-10T07:49:55.224788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:49.959392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T07:49:55.239360+0000","last_change":"2024-09-10T07:49:55.229860+0000","last_active":"2024-09-10T07:49:55.239360+0000","last_peered":"2024-09-10T07:49:55.239360+0000","last_clean":"2024-09-10T07:49:55.239360+0000","last_became_active":"2024-09-10T07:49:55.229545+0000","last_became_peered":"2024-09-10T07:49:55.229545+0000","last_unstale":"2024-09-10T07:49:55.239360+0000","last_undegraded":"2024-09-10T07:49:55.239360+0000","last_fullsized":"2024-09-10T07:49:55.239360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:54.201757+0000","last_clean_scrub_stamp":"2024-09-10T07:49:54.201757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:17.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.314774+0000","last_change":"2024-09-10T07:49:54.238177+0000","last_active":"2024-09-10T07:49:56.314774+0000","last_peered":"2024-09-10T07:49:56.314774+0000","last_clean":"2024-09-10T07:49:56.314774+0000","last_became_active":"2024-09-10T07:49:54.237506+0000","last_became_peered":"2024-09-10T07:49:54.237506+0000","last_unstale":"2024-09-10T07:49:56.314774+0000","last_undegraded":"2024-09-10T07:49:56.314774+0000","last_fullsized":"2024-09-10T07:49:56.314774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:49:53.197418+0000","last_clean_scrub_stamp":"2024-09-10T07:49:53.197418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:01.906396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":245,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T07:49:56.238781+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:49:56.238781+0000","last_peered":"2024-09-10T07:49:56.238781+0000","last_clean":"2024-09-10T07:49:56.238781+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:49:56.238781+0000","last_undegraded":"2024-09-10T07:49:56.238781+0000","last_fullsized":"2024-09-10T07:49:56.238781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476783,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":749568,"data_stored":313739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":765952,"data_stored":315421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476781,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1343488,"data_stored":904599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":757760,"data_stored":315261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":753664,"data_stored":313741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":733184,"data_stored":306874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1343488,"data_stored":904051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":765952,"data_stored":320565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:49:59.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:49:59.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:50:00.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.068+0000 7fc0efa00640 1 Processor -- start 2024-09-10T07:50:00.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.068+0000 7fc0efa00640 1 -- start start 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0efa00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0e80b4c90 0x7fc0e815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0efa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e815eff0 con 0x7fc0e8072c50 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0efa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e815f2b0 con 0x7fc0e815b230 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0efa00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e815fbe0 con 0x7fc0e80b4c90 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0eda00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc0e8072c50 0x7fc0e8071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43080/0 (socket says 172.21.15.3:43080) 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.069+0000 7fc0eda00640 1 -- 172.21.15.3:0/2164049258 learned_addr learned my addr 172.21.15.3:0/2164049258 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:00.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0dfe00640 1 --2- 172.21.15.3:0/2164049258 >> v2:172.21.15.70:3300/0 conn(0x7fc0e80b4c90 0x7fc0e815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0e7e00640 1 -- 172.21.15.3:0/2164049258 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526337446 0 0) 0x7fc0e815eff0 con 0x7fc0e8072c50 2024-09-10T07:50:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0e7e00640 1 -- 172.21.15.3:0/2164049258 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0cc003520 con 0x7fc0e8072c50 2024-09-10T07:50:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0dfe00640 1 -- 172.21.15.3:0/2164049258 >> v1:172.21.15.70:6789/0 conn(0x7fc0e815b230 legacy=0x7fc0e80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0dfe00640 1 -- 172.21.15.3:0/2164049258 >> v1:172.21.15.3:6789/0 conn(0x7fc0e8072c50 legacy=0x7fc0e8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.070+0000 7fc0dfe00640 1 -- 172.21.15.3:0/2164049258 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e815f590 con 0x7fc0e80b4c90 2024-09-10T07:50:00.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.071+0000 7fc0dfe00640 1 --2- 172.21.15.3:0/2164049258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e815c390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328166ae50f3b10c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0e7e00640 1 -- 172.21.15.3:0/2164049258 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0037070 con 0x7fc0e80b4c90 2024-09-10T07:50:00.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0e7e00640 1 -- 172.21.15.3:0/2164049258 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d0032440 con 0x7fc0e80b4c90 2024-09-10T07:50:00.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0e7e00640 1 -- 172.21.15.3:0/2164049258 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0032810 con 0x7fc0e80b4c90 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 -- 172.21.15.3:0/2164049258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 msgr2=0x7fc0e815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2164049258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e815c390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002a230 comp rx=0 tx=0).stop 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 -- 172.21.15.3:0/2164049258 shutdown_connections 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2164049258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e815c390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 -- 172.21.15.3:0/2164049258 >> 172.21.15.3:0/2164049258 conn(0x7fc0e80b5cb0 msgr2=0x7fc0e806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 -- 172.21.15.3:0/2164049258 shutdown_connections 2024-09-10T07:50:00.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.072+0000 7fc0efa00640 1 -- 172.21.15.3:0/2164049258 wait complete. 2024-09-10T07:50:00.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.074+0000 7fc0efa00640 1 Processor -- start 2024-09-10T07:50:00.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.074+0000 7fc0efa00640 1 -- start start 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0e8072c50 0x7fc0e814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e815b610 con 0x7fc0e815b230 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e815c8d0 con 0x7fc0e80b4c90 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0efa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e815b790 con 0x7fc0e8072c50 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40230/0 (socket says 172.21.15.3:40230) 2024-09-10T07:50:00.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.075+0000 7fc0eda00640 1 -- 172.21.15.3:0/2847138004 learned_addr learned my addr 172.21.15.3:0/2847138004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0ed000640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0e8072c50 0x7fc0e814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 -- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0e8072c50 msgr2=0x7fc0e814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0e8072c50 0x7fc0e814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 -- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 msgr2=0x7fc0e8159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0dfe00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e8159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 -- 172.21.15.3:0/2847138004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e814f380 con 0x7fc0e815b230 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0dfe00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0eda00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000cad0 tx=0x7fc0e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0017070 con 0x7fc0e815b230 2024-09-10T07:50:00.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e000ac30 con 0x7fc0e815b230 2024-09-10T07:50:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0012400 con 0x7fc0e815b230 2024-09-10T07:50:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.076+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e8150e30 con 0x7fc0e815b230 2024-09-10T07:50:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.077+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e8157a90 con 0x7fc0e815b230 2024-09-10T07:50:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.078+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e806ea30 con 0x7fc0e815b230 2024-09-10T07:50:00.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.079+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0e001a070 con 0x7fc0e815b230 2024-09-10T07:50:00.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.080+0000 7fc0e5600640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 0x7fc0c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.080+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fc0e008ac40 con 0x7fc0e815b230 2024-09-10T07:50:00.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.080+0000 7fc0ed000640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 0x7fc0c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.081+0000 7fc0ed000640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 0x7fc0c406e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8000f60 tx=0x7fc0d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.084+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0e00595f0 con 0x7fc0e815b230 2024-09-10T07:50:00.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.338+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc0e80b4200 con 0x7fc0e815b230 2024-09-10T07:50:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.340+0000 7fc0e5600640 1 -- 172.21.15.3:0/2847138004 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fc0e0058ef0 con 0x7fc0e815b230 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:50:00.340 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:50:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.342+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 msgr2=0x7fc0c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.342+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 0x7fc0c406e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8000f60 tx=0x7fc0d8009040 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.342+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 msgr2=0x7fc0e81522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.342+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000cad0 tx=0x7fc0e000cf90 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.342+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 shutdown_connections 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc0c406be30 0x7fc0c406e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0e815b230 0x7fc0e81522f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e80b4c90 0x7fc0e8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 --2- 172.21.15.3:0/2847138004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0e8072c50 0x7fc0e814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 >> 172.21.15.3:0/2847138004 conn(0x7fc0e80b5cb0 msgr2=0x7fc0e80714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 shutdown_connections 2024-09-10T07:50:00.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.343+0000 7fc0efa00640 1 -- 172.21.15.3:0/2847138004 wait complete. 2024-09-10T07:50:00.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.517+0000 7f5d11200640 1 Processor -- start 2024-09-10T07:50:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.517+0000 7f5d11200640 1 -- start start 2024-09-10T07:50:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d11200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d0c06d870 0x7f5d0c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d11200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c1603c0 con 0x7f5d0c15e4f0 2024-09-10T07:50:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d11200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c06c0c0 con 0x7f5d0c06bb20 2024-09-10T07:50:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d11200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5d0c06ccd0 con 0x7f5d0c06d870 2024-09-10T07:50:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d0a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d0c06d870 0x7f5d0c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d09e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d0c15e4f0 0x7f5d0c15fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43090/0 (socket says 172.21.15.3:43090) 2024-09-10T07:50:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d09e00640 1 -- 172.21.15.3:0/4213157637 learned_addr learned my addr 172.21.15.3:0/4213157637 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d0a800640 1 -- 172.21.15.3:0/4213157637 >> v1:172.21.15.70:6789/0 conn(0x7f5d0c06bb20 legacy=0x7f5d0c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d0a800640 1 -- 172.21.15.3:0/4213157637 >> v1:172.21.15.3:6789/0 conn(0x7f5d0c15e4f0 legacy=0x7f5d0c15fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d0a800640 1 -- 172.21.15.3:0/4213157637 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c06c3a0 con 0x7f5d0c06d870 2024-09-10T07:50:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.519+0000 7f5d09400640 1 -- 172.21.15.3:0/4213157637 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424130175 0 0) 0x7f5d0c1603c0 con 0x7f5d0c15e4f0 2024-09-10T07:50:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.520+0000 7f5d0a800640 1 --2- 172.21.15.3:0/4213157637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c15f570 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009d90 tx=0x7f5d0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27df1858b02ebf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d09400640 1 -- 172.21.15.3:0/4213157637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04007740 con 0x7f5d0c06d870 2024-09-10T07:50:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d09400640 1 -- 172.21.15.3:0/4213157637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d040078e0 con 0x7f5d0c06d870 2024-09-10T07:50:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d09400640 1 -- 172.21.15.3:0/4213157637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04007cb0 con 0x7f5d0c06d870 2024-09-10T07:50:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 -- 172.21.15.3:0/4213157637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 msgr2=0x7f5d0c15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 --2- 172.21.15.3:0/4213157637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c15f570 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009d90 tx=0x7f5d0402a6f0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 -- 172.21.15.3:0/4213157637 shutdown_connections 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 --2- 172.21.15.3:0/4213157637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c15f570 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 -- 172.21.15.3:0/4213157637 >> 172.21.15.3:0/4213157637 conn(0x7f5d0c06e5b0 msgr2=0x7f5d0c06f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 -- 172.21.15.3:0/4213157637 shutdown_connections 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.521+0000 7f5d11200640 1 -- 172.21.15.3:0/4213157637 wait complete. 2024-09-10T07:50:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.523+0000 7f5d11200640 1 Processor -- start 2024-09-10T07:50:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.523+0000 7f5d11200640 1 -- start start 2024-09-10T07:50:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 0x7f5d0c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d0c15e4f0 0x7f5d0c1e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c15e8d0 con 0x7f5d0c06d870 2024-09-10T07:50:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c06c960 con 0x7f5d0c06bb20 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d11200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c0b6dc0 con 0x7f5d0c15e4f0 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40252/0 (socket says 172.21.15.3:40252) 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d03e00640 1 -- 172.21.15.3:0/971073437 learned_addr learned my addr 172.21.15.3:0/971073437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d09e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 0x7f5d0c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.524+0000 7f5d0a800640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d0c15e4f0 0x7f5d0c1e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 -- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d0c15e4f0 msgr2=0x7f5d0c1e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d0c15e4f0 0x7f5d0c1e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 -- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 msgr2=0x7f5d0c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 0x7f5d0c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 -- 172.21.15.3:0/971073437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c159c70 con 0x7f5d0c06d870 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d09e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 0x7f5d0c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d03e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400c450 tx=0x7f5cf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40052d0 con 0x7f5d0c06d870 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf4005470 con 0x7f5d0c06d870 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c159f50 con 0x7f5d0c06d870 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.525+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4012870 con 0x7f5d0c06d870 2024-09-10T07:50:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.526+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c15a510 con 0x7f5d0c06d870 2024-09-10T07:50:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.526+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c15a710 con 0x7f5d0c06d870 2024-09-10T07:50:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.528+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cf4012a10 con 0x7f5d0c06d870 2024-09-10T07:50:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.528+0000 7f5d02000640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 0x7f5ce806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.529+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf408a9b0 con 0x7f5d0c06d870 2024-09-10T07:50:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.529+0000 7f5d09e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 0x7f5ce806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.529+0000 7f5d09e00640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 0x7f5ce806e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000f60 tx=0x7f5cfc007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:00.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.536+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d0c15a710 con 0x7f5d0c06d870 2024-09-10T07:50:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.755+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d0c0b2f50 con 0x7f5d0c06d870 2024-09-10T07:50:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.756+0000 7f5d02000640 1 -- 172.21.15.3:0/971073437 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5217 (secure 0 0 0) 0x7f5cf4058c70 con 0x7f5d0c06d870 2024-09-10T07:50:00.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:00.756 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":51,"btime":"2024-09-10T07:49:58:246472+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6111,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:49:57.665936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114]},"id":4}]} 2024-09-10T07:50:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 51 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 msgr2=0x7f5ce806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 0x7f5ce806e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000f60 tx=0x7f5cfc007e40 comp rx=0 tx=0).stop 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 msgr2=0x7f5d0c1591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400c450 tx=0x7f5cf400c910 comp rx=0 tx=0).stop 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 shutdown_connections 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5ce806be60 0x7f5ce806e320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d0c15e4f0 0x7f5d0c1e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d0c06d870 0x7f5d0c1591e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 --2- 172.21.15.3:0/971073437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0c06bb20 0x7f5d0c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 >> 172.21.15.3:0/971073437 conn(0x7f5d0c06e5b0 msgr2=0x7f5d0c0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 shutdown_connections 2024-09-10T07:50:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:00.758+0000 7f5d11200640 1 -- 172.21.15.3:0/971073437 wait complete. 2024-09-10T07:50:00.803 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:50:00.803+0000 7fe5eaa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:50:01.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:49:54.647917+0000', 'modified': '2024-09-10T07:49:57.665936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6114}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6114': {'gid': 6114, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6836/1359877653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1359877653}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 1359877653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6114, 'qdb_cluster': [6114]} 2024-09-10T07:50:01.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:50:01.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:50:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.341+0000 7fd768200640 1 Processor -- start 2024-09-10T07:50:01.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.341+0000 7fd768200640 1 -- start start 2024-09-10T07:50:01.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.342+0000 7fd768200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd760161b30 0x7fd76015f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.342+0000 7fd768200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd76015fdb0 0x7fd76006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.342+0000 7fd768200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76015fbb0 con 0x7fd760160220 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.342+0000 7fd768200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600b36a0 con 0x7fd760161b30 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.342+0000 7fd768200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600b3980 con 0x7fd76015fdb0 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd760161b30 0x7fd76015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd760161b30 0x7fd76015f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40264/0 (socket says 172.21.15.3:40264) 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 -- 172.21.15.3:0/1298020653 learned_addr learned my addr 172.21.15.3:0/1298020653 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd764e00640 1 --2- 172.21.15.3:0/1298020653 >> v2:172.21.15.70:3300/0 conn(0x7fd76015fdb0 0x7fd76006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 -- 172.21.15.3:0/1298020653 >> v2:172.21.15.70:3300/0 conn(0x7fd76015fdb0 msgr2=0x7fd76006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 --2- 172.21.15.3:0/1298020653 >> v2:172.21.15.70:3300/0 conn(0x7fd76015fdb0 0x7fd76006bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 -- 172.21.15.3:0/1298020653 >> v1:172.21.15.70:6789/0 conn(0x7fd760160220 legacy=0x7fd7600b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.343+0000 7fd765800640 1 -- 172.21.15.3:0/1298020653 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600b3c60 con 0x7fd760161b30 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.344+0000 7fd765800640 1 --2- 172.21.15.3:0/1298020653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760161b30 0x7fd76015f670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd76006cbd0 tx=0x7fd75402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24729db32fd4d76d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.344+0000 7fd75fe00640 1 -- 172.21.15.3:0/1298020653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754037070 con 0x7fd760161b30 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.344+0000 7fd75fe00640 1 -- 172.21.15.3:0/1298020653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd754032440 con 0x7fd760161b30 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.344+0000 7fd75fe00640 1 -- 172.21.15.3:0/1298020653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754032810 con 0x7fd760161b30 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 -- 172.21.15.3:0/1298020653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760161b30 msgr2=0x7fd76015f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 --2- 172.21.15.3:0/1298020653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760161b30 0x7fd76015f670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd76006cbd0 tx=0x7fd75402a270 comp rx=0 tx=0).stop 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 -- 172.21.15.3:0/1298020653 shutdown_connections 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 --2- 172.21.15.3:0/1298020653 >> v2:172.21.15.70:3300/0 conn(0x7fd76015fdb0 0x7fd76006bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 --2- 172.21.15.3:0/1298020653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760161b30 0x7fd76015f670 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 -- 172.21.15.3:0/1298020653 >> 172.21.15.3:0/1298020653 conn(0x7fd76006d000 msgr2=0x7fd76006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 -- 172.21.15.3:0/1298020653 shutdown_connections 2024-09-10T07:50:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.345+0000 7fd768200640 1 -- 172.21.15.3:0/1298020653 wait complete. 2024-09-10T07:50:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.347+0000 7fd768200640 1 Processor -- start 2024-09-10T07:50:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.347+0000 7fd768200640 1 -- start start 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd76015fdb0 0x7fd7600b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 0x7fd760071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd760161f10 con 0x7fd760160220 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b4220 con 0x7fd76015fdb0 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd768200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7600aefa0 con 0x7fd760161b30 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd764e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd764e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40274/0 (socket says 172.21.15.3:40274) 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd764e00640 1 -- 172.21.15.3:0/412833321 learned_addr learned my addr 172.21.15.3:0/412833321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd765800640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd76015fdb0 0x7fd7600b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd766200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 0x7fd760071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd764e00640 1 -- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 msgr2=0x7fd760071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.348+0000 7fd764e00640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 0x7fd760071790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd764e00640 1 -- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd76015fdb0 msgr2=0x7fd7600b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd764e00640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd76015fdb0 0x7fd7600b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd764e00640 1 -- 172.21.15.3:0/412833321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600714a0 con 0x7fd760160220 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd766200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 0x7fd760071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd764e00640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00d620 tx=0x7fd74c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd75d600640 1 -- 172.21.15.3:0/412833321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00b620 con 0x7fd760160220 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd75d600640 1 -- 172.21.15.3:0/412833321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74c019070 con 0x7fd760160220 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd760070f00 con 0x7fd760160220 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd75d600640 1 -- 172.21.15.3:0/412833321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00ba00 con 0x7fd760160220 2024-09-10T07:50:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.349+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7600b1eb0 con 0x7fd760160220 2024-09-10T07:50:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.351+0000 7fd75d600640 1 -- 172.21.15.3:0/412833321 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd74c00bba0 con 0x7fd760160220 2024-09-10T07:50:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.352+0000 7fd75d600640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 0x7fd74006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.352+0000 7fd75d600640 1 -- 172.21.15.3:0/412833321 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fd74c08b1b0 con 0x7fd760160220 2024-09-10T07:50:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.352+0000 7fd765800640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 0x7fd74006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.353+0000 7fd76b24c740 1 Processor -- start 2024-09-10T07:50:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.353+0000 7fd76b24c740 1 -- start start 2024-09-10T07:50:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.353+0000 7fd765800640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 0x7fd74006e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd754034000 tx=0x7fd75402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 0x5636af54b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 0x5636af545570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636af4a93b0 con 0x5636af54acf0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5636af4ae410 con 0x5636af5484b0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636af456620 con 0x5636af54c480 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd765800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd765800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40278/0 (socket says 172.21.15.3:40278) 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd765800640 1 -- 172.21.15.3:0/2706148848 learned_addr learned my addr 172.21.15.3:0/2706148848 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd764e00640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 0x5636af54b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.354+0000 7fd766200640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 0x5636af545570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 -- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 msgr2=0x5636af545570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 0x5636af545570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 -- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 msgr2=0x5636af54b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 0x5636af54b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 -- 172.21.15.3:0/2706148848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636af1c12d0 con 0x5636af54acf0 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd766200640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 0x5636af545570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd764e00640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 0x5636af54b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd765800640 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd754009930 tx=0x7fd75402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5791931ace5cb73a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd752000640 1 -- 172.21.15.3:0/2706148848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754037040 con 0x5636af54acf0 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd752000640 1 -- 172.21.15.3:0/2706148848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75404e070 con 0x5636af54acf0 2024-09-10T07:50:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.355+0000 7fd752000640 1 -- 172.21.15.3:0/2706148848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754032df0 con 0x5636af54acf0 2024-09-10T07:50:01.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 -- 172.21.15.3:0/2706148848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 msgr2=0x5636af590490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd754009930 tx=0x7fd75402c000 comp rx=0 tx=0).stop 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 -- 172.21.15.3:0/2706148848 shutdown_connections 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54c480 0x5636af545570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af5484b0 0x5636af54b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2706148848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af54acf0 0x5636af590490 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 -- 172.21.15.3:0/2706148848 >> 172.21.15.3:0/2706148848 conn(0x5636af547740 msgr2=0x5636af58f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 -- 172.21.15.3:0/2706148848 shutdown_connections 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.356+0000 7fd76b24c740 1 -- 172.21.15.3:0/2706148848 wait complete. 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.357+0000 7fd76b24c740 1 -- start start 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.357+0000 7fd76b24c740 1 Processor -- start 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af589740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636af469780 con 0x5636af574670 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5636af4aa3d0 con 0x5636af54c480 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636af546be0 con 0x5636af54acf0 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd76b24c740 10 client.0 _ll_register_callbacks cb 0x5636af4a75d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40290/0 (socket says 172.21.15.3:40290) 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd766200640 1 -- 172.21.15.3:0/2530458826 learned_addr learned my addr 172.21.15.3:0/2530458826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd764e00640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.358+0000 7fd765800640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af589740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 -- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 msgr2=0x5636af589740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af589740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 -- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 msgr2=0x5636af58cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 -- 172.21.15.3:0/2530458826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636af28b6b0 con 0x5636af574670 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd764e00640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd766200640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd758003eb0 tx=0x7fd758001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e98a3c8b3eff69c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd746a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75801b060 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fd758016b10 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd75404e070 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd7580124d0 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd76b24c740 10 client.6225 fetch_fsmap learned FSMap version 51 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.359+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5636af5735d0 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75404e070 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fd758012670 con 0x5636af574670 2024-09-10T07:50:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 10 client.6225 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 10 client.6225 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 20 client.6225 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af5a03d0 0x5636af5ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 4 client.6225 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636af4bbb40 con 0x5636af5a03d0 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.360+0000 7fd765800640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af5a03d0 0x5636af5ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.361+0000 7fd765800640 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af5a03d0 0x5636af5ae4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52dc29539ccb083a server_cookie=ff2a5a336e6a4542 in_seq=0 out_seq=0 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.361+0000 7fd746a00640 10 client.6225 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.361+0000 7fd746a00640 1 -- 172.21.15.3:0/2530458826 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636af4bbb40 con 0x5636af5a03d0 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.361+0000 7fd746a00640 10 client.6225 handle_command_reply: tid=0 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 client.6225 shutdown 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 msgr2=0x5636af544780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd758003eb0 tx=0x7fd758001af0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 shutdown_connections 2024-09-10T07:50:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af5a03d0 0x5636af5ae4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af544780 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2530458826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af589740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd746a00640 0 client.6225 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 >> 172.21.15.3:0/2530458826 conn(0x5636af5484b0 msgr2=0x5636af58eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 shutdown_connections 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 1 -- 172.21.15.3:0/2530458826 wait complete. 2024-09-10T07:50:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.362+0000 7fd76b24c740 20 client.6225 trim_cache size 0 max 16384 2024-09-10T07:50:01.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.381+0000 7fd76b24c740 1 Processor -- start 2024-09-10T07:50:01.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.381+0000 7fd76b24c740 1 -- start start 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af58d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd760200b00 con 0x5636af574670 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601f8830 con 0x5636af54c480 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd7601f7b90 con 0x5636af54acf0 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40306/0 (socket says 172.21.15.3:40306) 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 -- 172.21.15.3:0/4219681475 learned_addr learned my addr 172.21.15.3:0/4219681475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd764e00640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd765800640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af58d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 -- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 msgr2=0x5636af58d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af58d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 -- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 msgr2=0x5636af58df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd766200640 1 -- 172.21.15.3:0/4219681475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636af589a90 con 0x5636af574670 2024-09-10T07:50:01.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.382+0000 7fd764e00640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd766200640 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd758003a50 tx=0x7fd758010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2ab7c8de655cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd751600640 1 -- 172.21.15.3:0/4219681475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758003bd0 con 0x5636af574670 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd751600640 1 -- 172.21.15.3:0/4219681475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7580053f0 con 0x5636af574670 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd751600640 1 -- 172.21.15.3:0/4219681475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758012670 con 0x5636af574670 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 -- 172.21.15.3:0/4219681475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 msgr2=0x5636af58eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd758003a50 tx=0x7fd758010000 comp rx=0 tx=0).stop 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 -- 172.21.15.3:0/4219681475 shutdown_connections 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af58eec0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af58df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 --2- 172.21.15.3:0/4219681475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af58d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.383+0000 7fd76b24c740 1 -- 172.21.15.3:0/4219681475 >> 172.21.15.3:0/4219681475 conn(0x5636af5484b0 msgr2=0x5636af544530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.384+0000 7fd76b24c740 1 -- 172.21.15.3:0/4219681475 shutdown_connections 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.384+0000 7fd76b24c740 1 -- 172.21.15.3:0/4219681475 wait complete. 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.384+0000 7fd76b24c740 1 -- start start 2024-09-10T07:50:01.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.384+0000 7fd76b24c740 1 Processor -- start 2024-09-10T07:50:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af5a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636af574150 con 0x5636af574670 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601f81e0 con 0x5636af54c480 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636af574b40 con 0x5636af54acf0 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.385+0000 7fd76b24c740 10 client.0 _ll_register_callbacks cb 0x7fd758003af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd764e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd765800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af5a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd764e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:59280/0 (socket says 172.21.15.3:59280) 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40322/0 (socket says 172.21.15.3:40322) 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 -- 172.21.15.3:0/2980928269 learned_addr learned my addr 172.21.15.3:0/2980928269 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd765800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af5a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34252/0 (socket says 172.21.15.3:34252) 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 -- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 msgr2=0x5636af5a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af5a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 -- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 msgr2=0x5636af60ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 -- 172.21.15.3:0/2980928269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636af545cd0 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd764e00640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd766200640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd7580058a0 tx=0x7fd758013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996d2fe13912a49b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.386+0000 7fd746a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75801b070 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7580053f0 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fd758007040 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd758007240 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 10 client.6231 fetch_fsmap learned FSMap version 51 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5636af588440 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7580053f0 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fd7580073e0 con 0x5636af574670 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 10 client.6231 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T07:50:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 10 client.6231 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 20 client.6231 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af56d1b0 0x5636af570130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 4 client.6231 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.387+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "status"}) -- 0x5636af5474e0 con 0x5636af56d1b0 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd765800640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af56d1b0 0x5636af570130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd765800640 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af56d1b0 0x5636af570130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b58b2c84b20a529 server_cookie=df11b11488be6460 in_seq=0 out_seq=0 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd746a00640 10 client.6231 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd746a00640 1 -- 172.21.15.3:0/2980928269 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5636af5474e0 con 0x5636af56d1b0 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd746a00640 10 client.6231 handle_command_reply: tid=0 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.388+0000 7fd76b24c740 1 client.6231 shutdown 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 msgr2=0x5636af56cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd7580058a0 tx=0x7fd758013000 comp rx=0 tx=0).stop 2024-09-10T07:50:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 shutdown_connections 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5636af56d1b0 0x5636af570130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636af574670 0x5636af56cc70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636af54c480 0x5636af60ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 --2- 172.21.15.3:0/2980928269 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636af54acf0 0x5636af5a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd746a00640 0 client.6231 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 >> 172.21.15.3:0/2980928269 conn(0x5636af5484b0 msgr2=0x5636af59ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 shutdown_connections 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 1 -- 172.21.15.3:0/2980928269 wait complete. 2024-09-10T07:50:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd76b24c740 20 client.6231 trim_cache size 0 max 16384 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6114, 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 6.1773239819999999, 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 51, 2024-09-10T07:50:01.394 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 64, 2024-09-10T07:50:01.395 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 64, 2024-09-10T07:50:01.395 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 11.256618614000001 2024-09-10T07:50:01.395 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:01.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 msgr2=0x7fd74006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 0x7fd74006e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd754034000 tx=0x7fd75402e000 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 msgr2=0x7fd7600736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.389+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00d620 tx=0x7fd74c00dae0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 shutdown_connections 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd74006be60 0x7fd74006e320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd760161b30 0x7fd760071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd760160220 0x7fd7600736a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 --2- 172.21.15.3:0/412833321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd76015fdb0 0x7fd7600b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 >> 172.21.15.3:0/412833321 conn(0x7fd76006d000 msgr2=0x7fd7601594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 shutdown_connections 2024-09-10T07:50:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.390+0000 7fd768200640 1 -- 172.21.15.3:0/412833321 wait complete. 2024-09-10T07:50:01.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:50:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.562+0000 7f2510600640 1 Processor -- start 2024-09-10T07:50:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.563+0000 7f2510600640 1 -- start start 2024-09-10T07:50:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f2510600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2508160710 0x7f25080cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f2510600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f2510600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25080cfdd0 con 0x7f2508160af0 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f2510600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f250815cc90 con 0x7f2508160710 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f2510600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f250815cf70 con 0x7f25080d07c0 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f250dc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2508160710 0x7f25080cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f250e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f250e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34266/0 (socket says 172.21.15.3:34266) 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f250e600640 1 -- 172.21.15.3:0/3959499454 learned_addr learned my addr 172.21.15.3:0/3959499454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.564+0000 7f250d200640 1 --1- 172.21.15.3:0/3959499454 >> v1:172.21.15.70:6789/0 conn(0x7f2508160af0 0x7f25080d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:56954/0 (socket says 172.21.15.3:56954) 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.565+0000 7f250e600640 1 -- 172.21.15.3:0/3959499454 >> v2:172.21.15.70:3300/0 conn(0x7f2508160710 msgr2=0x7f25080cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.565+0000 7f250e600640 1 --2- 172.21.15.3:0/3959499454 >> v2:172.21.15.70:3300/0 conn(0x7f2508160710 0x7f25080cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.565+0000 7f250e600640 1 -- 172.21.15.3:0/3959499454 >> v1:172.21.15.70:6789/0 conn(0x7f2508160af0 legacy=0x7f25080d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.565+0000 7f250e600640 1 -- 172.21.15.3:0/3959499454 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250815d250 con 0x7f25080d07c0 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.566+0000 7f250e600640 1 --2- 172.21.15.3:0/3959499454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2500009e50 tx=0x7f250002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4510369942a160aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.566+0000 7f2507e00640 1 -- 172.21.15.3:0/3959499454 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500036070 con 0x7f25080d07c0 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.566+0000 7f2507e00640 1 -- 172.21.15.3:0/3959499454 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2500031600 con 0x7f25080d07c0 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.566+0000 7f2507e00640 1 -- 172.21.15.3:0/3959499454 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000319d0 con 0x7f25080d07c0 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 -- 172.21.15.3:0/3959499454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 msgr2=0x7f2508072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 --2- 172.21.15.3:0/3959499454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2500009e50 tx=0x7f250002a710 comp rx=0 tx=0).stop 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 -- 172.21.15.3:0/3959499454 shutdown_connections 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 --2- 172.21.15.3:0/3959499454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f2508072070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 --2- 172.21.15.3:0/3959499454 >> v2:172.21.15.70:3300/0 conn(0x7f2508160710 0x7f25080cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 -- 172.21.15.3:0/3959499454 >> 172.21.15.3:0/3959499454 conn(0x7f250806d000 msgr2=0x7f250806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 -- 172.21.15.3:0/3959499454 shutdown_connections 2024-09-10T07:50:01.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.567+0000 7f2510600640 1 -- 172.21.15.3:0/3959499454 wait complete. 2024-09-10T07:50:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.568+0000 7f2510600640 1 Processor -- start 2024-09-10T07:50:01.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.569+0000 7f2510600640 1 -- start start 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f25081524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2508160710 0x7f25080ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f250815d770 con 0x7f2508160af0 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25080cf800 con 0x7f2508160710 2024-09-10T07:50:01.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f2510600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25080b4430 con 0x7f25080d07c0 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40328/0 (socket says 172.21.15.3:40328) 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 -- 172.21.15.3:0/1392228377 learned_addr learned my addr 172.21.15.3:0/1392228377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250dc00640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f25081524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250d200640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2508160710 0x7f25080ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 -- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 msgr2=0x7f25081524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f25081524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 -- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2508160710 msgr2=0x7f25080ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2508160710 0x7f25080ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.570+0000 7f250e600640 1 -- 172.21.15.3:0/1392228377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25080b4710 con 0x7f2508160af0 2024-09-10T07:50:01.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f250e600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2500034040 tx=0x7f2500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500036050 con 0x7f2508160af0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2500042070 con 0x7f2508160af0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500009510 con 0x7f2508160af0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25080b49f0 con 0x7f2508160af0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.571+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25080b4fb0 con 0x7f2508160af0 2024-09-10T07:50:01.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.572+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25080afd50 con 0x7f2508160af0 2024-09-10T07:50:01.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.573+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2500007bf0 con 0x7f2508160af0 2024-09-10T07:50:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.574+0000 7f2505600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 0x7f24e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.574+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f25000abad0 con 0x7f2508160af0 2024-09-10T07:50:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.574+0000 7f250dc00640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 0x7f24e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:01.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.575+0000 7f250dc00640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 0x7f24e806e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009b40 tx=0x7f24fc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.578+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2500079f50 con 0x7f2508160af0 2024-09-10T07:50:01.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.801+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2508070df0 con 0x7f2508160af0 2024-09-10T07:50:01.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.801+0000 7f2505600640 1 -- 172.21.15.3:0/1392228377 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f250007e250 con 0x7f2508160af0 2024-09-10T07:50:01.801 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 msgr2=0x7f24e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 0x7f24e806e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009b40 tx=0x7f24fc009190 comp rx=0 tx=0).stop 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 msgr2=0x7f25080b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2500034040 tx=0x7f2500004a40 comp rx=0 tx=0).stop 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 shutdown_connections 2024-09-10T07:50:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f24e806be60 0x7f24e806e320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2508160af0 0x7f25080b3be0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2508160710 0x7f25080ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 --2- 172.21.15.3:0/1392228377 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25080d07c0 0x7f25081524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 >> 172.21.15.3:0/1392228377 conn(0x7f250806d000 msgr2=0x7f25080cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.803+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 shutdown_connections 2024-09-10T07:50:01.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:01.804+0000 7f2510600640 1 -- 172.21.15.3:0/1392228377 wait complete. 2024-09-10T07:50:01.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:50:01.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:50:01.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:50:01.820 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:50:01.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:01.820 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:50:01.856 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42494sec preferred_lft 42494sec 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:50:01.857 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:50:01.858 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:50:01.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:01.859 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:50:01.950 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:50:01.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:01.953 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:50:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:50:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:50:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:50:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:50:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:50:01.990 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:50:02.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:50:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:02.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:02.187 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:50:02.221 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:50:02.222 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:50:02.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:50:02.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:02.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:50:02.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:50:02.320 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:50:02.355 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:50:02.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:50:02.355 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:50:02.431 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:50:02.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:02.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:50:02.560 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:50:02.560 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:50:02.560 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:50:02.576 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:50:02.576 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:50:02.576 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:50:02.576 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:50:02.576 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:50:02.598 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:50:02.598 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:50:02.598 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:50:02.598 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:50:02.598 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:50:02.599 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:50:02.599 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:50:02.599 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:50:02.600 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:50:02.600 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:50:02.600 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:50:02.600 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:50:02.600 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:50:02.603 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:50:02.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:02.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:50:02.661 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:50:02.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:50:02.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:02.759 INFO:teuthology.orchestra.run.smithi003.stdout:client5591 2024-09-10T07:50:02.759 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:50:02.759 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:02.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:50:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.994+0000 7face8c00640 1 Processor -- start 2024-09-10T07:50:02.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.994+0000 7face8c00640 1 -- start start 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.995+0000 7face8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 0x7face40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.995+0000 7face8c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face40d4cb0 0x7face40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.995+0000 7face8c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face40d3de0 con 0x7face40d3860 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.995+0000 7face8c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7face4161f50 con 0x7face40d4cb0 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.995+0000 7face8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7face40d40c0 con 0x7face4161b70 2024-09-10T07:50:02.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face2a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7face40d3860 0x7face40d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43102/0 (socket says 172.21.15.3:43102) 2024-09-10T07:50:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face2a00640 1 -- 172.21.15.3:0/2387677321 learned_addr learned my addr 172.21.15.3:0/2387677321 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3e00640 1 --2- 172.21.15.3:0/2387677321 >> v2:172.21.15.3:3300/0 conn(0x7face40d4cb0 0x7face40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3400640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 0x7face40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:02.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face2000640 1 -- 172.21.15.3:0/2387677321 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485671854 0 0) 0x7face40d3de0 con 0x7face40d3860 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face2000640 1 -- 172.21.15.3:0/2387677321 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facc8003520 con 0x7face40d3860 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3e00640 1 -- 172.21.15.3:0/2387677321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 msgr2=0x7face40d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3e00640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 0x7face40d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3e00640 1 -- 172.21.15.3:0/2387677321 >> v1:172.21.15.3:6789/0 conn(0x7face40d3860 legacy=0x7face40d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3e00640 1 -- 172.21.15.3:0/2387677321 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face40cf690 con 0x7face40d4cb0 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.996+0000 7face3400640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 0x7face40d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:02.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.997+0000 7face3e00640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d4cb0 0x7face40cfe90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7facdc009d90 tx=0x7facdc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874e65f43ad7ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.997+0000 7face2000640 1 -- 172.21.15.3:0/2387677321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc007740 con 0x7face40d4cb0 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.997+0000 7face2000640 1 -- 172.21.15.3:0/2387677321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc0078e0 con 0x7face40d4cb0 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.997+0000 7face2000640 1 -- 172.21.15.3:0/2387677321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc007c90 con 0x7face40d4cb0 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 -- 172.21.15.3:0/2387677321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d4cb0 msgr2=0x7face40cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d4cb0 0x7face40cfe90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7facdc009d90 tx=0x7facdc02a660 comp rx=0 tx=0).stop 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 -- 172.21.15.3:0/2387677321 shutdown_connections 2024-09-10T07:50:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d4cb0 0x7face40cfe90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 --2- 172.21.15.3:0/2387677321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face4161b70 0x7face40d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 -- 172.21.15.3:0/2387677321 >> 172.21.15.3:0/2387677321 conn(0x7face4062630 msgr2=0x7face4062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 -- 172.21.15.3:0/2387677321 shutdown_connections 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:02.998+0000 7face8c00640 1 -- 172.21.15.3:0/2387677321 wait complete. 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.000+0000 7face8c00640 1 Processor -- start 2024-09-10T07:50:02.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.000+0000 7face8c00640 1 -- start start 2024-09-10T07:50:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 0x7face41f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face4161b70 0x7face4144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7face40cf2d0 con 0x7face40d3860 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7face40c9fc0 con 0x7face4161b70 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7face40cf450 con 0x7face40d4cb0 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40348/0 (socket says 172.21.15.3:40348) 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.001+0000 7face3400640 1 -- 172.21.15.3:0/4028519452 learned_addr learned my addr 172.21.15.3:0/4028519452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face2a00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 0x7face41f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3e00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face4161b70 0x7face4144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 -- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 msgr2=0x7face41f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 0x7face41f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 -- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face4161b70 msgr2=0x7face4144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face4161b70 0x7face4144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 -- 172.21.15.3:0/4028519452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face415a9c0 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face2a00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 0x7face41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face3400640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7facd400e990 tx=0x7facd400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4017070 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd400c9e0 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd400cdb0 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.002+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face4156e30 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.003+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face40ce4a0 con 0x7face40d3860 2024-09-10T07:50:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.004+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face40b7540 con 0x7face40d3860 2024-09-10T07:50:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.006+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facd401a070 con 0x7face40d3860 2024-09-10T07:50:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.006+0000 7facdb400640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 0x7facc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.007+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7facd408ad70 con 0x7face40d3860 2024-09-10T07:50:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.007+0000 7face2a00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 0x7facc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.007+0000 7face2a00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 0x7facc406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7face40ca520 tx=0x7facd0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:03.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.011+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7facd4059710 con 0x7face40d3860 2024-09-10T07:50:03.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.232+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7face40d0940 con 0x7face40d3860 2024-09-10T07:50:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.273+0000 7facdb400640 1 -- 172.21.15.3:0/4028519452 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v52) ==== 105+0+0 (secure 0 0 0) 0x7facd4059010 con 0x7face40d3860 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 msgr2=0x7facc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 0x7facc406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7face40ca520 tx=0x7facd0009040 comp rx=0 tx=0).stop 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 msgr2=0x7face41f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7facd400e990 tx=0x7facd400ee50 comp rx=0 tx=0).stop 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 shutdown_connections 2024-09-10T07:50:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7facc406be30 0x7facc406e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face4161b70 0x7face4144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7face40d4cb0 0x7face41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 --2- 172.21.15.3:0/4028519452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face40d3860 0x7face41f73f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 >> 172.21.15.3:0/4028519452 conn(0x7face4062630 msgr2=0x7face41605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.276+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 shutdown_connections 2024-09-10T07:50:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.277+0000 7face8c00640 1 -- 172.21.15.3:0/4028519452 wait complete. 2024-09-10T07:50:03.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:03.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.459+0000 7fed66800640 1 Processor -- start 2024-09-10T07:50:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.459+0000 7fed66800640 1 -- start start 2024-09-10T07:50:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 0x7fed6006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed66800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed600b36a0 con 0x7fed601601f0 2024-09-10T07:50:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed66800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed600b3980 con 0x7fed60161b30 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed600b3c60 con 0x7fed6015fdb0 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed5fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed601601f0 0x7fed600b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43108/0 (socket says 172.21.15.3:43108) 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed5fe00640 1 -- 172.21.15.3:0/2880854692 learned_addr learned my addr 172.21.15.3:0/2880854692 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.461+0000 7fed5ea00640 1 --2- 172.21.15.3:0/2880854692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 0x7fed6006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.462+0000 7fed5ea00640 1 -- 172.21.15.3:0/2880854692 >> v1:172.21.15.70:6789/0 conn(0x7fed60161b30 legacy=0x7fed6015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.462+0000 7fed5e000640 1 -- 172.21.15.3:0/2880854692 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616090840 0 0) 0x7fed600b3980 con 0x7fed60161b30 2024-09-10T07:50:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.462+0000 7fed5ea00640 1 -- 172.21.15.3:0/2880854692 >> v1:172.21.15.3:6789/0 conn(0x7fed601601f0 legacy=0x7fed600b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.462+0000 7fed5ea00640 1 -- 172.21.15.3:0/2880854692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600b3f40 con 0x7fed6015fdb0 2024-09-10T07:50:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.463+0000 7fed5ea00640 1 --2- 172.21.15.3:0/2880854692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 0x7fed6006bb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097d0 tx=0x7fed4c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d4a1c996428d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.463+0000 7fed5e000640 1 -- 172.21.15.3:0/2880854692 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c037070 con 0x7fed6015fdb0 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.463+0000 7fed5e000640 1 -- 172.21.15.3:0/2880854692 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4c02ad90 con 0x7fed6015fdb0 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.463+0000 7fed5e000640 1 -- 172.21.15.3:0/2880854692 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c032660 con 0x7fed6015fdb0 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 -- 172.21.15.3:0/2880854692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 msgr2=0x7fed6006bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 --2- 172.21.15.3:0/2880854692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 0x7fed6006bb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097d0 tx=0x7fed4c02a100 comp rx=0 tx=0).stop 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 -- 172.21.15.3:0/2880854692 shutdown_connections 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 --2- 172.21.15.3:0/2880854692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6015fdb0 0x7fed6006bb20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 -- 172.21.15.3:0/2880854692 >> 172.21.15.3:0/2880854692 conn(0x7fed6006d000 msgr2=0x7fed6006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 -- 172.21.15.3:0/2880854692 shutdown_connections 2024-09-10T07:50:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.464+0000 7fed66800640 1 -- 172.21.15.3:0/2880854692 wait complete. 2024-09-10T07:50:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.465+0000 7fed66800640 1 Processor -- start 2024-09-10T07:50:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.466+0000 7fed66800640 1 -- start start 2024-09-10T07:50:03.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.466+0000 7fed66800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed601601f0 0x7fed60159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed66800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed60161b30 0x7fed60144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed66800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed600aefa0 con 0x7fed6015fdb0 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed60161f10 con 0x7fed60161b30 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed66800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed600b3300 con 0x7fed601601f0 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40364/0 (socket says 172.21.15.3:40364) 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 -- 172.21.15.3:0/395103317 learned_addr learned my addr 172.21.15.3:0/395103317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5ea00640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed601601f0 0x7fed60159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5fe00640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed60161b30 0x7fed60144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 -- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed601601f0 msgr2=0x7fed60159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed601601f0 0x7fed60159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 -- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed60161b30 msgr2=0x7fed60144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed60161b30 0x7fed60144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.467+0000 7fed5f400640 1 -- 172.21.15.3:0/395103317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6015c250 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed5f400640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed50001eb0 tx=0x7fed5000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50017070 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50005480 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50003810 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed600cec70 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.468+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed600cf230 con 0x7fed6015fdb0 2024-09-10T07:50:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.469+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6015f670 con 0x7fed6015fdb0 2024-09-10T07:50:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.470+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed5001a070 con 0x7fed6015fdb0 2024-09-10T07:50:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.471+0000 7fed57400640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.471+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fed5008a260 con 0x7fed6015fdb0 2024-09-10T07:50:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.472+0000 7fed5ea00640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.472+0000 7fed5ea00640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:03.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.480+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed50058520 con 0x7fed6015fdb0 2024-09-10T07:50:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.700+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed600b2e30 con 0x7fed6015fdb0 2024-09-10T07:50:03.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.701+0000 7fed57400640 1 -- 172.21.15.3:0/395103317 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5233 (secure 0 0 0) 0x7fed5005c820 con 0x7fed6015fdb0 2024-09-10T07:50:03.701 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:03.701 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":53,"btime":"2024-09-10T07:50:03:277247+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:03.277230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6114,"mds_1":6111},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114]},"id":4}]} 2024-09-10T07:50:03.701 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 53 2024-09-10T07:50:03.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 msgr2=0x7fed4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 msgr2=0x7fed60143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed50001eb0 tx=0x7fed5000c9b0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 shutdown_connections 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed60161b30 0x7fed60144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed601601f0 0x7fed60159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 --2- 172.21.15.3:0/395103317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6015fdb0 0x7fed60143520 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.703+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 >> 172.21.15.3:0/395103317 conn(0x7fed6006d000 msgr2=0x7fed6015d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.704+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 shutdown_connections 2024-09-10T07:50:03.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:03.704+0000 7fed66800640 1 -- 172.21.15.3:0/395103317 wait complete. 2024-09-10T07:50:04.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:49:54.647917+0000', 'modified': '2024-09-10T07:50:03.277230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6114, 'mds_1': 6111}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6111': {'gid': 6111, 'name': 'a', 'rank': 1, 'incarnation': 53, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6837/1603892470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 1603892470}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 1603892470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6114': {'gid': 6114, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6836/1359877653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1359877653}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 1359877653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6114, 'qdb_cluster': [6114]} 2024-09-10T07:50:04.124 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6111:up:creating 2024-09-10T07:50:05.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:05.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.296+0000 7f1458000640 1 Processor -- start 2024-09-10T07:50:05.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.296+0000 7f1458000640 1 -- start start 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1458000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f14500730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1458000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1450073620 0x7f1450154e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1458000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14500b5d80 con 0x7f14500d0fd0 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1458000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f14501630e0 con 0x7f1450073620 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1458000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f14500b6060 con 0x7f1450162c80 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1455600640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14500d0fd0 0x7f14500b52c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51212/0 (socket says 172.21.15.3:51212) 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.297+0000 7f1455600640 1 -- 172.21.15.3:0/2244702709 learned_addr learned my addr 172.21.15.3:0/2244702709 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1456000640 1 --2- 172.21.15.3:0/2244702709 >> v2:172.21.15.3:3300/0 conn(0x7f1450073620 0x7f1450154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1454c00640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f14500730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f144fe00640 1 -- 172.21.15.3:0/2244702709 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121984598 0 0) 0x7f14500b5d80 con 0x7f14500d0fd0 2024-09-10T07:50:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f144fe00640 1 -- 172.21.15.3:0/2244702709 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1438003520 con 0x7f14500d0fd0 2024-09-10T07:50:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1456000640 1 -- 172.21.15.3:0/2244702709 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 msgr2=0x7f14500730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1456000640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f14500730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1456000640 1 -- 172.21.15.3:0/2244702709 >> v1:172.21.15.3:6789/0 conn(0x7f14500d0fd0 legacy=0x7f14500b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.298+0000 7f1456000640 1 -- 172.21.15.3:0/2244702709 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14501621a0 con 0x7f1450073620 2024-09-10T07:50:05.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.299+0000 7f1456000640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450154e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1448009dc0 tx=0x7f144802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149013d42bcb8e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:05.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.299+0000 7f144fe00640 1 -- 172.21.15.3:0/2244702709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144802ae60 con 0x7f1450073620 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.299+0000 7f144fe00640 1 -- 172.21.15.3:0/2244702709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448007740 con 0x7f1450073620 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.299+0000 7f144fe00640 1 -- 172.21.15.3:0/2244702709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448007b10 con 0x7f1450073620 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 -- 172.21.15.3:0/2244702709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 msgr2=0x7f1450154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450154e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1448009dc0 tx=0x7f144802a680 comp rx=0 tx=0).stop 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 -- 172.21.15.3:0/2244702709 shutdown_connections 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450154e60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 --2- 172.21.15.3:0/2244702709 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f14500730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 -- 172.21.15.3:0/2244702709 >> 172.21.15.3:0/2244702709 conn(0x7f145006bb20 msgr2=0x7f145006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 -- 172.21.15.3:0/2244702709 shutdown_connections 2024-09-10T07:50:05.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.300+0000 7f1458000640 1 -- 172.21.15.3:0/2244702709 wait complete. 2024-09-10T07:50:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.302+0000 7f1458000640 1 Processor -- start 2024-09-10T07:50:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.302+0000 7f1458000640 1 -- start start 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14500d0fd0 0x7f145009da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f145014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14500d13b0 con 0x7f1450073620 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1450162760 con 0x7f14500d0fd0 2024-09-10T07:50:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1458000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1450159b10 con 0x7f1450162c80 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1455600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1455600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52068/0 (socket says 172.21.15.3:52068) 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1455600640 1 -- 172.21.15.3:0/2383611706 learned_addr learned my addr 172.21.15.3:0/2383611706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1454c00640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14500d0fd0 0x7f145009da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1456000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f145014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.303+0000 7f1455600640 1 -- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 msgr2=0x7f145014c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1455600640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f145014c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1455600640 1 -- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14500d0fd0 msgr2=0x7f145009da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1455600640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14500d0fd0 0x7f145009da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1455600640 1 -- 172.21.15.3:0/2383611706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1450159df0 con 0x7f1450073620 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1456000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f145014c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:05.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1455600640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f144400c390 tx=0x7f144400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444017070 con 0x7f1450073620 2024-09-10T07:50:05.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1444005ce0 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440072f0 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145015a0d0 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.304+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1450159770 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.305+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145006f250 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.306+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f144401a070 con 0x7f1450073620 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.307+0000 7f144d600640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 0x7f143006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.307+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f144408a340 con 0x7f1450073620 2024-09-10T07:50:05.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.307+0000 7f1454c00640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 0x7f143006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:05.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.308+0000 7f1454c00640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 0x7f143006e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14501f7a80 tx=0x7f143c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.311+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1444058c50 con 0x7f1450073620 2024-09-10T07:50:05.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.533+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145015a460 con 0x7f1450073620 2024-09-10T07:50:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.533+0000 7f144d600640 1 -- 172.21.15.3:0/2383611706 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5255 (secure 0 0 0) 0x7f1444058550 con 0x7f1450073620 2024-09-10T07:50:05.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:05.533 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":56,"btime":"2024-09-10T07:50:05:287301+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:05.287295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[5857,6111,6114]},"id":4}]} 2024-09-10T07:50:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 56 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 msgr2=0x7f143006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 0x7f143006e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14501f7a80 tx=0x7f143c008040 comp rx=0 tx=0).stop 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 msgr2=0x7f1450159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f144400c390 tx=0x7f144400c850 comp rx=0 tx=0).stop 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 shutdown_connections 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f143006be30 0x7f143006e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1450162c80 0x7f145014c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14500d0fd0 0x7f145009da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 --2- 172.21.15.3:0/2383611706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1450073620 0x7f1450159230 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.535+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 >> 172.21.15.3:0/2383611706 conn(0x7f145006bb20 msgr2=0x7f14501608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.536+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 shutdown_connections 2024-09-10T07:50:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:05.536+0000 7f1458000640 1 -- 172.21.15.3:0/2383611706 wait complete. 2024-09-10T07:50:05.803 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:50:05.804+0000 7fe5eaa00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:50:05.804 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:50:05.804+0000 7fe5eaa00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T07:50:05.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:49:54.647917+0000', 'modified': '2024-09-10T07:50:05.287295+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6114, 'mds_1': 6111, 'mds_2': 5857}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5857': {'gid': 5857, 'name': 'd', 'rank': 2, 'incarnation': 55, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.70:6835/4215641179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 4215641179}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 4215641179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6111': {'gid': 6111, 'name': 'a', 'rank': 1, 'incarnation': 53, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6837/1603892470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 1603892470}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 1603892470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6114': {'gid': 6114, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6836/1359877653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1359877653}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 1359877653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6114, 'qdb_cluster': [5857, 6111, 6114]} 2024-09-10T07:50:05.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:50:05.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T07:50:06.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.114+0000 7f7697e00640 1 Processor -- start 2024-09-10T07:50:06.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.114+0000 7f7697e00640 1 -- start start 2024-09-10T07:50:06.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.115+0000 7f7697e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f769015fdb0 0x7f769006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.115+0000 7f7697e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76901601f0 0x7f769006c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.115+0000 7f7697e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76900b36a0 con 0x7f7690161b30 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.115+0000 7f7697e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f76900b3980 con 0x7f76901601f0 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.115+0000 7f7697e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76900b3c60 con 0x7f769015fdb0 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76901601f0 0x7f769006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76901601f0 0x7f769006c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52084/0 (socket says 172.21.15.3:52084) 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 -- 172.21.15.3:0/2370559631 learned_addr learned my addr 172.21.15.3:0/2370559631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7694a00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f769015fdb0 0x7f769006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 -- 172.21.15.3:0/2370559631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f769015fdb0 msgr2=0x7f769006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f769015fdb0 0x7f769006bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 -- 172.21.15.3:0/2370559631 >> v1:172.21.15.70:6789/0 conn(0x7f7690161b30 legacy=0x7f769015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.116+0000 7f7695e00640 1 -- 172.21.15.3:0/2370559631 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76900b3f40 con 0x7f76901601f0 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.117+0000 7f7695e00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76901601f0 0x7f769006c060 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7688009e50 tx=0x7f768802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87798c885c49475 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.117+0000 7f768fe00640 1 -- 172.21.15.3:0/2370559631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688036070 con 0x7f76901601f0 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f768fe00640 1 -- 172.21.15.3:0/2370559631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688031600 con 0x7f76901601f0 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f768fe00640 1 -- 172.21.15.3:0/2370559631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880319d0 con 0x7f76901601f0 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 -- 172.21.15.3:0/2370559631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76901601f0 msgr2=0x7f769006c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76901601f0 0x7f769006c060 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7688009e50 tx=0x7f768802a710 comp rx=0 tx=0).stop 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 -- 172.21.15.3:0/2370559631 shutdown_connections 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76901601f0 0x7f769006c060 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 --2- 172.21.15.3:0/2370559631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f769015fdb0 0x7f769006bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 -- 172.21.15.3:0/2370559631 >> 172.21.15.3:0/2370559631 conn(0x7f769006d000 msgr2=0x7f769006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 -- 172.21.15.3:0/2370559631 shutdown_connections 2024-09-10T07:50:06.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.118+0000 7f7697e00640 1 -- 172.21.15.3:0/2370559631 wait complete. 2024-09-10T07:50:06.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.120+0000 7f7697e00640 1 Processor -- start 2024-09-10T07:50:06.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.120+0000 7f7697e00640 1 -- start start 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76901601f0 0x7f769015b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7690161b30 0x7f769015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76900aefa0 con 0x7f769015fdb0 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7690161f10 con 0x7f7690161b30 2024-09-10T07:50:06.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7697e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76900b3300 con 0x7f76901601f0 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7695400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7695400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52090/0 (socket says 172.21.15.3:52090) 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7695400640 1 -- 172.21.15.3:0/3071361712 learned_addr learned my addr 172.21.15.3:0/3071361712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7695e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7690161b30 0x7f769015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.121+0000 7f7694a00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76901601f0 0x7f769015b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 -- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76901601f0 msgr2=0x7f769015b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76901601f0 0x7f769015b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 -- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7690161b30 msgr2=0x7f769015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7690161b30 0x7f769015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 -- 172.21.15.3:0/3071361712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769014d770 con 0x7f769015fdb0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7695400640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f768400c400 tx=0x7f768400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f768d600640 1 -- 172.21.15.3:0/3071361712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76840052d0 con 0x7f769015fdb0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f768d600640 1 -- 172.21.15.3:0/3071361712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684005490 con 0x7f769015fdb0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f768d600640 1 -- 172.21.15.3:0/3071361712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684007260 con 0x7f769015fdb0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.122+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76900ce9f0 con 0x7f769015fdb0 2024-09-10T07:50:06.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.123+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769014cae0 con 0x7f769015fdb0 2024-09-10T07:50:06.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.125+0000 7f768d600640 1 -- 172.21.15.3:0/3071361712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7684007400 con 0x7f769015fdb0 2024-09-10T07:50:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.125+0000 7f768d600640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 0x7f767006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.126+0000 7f768d600640 1 -- 172.21.15.3:0/3071361712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f768408a300 con 0x7f769015fdb0 2024-09-10T07:50:06.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.126+0000 7f7694a00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 0x7f767006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.126+0000 7f769acba740 1 Processor -- start 2024-09-10T07:50:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.126+0000 7f769acba740 1 -- start start 2024-09-10T07:50:06.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.126+0000 7f7694a00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 0x7f767006e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76900d0db0 tx=0x7f767c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488498c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648849444b0 0x5648849470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5648848a53b0 con 0x564884948480 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5648848aa410 con 0x5648849444b0 2024-09-10T07:50:06.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564884852620 con 0x564884946cf0 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f7695e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f7695e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52106/0 (socket says 172.21.15.3:52106) 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.127+0000 7f7695e00640 1 -- 172.21.15.3:0/2796986258 learned_addr learned my addr 172.21.15.3:0/2796986258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695400640 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488498c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7694a00640 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648849444b0 0x5648849470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 -- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 msgr2=0x56488498c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488498c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 -- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648849444b0 msgr2=0x5648849470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648849444b0 0x5648849470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 -- 172.21.15.3:0/2796986258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648845bd2d0 con 0x564884948480 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7695e00640 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7688009f70 tx=0x7f768802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8baaf8b6dcef218 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7681c00640 1 -- 172.21.15.3:0/2796986258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688036040 con 0x564884948480 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7681c00640 1 -- 172.21.15.3:0/2796986258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688037040 con 0x564884948480 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.128+0000 7f7681c00640 1 -- 172.21.15.3:0/2796986258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880319a0 con 0x564884948480 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 -- 172.21.15.3:0/2796986258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 msgr2=0x564884941570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7688009f70 tx=0x7f768802e000 comp rx=0 tx=0).stop 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 -- 172.21.15.3:0/2796986258 shutdown_connections 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884941570 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648849444b0 0x5648849470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 --2- 172.21.15.3:0/2796986258 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488498c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 -- 172.21.15.3:0/2796986258 >> 172.21.15.3:0/2796986258 conn(0x564884943740 msgr2=0x56488498b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 -- 172.21.15.3:0/2796986258 shutdown_connections 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.129+0000 7f769acba740 1 -- 172.21.15.3:0/2796986258 wait complete. 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.130+0000 7f769acba740 1 -- start start 2024-09-10T07:50:06.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.130+0000 7f769acba740 1 Processor -- start 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488499b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884968f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564884865780 con 0x564884948480 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5648848a63d0 con 0x56488496a4b0 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564884942be0 con 0x564884946cf0 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f769acba740 10 client.0 _ll_register_callbacks cb 0x5648848a35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52114/0 (socket says 172.21.15.3:52114) 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f7694a00640 1 -- 172.21.15.3:0/1638427446 learned_addr learned my addr 172.21.15.3:0/1638427446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f7695400640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488499b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.131+0000 7f7695e00640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884968f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 -- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 msgr2=0x56488499b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488499b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 -- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 msgr2=0x564884968f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884968f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 -- 172.21.15.3:0/1638427446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648846876b0 con 0x564884948480 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7695e00640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884968f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7694a00640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f767c0166e0 tx=0x7f767c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79079b4d60ad514b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7676000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c026020 con 0x564884948480 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7688037040 con 0x564884948480 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f767c014780 con 0x564884948480 2024-09-10T07:50:06.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f767c030050 con 0x564884948480 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f769acba740 10 client.6270 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.132+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x56488499bed0 con 0x564884948480 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688037040 con 0x564884948480 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7f767c033020 con 0x564884948480 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 10 client.6270 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 10 client.6270 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 20 client.6270 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x56488496ff20 0x564884971120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 4 client.6270 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648848b7b40 con 0x56488496ff20 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.133+0000 7f7695400640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x56488496ff20 0x564884971120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.134+0000 7f7695400640 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x56488496ff20 0x564884971120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbe8fe9baba68d19 server_cookie=16f62bb19b84cf03 in_seq=0 out_seq=0 2024-09-10T07:50:06.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.134+0000 7f7676000640 10 client.6270 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f7676000640 1 -- 172.21.15.3:0/1638427446 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648848b7b40 con 0x56488496ff20 2024-09-10T07:50:06.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f7676000640 10 client.6270 handle_command_reply: tid=0 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 client.6270 shutdown 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 msgr2=0x564884969f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f767c0166e0 tx=0x7f767c004730 comp rx=0 tx=0).stop 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 shutdown_connections 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x56488496ff20 0x564884971120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884968f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884969f70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 --2- 172.21.15.3:0/1638427446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x56488499b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f7676000640 0 client.6270 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.135+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 >> 172.21.15.3:0/1638427446 conn(0x5648849444b0 msgr2=0x5648849899e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.136+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 shutdown_connections 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.136+0000 7f769acba740 1 -- 172.21.15.3:0/1638427446 wait complete. 2024-09-10T07:50:06.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.136+0000 7f769acba740 20 client.6270 trim_cache size 0 max 16384 2024-09-10T07:50:06.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.154+0000 7f769acba740 1 Processor -- start 2024-09-10T07:50:06.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.154+0000 7f769acba740 1 -- start start 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 0x564884966ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 0x564884971350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76901fffd0 con 0x564884948480 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7690200540 con 0x564884946cf0 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.155+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7690157a20 con 0x56488496a4b0 2024-09-10T07:50:06.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52124/0 (socket says 172.21.15.3:52124) 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 -- 172.21.15.3:0/4163494378 learned_addr learned my addr 172.21.15.3:0/4163494378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7695e00640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 0x564884971350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7695400640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 0x564884966ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 -- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 msgr2=0x564884971350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 0x564884971350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 -- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 msgr2=0x564884966ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 0x564884966ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 -- 172.21.15.3:0/4163494378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648849434e0 con 0x564884948480 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7695400640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 0x564884966ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7695e00640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 0x564884971350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.156+0000 7f7694a00640 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f767c007fd0 tx=0x7f767c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86939bae6f57a187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f7681200640 1 -- 172.21.15.3:0/4163494378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c026040 con 0x564884948480 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f7681200640 1 -- 172.21.15.3:0/4163494378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c039070 con 0x564884948480 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f7681200640 1 -- 172.21.15.3:0/4163494378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c01c070 con 0x564884948480 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 -- 172.21.15.3:0/4163494378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 msgr2=0x564884989bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f767c007fd0 tx=0x7f767c021000 comp rx=0 tx=0).stop 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 -- 172.21.15.3:0/4163494378 shutdown_connections 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56488496a4b0 0x564884971350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884989bd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 --2- 172.21.15.3:0/4163494378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564884946cf0 0x564884966ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 -- 172.21.15.3:0/4163494378 >> 172.21.15.3:0/4163494378 conn(0x5648849444b0 msgr2=0x5648849889e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 -- 172.21.15.3:0/4163494378 shutdown_connections 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.157+0000 7f769acba740 1 -- 172.21.15.3:0/4163494378 wait complete. 2024-09-10T07:50:06.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.158+0000 7f769acba740 1 -- start start 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.158+0000 7f769acba740 1 Processor -- start 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x564884969aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884966a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564884967890 con 0x564884948480 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56488496fda0 con 0x56488496a4b0 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564884967e80 con 0x564884946cf0 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f769acba740 10 client.0 _ll_register_callbacks cb 0x7f767c010f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f7694a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52132/0 (socket says 172.21.15.3:52132) 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.159+0000 7f7694a00640 1 -- 172.21.15.3:0/4082488573 learned_addr learned my addr 172.21.15.3:0/4082488573 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 -- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 msgr2=0x564884969aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x564884969aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 -- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 msgr2=0x564884966a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884966a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 -- 172.21.15.3:0/4082488573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488499b3b0 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7694a00640 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f767c000c00 tx=0x7f767c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e483b7c6f521b10f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7676000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c041070 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f767c005ba0 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f767c039070 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f767c026050 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f769acba740 10 client.6276 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.160+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x564884a0d170 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c039070 con 0x564884948480 2024-09-10T07:50:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7f767c04f020 con 0x564884948480 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 10 client.6276 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 10 client.6276 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 20 client.6276 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x564884a0a2c0 0x564884999ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 4 client.6276 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "status"}) -- 0x5648848b1e90 con 0x564884a0a2c0 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.161+0000 7f7695400640 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x564884a0a2c0 0x564884999ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f7695400640 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x564884a0a2c0 0x564884999ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5b183c9bbc2ebe5 server_cookie=699e35987d2a3c2a in_seq=0 out_seq=0 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f7676000640 10 client.6276 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f7676000640 1 -- 172.21.15.3:0/4082488573 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5648848b1e90 con 0x564884a0a2c0 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f7676000640 10 client.6276 handle_command_reply: tid=0 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 client.6276 shutdown 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 msgr2=0x564884a090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f767c000c00 tx=0x7f767c015000 comp rx=0 tx=0).stop 2024-09-10T07:50:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 shutdown_connections 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x564884a0a2c0 0x564884999ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56488496a4b0 0x564884966a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564884948480 0x564884a090c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 --2- 172.21.15.3:0/4082488573 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564884946cf0 0x564884969aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f7676000640 0 client.6276 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 >> 172.21.15.3:0/4082488573 conn(0x5648849444b0 msgr2=0x5648849686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.162+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 shutdown_connections 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f769acba740 1 -- 172.21.15.3:0/4082488573 wait complete. 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f769acba740 20 client.6276 trim_cache size 0 max 16384 2024-09-10T07:50:06.211 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5857, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.875973004, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 56, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 64, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 64, 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.034836832 2024-09-10T07:50:06.212 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 msgr2=0x7f767006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 0x7f767006e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76900d0db0 tx=0x7f767c008040 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 msgr2=0x7f76900cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f768400c400 tx=0x7f768400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 shutdown_connections 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f767006be60 0x7f767006e320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7690161b30 0x7f769015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76901601f0 0x7f769015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 --2- 172.21.15.3:0/3071361712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f769015fdb0 0x7f76900cf4d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 >> 172.21.15.3:0/3071361712 conn(0x7f769006d000 msgr2=0x7f76900b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.163+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 shutdown_connections 2024-09-10T07:50:06.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.164+0000 7f7697e00640 1 -- 172.21.15.3:0/3071361712 wait complete. 2024-09-10T07:50:06.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:50:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.374+0000 7fa105a00640 1 Processor -- start 2024-09-10T07:50:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.374+0000 7fa105a00640 1 -- start start 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa105a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa1000b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa105a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1000b4250 con 0x7fa1000d08f0 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa105a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1000ccd80 con 0x7fa10015b4f0 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa105a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa1000b44f0 con 0x7fa10015b110 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa0ff400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1000d08f0 0x7fa1000cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51222/0 (socket says 172.21.15.3:51222) 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa0ff400640 1 -- 172.21.15.3:0/1526375581 learned_addr learned my addr 172.21.15.3:0/1526375581 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa1000b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751007869 0 0) 0x7fa1000b4250 con 0x7fa1000d08f0 2024-09-10T07:50:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.376+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e4003520 con 0x7fa1000d08f0 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3778531958 0 0) 0x7fa0e4003520 con 0x7fa1000d08f0 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 msgr2=0x7fa1000b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa1000b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 >> v1:172.21.15.70:6789/0 conn(0x7fa10015b4f0 legacy=0x7fa1000d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1000cd060 con 0x7fa1000d08f0 2024-09-10T07:50:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fa0f8001a10 con 0x7fa1000d08f0 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 >> v1:172.21.15.3:6789/0 conn(0x7fa1000d08f0 legacy=0x7fa1000cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40039e0 0x7fa0e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa0e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e4004d40 0x7fa0e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000b44f0 con 0x7fa0e4004d40 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e400ba70 con 0x7fa0e40039e0 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e400de30 con 0x7fa10015b110 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e4004d40 0x7fa0e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40039e0 0x7fa0e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.377+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa0e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.378+0000 7fa0ff400640 1 -- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 msgr2=0x7fa0e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.378+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa10015b110 0x7fa0e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.378+0000 7fa0ff400640 1 -- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40039e0 msgr2=0x7fa0e4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.378+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40039e0 0x7fa0e4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.378+0000 7fa0ff400640 1 -- 172.21.15.3:0/1526375581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e4003520 con 0x7fa0e4004d40 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e4004d40 0x7fa0e40051c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a780 tx=0x7fa0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcead2eb05d2ac06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800a3d0 con 0x7fa0e4004d40 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f802f070 con 0x7fa0e4004d40 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa0fe000640 1 -- 172.21.15.3:0/1526375581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f802bb60 con 0x7fa0e4004d40 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa105a00640 1 -- 172.21.15.3:0/1526375581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e4004d40 msgr2=0x7fa0e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa105a00640 1 --2- 172.21.15.3:0/1526375581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0e4004d40 0x7fa0e40051c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a780 tx=0x7fa0e4005830 comp rx=0 tx=0).stop 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.379+0000 7fa0ff400640 1 -- 172.21.15.3:0/1526375581 reap_dead start 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.380+0000 7fa105a00640 1 -- 172.21.15.3:0/1526375581 shutdown_connections 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.380+0000 7fa105a00640 1 -- 172.21.15.3:0/1526375581 >> 172.21.15.3:0/1526375581 conn(0x7fa10006d000 msgr2=0x7fa10006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.380+0000 7fa105a00640 1 -- 172.21.15.3:0/1526375581 shutdown_connections 2024-09-10T07:50:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.380+0000 7fa105a00640 1 -- 172.21.15.3:0/1526375581 wait complete. 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.381+0000 7fa105a00640 1 Processor -- start 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.381+0000 7fa105a00640 1 -- start start 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0e4004d40 0x7fa100146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 0x7fa1000ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa100073d10 con 0x7fa100158180 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000cd620 con 0x7fa10015e2f0 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa105a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa100073e90 con 0x7fa0e4004d40 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.382+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52164/0 (socket says 172.21.15.3:52164) 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 -- 172.21.15.3:0/3017901874 learned_addr learned my addr 172.21.15.3:0/3017901874 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0fea00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0e4004d40 0x7fa100146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 0x7fa1000ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 -- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0e4004d40 msgr2=0x7fa100146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0e4004d40 0x7fa100146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 -- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 msgr2=0x7fa1000ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 0x7fa1000ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 -- 172.21.15.3:0/3017901874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100146d90 con 0x7fa100158180 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 0x7fa1000ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0ff400640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a780 tx=0x7fa1000cd230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.383+0000 7fa0f6a00640 1 -- 172.21.15.3:0/3017901874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8044070 con 0x7fa100158180 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.384+0000 7fa0f6a00640 1 -- 172.21.15.3:0/3017901874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f80011c0 con 0x7fa100158180 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.384+0000 7fa0f6a00640 1 -- 172.21.15.3:0/3017901874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8001f20 con 0x7fa100158180 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.384+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10015e860 con 0x7fa100158180 2024-09-10T07:50:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.384+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10015ee20 con 0x7fa100158180 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.386+0000 7fa0f6a00640 1 -- 172.21.15.3:0/3017901874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0f8047070 con 0x7fa100158180 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.387+0000 7fa0f6a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 0x7fa0dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.387+0000 7fa0f6a00640 1 -- 172.21.15.3:0/3017901874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f80aadf0 con 0x7fa100158180 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.387+0000 7fa0fea00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 0x7fa0dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.388+0000 7fa1087fe740 1 Processor -- start 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.388+0000 7fa1087fe740 1 -- start start 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.388+0000 7fa0fea00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 0x7fa0dc06e320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009b00 tx=0x7fa0f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b9c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b544b0 0x55fbb7b570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7ab53b0 con 0x55fbb7b58480 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7aba410 con 0x55fbb7b56cf0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fbb7a62620 con 0x55fbb7b544b0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52168/0 (socket says 172.21.15.3:52168) 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 -- 172.21.15.3:0/2418191465 learned_addr learned my addr 172.21.15.3:0/2418191465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b544b0 0x55fbb7b570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0fea00640 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b9c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 -- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b544b0 msgr2=0x55fbb7b570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b544b0 0x55fbb7b570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 -- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 msgr2=0x55fbb7b9c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b9c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.389+0000 7fa0ff400640 1 -- 172.21.15.3:0/2418191465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb77cd2d0 con 0x55fbb7b58480 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa0ff400640 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8078a70 tx=0x7fa0f8079790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e10c1e5c4122592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa0ef400640 1 -- 172.21.15.3:0/2418191465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a7070 con 0x55fbb7b58480 2024-09-10T07:50:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa0ef400640 1 -- 172.21.15.3:0/2418191465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f802f020 con 0x55fbb7b58480 2024-09-10T07:50:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa0ef400640 1 -- 172.21.15.3:0/2418191465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80910d0 con 0x55fbb7b58480 2024-09-10T07:50:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa1087fe740 1 -- 172.21.15.3:0/2418191465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 msgr2=0x55fbb7b51570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.390+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8078a70 tx=0x7fa0f8079790 comp rx=0 tx=0).stop 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 -- 172.21.15.3:0/2418191465 shutdown_connections 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b58480 0x55fbb7b51570 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b544b0 0x55fbb7b570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2418191465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b9c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 -- 172.21.15.3:0/2418191465 >> 172.21.15.3:0/2418191465 conn(0x55fbb7b53740 msgr2=0x55fbb7b9b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 -- 172.21.15.3:0/2418191465 shutdown_connections 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 -- 172.21.15.3:0/2418191465 wait complete. 2024-09-10T07:50:06.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.391+0000 7fa1087fe740 1 -- start start 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.392+0000 7fa1087fe740 1 Processor -- start 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.392+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.392+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7b79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.392+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa1087fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7a75780 con 0x55fbb7b98ec0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7ab63d0 con 0x55fbb7b56cf0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fbb7aba410 con 0x55fbb7b58480 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa1087fe740 10 client.0 _ll_register_callbacks cb 0x55fbb7ab35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52182/0 (socket says 172.21.15.3:52182) 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 -- 172.21.15.3:0/1517250002 learned_addr learned my addr 172.21.15.3:0/1517250002 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7b79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 -- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 msgr2=0x55fbb7b79ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7b79ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 -- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 msgr2=0x55fbb7b77f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b77f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 -- 172.21.15.3:0/1517250002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb78976b0 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8081740 tx=0x7fa0f8009ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd51fc4a2e021f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.393+0000 7fa0e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a7040 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f802f020 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f807d050 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fa0f80910d0 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 10 client.6288 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55fbb7b784a0 con 0x55fbb7b98ec0 2024-09-10T07:50:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f802f020 con 0x55fbb7b98ec0 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7fa0f8092070 con 0x55fbb7b98ec0 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 10 client.6288 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 10 client.6288 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 20 client.6288 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7b7e6d0 0x55fbb7b7eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 4 client.6288 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.394+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb7ac7b40 con 0x55fbb7b7e6d0 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.395+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7b7e6d0 0x55fbb7b7eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.395+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7b7e6d0 0x55fbb7b7eab0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9fc9b37e6764856 server_cookie=543b1e6099a38664 in_seq=0 out_seq=0 2024-09-10T07:50:06.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.395+0000 7fa0e3e00640 10 client.6288 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1517250002 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbb7ac7b40 con 0x55fbb7b7e6d0 2024-09-10T07:50:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa0e3e00640 10 client.6288 handle_command_reply: tid=0 2024-09-10T07:50:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 client.6288 shutdown 2024-09-10T07:50:06.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 msgr2=0x55fbb7b69e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8081740 tx=0x7fa0f8009ff0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 shutdown_connections 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7b7e6d0 0x55fbb7b7eab0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b98ec0 0x55fbb7b69e90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa0e3e00640 0 client.6288 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7b79ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1517250002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.396+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 >> 172.21.15.3:0/1517250002 conn(0x55fbb7b544b0 msgr2=0x55fbb7b9af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.397+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 shutdown_connections 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.397+0000 7fa1087fe740 1 -- 172.21.15.3:0/1517250002 wait complete. 2024-09-10T07:50:06.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.397+0000 7fa1087fe740 20 client.6288 trim_cache size 0 max 16384 2024-09-10T07:50:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.415+0000 7fa1087fe740 1 Processor -- start 2024-09-10T07:50:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 -- start start 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 0x55fbb7bbf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 0x55fbb7bbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1001fbea0 con 0x55fbb7b56cf0 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa1001fc570 con 0x55fbb7b58480 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.416+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa100148390 con 0x55fbb7b98ec0 2024-09-10T07:50:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52196/0 (socket says 172.21.15.3:52196) 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 -- 172.21.15.3:0/2743506954 learned_addr learned my addr 172.21.15.3:0/2743506954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0ff400640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 0x55fbb7bbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 0x55fbb7bbf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 -- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 msgr2=0x55fbb7bbf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 0x55fbb7bbf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 -- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 msgr2=0x55fbb7bbf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 0x55fbb7bbf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 -- 172.21.15.3:0/2743506954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb7b78350 con 0x55fbb7b56cf0 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0ff400640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 0x55fbb7bbf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 0x55fbb7bbf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0fea00640 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0004180 tx=0x7fa0f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff31901c212fba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.417+0000 7fa0eea00640 1 -- 172.21.15.3:0/2743506954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0033070 con 0x55fbb7b56cf0 2024-09-10T07:50:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa0eea00640 1 -- 172.21.15.3:0/2743506954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0003c70 con 0x55fbb7b56cf0 2024-09-10T07:50:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa0eea00640 1 -- 172.21.15.3:0/2743506954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0003e20 con 0x55fbb7b56cf0 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 -- 172.21.15.3:0/2743506954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 msgr2=0x55fbb7b7da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0004180 tx=0x7fa0f0017000 comp rx=0 tx=0).stop 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 -- 172.21.15.3:0/2743506954 shutdown_connections 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b98ec0 0x55fbb7bbf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b58480 0x55fbb7bbf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 --2- 172.21.15.3:0/2743506954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7da60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 -- 172.21.15.3:0/2743506954 >> 172.21.15.3:0/2743506954 conn(0x55fbb7b544b0 msgr2=0x55fbb7b7f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 -- 172.21.15.3:0/2743506954 shutdown_connections 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.418+0000 7fa1087fe740 1 -- 172.21.15.3:0/2743506954 wait complete. 2024-09-10T07:50:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.419+0000 7fa1087fe740 1 -- start start 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.419+0000 7fa1087fe740 1 Processor -- start 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7bbe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b98ec0 0x55fbb7c19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7b77d40 con 0x55fbb7b56cf0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fbb7ab53b0 con 0x55fbb7b98ec0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fbb7b6a850 con 0x55fbb7b58480 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa1087fe740 10 client.0 _ll_register_callbacks cb 0x7fa0f80a5070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52210/0 (socket says 172.21.15.3:52210) 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0fea00640 1 -- 172.21.15.3:0/1309590636 learned_addr learned my addr 172.21.15.3:0/1309590636 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0ff400640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b98ec0 0x55fbb7c19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7bbe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0fea00640 1 -- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 msgr2=0x55fbb7bbe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.420+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7bbe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0fea00640 1 -- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b98ec0 msgr2=0x55fbb7c19090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b98ec0 0x55fbb7c19090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0fea00640 1 -- 172.21.15.3:0/1309590636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb7bbfa10 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0fea00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0008010 tx=0x7fa0f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1a21683ca944fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0033070 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f0025790 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f0003c70 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fa0f0040050 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa1087fe740 10 client.6294 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55fbb7bbebc0 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.421+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0003c70 con 0x55fbb7b56cf0 2024-09-10T07:50:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7fa0f0043020 con 0x55fbb7b56cf0 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 10 client.6294 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 10 client.6294 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 20 client.6294 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7badbc0 0x55fbb7b80cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 4 client.6294 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "status"}) -- 0x55fbb7ac1e90 con 0x55fbb7badbc0 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7badbc0 0x55fbb7b80cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa0f7e00640 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7badbc0 0x55fbb7b80cd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aeda5f0749c30f37 server_cookie=695a48bfa757ce0 in_seq=0 out_seq=0 2024-09-10T07:50:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.422+0000 7fa0e3e00640 10 client.6294 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa0e3e00640 1 -- 172.21.15.3:0/1309590636 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fbb7ac1e90 con 0x55fbb7badbc0 2024-09-10T07:50:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa0e3e00640 10 client.6294 handle_command_reply: tid=0 2024-09-10T07:50:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 client.6294 shutdown 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 msgr2=0x55fbb7b7d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0008010 tx=0x7fa0f0004730 comp rx=0 tx=0).stop 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 shutdown_connections 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55fbb7badbc0 0x55fbb7b80cd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fbb7b98ec0 0x55fbb7c19090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fbb7b58480 0x55fbb7bbe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 --2- 172.21.15.3:0/1309590636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fbb7b56cf0 0x55fbb7b7d0c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 >> 172.21.15.3:0/1309590636 conn(0x55fbb7b544b0 msgr2=0x55fbb7b9c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa0e3e00640 0 client.6294 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 shutdown_connections 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 1 -- 172.21.15.3:0/1309590636 wait complete. 2024-09-10T07:50:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.423+0000 7fa1087fe740 20 client.6294 trim_cache size 0 max 16384 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6111, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.1445856459999999, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 56, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 64, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 64, 2024-09-10T07:50:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.290901607999999 2024-09-10T07:50:06.429 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 msgr2=0x7fa0dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 0x7fa0dc06e320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009b00 tx=0x7fa0f0008040 comp rx=0 tx=0).stop 2024-09-10T07:50:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 msgr2=0x7fa1000ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a780 tx=0x7fa1000cd230 comp rx=0 tx=0).stop 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 shutdown_connections 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa0dc06be60 0x7fa0dc06e320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa100158180 0x7fa1000ad830 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa10015e2f0 0x7fa1000ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 --2- 172.21.15.3:0/3017901874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa0e4004d40 0x7fa100146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 >> 172.21.15.3:0/3017901874 conn(0x7fa10006d000 msgr2=0x7fa1000b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 shutdown_connections 2024-09-10T07:50:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.424+0000 7fa105a00640 1 -- 172.21.15.3:0/3017901874 wait complete. 2024-09-10T07:50:06.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:50:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.596+0000 7f10b5a00640 1 Processor -- start 2024-09-10T07:50:06.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.596+0000 7f10b5a00640 1 -- start start 2024-09-10T07:50:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.597+0000 7f10b5a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f10b0160710 0x7f10b00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.597+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b00d07c0 0x7f10b0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.597+0000 7f10b5a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10b00cfdd0 con 0x7f10b0160af0 2024-09-10T07:50:06.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.597+0000 7f10b5a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f10b015cc90 con 0x7f10b0160710 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.597+0000 7f10b5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f10b015cf70 con 0x7f10b00d07c0 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f10b0160710 0x7f10b00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f10b0160710 0x7f10b00cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52212/0 (socket says 172.21.15.3:52212) 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 -- 172.21.15.3:0/2062026705 learned_addr learned my addr 172.21.15.3:0/2062026705 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10ae000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f10b0160af0 0x7f10b00d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51224/0 (socket says 172.21.15.3:51224) 2024-09-10T07:50:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10af400640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b00d07c0 0x7f10b0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 -- 172.21.15.3:0/2062026705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b00d07c0 msgr2=0x7f10b0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b00d07c0 0x7f10b0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 -- 172.21.15.3:0/2062026705 >> v1:172.21.15.3:6789/0 conn(0x7f10b0160af0 legacy=0x7f10b00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10aea00640 1 -- 172.21.15.3:0/2062026705 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b015d250 con 0x7f10b0160710 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.598+0000 7f10ad600640 1 -- 172.21.15.3:0/2062026705 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2458751356 0 0) 0x7f10b00cfdd0 con 0x7f10b0160af0 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.599+0000 7f10aea00640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0160710 0x7f10b00cf2c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009870 tx=0x7f10a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70199d6d832981f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.599+0000 7f10ad600640 1 -- 172.21.15.3:0/2062026705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037070 con 0x7f10b0160710 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.599+0000 7f10ad600640 1 -- 172.21.15.3:0/2062026705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0032440 con 0x7f10b0160710 2024-09-10T07:50:06.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.599+0000 7f10ad600640 1 -- 172.21.15.3:0/2062026705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0032810 con 0x7f10b0160710 2024-09-10T07:50:06.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 -- 172.21.15.3:0/2062026705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0160710 msgr2=0x7f10b00cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0160710 0x7f10b00cf2c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009870 tx=0x7f10a002a250 comp rx=0 tx=0).stop 2024-09-10T07:50:06.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 -- 172.21.15.3:0/2062026705 shutdown_connections 2024-09-10T07:50:06.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b00d07c0 0x7f10b0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 --2- 172.21.15.3:0/2062026705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0160710 0x7f10b00cf2c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 -- 172.21.15.3:0/2062026705 >> 172.21.15.3:0/2062026705 conn(0x7f10b006d000 msgr2=0x7f10b006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 -- 172.21.15.3:0/2062026705 shutdown_connections 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.600+0000 7f10b5a00640 1 -- 172.21.15.3:0/2062026705 wait complete. 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.602+0000 7f10b5a00640 1 Processor -- start 2024-09-10T07:50:06.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.602+0000 7f10b5a00640 1 -- start start 2024-09-10T07:50:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b0160710 0x7f10b015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b0160af0 0x7f10b014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b015d770 con 0x7f10b00d07c0 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00cf800 con 0x7f10b0160710 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10b5a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f10b015f0b0 con 0x7f10b0160af0 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52222/0 (socket says 172.21.15.3:52222) 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 -- 172.21.15.3:0/489225170 learned_addr learned my addr 172.21.15.3:0/489225170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10af400640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b0160af0 0x7f10b014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10ae000640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b0160710 0x7f10b015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 -- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b0160af0 msgr2=0x7f10b014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b0160af0 0x7f10b014e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 -- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b0160710 msgr2=0x7f10b015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b0160710 0x7f10b015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.603+0000 7f10aea00640 1 -- 172.21.15.3:0/489225170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b01e5fb0 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10aea00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10a6a00640 1 -- 172.21.15.3:0/489225170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037040 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10a6a00640 1 -- 172.21.15.3:0/489225170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0042070 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10a6a00640 1 -- 172.21.15.3:0/489225170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0030880 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01e6290 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.604+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b014e710 con 0x7f10b00d07c0 2024-09-10T07:50:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.606+0000 7f10a6a00640 1 -- 172.21.15.3:0/489225170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10a0030a20 con 0x7f10b00d07c0 2024-09-10T07:50:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.607+0000 7f10a6a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 0x7f109006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.607+0000 7f10a6a00640 1 -- 172.21.15.3:0/489225170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f10a00ab6e0 con 0x7f10b00d07c0 2024-09-10T07:50:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.607+0000 7f10ae000640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 0x7f109006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.607+0000 7f10b89fe740 1 Processor -- start 2024-09-10T07:50:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 -- start start 2024-09-10T07:50:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10ae000640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 0x7f109006e320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10b0072810 tx=0x7f109c007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b7cf0 0x55e69c2fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c2b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e69c2163b0 con 0x55e69c2b54b0 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e69c21b410 con 0x55e69c2b9480 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.608+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e69c1c3620 con 0x55e69c2b7cf0 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52228/0 (socket says 172.21.15.3:52228) 2024-09-10T07:50:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 -- 172.21.15.3:0/558353592 learned_addr learned my addr 172.21.15.3:0/558353592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10aea00640 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b7cf0 0x55e69c2fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10af400640 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c2b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 -- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b7cf0 msgr2=0x55e69c2fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b7cf0 0x55e69c2fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 -- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 msgr2=0x55e69c2b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c2b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 -- 172.21.15.3:0/558353592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e69bf2e2d0 con 0x55e69c2b54b0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.609+0000 7f10ae000640 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f109c005dc0 tx=0x7f109c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb01e999891ae67a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f109b400640 1 -- 172.21.15.3:0/558353592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c015110 con 0x55e69c2b54b0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f109b400640 1 -- 172.21.15.3:0/558353592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c0159c0 con 0x55e69c2b54b0 2024-09-10T07:50:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f109b400640 1 -- 172.21.15.3:0/558353592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c015d70 con 0x55e69c2b54b0 2024-09-10T07:50:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 -- 172.21.15.3:0/558353592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 msgr2=0x55e69c2b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f109c005dc0 tx=0x7f109c012000 comp rx=0 tx=0).stop 2024-09-10T07:50:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 -- 172.21.15.3:0/558353592 shutdown_connections 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c2b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b54b0 0x55e69c2b80d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 --2- 172.21.15.3:0/558353592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b7cf0 0x55e69c2fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 -- 172.21.15.3:0/558353592 >> 172.21.15.3:0/558353592 conn(0x55e69c2b4740 msgr2=0x55e69c2fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 -- 172.21.15.3:0/558353592 shutdown_connections 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.610+0000 7f10b89fe740 1 -- 172.21.15.3:0/558353592 wait complete. 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.611+0000 7f10b89fe740 1 -- start start 2024-09-10T07:50:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.611+0000 7f10b89fe740 1 Processor -- start 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c311ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 0x55e69c2db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e69c1d6780 con 0x55e69c2b7cf0 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e69c2173d0 con 0x55e69c2b9480 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e69c2b3be0 con 0x55e69c2e26c0 2024-09-10T07:50:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.612+0000 7f10b89fe740 10 client.0 _ll_register_callbacks cb 0x55e69c2145d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52244/0 (socket says 172.21.15.3:52244) 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 -- 172.21.15.3:0/2728814964 learned_addr learned my addr 172.21.15.3:0/2728814964 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10ae000640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c311ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10af400640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 0x55e69c2db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 -- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 msgr2=0x55e69c2db350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 0x55e69c2db350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 -- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 msgr2=0x55e69c311ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c311ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 -- 172.21.15.3:0/2728814964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e69bff86b0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10ae000640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c311ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10af400640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 0x55e69c2db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f10aea00640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f10a0075d90 tx=0x7f10a0032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b847828bdc7c2aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.613+0000 7f108fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a007dd30 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f10a0030d10 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f109c0159c0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f10a0082560 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 10 client.6309 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55e69c2da390 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c0159c0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7f10a003a070 con 0x55e69c2b7cf0 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 10 client.6309 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 10 client.6309 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 20 client.6309 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c2df8f0 0x55e69c314fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 4 client.6309 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e69c228b40 con 0x55e69c2df8f0 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.614+0000 7f10ae000640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c2df8f0 0x55e69c314fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.615+0000 7f10ae000640 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c2df8f0 0x55e69c314fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be87aaca7fb724b server_cookie=3b1ff462714bcbeb in_seq=0 out_seq=0 2024-09-10T07:50:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.615+0000 7f108fe00640 10 client.6309 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f108fe00640 1 -- 172.21.15.3:0/2728814964 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e69c228b40 con 0x55e69c2df8f0 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f108fe00640 10 client.6309 handle_command_reply: tid=0 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 client.6309 shutdown 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 msgr2=0x55e69c2e0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f10a0075d90 tx=0x7f10a0032440 comp rx=0 tx=0).stop 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 shutdown_connections 2024-09-10T07:50:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c2df8f0 0x55e69c314fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2e26c0 0x55e69c2db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c311ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 --2- 172.21.15.3:0/2728814964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c2e0b30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 >> 172.21.15.3:0/2728814964 conn(0x55e69c2b54b0 msgr2=0x55e69c2fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.616+0000 7f108fe00640 0 client.6309 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.617+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 shutdown_connections 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.617+0000 7f10b89fe740 1 -- 172.21.15.3:0/2728814964 wait complete. 2024-09-10T07:50:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.617+0000 7f10b89fe740 20 client.6309 trim_cache size 0 max 16384 2024-09-10T07:50:06.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.635+0000 7f10b89fe740 1 Processor -- start 2024-09-10T07:50:06.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 -- start start 2024-09-10T07:50:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b7cf0 0x55e69c324f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b9480 0x55e69c3254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00b4a90 con 0x55e69c2dcd60 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00b5160 con 0x55e69c2b7cf0 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.636+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f10b01e6ef0 con 0x55e69c2b9480 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52260/0 (socket says 172.21.15.3:52260) 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 -- 172.21.15.3:0/1610114881 learned_addr learned my addr 172.21.15.3:0/1610114881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10ae000640 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b9480 0x55e69c3254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10aea00640 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b7cf0 0x55e69c324f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 -- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b9480 msgr2=0x55e69c3254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b9480 0x55e69c3254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 -- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b7cf0 msgr2=0x55e69c324f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b7cf0 0x55e69c324f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 -- 172.21.15.3:0/1610114881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e69c2b3760 con 0x55e69c2dcd60 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f10af400640 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f10a80076b0 tx=0x7f10a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b0339c19590e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f109aa00640 1 -- 172.21.15.3:0/1610114881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a801d070 con 0x55e69c2dcd60 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f109aa00640 1 -- 172.21.15.3:0/1610114881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a80148d0 con 0x55e69c2dcd60 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.637+0000 7f109aa00640 1 -- 172.21.15.3:0/1610114881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8014cc0 con 0x55e69c2dcd60 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 -- 172.21.15.3:0/1610114881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 msgr2=0x55e69c30d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f10a80076b0 tx=0x7f10a8009750 comp rx=0 tx=0).stop 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 -- 172.21.15.3:0/1610114881 shutdown_connections 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2dcd60 0x55e69c30d9d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2b9480 0x55e69c3254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1610114881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b7cf0 0x55e69c324f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 -- 172.21.15.3:0/1610114881 >> 172.21.15.3:0/1610114881 conn(0x55e69c2b54b0 msgr2=0x55e69c31b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 -- 172.21.15.3:0/1610114881 shutdown_connections 2024-09-10T07:50:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.638+0000 7f10b89fe740 1 -- 172.21.15.3:0/1610114881 wait complete. 2024-09-10T07:50:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.639+0000 7f10b89fe740 1 -- start start 2024-09-10T07:50:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.639+0000 7f10b89fe740 1 Processor -- start 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c3695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2dcd60 0x55e69c324f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e69c2df770 con 0x55e69c2b7cf0 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e69c2e02c0 con 0x55e69c2b9480 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e69c2dfdc0 con 0x55e69c2dcd60 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10b89fe740 10 client.0 _ll_register_callbacks cb 0x7f10a0079750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52264/0 (socket says 172.21.15.3:52264) 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10aea00640 1 -- 172.21.15.3:0/1874687430 learned_addr learned my addr 172.21.15.3:0/1874687430 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10af400640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2dcd60 0x55e69c324f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.640+0000 7f10ae000640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c3695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 -- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2dcd60 msgr2=0x55e69c324f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2dcd60 0x55e69c324f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 -- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 msgr2=0x55e69c3695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c3695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 -- 172.21.15.3:0/1874687430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e69c30e9e0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10aea00640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f10a0077c30 tx=0x7f10a0049070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c62360e5623a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f108fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0095060 con 0x55e69c2b7cf0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f10a00820d0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a80148d0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f10a00794c0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10b89fe740 10 client.6315 fetch_fsmap learned FSMap version 56 2024-09-10T07:50:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.641+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55e69c2fb220 con 0x55e69c2b7cf0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a80148d0 con 0x55e69c2b7cf0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) ==== 3542+0+0 (secure 0 0 0) 0x7f10a003a070 con 0x55e69c2b7cf0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 10 client.6315 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 10 client.6315 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 20 client.6315 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c380b60 0x55e69c380f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 4 client.6315 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "status"}) -- 0x55e69c222e90 con 0x55e69c380b60 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10ae000640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c380b60 0x55e69c380f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f10ae000640 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c380b60 0x55e69c380f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d380dd8218524e server_cookie=c6165122adfb7726 in_seq=0 out_seq=0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.642+0000 7f108fe00640 10 client.6315 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f108fe00640 1 -- 172.21.15.3:0/1874687430 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e69c222e90 con 0x55e69c380b60 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f108fe00640 10 client.6315 handle_command_reply: tid=0 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 client.6315 shutdown 2024-09-10T07:50:06.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 msgr2=0x55e69c383a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f10a0077c30 tx=0x7f10a0049070 comp rx=0 tx=0).stop 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 shutdown_connections 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55e69c380b60 0x55e69c380f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e69c2dcd60 0x55e69c324f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e69c2b9480 0x55e69c3695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 --2- 172.21.15.3:0/1874687430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e69c2b7cf0 0x55e69c383a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 >> 172.21.15.3:0/1874687430 conn(0x55e69c2b54b0 msgr2=0x55e69c2fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f108fe00640 0 client.6315 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 shutdown_connections 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 1 -- 172.21.15.3:0/1874687430 wait complete. 2024-09-10T07:50:06.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.643+0000 7f10b89fe740 20 client.6315 trim_cache size 0 max 16384 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6114, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 11.431674105999999, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 56, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 64, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 64, 2024-09-10T07:50:06.647 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.510969419999999 2024-09-10T07:50:06.648 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:06.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 msgr2=0x7f109006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 0x7f109006e320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10b0072810 tx=0x7f109c007c00 comp rx=0 tx=0).stop 2024-09-10T07:50:06.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 msgr2=0x7f10b00ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a00046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 shutdown_connections 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f109006be60 0x7f109006e320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f10b0160af0 0x7f10b014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b0160710 0x7f10b015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 --2- 172.21.15.3:0/489225170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b00d07c0 0x7f10b00ab130 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 >> 172.21.15.3:0/489225170 conn(0x7f10b006d000 msgr2=0x7f10b00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 shutdown_connections 2024-09-10T07:50:06.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.644+0000 7f10b5a00640 1 -- 172.21.15.3:0/489225170 wait complete. 2024-09-10T07:50:06.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:06.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.815+0000 7f29ea600640 1 Processor -- start 2024-09-10T07:50:06.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.815+0000 7f29ea600640 1 -- start start 2024-09-10T07:50:06.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29ea600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29ea600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29e40d05f0 0x7f29e4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29ea600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40cfdd0 con 0x7f29e4160710 2024-09-10T07:50:06.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29ea600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f29e415cc90 con 0x7f29e40d05f0 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29ea600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29e415cf70 con 0x7f29e4160af0 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29e40d05f0 0x7f29e4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29e40d05f0 0x7f29e4072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52272/0 (socket says 172.21.15.3:52272) 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 -- 172.21.15.3:0/2800299767 learned_addr learned my addr 172.21.15.3:0/2800299767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29dbe00640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 -- 172.21.15.3:0/2800299767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 msgr2=0x7f29e40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 -- 172.21.15.3:0/2800299767 >> v1:172.21.15.70:6789/0 conn(0x7f29e4160710 legacy=0x7f29e40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29e3e00640 1 -- 172.21.15.3:0/2800299767 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e415d250 con 0x7f29e40d05f0 2024-09-10T07:50:06.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.817+0000 7f29dbe00640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.818+0000 7f29e3e00640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e40d05f0 0x7f29e4072070 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e50 tx=0x7f29dc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c316c77b8cad34cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29e2a00640 1 -- 172.21.15.3:0/2800299767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc036070 con 0x7f29e40d05f0 2024-09-10T07:50:06.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29e2a00640 1 -- 172.21.15.3:0/2800299767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc031600 con 0x7f29e40d05f0 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29e2a00640 1 -- 172.21.15.3:0/2800299767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0319d0 con 0x7f29e40d05f0 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 -- 172.21.15.3:0/2800299767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e40d05f0 msgr2=0x7f29e4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e40d05f0 0x7f29e4072070 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e50 tx=0x7f29dc02a6c0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 -- 172.21.15.3:0/2800299767 shutdown_connections 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e40d05f0 0x7f29e4072070 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 --2- 172.21.15.3:0/2800299767 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 -- 172.21.15.3:0/2800299767 >> 172.21.15.3:0/2800299767 conn(0x7f29e406d000 msgr2=0x7f29e406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 -- 172.21.15.3:0/2800299767 shutdown_connections 2024-09-10T07:50:06.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.819+0000 7f29ea600640 1 -- 172.21.15.3:0/2800299767 wait complete. 2024-09-10T07:50:06.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.821+0000 7f29ea600640 1 Processor -- start 2024-09-10T07:50:06.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.821+0000 7f29ea600640 1 -- start start 2024-09-10T07:50:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 0x7f29e41507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e4150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29e415d770 con 0x7f29e4160710 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40d0a10 con 0x7f29e40d05f0 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29ea600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29e41539c0 con 0x7f29e4160af0 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52278/0 (socket says 172.21.15.3:52278) 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.822+0000 7f29dbe00640 1 -- 172.21.15.3:0/3704402165 learned_addr learned my addr 172.21.15.3:0/3704402165 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29e3e00640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e4150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 -- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 msgr2=0x7f29e4150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29e3400640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 0x7f29e41507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e4150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 -- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 msgr2=0x7f29e41507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 0x7f29e41507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 -- 172.21.15.3:0/3704402165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e415b980 con 0x7f29e4160710 2024-09-10T07:50:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29e3e00640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e4150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29e3400640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 0x7f29e41507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29dbe00640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c820 tx=0x7f29cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc015070 con 0x7f29e4160710 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29cc007a30 con 0x7f29e4160710 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc007de0 con 0x7f29e4160710 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.823+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e415bc60 con 0x7f29e4160710 2024-09-10T07:50:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.824+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e4155e20 con 0x7f29e4160710 2024-09-10T07:50:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.825+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40cfaf0 con 0x7f29e4160710 2024-09-10T07:50:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.825+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29cc018070 con 0x7f29e4160710 2024-09-10T07:50:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.827+0000 7f29db400640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.827+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5709+0+0 (secure 0 0 0) 0x7f29cc08ae20 con 0x7f29e4160710 2024-09-10T07:50:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.827+0000 7f29e3400640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.828+0000 7f29e3400640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 0x7f29c006e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003eb0 tx=0x7f29d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:06.831+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f29cc059730 con 0x7f29e4160710 2024-09-10T07:50:07.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.051+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29e415be30 con 0x7f29e4160710 2024-09-10T07:50:07.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.052+0000 7f29db400640 1 -- 172.21.15.3:0/3704402165 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5255 (secure 0 0 0) 0x7f29cc059030 con 0x7f29e4160710 2024-09-10T07:50:07.051 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:07.051 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":56,"btime":"2024-09-10T07:50:05:287301+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:05.287295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[5857,6111,6114]},"id":4}]} 2024-09-10T07:50:07.052 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 56 2024-09-10T07:50:07.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 msgr2=0x7f29c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 0x7f29c006e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003eb0 tx=0x7f29d4009040 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 msgr2=0x7f29e41558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c820 tx=0x7f29cc00cce0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 shutdown_connections 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e4160af0 0x7f29e4150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29e4160710 0x7f29e41558e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 --2- 172.21.15.3:0/3704402165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e40d05f0 0x7f29e41507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 >> 172.21.15.3:0/3704402165 conn(0x7f29e406d000 msgr2=0x7f29e40b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 shutdown_connections 2024-09-10T07:50:07.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:07.054+0000 7f29ea600640 1 -- 172.21.15.3:0/3704402165 wait complete. 2024-09-10T07:50:07.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:07.071 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:50:07.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:07.117 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:50:07.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:07.199 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:50:07.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:07.276 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:50:07.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:07.353 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:50:07.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:50:07.646 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:50:07.646 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:50:07.646 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:50:09.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:50:09.834 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:50:09.835 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:50:09.835 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:50:11.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:50:12.023 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:50:12.024 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:50:12.024 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:50:14.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:50:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:50:14.210 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:50:14.211 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:50:16.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:50:16.398 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:50:16.398 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:50:16.399 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:50:16.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:50:16.575 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"0","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}}] 2024-09-10T07:50:16.575 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:50:16.575 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:50:16.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-10T07:50:16.779 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:50:16.780 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:50:16.780 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:50:16.780 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-10T07:50:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.938+0000 7f9409c00640 1 Processor -- start 2024-09-10T07:50:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.938+0000 7f9409c00640 1 -- start start 2024-09-10T07:50:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9409c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94041451d0 0x7f940406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9409c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94040cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9409c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040cd900 con 0x7f9404145730 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9409c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f940406d7c0 con 0x7f94041451d0 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9409c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f94040cdbe0 con 0x7f94040cf270 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94041451d0 0x7f940406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f94041451d0 0x7f940406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53208/0 (socket says 172.21.15.3:53208) 2024-09-10T07:50:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 -- 172.21.15.3:0/1720523513 learned_addr learned my addr 172.21.15.3:0/1720523513 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 -- 172.21.15.3:0/1720523513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 msgr2=0x7f94040cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 --2- 172.21.15.3:0/1720523513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94040cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 -- 172.21.15.3:0/1720523513 >> v1:172.21.15.70:6789/0 conn(0x7f9404145730 legacy=0x7f94040b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.940+0000 7f9402a00640 1 -- 172.21.15.3:0/1720523513 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940406da10 con 0x7f94041451d0 2024-09-10T07:50:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.941+0000 7f9402a00640 1 --2- 172.21.15.3:0/1720523513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f940406cfc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f94040732d0 tx=0x7f93f402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5df49b5b54a840 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9402000640 1 -- 172.21.15.3:0/1720523513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4037070 con 0x7f94041451d0 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9402000640 1 -- 172.21.15.3:0/1720523513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f4032440 con 0x7f94041451d0 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9402000640 1 -- 172.21.15.3:0/1720523513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4032810 con 0x7f94041451d0 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 -- 172.21.15.3:0/1720523513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 msgr2=0x7f940406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 --2- 172.21.15.3:0/1720523513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f940406cfc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f94040732d0 tx=0x7f93f402a160 comp rx=0 tx=0).stop 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 -- 172.21.15.3:0/1720523513 shutdown_connections 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 --2- 172.21.15.3:0/1720523513 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94040cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 --2- 172.21.15.3:0/1720523513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f940406cfc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 -- 172.21.15.3:0/1720523513 >> 172.21.15.3:0/1720523513 conn(0x7f94040b5cb0 msgr2=0x7f940406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 -- 172.21.15.3:0/1720523513 shutdown_connections 2024-09-10T07:50:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.942+0000 7f9409c00640 1 -- 172.21.15.3:0/1720523513 wait complete. 2024-09-10T07:50:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.944+0000 7f9409c00640 1 Processor -- start 2024-09-10T07:50:16.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.944+0000 7f9409c00640 1 -- start start 2024-09-10T07:50:16.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94041349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 0x7f94040ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f940406df30 con 0x7f94041451d0 2024-09-10T07:50:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94040cfc10 con 0x7f9404145730 2024-09-10T07:50:16.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9409c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f940406e0b0 con 0x7f94040cf270 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53216/0 (socket says 172.21.15.3:53216) 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f93fbe00640 1 -- 172.21.15.3:0/3926549768 learned_addr learned my addr 172.21.15.3:0/3926549768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.945+0000 7f9402a00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94041349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f9403400640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 0x7f94040ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 -- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 msgr2=0x7f94041349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94041349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 -- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 msgr2=0x7f94040ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 0x7f94040ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 -- 172.21.15.3:0/3926549768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040ae0d0 con 0x7f94041451d0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f9403400640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 0x7f94040ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93fbe00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00b410 tx=0x7f93ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93faa00640 1 -- 172.21.15.3:0/3926549768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec015020 con 0x7f94041451d0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93faa00640 1 -- 172.21.15.3:0/3926549768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ec011830 con 0x7f94041451d0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f93faa00640 1 -- 172.21.15.3:0/3926549768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec011c20 con 0x7f94041451d0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9404134f00 con 0x7f94041451d0 2024-09-10T07:50:16.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.946+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94040aacc0 con 0x7f94041451d0 2024-09-10T07:50:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.949+0000 7f93faa00640 1 -- 172.21.15.3:0/3926549768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93ec018020 con 0x7f94041451d0 2024-09-10T07:50:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.950+0000 7f93faa00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 0x7f93e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.950+0000 7f93faa00640 1 -- 172.21.15.3:0/3926549768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f93ec08a890 con 0x7f94041451d0 2024-09-10T07:50:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.950+0000 7f9402a00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 0x7f93e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.951+0000 7f940ca66740 1 Processor -- start 2024-09-10T07:50:16.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.951+0000 7f940ca66740 1 -- start start 2024-09-10T07:50:16.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.951+0000 7f9402a00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 0x7f93e006e320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x56529881f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987d4480 0x56529881f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56529873c660 con 0x5652987d4480 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652986e4680 con 0x5652987ee420 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x565298738620 con 0x5652987ee800 2024-09-10T07:50:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x56529881f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f9403400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987d4480 0x56529881f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f9402a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f9402a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:58658/0 (socket says 172.21.15.3:58658) 2024-09-10T07:50:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.952+0000 7f9402a00640 1 -- 172.21.15.3:0/2892943143 learned_addr learned my addr 172.21.15.3:0/2892943143 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 -- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 msgr2=0x56529881f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x56529881f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 -- 172.21.15.3:0/2892943143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987d4480 msgr2=0x56529881f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987d4480 0x56529881f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 -- 172.21.15.3:0/2892943143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652985197d0 con 0x5652987ee420 2024-09-10T07:50:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f93fbe00640 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x56529881f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f9402a00640 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000c00 tx=0x7f93f402c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce021faaa2d4ca96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f93f3400640 1 -- 172.21.15.3:0/2892943143 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4037040 con 0x5652987ee420 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f93f3400640 1 -- 172.21.15.3:0/2892943143 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f404e070 con 0x5652987ee420 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.953+0000 7f93f3400640 1 -- 172.21.15.3:0/2892943143 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4049480 con 0x5652987ee420 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 -- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 msgr2=0x56529881ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000c00 tx=0x7f93f402c000 comp rx=0 tx=0).stop 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 -- 172.21.15.3:0/2892943143 shutdown_connections 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987d4480 0x56529881f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x56529881f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 --2- 172.21.15.3:0/2892943143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 -- 172.21.15.3:0/2892943143 >> 172.21.15.3:0/2892943143 conn(0x5652987d5220 msgr2=0x56529881d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:16.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 -- 172.21.15.3:0/2892943143 shutdown_connections 2024-09-10T07:50:16.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.954+0000 7f940ca66740 1 -- 172.21.15.3:0/2892943143 wait complete. 2024-09-10T07:50:16.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.955+0000 7f940ca66740 1 -- start start 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.955+0000 7f940ca66740 1 Processor -- start 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x565298804b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x565298802c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652987d33e0 con 0x5652988062c0 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652986c54c0 con 0x5652987ee420 2024-09-10T07:50:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652987d36c0 con 0x5652987ee800 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f940ca66740 10 client.0 _ll_register_callbacks cb 0x565298734e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f9403400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f9403400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53244/0 (socket says 172.21.15.3:53244) 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f9403400640 1 -- 172.21.15.3:0/744679778 learned_addr learned my addr 172.21.15.3:0/744679778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f9402a00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x565298804b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.956+0000 7f93fbe00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x565298802c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 -- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 msgr2=0x565298802c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x565298802c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 -- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 msgr2=0x565298806720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 -- 172.21.15.3:0/744679778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529844f3f0 con 0x5652987ee420 2024-09-10T07:50:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9403400640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f9402a00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x565298804b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009630 tx=0x7f93f4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121028ccfd8d0670 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f93e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4037040 con 0x5652987ee420 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mon.1 v2:172.21.15.70:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f93f4049890 con 0x5652987ee420 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.957+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f404e070 con 0x5652987ee420 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f93f4063050 con 0x5652987ee420 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 10 client.6025 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x56529840f0f0 con 0x5652987ee420 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mon.1 v2:172.21.15.70:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f404e070 con 0x5652987ee420 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mon.1 v2:172.21.15.70:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f93f4066020 con 0x5652987ee420 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 10 client.6025 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 10 client.6025 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 20 client.6025 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5652988195c0 0x5652988199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 4 client.6025 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565298749d90 con 0x5652988195c0 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.958+0000 7f93fbe00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5652988195c0 0x5652988199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.959+0000 7f93fbe00640 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5652988195c0 0x5652988199a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e1073861bebe80 server_cookie=d05ad0b9b312a854 in_seq=0 out_seq=0 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.959+0000 7f93e7e00640 10 client.6025 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.959+0000 7f93e7e00640 1 -- 172.21.15.3:0/744679778 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565298749d90 con 0x5652988195c0 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.959+0000 7f93e7e00640 10 client.6025 handle_command_reply: tid=0 2024-09-10T07:50:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 client.6025 shutdown 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 msgr2=0x565298804b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x565298804b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009630 tx=0x7f93f4004730 comp rx=0 tx=0).stop 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 shutdown_connections 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x5652988195c0 0x5652988199a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x565298806720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x565298802c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 --2- 172.21.15.3:0/744679778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x565298804b30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 >> 172.21.15.3:0/744679778 conn(0x5652987d4480 msgr2=0x5652987d87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f93e7e00640 0 client.6025 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 shutdown_connections 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 1 -- 172.21.15.3:0/744679778 wait complete. 2024-09-10T07:50:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.960+0000 7f940ca66740 20 client.6025 trim_cache size 0 max 16384 2024-09-10T07:50:16.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.975+0000 7f940ca66740 1 Processor -- start 2024-09-10T07:50:16.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.975+0000 7f940ca66740 1 -- start start 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x5652987d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94041fca80 con 0x5652988062c0 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94040722d0 con 0x5652987ee420 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f94040729a0 con 0x5652987ee800 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.976+0000 7f9403400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53252/0 (socket says 172.21.15.3:53252) 2024-09-10T07:50:16.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 -- 172.21.15.3:0/2740526884 learned_addr learned my addr 172.21.15.3:0/2740526884 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f93fbe00640 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x5652987d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9402a00640 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 -- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 msgr2=0x5652987d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x5652987d87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 -- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 msgr2=0x56529881ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 -- 172.21.15.3:0/2740526884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529881dc90 con 0x5652988062c0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f9403400640 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f93fc010770 tx=0x7f93fc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a133d6be22662d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f93f2a00640 1 -- 172.21.15.3:0/2740526884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00f620 con 0x5652988062c0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f93f2a00640 1 -- 172.21.15.3:0/2740526884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc00f7e0 con 0x5652988062c0 2024-09-10T07:50:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.977+0000 7f93f2a00640 1 -- 172.21.15.3:0/2740526884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00fbd0 con 0x5652988062c0 2024-09-10T07:50:16.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 -- 172.21.15.3:0/2740526884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 msgr2=0x5652987d8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f93fc010770 tx=0x7f93fc010860 comp rx=0 tx=0).stop 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 -- 172.21.15.3:0/2740526884 shutdown_connections 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652988062c0 0x5652987d8d10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652987ee800 0x5652987d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 --2- 172.21.15.3:0/2740526884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x56529881ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 -- 172.21.15.3:0/2740526884 >> 172.21.15.3:0/2740526884 conn(0x5652987d4480 msgr2=0x565298816c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 -- 172.21.15.3:0/2740526884 shutdown_connections 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.978+0000 7f940ca66740 1 -- 172.21.15.3:0/2740526884 wait complete. 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.979+0000 7f940ca66740 1 -- start start 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.979+0000 7f940ca66740 1 Processor -- start 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x5652987d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652988062c0 0x5652988813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565298805c70 con 0x5652987ee800 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94041f6680 con 0x5652987ee420 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x565298805600 con 0x5652988062c0 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f940ca66740 10 client.0 _ll_register_callbacks cb 0x7f93fc0075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53266/0 (socket says 172.21.15.3:53266) 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f93fbe00640 1 -- 172.21.15.3:0/1841063301 learned_addr learned my addr 172.21.15.3:0/1841063301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f9402a00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x5652987d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f9403400640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652988062c0 0x5652988813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.980+0000 7f93fbe00640 1 -- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652988062c0 msgr2=0x5652988813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93fbe00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652988062c0 0x5652988813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93fbe00640 1 -- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 msgr2=0x5652987d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93fbe00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x5652987d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93fbe00640 1 -- 172.21.15.3:0/1841063301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652988418e0 con 0x5652987ee800 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f9402a00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x5652987d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93fbe00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00bdd0 tx=0x7f93ec061860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e19f9d3fe7aa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec057f50 con 0x5652987ee800 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f93ec0580f0 con 0x5652987ee800 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93fc00f7e0 con 0x5652987ee800 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f93ec05f0b0 con 0x5652987ee800 2024-09-10T07:50:16.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f940ca66740 10 client.6339 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.981+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x565298881950 con 0x5652987ee800 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc00f7e0 con 0x5652987ee800 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f93ec05f250 con 0x5652987ee800 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 10 client.6339 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 10 client.6339 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 20 client.6339 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x56529881de60 0x56529881e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 4 client.6339 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652987440e0 con 0x56529881de60 2024-09-10T07:50:16.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f9402a00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x56529881de60 0x56529881e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:16.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f9402a00640 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x56529881de60 0x56529881e270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2246013dbb169d3 server_cookie=76cdda98ae5f3d56 in_seq=0 out_seq=0 2024-09-10T07:50:16.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:16.982+0000 7f93e7e00640 10 client.6339 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.000+0000 7f93e7e00640 1 -- 172.21.15.3:0/1841063301 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652987440e0 con 0x56529881de60 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.000+0000 7f93e7e00640 10 client.6339 handle_command_reply: tid=0 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.000+0000 7f940ca66740 1 client.6339 shutdown 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 msgr2=0x5652988413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00bdd0 tx=0x7f93ec061860 comp rx=0 tx=0).stop 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 shutdown_connections 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x56529881de60 0x56529881e270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652988062c0 0x5652988813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652987ee800 0x5652988413a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f93e7e00640 0 client.6339 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 --2- 172.21.15.3:0/1841063301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652987ee420 0x5652987d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 >> 172.21.15.3:0/1841063301 conn(0x5652987d4480 msgr2=0x565298816b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 shutdown_connections 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 1 -- 172.21.15.3:0/1841063301 wait complete. 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.001+0000 7f940ca66740 20 client.6339 trim_cache size 0 max 16384 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:50:17.002 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 msgr2=0x7f93e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 0x7f93e006e320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).stop 2024-09-10T07:50:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 msgr2=0x7f94040ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00b410 tx=0x7f93ec00b8d0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 shutdown_connections 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f93e006be60 0x7f93e006e320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9404145730 0x7f94040ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94041451d0 0x7f94040ac0c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 --2- 172.21.15.3:0/3926549768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94040cf270 0x7f94041349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 >> 172.21.15.3:0/3926549768 conn(0x7f94040b5cb0 msgr2=0x7f94040b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 shutdown_connections 2024-09-10T07:50:17.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.002+0000 7f9409c00640 1 -- 172.21.15.3:0/3926549768 wait complete. 2024-09-10T07:50:17.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2024-09-10T07:50:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.175+0000 7fcdbfe00640 1 Processor -- start 2024-09-10T07:50:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.175+0000 7fcdbfe00640 1 -- start start 2024-09-10T07:50:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.176+0000 7fcdbfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdb815b230 0x7fcdb815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.176+0000 7fcdbfe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb815eff0 con 0x7fcdb80b4c90 2024-09-10T07:50:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.176+0000 7fcdbfe00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb815f2b0 con 0x7fcdb8072c50 2024-09-10T07:50:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbfe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb815fbe0 con 0x7fcdb815b230 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdb815b230 0x7fcdb815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdb815b230 0x7fcdb815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53274/0 (socket says 172.21.15.3:53274) 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 -- 172.21.15.3:0/3851701003 learned_addr learned my addr 172.21.15.3:0/3851701003 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbde00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcdb80b4c90 0x7fcdb8071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59750/0 (socket says 172.21.15.3:59750) 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 -- 172.21.15.3:0/3851701003 >> v1:172.21.15.70:6789/0 conn(0x7fcdb8072c50 legacy=0x7fcdb80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 -- 172.21.15.3:0/3851701003 >> v1:172.21.15.3:6789/0 conn(0x7fcdb80b4c90 legacy=0x7fcdb8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.177+0000 7fcdbd400640 1 -- 172.21.15.3:0/3851701003 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb815f590 con 0x7fcdb815b230 2024-09-10T07:50:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.178+0000 7fcdbd400640 1 --2- 172.21.15.3:0/3851701003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb815b230 0x7fcdb815c390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009870 tx=0x7fcdac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b44f2e38a41224 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.178+0000 7fcdb7e00640 1 -- 172.21.15.3:0/3851701003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac037070 con 0x7fcdb815b230 2024-09-10T07:50:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.178+0000 7fcdb7e00640 1 -- 172.21.15.3:0/3851701003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac02ae50 con 0x7fcdb815b230 2024-09-10T07:50:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.178+0000 7fcdb7e00640 1 -- 172.21.15.3:0/3851701003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac032860 con 0x7fcdb815b230 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 -- 172.21.15.3:0/3851701003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb815b230 msgr2=0x7fcdb815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/3851701003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb815b230 0x7fcdb815c390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009870 tx=0x7fcdac02a300 comp rx=0 tx=0).stop 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 -- 172.21.15.3:0/3851701003 shutdown_connections 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/3851701003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb815b230 0x7fcdb815c390 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 -- 172.21.15.3:0/3851701003 >> 172.21.15.3:0/3851701003 conn(0x7fcdb80b5cb0 msgr2=0x7fcdb806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 -- 172.21.15.3:0/3851701003 shutdown_connections 2024-09-10T07:50:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.179+0000 7fcdbfe00640 1 -- 172.21.15.3:0/3851701003 wait complete. 2024-09-10T07:50:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.181+0000 7fcdbfe00640 1 Processor -- start 2024-09-10T07:50:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.181+0000 7fcdbfe00640 1 -- start start 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb8072c50 0x7fcdb8150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdb815b230 0x7fcdb81510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb815b610 con 0x7fcdb80b4c90 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb815c8d0 con 0x7fcdb8072c50 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbfe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb815b790 con 0x7fcdb815b230 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53278/0 (socket says 172.21.15.3:53278) 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbca00640 1 -- 172.21.15.3:0/4247698067 learned_addr learned my addr 172.21.15.3:0/4247698067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb8072c50 0x7fcdb8150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.182+0000 7fcdbde00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdb815b230 0x7fcdb81510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 -- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdb815b230 msgr2=0x7fcdb81510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdb815b230 0x7fcdb81510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 -- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb8072c50 msgr2=0x7fcdb8150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb8072c50 0x7fcdb8150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 -- 172.21.15.3:0/4247698067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb81451a0 con 0x7fcdb80b4c90 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbca00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c400 tx=0x7fcda400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdb5600640 1 -- 172.21.15.3:0/4247698067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40052d0 con 0x7fcdb80b4c90 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdb5600640 1 -- 172.21.15.3:0/4247698067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda4005490 con 0x7fcdb80b4c90 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb814f4c0 con 0x7fcdb80b4c90 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdb5600640 1 -- 172.21.15.3:0/4247698067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4007260 con 0x7fcdb80b4c90 2024-09-10T07:50:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.183+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb814fa80 con 0x7fcdb80b4c90 2024-09-10T07:50:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.185+0000 7fcdb5600640 1 -- 172.21.15.3:0/4247698067 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcda4007400 con 0x7fcdb80b4c90 2024-09-10T07:50:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.187+0000 7fcdb5600640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.187+0000 7fcdb5600640 1 -- 172.21.15.3:0/4247698067 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fcda408a3d0 con 0x7fcdb80b4c90 2024-09-10T07:50:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.187+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.187+0000 7fcdc2c94740 1 Processor -- start 2024-09-10T07:50:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.188+0000 7fcdc2c94740 1 -- start start 2024-09-10T07:50:17.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.188+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.188+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9800 0x55cc0d6fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6af480 0x55cc0d6fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc0d617660 con 0x55cc0d6c9420 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc0d5bf680 con 0x55cc0d6c9800 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc0d613620 con 0x55cc0d6af480 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53294/0 (socket says 172.21.15.3:53294) 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 -- 172.21.15.3:0/851322211 learned_addr learned my addr 172.21.15.3:0/851322211 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbca00640 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9800 0x55cc0d6fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbde00640 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6af480 0x55cc0d6fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 -- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6af480 msgr2=0x55cc0d6fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6af480 0x55cc0d6fa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 -- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9800 msgr2=0x55cc0d6fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9800 0x55cc0d6fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.189+0000 7fcdbd400640 1 -- 172.21.15.3:0/851322211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc0d3f47d0 con 0x55cc0d6c9420 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcdbd400640 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcdac03e040 tx=0x7fcdac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a7b83001bd2a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcda9c00640 1 -- 172.21.15.3:0/851322211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac037040 con 0x55cc0d6c9420 2024-09-10T07:50:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcda9c00640 1 -- 172.21.15.3:0/851322211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac04d070 con 0x55cc0d6c9420 2024-09-10T07:50:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcda9c00640 1 -- 172.21.15.3:0/851322211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac03c440 con 0x55cc0d6c9420 2024-09-10T07:50:17.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcdc2c94740 1 -- 172.21.15.3:0/851322211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 msgr2=0x55cc0d6f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.190+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcdac03e040 tx=0x7fcdac02c000 comp rx=0 tx=0).stop 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 -- 172.21.15.3:0/851322211 shutdown_connections 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6af480 0x55cc0d6fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9800 0x55cc0d6fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/851322211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f9c40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 -- 172.21.15.3:0/851322211 >> 172.21.15.3:0/851322211 conn(0x55cc0d6b0220 msgr2=0x55cc0d6f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 -- 172.21.15.3:0/851322211 shutdown_connections 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 -- 172.21.15.3:0/851322211 wait complete. 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.191+0000 7fcdc2c94740 1 -- start start 2024-09-10T07:50:17.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.192+0000 7fcdc2c94740 1 Processor -- start 2024-09-10T07:50:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc0d6ae3e0 con 0x55cc0d6c9420 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55cc0d5a04c0 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc0d6ae6c0 con 0x55cc0d6c9800 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdc2c94740 10 client.0 _ll_register_callbacks cb 0x55cc0d60fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53310/0 (socket says 172.21.15.3:53310) 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 -- 172.21.15.3:0/3806885404 learned_addr learned my addr 172.21.15.3:0/3806885404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbde00640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbca00640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 -- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 msgr2=0x55cc0d6dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 -- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 msgr2=0x55cc0d6b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbd400640 1 -- 172.21.15.3:0/3806885404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc0d32a3f0 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.193+0000 7fcdbca00640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcdbd400640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495fa631a7b2b0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac057070 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac04d070 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fcdac03c910 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcdac062050 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcdc2c94740 10 client.6354 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55cc0d2ea0f0 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac04d070 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7fcdac065020 con 0x55cc0d6c9420 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.194+0000 7fcdc2c94740 10 client.6354 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdc2c94740 10 client.6354 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-09-10T07:50:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdc2c94740 20 client.6354 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6b5f40 0x55cc0d6f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdc2c94740 4 client.6354 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc0d624d90 con 0x55cc0d6b5f40 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdbca00640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6b5f40 0x55cc0d6f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcdbca00640 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6b5f40 0x55cc0d6f2e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1286d338d90f8e22 server_cookie=8dc44902ec1c00d4 in_seq=0 out_seq=0 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.195+0000 7fcd9e000640 10 client.6354 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcd9e000640 1 -- 172.21.15.3:0/3806885404 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc0d624d90 con 0x55cc0d6b5f40 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcd9e000640 10 client.6354 handle_command_reply: tid=0 2024-09-10T07:50:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 client.6354 shutdown 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 msgr2=0x55cc0d6f5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac004730 comp rx=0 tx=0).stop 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 shutdown_connections 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6b5f40 0x55cc0d6f2e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/3806885404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f5bb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcd9e000640 0 client.6354 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:17.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 >> 172.21.15.3:0/3806885404 conn(0x55cc0d6af480 msgr2=0x55cc0d6f6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 shutdown_connections 2024-09-10T07:50:17.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.196+0000 7fcdc2c94740 1 -- 172.21.15.3:0/3806885404 wait complete. 2024-09-10T07:50:17.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.197+0000 7fcdc2c94740 20 client.6354 trim_cache size 0 max 16384 2024-09-10T07:50:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.212+0000 7fcdc2c94740 1 Processor -- start 2024-09-10T07:50:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.212+0000 7fcdc2c94740 1 -- start start 2024-09-10T07:50:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8200db0 con 0x55cc0d6c9420 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb80b0c10 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb8200680 con 0x55cc0d6c9800 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53314/0 (socket says 172.21.15.3:53314) 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 -- 172.21.15.3:0/2653190334 learned_addr learned my addr 172.21.15.3:0/2653190334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbde00640 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbca00640 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 -- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 msgr2=0x55cc0d6f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 -- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 msgr2=0x55cc0d6b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.213+0000 7fcdbd400640 1 -- 172.21.15.3:0/2653190334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc0d6f31e0 con 0x55cc0d6c9420 2024-09-10T07:50:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcdbd400640 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcdac02a8f0 tx=0x7fcdac00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d878d7be695b9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcda9200640 1 -- 172.21.15.3:0/2653190334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac057040 con 0x55cc0d6c9420 2024-09-10T07:50:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcda9200640 1 -- 172.21.15.3:0/2653190334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac04d040 con 0x55cc0d6c9420 2024-09-10T07:50:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcda9200640 1 -- 172.21.15.3:0/2653190334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac05c440 con 0x55cc0d6c9420 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcdc2c94740 1 -- 172.21.15.3:0/2653190334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 msgr2=0x55cc0d6f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.214+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcdac02a8f0 tx=0x7fcdac00b050 comp rx=0 tx=0).stop 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 -- 172.21.15.3:0/2653190334 shutdown_connections 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d6b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d6f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/2653190334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6c9420 0x55cc0d6f3e70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 -- 172.21.15.3:0/2653190334 >> 172.21.15.3:0/2653190334 conn(0x55cc0d6af480 msgr2=0x55cc0d6f7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 -- 172.21.15.3:0/2653190334 shutdown_connections 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 -- 172.21.15.3:0/2653190334 wait complete. 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 -- start start 2024-09-10T07:50:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.215+0000 7fcdc2c94740 1 Processor -- start 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 0x55cc0d6b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d779b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc0d6a9a50 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb815d640 con 0x55cc0d6c9420 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55cc0d6f1c60 con 0x55cc0d6c9800 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.216+0000 7fcdc2c94740 10 client.0 _ll_register_callbacks cb 0x7fcdb0001510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53324/0 (socket says 172.21.15.3:53324) 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 -- 172.21.15.3:0/4080222916 learned_addr learned my addr 172.21.15.3:0/4080222916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 0x55cc0d6b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbca00640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d779b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 -- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 msgr2=0x55cc0d779b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d779b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 -- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 msgr2=0x55cc0d6b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 0x55cc0d6b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 -- 172.21.15.3:0/4080222916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc0d6c8d90 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 0x55cc0d6b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcdbde00640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00012e0 tx=0x7fcdb0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ccf6bf4d872333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.217+0000 7fcd9e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0007a30 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fcdb0017b20 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac04d040 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcdb0017d20 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 10 client.6360 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55cc0d6fc060 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac04d040 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7fcdb0015440 con 0x55cc0d6ddfb0 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 10 client.6360 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 10 client.6360 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 20 client.6360 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6ae180 0x55cc0d7121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 4 client.6360 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc0d6b8f80 con 0x55cc0d6ae180 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.218+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6ae180 0x55cc0d7121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.219+0000 7fcdbd400640 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6ae180 0x55cc0d7121f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5c4ca053e45fdbb server_cookie=4da691f92ed9f339 in_seq=0 out_seq=0 2024-09-10T07:50:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.219+0000 7fcd9e000640 10 client.6360 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcd9e000640 1 -- 172.21.15.3:0/4080222916 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc0d6b8f80 con 0x55cc0d6ae180 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcd9e000640 10 client.6360 handle_command_reply: tid=0 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 client.6360 shutdown 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 msgr2=0x55cc0d711cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00012e0 tx=0x7fcdb0007780 comp rx=0 tx=0).stop 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 shutdown_connections 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55cc0d6ae180 0x55cc0d7121f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc0d6ddfb0 0x55cc0d711cb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55cc0d6c9800 0x55cc0d779b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 --2- 172.21.15.3:0/4080222916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55cc0d6c9420 0x55cc0d6b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcd9e000640 0 client.6360 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 >> 172.21.15.3:0/4080222916 conn(0x55cc0d6af480 msgr2=0x55cc0d6f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 shutdown_connections 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 1 -- 172.21.15.3:0/4080222916 wait complete. 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.230+0000 7fcdc2c94740 20 client.6360 trim_cache size 0 max 16384 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:17.231 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:50:17.232 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:50:17.232 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 msgr2=0x7fcd9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 msgr2=0x7fcdb8144c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c400 tx=0x7fcda400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 shutdown_connections 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdb815b230 0x7fcdb81510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb80b4c90 0x7fcdb8144c60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 --2- 172.21.15.3:0/4247698067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb8072c50 0x7fcdb8150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 >> 172.21.15.3:0/4247698067 conn(0x7fcdb80b5cb0 msgr2=0x7fcdb81521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.231+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 shutdown_connections 2024-09-10T07:50:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.232+0000 7fcdbfe00640 1 -- 172.21.15.3:0/4247698067 wait complete. 2024-09-10T07:50:17.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2024-09-10T07:50:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.405+0000 7f9f87a00640 1 Processor -- start 2024-09-10T07:50:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.405+0000 7f9f87a00640 1 -- start start 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.406+0000 7f9f87a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9f800cf980 0x7f9f8006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.406+0000 7f9f87a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f800cd900 con 0x7f9f801451d0 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.406+0000 7f9f87a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f800cdbe0 con 0x7f9f80145730 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.406+0000 7f9f87a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8006d7c0 con 0x7f9f800cf980 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.406+0000 7f9f85000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f801451d0 0x7f9f800b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59764/0 (socket says 172.21.15.3:59764) 2024-09-10T07:50:17.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f85000640 1 -- 172.21.15.3:0/1409767385 learned_addr learned my addr 172.21.15.3:0/1409767385 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f85a00640 1 --2- 172.21.15.3:0/1409767385 >> v2:172.21.15.70:3300/0 conn(0x7f9f800cf980 0x7f9f8006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067490797 0 0) 0x7f9f800cd900 con 0x7f9f801451d0 2024-09-10T07:50:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f68003520 con 0x7f9f801451d0 2024-09-10T07:50:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f85a00640 1 -- 172.21.15.3:0/1409767385 >> v1:172.21.15.70:6789/0 conn(0x7f9f80145730 legacy=0x7f9f800cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510092936 0 0) 0x7f9f800cdbe0 con 0x7f9f80145730 2024-09-10T07:50:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f85a00640 1 -- 172.21.15.3:0/1409767385 >> v1:172.21.15.3:6789/0 conn(0x7f9f801451d0 legacy=0x7f9f800b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.407+0000 7f9f85a00640 1 -- 172.21.15.3:0/1409767385 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8006da10 con 0x7f9f800cf980 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.408+0000 7f9f85a00640 1 --2- 172.21.15.3:0/1409767385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f800cf980 0x7f9f8006cfc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f8006d940 tx=0x7f9f7802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=572f90428a931ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.408+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007740 con 0x7f9f800cf980 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.408+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f780078e0 con 0x7f9f800cf980 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.408+0000 7f9f7f400640 1 -- 172.21.15.3:0/1409767385 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007c90 con 0x7f9f800cf980 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 -- 172.21.15.3:0/1409767385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f800cf980 msgr2=0x7f9f8006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 --2- 172.21.15.3:0/1409767385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f800cf980 0x7f9f8006cfc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f8006d940 tx=0x7f9f7802a660 comp rx=0 tx=0).stop 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 -- 172.21.15.3:0/1409767385 shutdown_connections 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 --2- 172.21.15.3:0/1409767385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f800cf980 0x7f9f8006cfc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 -- 172.21.15.3:0/1409767385 >> 172.21.15.3:0/1409767385 conn(0x7f9f800b5cb0 msgr2=0x7f9f8006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 -- 172.21.15.3:0/1409767385 shutdown_connections 2024-09-10T07:50:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.409+0000 7f9f87a00640 1 -- 172.21.15.3:0/1409767385 wait complete. 2024-09-10T07:50:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.411+0000 7f9f87a00640 1 Processor -- start 2024-09-10T07:50:17.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.411+0000 7f9f87a00640 1 -- start start 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 0x7f9f8015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f80145730 0x7f9f800c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8006e0b0 con 0x7f9f801451d0 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8006df30 con 0x7f9f80145730 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f87a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8006e230 con 0x7f9f800cf980 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53330/0 (socket says 172.21.15.3:53330) 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f7fe00640 1 -- 172.21.15.3:0/2221987361 learned_addr learned my addr 172.21.15.3:0/2221987361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f85000640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 0x7f9f8015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.412+0000 7f9f85a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f80145730 0x7f9f800c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 -- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 msgr2=0x7f9f8015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 0x7f9f8015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 -- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f80145730 msgr2=0x7f9f800c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f80145730 0x7f9f800c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 -- 172.21.15.3:0/2221987361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f800c5530 con 0x7f9f801451d0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f85000640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 0x7f9f8015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c410 tx=0x7f9f6c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7cc00640 1 -- 172.21.15.3:0/2221987361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c017070 con 0x7f9f801451d0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7cc00640 1 -- 172.21.15.3:0/2221987361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c0035f0 con 0x7f9f801451d0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f7cc00640 1 -- 172.21.15.3:0/2221987361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0039e0 con 0x7f9f801451d0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f800c5810 con 0x7f9f801451d0 2024-09-10T07:50:17.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.413+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8015cdd0 con 0x7f9f801451d0 2024-09-10T07:50:17.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.416+0000 7f9f7cc00640 1 -- 172.21.15.3:0/2221987361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f6c01a070 con 0x7f9f801451d0 2024-09-10T07:50:17.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.417+0000 7f9f7cc00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 0x7f9f6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.417+0000 7f9f7cc00640 1 -- 172.21.15.3:0/2221987361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f9f6c08a280 con 0x7f9f801451d0 2024-09-10T07:50:17.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.417+0000 7f9f85000640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 0x7f9f6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.418+0000 7f9f8a7d5740 1 Processor -- start 2024-09-10T07:50:17.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.418+0000 7f9f8a7d5740 1 -- start start 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.418+0000 7f9f85000640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 0x7f9f6006e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f74007fd0 tx=0x7f9f74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d55c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25800 0x55ef88d56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef88c73660 con 0x55ef88d0b480 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef88c1b680 con 0x55ef88d25800 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef88c6f620 con 0x55ef88d25420 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53338/0 (socket says 172.21.15.3:53338) 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f85a00640 1 -- 172.21.15.3:0/511478053 learned_addr learned my addr 172.21.15.3:0/511478053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25800 0x55ef88d56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f85000640 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d55c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.419+0000 7f9f85a00640 1 -- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 msgr2=0x55ef88d55c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f85a00640 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d55c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f85a00640 1 -- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25800 msgr2=0x55ef88d56200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f85a00640 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25800 0x55ef88d56200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f85a00640 1 -- 172.21.15.3:0/511478053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef88a507d0 con 0x55ef88d0b480 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f85a00640 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9f7802ac50 tx=0x7f9f7802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b8ee0209407c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f71600640 1 -- 172.21.15.3:0/511478053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78003e40 con 0x55ef88d0b480 2024-09-10T07:50:17.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f71600640 1 -- 172.21.15.3:0/511478053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78004040 con 0x55ef88d0b480 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.420+0000 7f9f71600640 1 -- 172.21.15.3:0/511478053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f780043f0 con 0x55ef88d0b480 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/511478053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 msgr2=0x55ef88d56740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9f7802ac50 tx=0x7f9f7802e000 comp rx=0 tx=0).stop 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/511478053 shutdown_connections 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d0b480 0x55ef88d56740 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25800 0x55ef88d56200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/511478053 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d55c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/511478053 >> 172.21.15.3:0/511478053 conn(0x55ef88d0c220 msgr2=0x55ef88d54840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/511478053 shutdown_connections 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.421+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/511478053 wait complete. 2024-09-10T07:50:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.422+0000 7f9f8a7d5740 1 -- start start 2024-09-10T07:50:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.422+0000 7f9f8a7d5740 1 Processor -- start 2024-09-10T07:50:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d68b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d2f590 0x55ef88d4cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef88d0a3e0 con 0x55ef88d25800 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef88bfc4c0 con 0x55ef88d25420 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef88d0a6c0 con 0x55ef88d2f590 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f8a7d5740 10 client.0 _ll_register_callbacks cb 0x55ef88c6be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53346/0 (socket says 172.21.15.3:53346) 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d2f590 0x55ef88d4cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1634230406 learned_addr learned my addr 172.21.15.3:0/1634230406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.423+0000 7f9f85000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d68b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d2f590 msgr2=0x55ef88d4cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d2f590 0x55ef88d4cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 msgr2=0x55ef88d68b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d68b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1634230406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef889863f0 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f85000640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d68b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3e0 tx=0x7f9f6c058680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3708f2177c78bd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f66000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c070020 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f9f6c003d80 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78004040 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c075050 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f8a7d5740 10 client.6372 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55ef889460f0 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.424+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78004040 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f9f6c088030 con 0x55ef88d25800 2024-09-10T07:50:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 10 client.6372 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 10 client.6372 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 20 client.6372 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88d4a7e0 0x55ef88d4ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 4 client.6372 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef88c80d90 con 0x55ef88d4a7e0 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.425+0000 7f9f85000640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88d4a7e0 0x55ef88d4ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.426+0000 7f9f85000640 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88d4a7e0 0x55ef88d4ac00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9ce85bfc42769d1 server_cookie=d75e56e507bfba99 in_seq=0 out_seq=0 2024-09-10T07:50:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.426+0000 7f9f66000640 10 client.6372 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.426+0000 7f9f66000640 1 -- 172.21.15.3:0/1634230406 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef88c80d90 con 0x55ef88d4a7e0 2024-09-10T07:50:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.426+0000 7f9f66000640 10 client.6372 handle_command_reply: tid=0 2024-09-10T07:50:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 client.6372 shutdown 2024-09-10T07:50:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 msgr2=0x55ef88d29330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3e0 tx=0x7f9f6c058680 comp rx=0 tx=0).stop 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 shutdown_connections 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88d4a7e0 0x55ef88d4ac00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d2f590 0x55ef88d4cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d29330 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1634230406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d68b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 >> 172.21.15.3:0/1634230406 conn(0x55ef88d0b480 msgr2=0x55ef88d08f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f66000640 0 client.6372 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 shutdown_connections 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1634230406 wait complete. 2024-09-10T07:50:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.427+0000 7f9f8a7d5740 20 client.6372 trim_cache size 0 max 16384 2024-09-10T07:50:17.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.442+0000 7f9f8a7d5740 1 Processor -- start 2024-09-10T07:50:17.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 -- start start 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d2e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25800 0x55ef88d09150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f802008c0 con 0x55ef88d2f590 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801ffac0 con 0x55ef88d25420 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.443+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9f80200190 con 0x55ef88d25800 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53358/0 (socket says 172.21.15.3:53358) 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 -- 172.21.15.3:0/4199059216 learned_addr learned my addr 172.21.15.3:0/4199059216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85000640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d2e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25800 0x55ef88d09150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 -- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25800 msgr2=0x55ef88d09150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25800 0x55ef88d09150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 -- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 msgr2=0x55ef88d2e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d2e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 -- 172.21.15.3:0/4199059216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef88d06050 con 0x55ef88d2f590 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85000640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d2e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.444+0000 7f9f85a00640 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9f780076d0 tx=0x7f9f78030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db91c2ae0ec4c202 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f70c00640 1 -- 172.21.15.3:0/4199059216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007a30 con 0x55ef88d2f590 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f70c00640 1 -- 172.21.15.3:0/4199059216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78007bd0 con 0x55ef88d2f590 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f70c00640 1 -- 172.21.15.3:0/4199059216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78046480 con 0x55ef88d2f590 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/4199059216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 msgr2=0x55ef88d28c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9f780076d0 tx=0x7f9f78030000 comp rx=0 tx=0).stop 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/4199059216 shutdown_connections 2024-09-10T07:50:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d2f590 0x55ef88d28c00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25800 0x55ef88d09150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/4199059216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d25420 0x55ef88d2e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/4199059216 >> 172.21.15.3:0/4199059216 conn(0x55ef88d0b480 msgr2=0x55ef88d52ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/4199059216 shutdown_connections 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.445+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/4199059216 wait complete. 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.446+0000 7f9f8a7d5740 1 -- start start 2024-09-10T07:50:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.446+0000 7f9f8a7d5740 1 Processor -- start 2024-09-10T07:50:17.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d2f590 0x55ef88dd2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef88d4df70 con 0x55ef88d25800 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801f7820 con 0x55ef88d2f590 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef88d09ba0 con 0x55ef88d25420 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f8a7d5740 10 client.0 _ll_register_callbacks cb 0x7f9f6c0581c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53370/0 (socket says 172.21.15.3:53370) 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1143462828 learned_addr learned my addr 172.21.15.3:0/1143462828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.447+0000 7f9f85000640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f85a00640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d2f590 0x55ef88dd2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 msgr2=0x55ef88d23ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d23ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d2f590 msgr2=0x55ef88dd2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d2f590 0x55ef88dd2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 -- 172.21.15.3:0/1143462828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef88d50e80 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f7fe00640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c058520 tx=0x7f9f6c088630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96dedda5a805c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f66000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c070070 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f9f6c0579e0 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.448+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78007bd0 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9f6c080050 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 10 client.6378 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55ef88d24540 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78007bd0 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f9f6c088040 con 0x55ef88d25800 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 10 client.6378 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 10 client.6378 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 20 client.6378 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88dd3030 0x55ef88dd3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 4 client.6378 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:50:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef88c7b0e0 con 0x55ef88dd3030 2024-09-10T07:50:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.449+0000 7f9f85000640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88dd3030 0x55ef88dd3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.450+0000 7f9f85000640 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88dd3030 0x55ef88dd3410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffa4ba60310119c2 server_cookie=355eb13d62126e52 in_seq=0 out_seq=0 2024-09-10T07:50:17.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.450+0000 7f9f66000640 10 client.6378 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f66000640 1 -- 172.21.15.3:0/1143462828 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef88c7b0e0 con 0x55ef88dd3030 2024-09-10T07:50:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f66000640 10 client.6378 handle_command_reply: tid=0 2024-09-10T07:50:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 client.6378 shutdown 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 msgr2=0x55ef88d24000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c058520 tx=0x7f9f6c088630 comp rx=0 tx=0).stop 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 shutdown_connections 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55ef88dd3030 0x55ef88dd3410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef88d2f590 0x55ef88dd2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef88d25800 0x55ef88d24000 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 --2- 172.21.15.3:0/1143462828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef88d25420 0x55ef88d23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f66000640 0 client.6378 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 >> 172.21.15.3:0/1143462828 conn(0x55ef88d0b480 msgr2=0x55ef88d2edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 shutdown_connections 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.460+0000 7f9f8a7d5740 1 -- 172.21.15.3:0/1143462828 wait complete. 2024-09-10T07:50:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f8a7d5740 20 client.6378 trim_cache size 0 max 16384 2024-09-10T07:50:17.462 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:17.462 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:50:17.462 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:50:17.462 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 msgr2=0x7f9f6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 0x7f9f6006e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f74007fd0 tx=0x7f9f74009040 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 msgr2=0x7f9f8015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c410 tx=0x7f9f6c00c8d0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 shutdown_connections 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9f6006be30 0x7f9f6006e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f80145730 0x7f9f800c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801451d0 0x7f9f8015bdb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 --2- 172.21.15.3:0/2221987361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9f800cf980 0x7f9f8015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.461+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 >> 172.21.15.3:0/2221987361 conn(0x7f9f800b5cb0 msgr2=0x7f9f800cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.462+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 shutdown_connections 2024-09-10T07:50:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17.462+0000 7f9f87a00640 1 -- 172.21.15.3:0/2221987361 wait complete. 2024-09-10T07:50:17.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:17.476 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-10T07:50:17.518 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/ 2024-09-10T07:50:17.518 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4 2024-09-10T07:50:17.519 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5 2024-09-10T07:50:17.519 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6 2024-09-10T07:50:17.519 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-10T07:50:17.519 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-10T07:50:17.520 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:17.661 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627778 2024-09-10T07:50:17.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:17.672 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:17.673 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:17.673 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:17.673 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:17.673 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-10T07:50:17.673 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:17.812 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627779 2024-09-10T07:50:17.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:17.823 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:17.823 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:17.824 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:17.824 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:17.824 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-10T07:50:17.824 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:17.964 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627780 2024-09-10T07:50:17.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:17.976 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:17.976 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:17.976 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:17.977 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:17.977 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-10T07:50:17.977 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:18.117 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627781 2024-09-10T07:50:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:18.128 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:18.129 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:18.129 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:18.129 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:18.129 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-10T07:50:18.129 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:18.271 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627782 2024-09-10T07:50:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-10T07:50:18.282 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:50:18.423 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627783 2024-09-10T07:50:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:50:18.436 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-10T07:50:18.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.552+0000 7f4481c35f40 1 Processor -- start 2024-09-10T07:50:18.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 -- start start 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 0x55650df496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deca130 0x55650df4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55650df57900 con 0x55650deb2220 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55650df57620 con 0x55650deb1e40 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.553+0000 7f4481c35f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55650df57be0 con 0x55650deca130 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53376/0 (socket says 172.21.15.3:53376) 2024-09-10T07:50:18.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 -- 172.21.15.3:0/603481832 learned_addr learned my addr 172.21.15.3:0/603481832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447f800640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deca130 0x55650df4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 -- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deca130 msgr2=0x55650df4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447ee00640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 0x55650df496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deca130 0x55650df4d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 -- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 msgr2=0x55650df496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 0x55650df496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 -- 172.21.15.3:0/603481832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650ddaab30 con 0x55650deb2220 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447ee00640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 0x55650df496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447e400640 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f446c00a5e0 tx=0x7f446c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f292fbd755823ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.554+0000 7f447d000640 1 -- 172.21.15.3:0/603481832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c008a40 con 0x55650deb2220 2024-09-10T07:50:18.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f447d000640 1 -- 172.21.15.3:0/603481832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c008c00 con 0x55650deb2220 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f447d000640 1 -- 172.21.15.3:0/603481832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c013440 con 0x55650deb2220 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 -- 172.21.15.3:0/603481832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 msgr2=0x55650df4d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f446c00a5e0 tx=0x7f446c005f90 comp rx=0 tx=0).stop 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 -- 172.21.15.3:0/603481832 shutdown_connections 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deca130 0x55650df4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deb2220 0x55650df4d280 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 --2- 172.21.15.3:0/603481832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb1e40 0x55650df496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 -- 172.21.15.3:0/603481832 >> 172.21.15.3:0/603481832 conn(0x55650deaed40 msgr2=0x55650de7f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 -- 172.21.15.3:0/603481832 shutdown_connections 2024-09-10T07:50:18.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.555+0000 7f4481c35f40 1 -- 172.21.15.3:0/603481832 wait complete. 2024-09-10T07:50:18.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.556+0000 7f4481c35f40 1 Processor -- start 2024-09-10T07:50:18.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.556+0000 7f4481c35f40 1 -- start start 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deb1e40 0x55650df49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb2220 0x55650df46240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 0x55650df4de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55650df505f0 con 0x55650deca130 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55650dead5b0 con 0x55650deb2220 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f4481c35f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55650dead280 con 0x55650deb1e40 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f447f800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 0x55650df4de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f447ee00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deb1e40 0x55650df49cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f447f800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 0x55650df4de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53392/0 (socket says 172.21.15.3:53392) 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f447f800640 1 -- 172.21.15.3:0/3001195045 learned_addr learned my addr 172.21.15.3:0/3001195045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.557+0000 7f447e400640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb2220 0x55650df46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 -- 172.21.15.3:0/3001195045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deb1e40 msgr2=0x55650df49cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55650deb1e40 0x55650df49cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 -- 172.21.15.3:0/3001195045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb2220 msgr2=0x55650df46240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55650deb2220 0x55650df46240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 -- 172.21.15.3:0/3001195045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650df46780 con 0x55650deca130 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f447f800640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 0x55650df4de10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f447800a4b0 tx=0x7f4478003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f4477400640 1 -- 172.21.15.3:0/3001195045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478068070 con 0x55650deca130 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f4477400640 1 -- 172.21.15.3:0/3001195045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4478059450 con 0x55650deca130 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f4477400640 1 -- 172.21.15.3:0/3001195045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478054040 con 0x55650deca130 2024-09-10T07:50:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55650df4a420 con 0x55650deca130 2024-09-10T07:50:18.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.558+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650df4a9e0 con 0x55650deca130 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.560+0000 7f4477400640 1 -- 172.21.15.3:0/3001195045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4478057070 con 0x55650deca130 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.561+0000 7f4477400640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f446006a810 0x7f446006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.561+0000 7f4477400640 1 -- 172.21.15.3:0/3001195045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f44780cd320 con 0x55650deca130 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.561+0000 7f447ee00640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f446006a810 0x7f446006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.561+0000 7f4481c35f40 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55650dfed2d0 0x55650df4b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.561+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- osd_op(unknown.0.0:1 8.12 8:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x55650df4ba00 con 0x55650dfed2d0 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.562+0000 7f447e400640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55650dfed2d0 0x55650df4b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.562+0000 7f447ee00640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f446006a810 0x7f446006ccd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4470007750 tx=0x7f4470005a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.562+0000 7f447e400640 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55650dfed2d0 0x55650df4b4c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f447e400640 1 -- 172.21.15.3:0/3001195045 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v66'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f446c008840 con 0x55650dfed2d0 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55650dfed2d0 msgr2=0x55650df4b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55650dfed2d0 0x55650df4b4c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f446006a810 msgr2=0x7f446006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f446006a810 0x7f446006ccd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4470007750 tx=0x7f4470005a20 comp rx=0 tx=0).stop 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 msgr2=0x55650df4de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 --2- 172.21.15.3:0/3001195045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55650deca130 0x55650df4de10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f447800a4b0 tx=0x7f4478003330 comp rx=0 tx=0).stop 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f447f800640 1 -- 172.21.15.3:0/3001195045 reap_dead start 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 shutdown_connections 2024-09-10T07:50:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 >> 172.21.15.3:0/3001195045 conn(0x55650deaed40 msgr2=0x55650de7f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 shutdown_connections 2024-09-10T07:50:18.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.564+0000 7f4481c35f40 1 -- 172.21.15.3:0/3001195045 wait complete. 2024-09-10T07:50:18.568 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-10T07:50:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.645+0000 7f50d8520f40 1 Processor -- start 2024-09-10T07:50:18.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.645+0000 7f50d8520f40 1 -- start start 2024-09-10T07:50:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a676f7a0 0x55c1a67fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6770e40 0x55c1a6788c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6770150 con 0x55c1a6771220 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c1a66e5b30 con 0x55c1a6770e40 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d8520f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c1a6816620 con 0x55c1a676f7a0 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53408/0 (socket says 172.21.15.3:53408) 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d5800640 1 -- 172.21.15.3:0/532081697 learned_addr learned my addr 172.21.15.3:0/532081697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50d4e00640 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a676f7a0 0x55c1a67fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.646+0000 7f50cfe00640 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6770e40 0x55c1a6788c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 -- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a676f7a0 msgr2=0x55c1a67fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a676f7a0 0x55c1a67fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 -- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6770e40 msgr2=0x55c1a6788c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6770e40 0x55c1a6788c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 -- 172.21.15.3:0/532081697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a67fb140 con 0x55c1a6771220 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50d5800640 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f50d000a4b0 tx=0x7f50d0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b690367a924157 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50cea00640 1 -- 172.21.15.3:0/532081697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0054040 con 0x55c1a6771220 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50cea00640 1 -- 172.21.15.3:0/532081697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d0050820 con 0x55c1a6771220 2024-09-10T07:50:18.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.647+0000 7f50cea00640 1 -- 172.21.15.3:0/532081697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0050c10 con 0x55c1a6771220 2024-09-10T07:50:18.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 -- 172.21.15.3:0/532081697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 msgr2=0x55c1a6789160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f50d000a4b0 tx=0x7f50d0050000 comp rx=0 tx=0).stop 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 -- 172.21.15.3:0/532081697 shutdown_connections 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a6771220 0x55c1a6789160 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6770e40 0x55c1a6788c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 --2- 172.21.15.3:0/532081697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a676f7a0 0x55c1a67fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 -- 172.21.15.3:0/532081697 >> 172.21.15.3:0/532081697 conn(0x55c1a676dd40 msgr2=0x55c1a676e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 -- 172.21.15.3:0/532081697 shutdown_connections 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.648+0000 7f50d8520f40 1 -- 172.21.15.3:0/532081697 wait complete. 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.649+0000 7f50d8520f40 1 Processor -- start 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.649+0000 7f50d8520f40 1 -- start start 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 0x55c1a689f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a6770e40 0x55c1a67f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6771220 0x55c1a6788b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c1a680f040 con 0x55c1a676f7a0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c1a680f600 con 0x55c1a6771220 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d8520f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c1a680f320 con 0x55c1a6770e40 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 0x55c1a689f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 0x55c1a689f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53410/0 (socket says 172.21.15.3:53410) 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 -- 172.21.15.3:0/3258356613 learned_addr learned my addr 172.21.15.3:0/3258356613 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50cfe00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a6770e40 0x55c1a67f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d5800640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6771220 0x55c1a6788b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 -- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a6770e40 msgr2=0x55c1a67f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1a6770e40 0x55c1a67f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 -- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6771220 msgr2=0x55c1a6788b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1a6771220 0x55c1a6788b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.650+0000 7f50d4e00640 1 -- 172.21.15.3:0/3258356613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a67f7a10 con 0x55c1a676f7a0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50d4e00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 0x55c1a689f830 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002d10 tx=0x7f50c800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50cd600640 1 -- 172.21.15.3:0/3258356613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8015020 con 0x55c1a676f7a0 2024-09-10T07:50:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50cd600640 1 -- 172.21.15.3:0/3258356613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c8011450 con 0x55c1a676f7a0 2024-09-10T07:50:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50cd600640 1 -- 172.21.15.3:0/3258356613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8011820 con 0x55c1a676f7a0 2024-09-10T07:50:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1a67f7cf0 con 0x55c1a676f7a0 2024-09-10T07:50:18.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.651+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a67f82b0 con 0x55c1a676f7a0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.653+0000 7f50cd600640 1 -- 172.21.15.3:0/3258356613 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50c8018020 con 0x55c1a676f7a0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50cd600640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f50b406a810 0x7f50b406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50cd600640 1 -- 172.21.15.3:0/3258356613 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f50c8089c90 con 0x55c1a676f7a0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50cfe00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f50b406a810 0x7f50b406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50d8520f40 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55c1a67890d0 0x55c1a68134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- osd_op(unknown.0.0:1 8.12 8:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x55c1a6813a00 con 0x55c1a67890d0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.654+0000 7f50d5800640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55c1a67890d0 0x55c1a68134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.655+0000 7f50cfe00640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f50b406a810 0x7f50b406ccd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c1a6818ad0 tx=0x7f50c000c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.655+0000 7f50d5800640 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55c1a67890d0 0x55c1a68134c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d5800640 1 -- 172.21.15.3:0/3258356613 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v66'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f50d0056030 con 0x55c1a67890d0 2024-09-10T07:50:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55c1a67890d0 msgr2=0x55c1a68134c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x55c1a67890d0 0x55c1a68134c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f50b406a810 msgr2=0x7f50b406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f50b406a810 0x7f50b406ccd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c1a6818ad0 tx=0x7f50c000c040 comp rx=0 tx=0).stop 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 msgr2=0x55c1a689f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d8520f40 1 --2- 172.21.15.3:0/3258356613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1a676f7a0 0x55c1a689f830 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002d10 tx=0x7f50c800c620 comp rx=0 tx=0).stop 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.656+0000 7f50d5800640 1 -- 172.21.15.3:0/3258356613 reap_dead start 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.657+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 shutdown_connections 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.657+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 >> 172.21.15.3:0/3258356613 conn(0x55c1a676dd40 msgr2=0x55c1a676e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.657+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 shutdown_connections 2024-09-10T07:50:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.657+0000 7f50d8520f40 1 -- 172.21.15.3:0/3258356613 wait complete. 2024-09-10T07:50:18.661 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-10T07:50:18.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.737+0000 7f7abdfcaf40 1 Processor -- start 2024-09-10T07:50:18.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.737+0000 7f7abdfcaf40 1 -- start start 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8ce40 0x561240e242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8d220 0x561240e34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561240d8c150 con 0x561240d8b7a0 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561240d01b30 con 0x561240d8d220 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561240e32620 con 0x561240d8ce40 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53422/0 (socket says 172.21.15.3:53422) 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.738+0000 7f7abb200640 1 -- 172.21.15.3:0/3356560181 learned_addr learned my addr 172.21.15.3:0/3356560181 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7aba800640 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8ce40 0x561240e242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abbc00640 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8d220 0x561240e34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 -- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8ce40 msgr2=0x561240e242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8ce40 0x561240e242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 -- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8d220 msgr2=0x561240e34000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8d220 0x561240e34000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 -- 172.21.15.3:0/3356560181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561240d89290 con 0x561240d8b7a0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7abb200640 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7aac001d80 tx=0x7f7aac001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedcedc4d98beea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7ab9400640 1 -- 172.21.15.3:0/3356560181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00bd70 con 0x561240d8b7a0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7ab9400640 1 -- 172.21.15.3:0/3356560181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac00f400 con 0x561240d8b7a0 2024-09-10T07:50:18.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.739+0000 7f7ab9400640 1 -- 172.21.15.3:0/3356560181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00f7f0 con 0x561240d8b7a0 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/3356560181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 msgr2=0x561240d87fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7aac001d80 tx=0x7f7aac001e70 comp rx=0 tx=0).stop 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/3356560181 shutdown_connections 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8d220 0x561240e34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8ce40 0x561240e242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/3356560181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8b7a0 0x561240d87fe0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/3356560181 >> 172.21.15.3:0/3356560181 conn(0x561240d89d40 msgr2=0x561240d5a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/3356560181 shutdown_connections 2024-09-10T07:50:18.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.740+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/3356560181 wait complete. 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.741+0000 7f7abdfcaf40 1 Processor -- start 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.741+0000 7f7abdfcaf40 1 -- start start 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8b7a0 0x561240e18b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8ce40 0x561240ecb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 0x561240e242b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561240d87d40 con 0x561240d8d220 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561240d87b20 con 0x561240d8ce40 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abdfcaf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561240d89750 con 0x561240d8b7a0 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 0x561240e242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 0x561240e242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53430/0 (socket says 172.21.15.3:53430) 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8b7a0 0x561240e18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 -- 172.21.15.3:0/1379666480 learned_addr learned my addr 172.21.15.3:0/1379666480 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7aba800640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8ce40 0x561240ecb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 -- 172.21.15.3:0/1379666480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8b7a0 msgr2=0x561240e18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561240d8b7a0 0x561240e18b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 -- 172.21.15.3:0/1379666480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8ce40 msgr2=0x561240ecb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561240d8ce40 0x561240ecb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.742+0000 7f7abbc00640 1 -- 172.21.15.3:0/1379666480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561240eca2f0 con 0x561240d8d220 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7abbc00640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 0x561240e242b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4045c50 tx=0x7f7ab4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7ab3e00640 1 -- 172.21.15.3:0/1379666480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4054040 con 0x561240d8d220 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7ab3e00640 1 -- 172.21.15.3:0/1379666480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab4068070 con 0x561240d8d220 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561240eca5d0 con 0x561240d8d220 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7ab3e00640 1 -- 172.21.15.3:0/1379666480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4059690 con 0x561240d8d220 2024-09-10T07:50:18.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.743+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561240ecab90 con 0x561240d8d220 2024-09-10T07:50:18.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.745+0000 7f7ab3e00640 1 -- 172.21.15.3:0/1379666480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ab4057070 con 0x561240d8d220 2024-09-10T07:50:18.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7ab3e00640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a9c06a840 0x7f7a9c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7ab3e00640 1 -- 172.21.15.3:0/1379666480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f7ab40cde00 con 0x561240d8d220 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7abb200640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a9c06a840 0x7f7a9c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x561240e247f0 0x561240e2e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- osd_op(unknown.0.0:1 8.5 8:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x561240e2ec00 con 0x561240e247f0 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.746+0000 7f7aba800640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x561240e247f0 0x561240e2e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.747+0000 7f7abb200640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a9c06a840 0x7f7a9c06cd00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7aac003040 tx=0x7f7aac009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.747+0000 7f7aba800640 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x561240e247f0 0x561240e2e6c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7aba800640 1 -- 172.21.15.3:0/1379666480 <== osd.1 v2:172.21.15.3:6800/2429800591 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v66'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7aa8009940 con 0x561240e247f0 2024-09-10T07:50:18.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x561240e247f0 msgr2=0x561240e2e6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x561240e247f0 0x561240e2e6c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a9c06a840 msgr2=0x7f7a9c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a9c06a840 0x7f7a9c06cd00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7aac003040 tx=0x7f7aac009000 comp rx=0 tx=0).stop 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 msgr2=0x561240e242b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.748+0000 7f7abdfcaf40 1 --2- 172.21.15.3:0/1379666480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561240d8d220 0x561240e242b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4045c50 tx=0x7f7ab4003330 comp rx=0 tx=0).stop 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.749+0000 7f7abbc00640 1 -- 172.21.15.3:0/1379666480 reap_dead start 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.749+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 shutdown_connections 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.749+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 >> 172.21.15.3:0/1379666480 conn(0x561240d89d40 msgr2=0x561240e2a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.749+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 shutdown_connections 2024-09-10T07:50:18.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.749+0000 7f7abdfcaf40 1 -- 172.21.15.3:0/1379666480 wait complete. 2024-09-10T07:50:18.753 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-10T07:50:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.830+0000 7f369248cf40 1 Processor -- start 2024-09-10T07:50:18.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.830+0000 7f369248cf40 1 -- start start 2024-09-10T07:50:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 0x561dbc67c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 0x561dbc682200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561dbc559b30 con 0x561dbc5e37a0 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561dbc55db70 con 0x561dbc5e5220 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f369248cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561dbc68a900 con 0x561dbc5e4e40 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f368f600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f368f600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53436/0 (socket says 172.21.15.3:53436) 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.831+0000 7f368f600640 1 -- 172.21.15.3:0/3241064681 learned_addr learned my addr 172.21.15.3:0/3241064681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368ec00640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 0x561dbc67c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f3690000640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 0x561dbc682200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 -- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 msgr2=0x561dbc67c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 0x561dbc67c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 -- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 msgr2=0x561dbc682200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 0x561dbc682200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 -- 172.21.15.3:0/3241064681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbc683580 con 0x561dbc5e37a0 2024-09-10T07:50:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368ec00640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 0x561dbc67c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f3690000640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 0x561dbc682200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368f600640 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3680002de0 tx=0x7f368000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e980ffee825e6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368d800640 1 -- 172.21.15.3:0/3241064681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368000f400 con 0x561dbc5e37a0 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368d800640 1 -- 172.21.15.3:0/3241064681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3680014070 con 0x561dbc5e37a0 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.832+0000 7f368d800640 1 -- 172.21.15.3:0/3241064681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368000f7e0 con 0x561dbc5e37a0 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 -- 172.21.15.3:0/3241064681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 msgr2=0x561dbc67cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3680002de0 tx=0x7f368000b5b0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 -- 172.21.15.3:0/3241064681 shutdown_connections 2024-09-10T07:50:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e5220 0x561dbc682200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e4e40 0x561dbc67c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 --2- 172.21.15.3:0/3241064681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e37a0 0x561dbc67cc70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 -- 172.21.15.3:0/3241064681 >> 172.21.15.3:0/3241064681 conn(0x561dbc5e1d40 msgr2=0x561dbc5e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 -- 172.21.15.3:0/3241064681 shutdown_connections 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.833+0000 7f369248cf40 1 -- 172.21.15.3:0/3241064681 wait complete. 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.834+0000 7f369248cf40 1 Processor -- start 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.834+0000 7f369248cf40 1 -- start start 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e37a0 0x561dbc5b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 0x561dbc71a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e5220 0x561dbc71a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561dbc5dff10 con 0x561dbc5e4e40 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x561dbc6832a0 con 0x561dbc5e37a0 2024-09-10T07:50:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f369248cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x561dbc5e01f0 con 0x561dbc5e5220 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 0x561dbc71a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 0x561dbc71a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53452/0 (socket says 172.21.15.3:53452) 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368ec00640 1 -- 172.21.15.3:0/446392611 learned_addr learned my addr 172.21.15.3:0/446392611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368f600640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e37a0 0x561dbc5b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f3690000640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e5220 0x561dbc71a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368ec00640 1 -- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e5220 msgr2=0x561dbc71a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.835+0000 7f368ec00640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x561dbc5e5220 0x561dbc71a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f368ec00640 1 -- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e37a0 msgr2=0x561dbc5b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f368ec00640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561dbc5e37a0 0x561dbc5b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f368ec00640 1 -- 172.21.15.3:0/446392611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbc67d1b0 con 0x561dbc5e4e40 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f368ec00640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 0x561dbc71a2d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f367c025a60 tx=0x7f367c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f3687e00640 1 -- 172.21.15.3:0/446392611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c063070 con 0x561dbc5e4e40 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f3687e00640 1 -- 172.21.15.3:0/446392611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c0547f0 con 0x561dbc5e4e40 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f3687e00640 1 -- 172.21.15.3:0/446392611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c059040 con 0x561dbc5e4e40 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561dbc682960 con 0x561dbc5e4e40 2024-09-10T07:50:18.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.836+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbc682680 con 0x561dbc5e4e40 2024-09-10T07:50:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.838+0000 7f3687e00640 1 -- 172.21.15.3:0/446392611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f367c066070 con 0x561dbc5e4e40 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.839+0000 7f3687e00640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f367006a810 0x7f367006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.839+0000 7f3687e00640 1 -- 172.21.15.3:0/446392611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f367c0cd620 con 0x561dbc5e4e40 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.839+0000 7f368f600640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f367006a810 0x7f367006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.839+0000 7f369248cf40 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x561dbc5b2660 0x561dbc6855d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.839+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 8.9 8:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x561dbc67bb20 con 0x561dbc5b2660 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.840+0000 7f3690000640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x561dbc5b2660 0x561dbc6855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.840+0000 7f368f600640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f367006a810 0x7f367006ccd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f368000b5f0 tx=0x7f368001d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.840+0000 7f3690000640 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x561dbc5b2660 0x561dbc6855d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f3690000640 1 -- 172.21.15.3:0/446392611 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v66'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f36880018e0 con 0x561dbc5b2660 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x561dbc5b2660 msgr2=0x561dbc6855d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x561dbc5b2660 0x561dbc6855d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f367006a810 msgr2=0x7f367006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f367006a810 0x7f367006ccd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f368000b5f0 tx=0x7f368001d040 comp rx=0 tx=0).stop 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 msgr2=0x561dbc71a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 --2- 172.21.15.3:0/446392611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbc5e4e40 0x561dbc71a2d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f367c025a60 tx=0x7f367c002fb0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f3690000640 1 -- 172.21.15.3:0/446392611 reap_dead start 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 shutdown_connections 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 >> 172.21.15.3:0/446392611 conn(0x561dbc5e1d40 msgr2=0x561dbc5e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 shutdown_connections 2024-09-10T07:50:18.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.842+0000 7f369248cf40 1 -- 172.21.15.3:0/446392611 wait complete. 2024-09-10T07:50:18.846 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-10T07:50:18.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.923+0000 7f91da7e1f40 1 Processor -- start 2024-09-10T07:50:18.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.923+0000 7f91da7e1f40 1 -- start start 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdf6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afdff9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558afdee4b30 con 0x558afdf70220 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558afdee8b70 con 0x558afdf6fe40 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558afe015900 con 0x558afdf6eeb0 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53456/0 (socket says 172.21.15.3:53456) 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 -- 172.21.15.3:0/1129954000 learned_addr learned my addr 172.21.15.3:0/1129954000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d7a00640 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdf6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d7000640 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afdff9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 -- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 msgr2=0x558afdf6b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdf6b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 -- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 msgr2=0x558afdff9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afdff9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.924+0000 7f91d8400640 1 -- 172.21.15.3:0/1129954000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afdffa140 con 0x558afdf70220 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.925+0000 7f91d8400640 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f91d00259e0 tx=0x7f91d0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395e6da0e765d1af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.925+0000 7f91d5c00640 1 -- 172.21.15.3:0/1129954000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0058b50 con 0x558afdf70220 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.925+0000 7f91d5c00640 1 -- 172.21.15.3:0/1129954000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0066070 con 0x558afdf70220 2024-09-10T07:50:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.925+0000 7f91d5c00640 1 -- 172.21.15.3:0/1129954000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0061400 con 0x558afdf70220 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1129954000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 msgr2=0x558afdf3ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f91d00259e0 tx=0x7f91d0051000 comp rx=0 tx=0).stop 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1129954000 shutdown_connections 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afdf3ce60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afdff9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1129954000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdf6b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1129954000 >> 172.21.15.3:0/1129954000 conn(0x558afdf6cd40 msgr2=0x558afdf6be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1129954000 shutdown_connections 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1129954000 wait complete. 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.926+0000 7f91da7e1f40 1 Processor -- start 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.927+0000 7f91da7e1f40 1 -- start start 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.927+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdff8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.927+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afe00c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91da7e1f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afe008990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558afdf6b670 con 0x558afdf70220 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558afdf6bac0 con 0x558afdf6fe40 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91da7e1f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558afdf6f290 con 0x558afdf6eeb0 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afe008990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afe008990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53466/0 (socket says 172.21.15.3:53466) 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 -- 172.21.15.3:0/1459243819 learned_addr learned my addr 172.21.15.3:0/1459243819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d7000640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afe00c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d7a00640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdff8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 -- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 msgr2=0x558afdff8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558afdf6eeb0 0x558afdff8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 -- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 msgr2=0x558afe00c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558afdf6fe40 0x558afe00c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.928+0000 7f91d8400640 1 -- 172.21.15.3:0/1459243819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afdf6f810 con 0x558afdf70220 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91d8400640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afe008990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f91d0063040 tx=0x7f91d0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91cfe00640 1 -- 172.21.15.3:0/1459243819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0005a80 con 0x558afdf70220 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91cfe00640 1 -- 172.21.15.3:0/1459243819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d006b600 con 0x558afdf70220 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91cfe00640 1 -- 172.21.15.3:0/1459243819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d006b9f0 con 0x558afdf70220 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558afe0a5960 con 0x558afdf70220 2024-09-10T07:50:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.929+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afdf6f530 con 0x558afdf70220 2024-09-10T07:50:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.931+0000 7f91cfe00640 1 -- 172.21.15.3:0/1459243819 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91d0060070 con 0x558afdf70220 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.931+0000 7f91cfe00640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91b806a810 0x7f91b806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91cfe00640 1 -- 172.21.15.3:0/1459243819 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f91d00cf400 con 0x558afdf70220 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91d7a00640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91b806a810 0x7f91b806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x558afe00c9a0 0x558afe0093e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- osd_op(unknown.0.0:1 8.5 8:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x558afe00f420 con 0x558afe00c9a0 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91d7000640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x558afe00c9a0 0x558afe0093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.932+0000 7f91d7a00640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91b806a810 0x7f91b806ccd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f91c8000f60 tx=0x7f91c8001670 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.933+0000 7f91d7000640 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x558afe00c9a0 0x558afe0093e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91d7000640 1 -- 172.21.15.3:0/1459243819 <== osd.1 v2:172.21.15.3:6800/2429800591 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v66'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91c40117e0 con 0x558afe00c9a0 2024-09-10T07:50:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x558afe00c9a0 msgr2=0x558afe0093e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x558afe00c9a0 0x558afe0093e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91b806a810 msgr2=0x7f91b806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f91b806a810 0x7f91b806ccd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f91c8000f60 tx=0x7f91c8001670 comp rx=0 tx=0).stop 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 msgr2=0x558afe008990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 --2- 172.21.15.3:0/1459243819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558afdf70220 0x558afe008990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f91d0063040 tx=0x7f91d0003330 comp rx=0 tx=0).stop 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91d8400640 1 -- 172.21.15.3:0/1459243819 reap_dead start 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 shutdown_connections 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.934+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 >> 172.21.15.3:0/1459243819 conn(0x558afdf6cd40 msgr2=0x558afe014670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.935+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 shutdown_connections 2024-09-10T07:50:18.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:18.935+0000 7f91da7e1f40 1 -- 172.21.15.3:0/1459243819 wait complete. 2024-09-10T07:50:18.938 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-10T07:50:19.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.016+0000 7f8a8d1dcf40 1 Processor -- start 2024-09-10T07:50:19.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.016+0000 7f8a8d1dcf40 1 -- start start 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554d9e40 0x55ea55571b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554da220 0x55ea555712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ea5557f620 con 0x55ea554f2130 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ea5544eb30 con 0x55ea554d9e40 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ea5557f900 con 0x55ea554da220 2024-09-10T07:50:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53476/0 (socket says 172.21.15.3:53476) 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2805075351 learned_addr learned my addr 172.21.15.3:0/2805075351 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a8a400640 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554d9e40 0x55ea55571b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.017+0000 7f8a89a00640 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554da220 0x55ea555712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554da220 msgr2=0x55ea555712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554da220 0x55ea555712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554d9e40 msgr2=0x55ea55571b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554d9e40 0x55ea55571b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2805075351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea554d5350 con 0x55ea554f2130 2024-09-10T07:50:19.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8a840015b0 tx=0x7f8a84001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8768b99bbb428fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a83e00640 1 -- 172.21.15.3:0/2805075351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84016070 con 0x55ea554f2130 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a83e00640 1 -- 172.21.15.3:0/2805075351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8400f710 con 0x55ea554f2130 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.018+0000 7f8a83e00640 1 -- 172.21.15.3:0/2805075351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400fb00 con 0x55ea554f2130 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2805075351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 msgr2=0x55ea554a77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8a840015b0 tx=0x7f8a84001a70 comp rx=0 tx=0).stop 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2805075351 shutdown_connections 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554f2130 0x55ea554a77a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554da220 0x55ea555712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2805075351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554d9e40 0x55ea55571b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2805075351 >> 172.21.15.3:0/2805075351 conn(0x55ea554d6d40 msgr2=0x55ea554d6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2805075351 shutdown_connections 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2805075351 wait complete. 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.019+0000 7f8a8d1dcf40 1 Processor -- start 2024-09-10T07:50:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 -- start start 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 0x55ea5556ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554da220 0x55ea55572ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554f2130 0x55ea554d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ea554d5dd0 con 0x55ea554d9e40 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ea554f4520 con 0x55ea554da220 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.020+0000 7f8a8d1dcf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ea554f4770 con 0x55ea554f2130 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 0x55ea5556ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 0x55ea5556ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53480/0 (socket says 172.21.15.3:53480) 2024-09-10T07:50:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 -- 172.21.15.3:0/2757402873 learned_addr learned my addr 172.21.15.3:0/2757402873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a89a00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554da220 0x55ea55572ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554f2130 0x55ea554d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 -- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554f2130 msgr2=0x55ea554d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ea554f2130 0x55ea554d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 -- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554da220 msgr2=0x55ea55572ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ea554da220 0x55ea55572ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.021+0000 7f8a8a400640 1 -- 172.21.15.3:0/2757402873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea554f1d60 con 0x55ea554d9e40 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a8a400640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 0x55ea5556ef00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0087e0 tx=0x7f8a7c0088d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a82a00640 1 -- 172.21.15.3:0/2757402873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c019020 con 0x55ea554d9e40 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a82a00640 1 -- 172.21.15.3:0/2757402873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a7c015880 con 0x55ea554d9e40 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea5556b6b0 con 0x55ea554d9e40 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a82a00640 1 -- 172.21.15.3:0/2757402873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c015c30 con 0x55ea554d9e40 2024-09-10T07:50:19.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.022+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea554d86b0 con 0x55ea554d9e40 2024-09-10T07:50:19.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.024+0000 7f8a82a00640 1 -- 172.21.15.3:0/2757402873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a7c01c020 con 0x55ea554d9e40 2024-09-10T07:50:19.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a82a00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a6c06a810 0x7f8a6c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a82a00640 1 -- 172.21.15.3:0/2757402873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f8a7c08a540 con 0x55ea554d9e40 2024-09-10T07:50:19.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a89a00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a6c06a810 0x7f8a6c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55ea5556f440 0x55ea55618d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 8.1d 8:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) -- 0x55ea554f2600 con 0x55ea5556f440 2024-09-10T07:50:19.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a89a00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a6c06a810 0x7f8a6c06ccd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a78056000 tx=0x7f8a78050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.025+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55ea5556f440 0x55ea55618d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.026+0000 7f8a8ae00640 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55ea5556f440 0x55ea55618d10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2757402873 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v66'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8a8400f070 con 0x55ea5556f440 2024-09-10T07:50:19.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55ea5556f440 msgr2=0x55ea55618d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55ea5556f440 0x55ea55618d10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a6c06a810 msgr2=0x7f8a6c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a6c06a810 0x7f8a6c06ccd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a78056000 tx=0x7f8a78050000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 msgr2=0x55ea5556ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 --2- 172.21.15.3:0/2757402873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ea554d9e40 0x55ea5556ef00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0087e0 tx=0x7f8a7c0088d0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8ae00640 1 -- 172.21.15.3:0/2757402873 reap_dead start 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 shutdown_connections 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.028+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 >> 172.21.15.3:0/2757402873 conn(0x55ea554d6d40 msgr2=0x55ea554d7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.029+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 shutdown_connections 2024-09-10T07:50:19.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.029+0000 7f8a8d1dcf40 1 -- 172.21.15.3:0/2757402873 wait complete. 2024-09-10T07:50:19.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:19.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2024-09-10T07:50:19.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.193+0000 7eff86200640 1 Processor -- start 2024-09-10T07:50:19.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.193+0000 7eff86200640 1 -- start start 2024-09-10T07:50:19.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.194+0000 7eff86200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7eff8015e7c0 0x7eff800d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.194+0000 7eff86200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff800cead0 0x7eff80072d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.194+0000 7eff86200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff80073280 con 0x7eff8006bb20 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.194+0000 7eff86200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7eff800cfca0 con 0x7eff800cead0 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.194+0000 7eff86200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7eff800cf090 con 0x7eff8015e7c0 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff800cead0 0x7eff80072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7eff800cead0 0x7eff80072d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53492/0 (socket says 172.21.15.3:53492) 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 -- 172.21.15.3:0/947103735 learned_addr learned my addr 172.21.15.3:0/947103735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7f400640 1 --2- 172.21.15.3:0/947103735 >> v2:172.21.15.70:3300/0 conn(0x7eff8015e7c0 0x7eff800d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 -- 172.21.15.3:0/947103735 >> v2:172.21.15.70:3300/0 conn(0x7eff8015e7c0 msgr2=0x7eff800d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 --2- 172.21.15.3:0/947103735 >> v2:172.21.15.70:3300/0 conn(0x7eff8015e7c0 0x7eff800d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 -- 172.21.15.3:0/947103735 >> v1:172.21.15.70:6789/0 conn(0x7eff8006bb20 legacy=0x7eff8006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.195+0000 7eff7ea00640 1 -- 172.21.15.3:0/947103735 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff800cf370 con 0x7eff800cead0 2024-09-10T07:50:19.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.196+0000 7eff7ea00640 1 --2- 172.21.15.3:0/947103735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff800cead0 0x7eff80072d40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7eff800ce6e0 tx=0x7eff6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893021eac947258c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.196+0000 7eff7e000640 1 -- 172.21.15.3:0/947103735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c037070 con 0x7eff800cead0 2024-09-10T07:50:19.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff7e000640 1 -- 172.21.15.3:0/947103735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6c032440 con 0x7eff800cead0 2024-09-10T07:50:19.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff7e000640 1 -- 172.21.15.3:0/947103735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c032810 con 0x7eff800cead0 2024-09-10T07:50:19.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 -- 172.21.15.3:0/947103735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff800cead0 msgr2=0x7eff80072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 --2- 172.21.15.3:0/947103735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff800cead0 0x7eff80072d40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7eff800ce6e0 tx=0x7eff6c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 -- 172.21.15.3:0/947103735 shutdown_connections 2024-09-10T07:50:19.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 --2- 172.21.15.3:0/947103735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff800cead0 0x7eff80072d40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 --2- 172.21.15.3:0/947103735 >> v2:172.21.15.70:3300/0 conn(0x7eff8015e7c0 0x7eff800d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 -- 172.21.15.3:0/947103735 >> 172.21.15.3:0/947103735 conn(0x7eff800b28a0 msgr2=0x7eff800b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 -- 172.21.15.3:0/947103735 shutdown_connections 2024-09-10T07:50:19.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.197+0000 7eff86200640 1 -- 172.21.15.3:0/947103735 wait complete. 2024-09-10T07:50:19.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.199+0000 7eff86200640 1 Processor -- start 2024-09-10T07:50:19.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.199+0000 7eff86200640 1 -- start start 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7eff8006bb20 0x7eff80155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff800cead0 0x7eff8009ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7eff8015eba0 con 0x7eff8015e7c0 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7eff800cf930 con 0x7eff800cead0 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff86200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7eff800b3ba0 con 0x7eff8006bb20 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53494/0 (socket says 172.21.15.3:53494) 2024-09-10T07:50:19.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff7fe00640 1 -- 172.21.15.3:0/1101064520 learned_addr learned my addr 172.21.15.3:0/1101064520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff7ea00640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff800cead0 0x7eff8009ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.200+0000 7eff7f400640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7eff8006bb20 0x7eff80155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 -- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7eff8006bb20 msgr2=0x7eff80155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7eff8006bb20 0x7eff80155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 -- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff800cead0 msgr2=0x7eff8009ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff800cead0 0x7eff8009ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 -- 172.21.15.3:0/1101064520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8015c500 con 0x7eff8015e7c0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff7fe00640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c780 tx=0x7eff7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff77400640 1 -- 172.21.15.3:0/1101064520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004b60 con 0x7eff8015e7c0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff77400640 1 -- 172.21.15.3:0/1101064520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff78004d00 con 0x7eff8015e7c0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8015c7e0 con 0x7eff8015e7c0 2024-09-10T07:50:19.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff77400640 1 -- 172.21.15.3:0/1101064520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800aa90 con 0x7eff8015e7c0 2024-09-10T07:50:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.201+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8015cda0 con 0x7eff8015e7c0 2024-09-10T07:50:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.203+0000 7eff77400640 1 -- 172.21.15.3:0/1101064520 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff7800ac30 con 0x7eff8015e7c0 2024-09-10T07:50:19.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.204+0000 7eff77400640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 0x7eff6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.204+0000 7eff77400640 1 -- 172.21.15.3:0/1101064520 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7eff7808b0b0 con 0x7eff8015e7c0 2024-09-10T07:50:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.204+0000 7eff7f400640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 0x7eff6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.205+0000 7eff7f400640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 0x7eff6006e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.205+0000 7eff89016740 1 Processor -- start 2024-09-10T07:50:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.205+0000 7eff89016740 1 -- start start 2024-09-10T07:50:19.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce8309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce7eb540 0x55abce7ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55abce7509c0 con 0x55abce817be0 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55abce6f88f0 con 0x55abce8325a0 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55abce74c980 con 0x55abce7eb540 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53498/0 (socket says 172.21.15.3:53498) 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff7f400640 1 -- 172.21.15.3:0/1725703724 learned_addr learned my addr 172.21.15.3:0/1725703724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.206+0000 7eff7fe00640 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce7eb540 0x55abce7ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7ea00640 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce8309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 -- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce7eb540 msgr2=0x55abce7ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce7eb540 0x55abce7ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 -- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 msgr2=0x55abce8309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce8309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 -- 172.21.15.3:0/1725703724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abce52ba50 con 0x55abce817be0 2024-09-10T07:50:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff7f400640 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7eff7000f9f0 tx=0x7eff70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c8f298543a940b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff6b400640 1 -- 172.21.15.3:0/1725703724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7001b050 con 0x55abce817be0 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff6b400640 1 -- 172.21.15.3:0/1725703724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70025070 con 0x55abce817be0 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.207+0000 7eff6b400640 1 -- 172.21.15.3:0/1725703724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70014de0 con 0x55abce817be0 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 -- 172.21.15.3:0/1725703724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 msgr2=0x55abce817fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7eff7000f9f0 tx=0x7eff70012000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 -- 172.21.15.3:0/1725703724 shutdown_connections 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce7eb540 0x55abce7ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce8309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 --2- 172.21.15.3:0/1725703724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce817fc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 -- 172.21.15.3:0/1725703724 >> 172.21.15.3:0/1725703724 conn(0x55abce7ea7d0 msgr2=0x55abce8329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 -- 172.21.15.3:0/1725703724 shutdown_connections 2024-09-10T07:50:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.208+0000 7eff89016740 1 -- 172.21.15.3:0/1725703724 wait complete. 2024-09-10T07:50:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.209+0000 7eff89016740 1 -- start start 2024-09-10T07:50:19.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.209+0000 7eff89016740 1 Processor -- start 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce817be0 0x55abce82d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce8325a0 0x55abce8064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55abce818500 con 0x55abce82a530 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55abce70bb40 con 0x55abce817be0 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55abce7e9c00 con 0x55abce8325a0 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff89016740 10 client.0 _ll_register_callbacks cb 0x55abce70dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:19.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53502/0 (socket says 172.21.15.3:53502) 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 -- 172.21.15.3:0/2307535928 learned_addr learned my addr 172.21.15.3:0/2307535928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7ea00640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce8325a0 0x55abce8064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7f400640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce817be0 0x55abce82d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 -- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce8325a0 msgr2=0x55abce8064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce8325a0 0x55abce8064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 -- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce817be0 msgr2=0x55abce82d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.210+0000 7eff7fe00640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce817be0 0x55abce82d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff7fe00640 1 -- 172.21.15.3:0/2307535928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abce4233e0 con 0x55abce82a530 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff7fe00640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7eff78055770 tx=0x7eff78059ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbd0a2caadc74bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:19.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78086030 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7eff78055980 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff70025070 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7eff78077050 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff89016740 10 client.6441 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55abce58c1e0 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.211+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70025070 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7eff78078240 con 0x55abce82a530 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 10 client.6441 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 10 client.6441 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 20 client.6441 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce82f540 0x55abce82f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 4 client.6441 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:19.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abce75e0f0 con 0x55abce82f540 2024-09-10T07:50:19.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff7f400640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce82f540 0x55abce82f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff7f400640 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce82f540 0x55abce82f940 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c33ca35279a756 server_cookie=324c268001b757e4 in_seq=0 out_seq=0 2024-09-10T07:50:19.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.212+0000 7eff5fe00640 10 client.6441 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.213+0000 7eff5fe00640 1 -- 172.21.15.3:0/2307535928 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abce75e0f0 con 0x55abce82f540 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.213+0000 7eff5fe00640 10 client.6441 handle_command_reply: tid=0 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.213+0000 7eff89016740 1 client.6441 shutdown 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.213+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 msgr2=0x55abce7ed010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.213+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7eff78055770 tx=0x7eff78059ad0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 shutdown_connections 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce82f540 0x55abce82f940 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce82a530 0x55abce7ed010 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce8325a0 0x55abce8064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 --2- 172.21.15.3:0/2307535928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce817be0 0x55abce82d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff5fe00640 0 client.6441 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 >> 172.21.15.3:0/2307535928 conn(0x55abce7eb540 msgr2=0x55abce8151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 shutdown_connections 2024-09-10T07:50:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 1 -- 172.21.15.3:0/2307535928 wait complete. 2024-09-10T07:50:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.214+0000 7eff89016740 20 client.6441 trim_cache size 0 max 16384 2024-09-10T07:50:19.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.233+0000 7eff89016740 1 Processor -- start 2024-09-10T07:50:19.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.233+0000 7eff89016740 1 -- start start 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce82a530 0x55abce82ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce836120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7eff80200d50 con 0x55abce817be0 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7eff801fff50 con 0x55abce8325a0 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.234+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7eff80200620 con 0x55abce82a530 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53506/0 (socket says 172.21.15.3:53506) 2024-09-10T07:50:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 -- 172.21.15.3:0/3955330635 learned_addr learned my addr 172.21.15.3:0/3955330635 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7fe00640 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce836120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7ea00640 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce82a530 0x55abce82ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 -- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce82a530 msgr2=0x55abce82ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce82a530 0x55abce82ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 -- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 msgr2=0x55abce836120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce836120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 -- 172.21.15.3:0/3955330635 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abce818330 con 0x55abce817be0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.235+0000 7eff7f400640 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7eff70021000 tx=0x7eff70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56632e34395197e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.236+0000 7eff6aa00640 1 -- 172.21.15.3:0/3955330635 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7001b050 con 0x55abce817be0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.236+0000 7eff6aa00640 1 -- 172.21.15.3:0/3955330635 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70025040 con 0x55abce817be0 2024-09-10T07:50:19.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.236+0000 7eff6aa00640 1 -- 172.21.15.3:0/3955330635 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70035880 con 0x55abce817be0 2024-09-10T07:50:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 -- 172.21.15.3:0/3955330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 msgr2=0x55abce815330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7eff70021000 tx=0x7eff70014000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 -- 172.21.15.3:0/3955330635 shutdown_connections 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce8325a0 0x55abce836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce82a530 0x55abce82ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 --2- 172.21.15.3:0/3955330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce817be0 0x55abce815330 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 -- 172.21.15.3:0/3955330635 >> 172.21.15.3:0/3955330635 conn(0x55abce7eb540 msgr2=0x55abce7e3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 -- 172.21.15.3:0/3955330635 shutdown_connections 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.237+0000 7eff89016740 1 -- 172.21.15.3:0/3955330635 wait complete. 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.238+0000 7eff89016740 1 -- start start 2024-09-10T07:50:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.238+0000 7eff89016740 1 Processor -- start 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce82a530 0x55abce8b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce8325a0 0x55abce816230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55abce82af30 con 0x55abce8325a0 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7eff8009e7b0 con 0x55abce82a530 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55abce70ab80 con 0x55abce817be0 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff89016740 10 client.0 _ll_register_callbacks cb 0x7eff7800ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:19.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff7f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff7f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:48514/0 (socket says 172.21.15.3:48514) 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff7f400640 1 -- 172.21.15.3:0/3822083887 learned_addr learned my addr 172.21.15.3:0/3822083887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff7ea00640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce82a530 0x55abce8b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.239+0000 7eff7fe00640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce8325a0 0x55abce816230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 -- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce82a530 msgr2=0x55abce8b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce82a530 0x55abce8b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 -- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce8325a0 msgr2=0x55abce816230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce8325a0 0x55abce816230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 -- 172.21.15.3:0/3822083887 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55abce3fc4a0 con 0x55abce817be0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff7f400640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55abce71a830 tx=0x7eff70004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1107d5fd6ec4e6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:50:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7001b040 con 0x55abce817be0 2024-09-10T07:50:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7eff70035890 con 0x55abce817be0 2024-09-10T07:50:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.240+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff70025040 con 0x55abce817be0 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7eff7000b580 con 0x55abce817be0 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 10 client.5753 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55abce82a080 con 0x55abce817be0 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70025040 con 0x55abce817be0 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7eff70035ad0 con 0x55abce817be0 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 10 client.5753 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 10 client.5753 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 20 client.5753 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce816770 0x55abce828a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 4 client.5753 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55abce758440 con 0x55abce816770 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.241+0000 7eff7ea00640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce816770 0x55abce828a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.242+0000 7eff7ea00640 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce816770 0x55abce828a00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ca1a19f7c9a12d server_cookie=98a398ffd697a21d in_seq=0 out_seq=0 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.242+0000 7eff5fe00640 10 client.5753 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff5fe00640 1 -- 172.21.15.3:0/3822083887 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55abce758440 con 0x55abce816770 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff5fe00640 10 client.5753 handle_command_reply: tid=0 2024-09-10T07:50:19.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 client.5753 shutdown 2024-09-10T07:50:19.244 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:50:19.243+0000 7fe5e5a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1) n(v3 rc2024-09-10T07:50:07.429717+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56487ae49b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:50:07.429717+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 msgr2=0x55abce8adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55abce71a830 tx=0x7eff70004730 comp rx=0 tx=0).stop 2024-09-10T07:50:19.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 shutdown_connections 2024-09-10T07:50:19.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55abce816770 0x55abce828a00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abce8325a0 0x55abce816230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55abce82a530 0x55abce8b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 --2- 172.21.15.3:0/3822083887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55abce817be0 0x55abce8adf60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 >> 172.21.15.3:0/3822083887 conn(0x55abce7eb540 msgr2=0x55abce817000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.243+0000 7eff5fe00640 0 client.5753 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.244+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 shutdown_connections 2024-09-10T07:50:19.246 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:50:19.244+0000 7f0cf8600640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1) n(v2 rc2024-09-10T07:50:07.429717+0000 4=0+4) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55dbf7cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:50:07.429717+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.244+0000 7eff89016740 1 -- 172.21.15.3:0/3822083887 wait complete. 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.244+0000 7eff89016740 20 client.5753 trim_cache size 0 max 16384 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "acf7472a-df27-4ea7-bd17-6e9ec4f9c1b0", 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-10T07:50:19.247 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 msgr2=0x7eff6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 0x7eff6006e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70008040 comp rx=0 tx=0).stop 2024-09-10T07:50:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 msgr2=0x7eff8009bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c780 tx=0x7eff7800cc40 comp rx=0 tx=0).stop 2024-09-10T07:50:19.248 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:50:19.245+0000 7fe5edc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1) n(v4 rc2024-09-10T07:50:07.429717+0000 5=0+5) (inest sync->mix g=1,2) caps={5591=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56487ae4a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:50:07.429717+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.249 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:50:19.244+0000 7f64bbc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1) n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562725276000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.249 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:50:19.244+0000 7f64bbc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/AsLsXsFsx@3},l=5591 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562725276580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 shutdown_connections 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7eff6006be30 0x7eff6006e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff8015e7c0 0x7eff8009bf70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff800cead0 0x7eff8009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 --2- 172.21.15.3:0/1101064520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7eff8006bb20 0x7eff80155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.245+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 >> 172.21.15.3:0/1101064520 conn(0x7eff800b28a0 msgr2=0x7eff8006ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.246+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 shutdown_connections 2024-09-10T07:50:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.246+0000 7eff86200640 1 -- 172.21.15.3:0/1101064520 wait complete. 2024-09-10T07:50:19.250 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:50:19.246+0000 7f0d00800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1) n(v1 rc2024-09-10T07:50:07.429717+0000 3=0+3) (inest sync->mix g=0,2) caps={5591=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x55dbf8cc6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:50:07.429717+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:50:19.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:19.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T07:50:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.420+0000 7f40fa400640 1 Processor -- start 2024-09-10T07:50:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.420+0000 7f40fa400640 1 -- start start 2024-09-10T07:50:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40fa400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40f40cf980 0x7f40f406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40fa400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f40cd900 con 0x7f40f41451d0 2024-09-10T07:50:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40fa400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f40cdbe0 con 0x7f40f4145730 2024-09-10T07:50:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40fa400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f40f406d7c0 con 0x7f40f40cf980 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40f3400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40f41451d0 0x7f40f40b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59784/0 (socket says 172.21.15.3:59784) 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40f3400640 1 -- 172.21.15.3:0/1027884587 learned_addr learned my addr 172.21.15.3:0/1027884587 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40f3e00640 1 --2- 172.21.15.3:0/1027884587 >> v2:172.21.15.3:3300/0 conn(0x7f40f40cf980 0x7f40f406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.422+0000 7f40f2000640 1 -- 172.21.15.3:0/1027884587 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668019732 0 0) 0x7f40f40cd900 con 0x7f40f41451d0 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.423+0000 7f40f2000640 1 -- 172.21.15.3:0/1027884587 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40d8003520 con 0x7f40f41451d0 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.423+0000 7f40f3e00640 1 -- 172.21.15.3:0/1027884587 >> v1:172.21.15.70:6789/0 conn(0x7f40f4145730 legacy=0x7f40f40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.423+0000 7f40f3e00640 1 -- 172.21.15.3:0/1027884587 >> v1:172.21.15.3:6789/0 conn(0x7f40f41451d0 legacy=0x7f40f40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.423+0000 7f40f3e00640 1 -- 172.21.15.3:0/1027884587 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f406da10 con 0x7f40f40cf980 2024-09-10T07:50:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40f3e00640 1 --2- 172.21.15.3:0/1027884587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f40cf980 0x7f40f406cfc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f40f406d940 tx=0x7f40ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35cb1e2c7d559ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40f2000640 1 -- 172.21.15.3:0/1027884587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007740 con 0x7f40f40cf980 2024-09-10T07:50:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40f2000640 1 -- 172.21.15.3:0/1027884587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec0078e0 con 0x7f40f40cf980 2024-09-10T07:50:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40f2000640 1 -- 172.21.15.3:0/1027884587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007c90 con 0x7f40f40cf980 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40fa400640 1 -- 172.21.15.3:0/1027884587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f40cf980 msgr2=0x7f40f406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.424+0000 7f40fa400640 1 --2- 172.21.15.3:0/1027884587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f40cf980 0x7f40f406cfc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f40f406d940 tx=0x7f40ec02a660 comp rx=0 tx=0).stop 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.425+0000 7f40fa400640 1 -- 172.21.15.3:0/1027884587 shutdown_connections 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.425+0000 7f40fa400640 1 --2- 172.21.15.3:0/1027884587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f40cf980 0x7f40f406cfc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.425+0000 7f40fa400640 1 -- 172.21.15.3:0/1027884587 >> 172.21.15.3:0/1027884587 conn(0x7f40f40b5cb0 msgr2=0x7f40f406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.425+0000 7f40fa400640 1 -- 172.21.15.3:0/1027884587 shutdown_connections 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.425+0000 7f40fa400640 1 -- 172.21.15.3:0/1027884587 wait complete. 2024-09-10T07:50:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.426+0000 7f40fa400640 1 Processor -- start 2024-09-10T07:50:19.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.426+0000 7f40fa400640 1 -- start start 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 0x7f40f41f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f4145730 0x7f40f4139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40f406e0b0 con 0x7f40f41451d0 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40f406df30 con 0x7f40f4145730 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40fa400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f40f406e230 con 0x7f40f40cf980 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53534/0 (socket says 172.21.15.3:53534) 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.427+0000 7f40f2a00640 1 -- 172.21.15.3:0/3871977340 learned_addr learned my addr 172.21.15.3:0/3871977340 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f3e00640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f4145730 0x7f40f4139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f3400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 0x7f40f41f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 -- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 msgr2=0x7f40f41f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 0x7f40f41f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 -- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f4145730 msgr2=0x7f40f4139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f4145730 0x7f40f4139a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 -- 172.21.15.3:0/3871977340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f40b2970 con 0x7f40f41451d0 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f3400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 0x7f40f41f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c3c0 tx=0x7f40e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.428+0000 7f40eb400640 1 -- 172.21.15.3:0/3871977340 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00052d0 con 0x7f40f41451d0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.429+0000 7f40eb400640 1 -- 172.21.15.3:0/3871977340 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e0005490 con 0x7f40f41451d0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.429+0000 7f40eb400640 1 -- 172.21.15.3:0/3871977340 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007260 con 0x7f40f41451d0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.429+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f413aea0 con 0x7f40f41451d0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.429+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41f55d0 con 0x7f40f41451d0 2024-09-10T07:50:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.431+0000 7f40eb400640 1 -- 172.21.15.3:0/3871977340 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40e0007400 con 0x7f40f41451d0 2024-09-10T07:50:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.431+0000 7f40eb400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 0x7f40d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.432+0000 7f40eb400640 1 -- 172.21.15.3:0/3871977340 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f40e008a2b0 con 0x7f40f41451d0 2024-09-10T07:50:19.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.432+0000 7f40f3400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 0x7f40d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.432+0000 7f40fd227740 1 Processor -- start 2024-09-10T07:50:19.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.432+0000 7f40fd227740 1 -- start start 2024-09-10T07:50:19.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.432+0000 7f40f3400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 0x7f40d406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f40e40011e0 tx=0x7f40e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b38136c480 0x55b3813b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b3812d4660 con 0x55b381386420 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b38127c680 con 0x55b381386800 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b3812d0620 con 0x55b38136c480 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53536/0 (socket says 172.21.15.3:53536) 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.433+0000 7f40f3400640 1 -- 172.21.15.3:0/195728547 learned_addr learned my addr 172.21.15.3:0/195728547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f2a00640 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 -- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b38136c480 msgr2=0x55b3813b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b38136c480 0x55b3813b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 -- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 msgr2=0x55b3813b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 -- 172.21.15.3:0/195728547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3810b17d0 con 0x55b381386420 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f2a00640 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40f3400640 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001eb0 tx=0x7f40e4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249d3d5b8bda37ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40dfe00640 1 -- 172.21.15.3:0/195728547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e400e7b0 con 0x55b381386420 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40dfe00640 1 -- 172.21.15.3:0/195728547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e400e950 con 0x55b381386420 2024-09-10T07:50:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.434+0000 7f40dfe00640 1 -- 172.21.15.3:0/195728547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e400ed00 con 0x55b381386420 2024-09-10T07:50:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 -- 172.21.15.3:0/195728547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 msgr2=0x55b3813b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001eb0 tx=0x7f40e4011000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 -- 172.21.15.3:0/195728547 shutdown_connections 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b38136c480 0x55b3813b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 --2- 172.21.15.3:0/195728547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b3813b6c40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 -- 172.21.15.3:0/195728547 >> 172.21.15.3:0/195728547 conn(0x55b38136d220 msgr2=0x55b3813b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 -- 172.21.15.3:0/195728547 shutdown_connections 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.435+0000 7f40fd227740 1 -- 172.21.15.3:0/195728547 wait complete. 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.436+0000 7f40fd227740 1 -- start start 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.436+0000 7f40fd227740 1 Processor -- start 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b381374e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b38136b3e0 con 0x55b3813850d0 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b38125d4c0 con 0x55b381386800 2024-09-10T07:50:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b38136b6c0 con 0x55b381386420 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40fd227740 10 client.0 _ll_register_callbacks cb 0x55b3812cce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53544/0 (socket says 172.21.15.3:53544) 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40f3e00640 1 -- 172.21.15.3:0/2711514826 learned_addr learned my addr 172.21.15.3:0/2711514826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40f3400640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b381374e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.437+0000 7f40f2a00640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 -- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 msgr2=0x55b381374e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b381374e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 -- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 msgr2=0x55b3813c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 -- 172.21.15.3:0/2711514826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b380fe73f0 con 0x55b3813850d0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3400640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b381374e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f2a00640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40f3e00640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009e50 tx=0x7f40ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8080429ac7630a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:19.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0047b0 con 0x55b3813850d0 2024-09-10T07:50:19.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f40ec007540 con 0x55b3813850d0 2024-09-10T07:50:19.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e400e950 con 0x55b3813850d0 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f40ec04c050 con 0x55b3813850d0 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40fd227740 10 client.6462 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.438+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b380fa70f0 con 0x55b3813850d0 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e400e950 con 0x55b3813850d0 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f40ec04f020 con 0x55b3813850d0 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 10 client.6462 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 10 client.6462 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 20 client.6462 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b381372dc0 0x55b3813c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 4 client.6462 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3812e1d90 con 0x55b381372dc0 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.439+0000 7f40f3400640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b381372dc0 0x55b3813c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.440+0000 7f40f3400640 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b381372dc0 0x55b3813c9160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74d21a634875af7 server_cookie=e228d7d446439966 in_seq=0 out_seq=0 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.440+0000 7f40dcc00640 10 client.6462 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.440+0000 7f40dcc00640 1 -- 172.21.15.3:0/2711514826 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3812e1d90 con 0x55b381372dc0 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.440+0000 7f40dcc00640 10 client.6462 handle_command_reply: tid=0 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.440+0000 7f40fd227740 1 client.6462 shutdown 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 msgr2=0x55b3813afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009e50 tx=0x7f40ec004a40 comp rx=0 tx=0).stop 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 shutdown_connections 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b381372dc0 0x55b3813c9160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3813850d0 0x55b3813afe70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b381386800 0x55b3813c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40dcc00640 0 client.6462 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 --2- 172.21.15.3:0/2711514826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b381374e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 >> 172.21.15.3:0/2711514826 conn(0x55b38136c480 msgr2=0x55b3813af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 shutdown_connections 2024-09-10T07:50:19.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 1 -- 172.21.15.3:0/2711514826 wait complete. 2024-09-10T07:50:19.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.441+0000 7f40fd227740 20 client.6462 trim_cache size 0 max 16384 2024-09-10T07:50:19.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.459+0000 7f40fd227740 1 Processor -- start 2024-09-10T07:50:19.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.459+0000 7f40fd227740 1 -- start start 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386800 0x55b381373450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4200c30 con 0x55b381386420 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41ffe50 con 0x55b3813850d0 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f40f4200500 con 0x55b381386800 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53558/0 (socket says 172.21.15.3:53558) 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.460+0000 7f40f2a00640 1 -- 172.21.15.3:0/3980479689 learned_addr learned my addr 172.21.15.3:0/3980479689 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f3e00640 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386800 0x55b381373450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f3400640 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 -- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386800 msgr2=0x55b381373450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386800 0x55b381373450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 -- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 msgr2=0x55b3813d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813d9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 -- 172.21.15.3:0/3980479689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3813b01e0 con 0x55b381386420 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40f2a00640 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f40e000cd80 tx=0x7f40e000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d051d6141842493 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40df400640 1 -- 172.21.15.3:0/3980479689 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00568c0 con 0x55b381386420 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40df400640 1 -- 172.21.15.3:0/3980479689 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e0054c30 con 0x55b381386420 2024-09-10T07:50:19.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.461+0000 7f40df400640 1 -- 172.21.15.3:0/3980479689 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0088d60 con 0x55b381386420 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 -- 172.21.15.3:0/3980479689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 msgr2=0x55b381385c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f40e000cd80 tx=0x7f40e000a000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 -- 172.21.15.3:0/3980479689 shutdown_connections 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386800 0x55b381373450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386420 0x55b381385c30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 --2- 172.21.15.3:0/3980479689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 -- 172.21.15.3:0/3980479689 >> 172.21.15.3:0/3980479689 conn(0x55b38136c480 msgr2=0x55b3813b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 -- 172.21.15.3:0/3980479689 shutdown_connections 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.462+0000 7f40fd227740 1 -- 172.21.15.3:0/3980479689 wait complete. 2024-09-10T07:50:19.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.463+0000 7f40fd227740 1 -- start start 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.463+0000 7f40fd227740 1 Processor -- start 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b3813ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b3813b7c60 con 0x55b381386800 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41fef10 con 0x55b3813850d0 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b381373e30 con 0x55b381386420 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40fd227740 10 client.0 _ll_register_callbacks cb 0x7f40ec009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53564/0 (socket says 172.21.15.3:53564) 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 -- 172.21.15.3:0/1057145950 learned_addr learned my addr 172.21.15.3:0/1057145950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3400640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f2a00640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b3813ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 -- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 msgr2=0x55b3813ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b3813ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 -- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 msgr2=0x55b3813ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3e00640 1 -- 172.21.15.3:0/1057145950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3813adc10 con 0x55b381386800 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.464+0000 7f40f3400640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40f3e00640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0079f0 tx=0x7f40ec02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bb7898bee837cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec003d30 con 0x55b381386800 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f40ec007d60 con 0x55b381386800 2024-09-10T07:50:19.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e0054c30 con 0x55b381386800 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f40ec05c050 con 0x55b381386800 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 10 client.6468 fetch_fsmap learned FSMap version 57 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b3814351f0 con 0x55b381386800 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e0054c30 con 0x55b381386800 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 57) ==== 3550+0+0 (secure 0 0 0) 0x7f40ec05f020 con 0x55b381386800 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 10 client.6468 fetch_fsmap finished waiting for FSMap version 57 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 10 client.6468 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 20 client.6468 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b3813cfc20 0x55b381434b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 4 client.6468 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:50:19.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.465+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3812dc0e0 con 0x55b3813cfc20 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40f3400640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b3813cfc20 0x55b381434b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40f3400640 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b3813cfc20 0x55b381434b80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9748109f9b58d353 server_cookie=6d02c3096c9bf54f in_seq=0 out_seq=0 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40dcc00640 10 client.6468 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40dcc00640 1 -- 172.21.15.3:0/1057145950 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b3812dc0e0 con 0x55b3813cfc20 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40dcc00640 10 client.6468 handle_command_reply: tid=0 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.466+0000 7f40fd227740 1 client.6468 shutdown 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 msgr2=0x55b381434640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0079f0 tx=0x7f40ec02f000 comp rx=0 tx=0).stop 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 shutdown_connections 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x55b3813cfc20 0x55b381434b80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b381386800 0x55b381434640 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b381386420 0x55b3813ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 --2- 172.21.15.3:0/1057145950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b3813850d0 0x55b3813ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40dcc00640 0 client.6468 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 >> 172.21.15.3:0/1057145950 conn(0x55b38136c480 msgr2=0x55b3813b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 shutdown_connections 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 1 -- 172.21.15.3:0/1057145950 wait complete. 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fd227740 20 client.6468 trim_cache size 0 max 16384 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:50:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "acf7472a-df27-4ea7-bd17-6e9ec4f9c1b0": { 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "acf7472a-df27-4ea7-bd17-6e9ec4f9c1b0", 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:50:19.474 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 msgr2=0x7f40d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 0x7f40d406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f40e40011e0 tx=0x7f40e4009040 comp rx=0 tx=0).stop 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 msgr2=0x7f40f40afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.467+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c3c0 tx=0x7f40e000c880 comp rx=0 tx=0).stop 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 shutdown_connections 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40d406be30 0x7f40d406e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f4145730 0x7f40f4139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40f41451d0 0x7f40f40afce0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 --2- 172.21.15.3:0/3871977340 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40f40cf980 0x7f40f41f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 >> 172.21.15.3:0/3871977340 conn(0x7f40f40b5cb0 msgr2=0x7f40f4140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 shutdown_connections 2024-09-10T07:50:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:19.468+0000 7f40fa400640 1 -- 172.21.15.3:0/3871977340 wait complete. 2024-09-10T07:50:19.482 INFO:tasks.cephfs.filesystem:scrub status for tag:acf7472a-df27-4ea7-bd17-6e9ec4f9c1b0 - {'path': '/d1/d2/d3', 'tag': 'acf7472a-df27-4ea7-bd17-6e9ec4f9c1b0', 'options': 'recursive,force'} 2024-09-10T07:50:20.621 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:50:20.621+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:50:21.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:50:21.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:50:26.439 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:50:26.440+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:50:49.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:49.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T07:50:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.655+0000 7fe9d5c00640 1 Processor -- start 2024-09-10T07:50:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.655+0000 7fe9d5c00640 1 -- start start 2024-09-10T07:50:49.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9d5c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9d00d45a0 0x7fe9d00d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9d5c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d00cfe90 0x7fe9d00d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9d5c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d00d4f00 con 0x7fe9d0161b70 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9d5c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d0161f50 con 0x7fe9d00d45a0 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9d5c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d00d3de0 con 0x7fe9d00cfe90 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9c7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9d00d45a0 0x7fe9d00d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9c7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9d00d45a0 0x7fe9d00d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50804/0 (socket says 172.21.15.3:50804) 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9c7e00640 1 -- 172.21.15.3:0/1592914469 learned_addr learned my addr 172.21.15.3:0/1592914469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.656+0000 7fe9cfe00640 1 --1- 172.21.15.3:0/1592914469 >> v1:172.21.15.3:6789/0 conn(0x7fe9d0161b70 0x7fe9d00d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33804/0 (socket says 172.21.15.3:33804) 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.657+0000 7fe9c7e00640 1 -- 172.21.15.3:0/1592914469 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d00cfe90 msgr2=0x7fe9d00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.657+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/1592914469 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d00cfe90 0x7fe9d00d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.657+0000 7fe9c7e00640 1 -- 172.21.15.3:0/1592914469 >> v1:172.21.15.3:6789/0 conn(0x7fe9d0161b70 legacy=0x7fe9d00d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.657+0000 7fe9c7e00640 1 -- 172.21.15.3:0/1592914469 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d00d40c0 con 0x7fe9d00d45a0 2024-09-10T07:50:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/1592914469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00d49c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5defbbfad47f2e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9cf400640 1 -- 172.21.15.3:0/1592914469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8037070 con 0x7fe9d00d45a0 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9cf400640 1 -- 172.21.15.3:0/1592914469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b8032440 con 0x7fe9d00d45a0 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9cf400640 1 -- 172.21.15.3:0/1592914469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8032810 con 0x7fe9d00d45a0 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9d5c00640 1 -- 172.21.15.3:0/1592914469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 msgr2=0x7fe9d00d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.658+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/1592914469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00d49c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b802a250 comp rx=0 tx=0).stop 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 -- 172.21.15.3:0/1592914469 shutdown_connections 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/1592914469 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d00cfe90 0x7fe9d00d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/1592914469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00d49c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 -- 172.21.15.3:0/1592914469 >> 172.21.15.3:0/1592914469 conn(0x7fe9d0062630 msgr2=0x7fe9d0062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 -- 172.21.15.3:0/1592914469 shutdown_connections 2024-09-10T07:50:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.659+0000 7fe9d5c00640 1 -- 172.21.15.3:0/1592914469 wait complete. 2024-09-10T07:50:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.660+0000 7fe9d5c00640 1 Processor -- start 2024-09-10T07:50:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.660+0000 7fe9d5c00640 1 -- start start 2024-09-10T07:50:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d00cfe90 0x7fe9d01f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d0161b70 0x7fe9d00b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00c9fc0 con 0x7fe9d00d45a0 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00cf970 con 0x7fe9d00cfe90 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9d5c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d00d3a30 con 0x7fe9d0161b70 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.661+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50812/0 (socket says 172.21.15.3:50812) 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2671030371 learned_addr learned my addr 172.21.15.3:0/2671030371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d00cfe90 0x7fe9d01f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9d5200640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d0161b70 0x7fe9d00b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d0161b70 msgr2=0x7fe9d00b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d0161b70 0x7fe9d00b9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d00cfe90 msgr2=0x7fe9d01f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d00cfe90 0x7fe9d01f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2671030371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d00b8040 con 0x7fe9d00d45a0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.662+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.663+0000 7fe9ccc00640 1 -- 172.21.15.3:0/2671030371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8041020 con 0x7fe9d00d45a0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.663+0000 7fe9ccc00640 1 -- 172.21.15.3:0/2671030371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b8030970 con 0x7fe9d00d45a0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.663+0000 7fe9ccc00640 1 -- 172.21.15.3:0/2671030371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8037040 con 0x7fe9d00d45a0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.663+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d00b8320 con 0x7fe9d00d45a0 2024-09-10T07:50:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.663+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d00b5060 con 0x7fe9d00d45a0 2024-09-10T07:50:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.666+0000 7fe9ccc00640 1 -- 172.21.15.3:0/2671030371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9b8044020 con 0x7fe9d00d45a0 2024-09-10T07:50:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.667+0000 7fe9ccc00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 0x7fe9ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.667+0000 7fe9ccc00640 1 -- 172.21.15.3:0/2671030371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fe9b80ab870 con 0x7fe9d00d45a0 2024-09-10T07:50:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.667+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 0x7fe9ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.667+0000 7fe9da0a8740 1 Processor -- start 2024-09-10T07:50:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.667+0000 7fe9da0a8740 1 -- start start 2024-09-10T07:50:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 0x7fe9ac06e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000f60 tx=0x7fe9c0007ce0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737b699e0 0x562737b68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562737ad1660 con 0x562737bb3ea0 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562737a79680 con 0x562737b699e0 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562737acd620 con 0x562737b834f0 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9d5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9d5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50826/0 (socket says 172.21.15.3:50826) 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.668+0000 7fe9d5200640 1 -- 172.21.15.3:0/4059352961 learned_addr learned my addr 172.21.15.3:0/4059352961 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737b699e0 0x562737b68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 -- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 msgr2=0x562737b838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 -- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737b699e0 msgr2=0x562737b68240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737b699e0 0x562737b68240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 -- 172.21.15.3:0/4059352961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627378ae7d0 con 0x562737bb3ea0 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9d5200640 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009f80 tx=0x7fe9c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d2a933e77b8f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.669+0000 7fe9c4c00640 1 -- 172.21.15.3:0/4059352961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8016420 con 0x562737bb3ea0 2024-09-10T07:50:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9c4c00640 1 -- 172.21.15.3:0/4059352961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c80165c0 con 0x562737bb3ea0 2024-09-10T07:50:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9c4c00640 1 -- 172.21.15.3:0/4059352961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80169b0 con 0x562737bb3ea0 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 -- 172.21.15.3:0/4059352961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 msgr2=0x562737bb43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009f80 tx=0x7fe9c800c8b0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 -- 172.21.15.3:0/4059352961 shutdown_connections 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737bb3ea0 0x562737bb43f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737b699e0 0x562737b68240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/4059352961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 -- 172.21.15.3:0/4059352961 >> 172.21.15.3:0/4059352961 conn(0x562737b6a410 msgr2=0x562737bb2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 -- 172.21.15.3:0/4059352961 shutdown_connections 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.670+0000 7fe9da0a8740 1 -- 172.21.15.3:0/4059352961 wait complete. 2024-09-10T07:50:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.671+0000 7fe9da0a8740 1 -- start start 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.671+0000 7fe9da0a8740 1 Processor -- start 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 0x562737baed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 0x562737bb2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562737b68940 con 0x562737b834f0 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x562737a5a4c0 con 0x562737baf290 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562737b68c20 con 0x562737bb3ea0 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9da0a8740 10 client.0 _ll_register_callbacks cb 0x562737ac9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50840/0 (socket says 172.21.15.3:50840) 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.672+0000 7fe9cfe00640 1 -- 172.21.15.3:0/3577036287 learned_addr learned my addr 172.21.15.3:0/3577036287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 0x562737baed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9d5200640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 0x562737bb2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 -- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 msgr2=0x562737baed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 0x562737baed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 -- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 msgr2=0x562737bb2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 0x562737bb2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 -- 172.21.15.3:0/3577036287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627377e43f0 con 0x562737b834f0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 0x562737baed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9d5200640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 0x562737bb2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001d10 tx=0x7fe9c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2045c6d309248e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9bd600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0027060 con 0x562737b834f0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fe9c0010040 con 0x562737b834f0 2024-09-10T07:50:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c80165c0 con 0x562737b834f0 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.673+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00076b0 con 0x562737b834f0 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 10 client.6483 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5627377a40f0 con 0x562737b834f0 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c80165c0 con 0x562737b834f0 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7fe9c0004c70 con 0x562737b834f0 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 10 client.6483 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 10 client.6483 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 20 client.6483 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bab1f0 0x562737bab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 4 client.6483 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:49.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562737aded90 con 0x562737bab1f0 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.674+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bab1f0 0x562737bab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.675+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bab1f0 0x562737bab600 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ba315ad9163b75 server_cookie=8255e498a0d06ed1 in_seq=0 out_seq=0 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.675+0000 7fe9bd600640 10 client.6483 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.675+0000 7fe9bd600640 1 -- 172.21.15.3:0/3577036287 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562737aded90 con 0x562737bab1f0 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.675+0000 7fe9bd600640 10 client.6483 handle_command_reply: tid=0 2024-09-10T07:50:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.675+0000 7fe9da0a8740 1 client.6483 shutdown 2024-09-10T07:50:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 msgr2=0x562737bc5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001d10 tx=0x7fe9c0000bc0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 shutdown_connections 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bab1f0 0x562737bab600 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737baf290 0x562737bb2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737bb3ea0 0x562737baed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/3577036287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bc5cb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9bd600640 0 client.6483 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 >> 172.21.15.3:0/3577036287 conn(0x562737b699e0 msgr2=0x562737b67140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 shutdown_connections 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 1 -- 172.21.15.3:0/3577036287 wait complete. 2024-09-10T07:50:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.676+0000 7fe9da0a8740 20 client.6483 trim_cache size 0 max 16384 2024-09-10T07:50:49.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.694+0000 7fe9da0a8740 1 Processor -- start 2024-09-10T07:50:49.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.694+0000 7fe9da0a8740 1 -- start start 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737bb2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562737bb59f0 con 0x562737baf290 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0200d00 con 0x562737bb3ea0 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d01fff00 con 0x562737b834f0 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50856/0 (socket says 172.21.15.3:50856) 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.695+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2307893120 learned_addr learned my addr 172.21.15.3:0/2307893120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9d5200640 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737bb2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 msgr2=0x562737b63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b63a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 msgr2=0x562737bb2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737bb2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 -- 172.21.15.3:0/2307893120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562737b83f70 con 0x562737baf290 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8075f30 tx=0x7fe9b8048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb8d7fa97f45122 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9bfe00640 1 -- 172.21.15.3:0/2307893120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b807de80 con 0x562737baf290 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9bfe00640 1 -- 172.21.15.3:0/2307893120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b80826b0 con 0x562737baf290 2024-09-10T07:50:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.696+0000 7fe9bfe00640 1 -- 172.21.15.3:0/2307893120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8077b70 con 0x562737baf290 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2307893120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 msgr2=0x562737bd69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8075f30 tx=0x7fe9b8048040 comp rx=0 tx=0).stop 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2307893120 shutdown_connections 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737bb2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737baf290 0x562737bd69a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2307893120 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737b834f0 0x562737b63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2307893120 >> 172.21.15.3:0/2307893120 conn(0x562737b699e0 msgr2=0x562737ba83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2307893120 shutdown_connections 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.697+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2307893120 wait complete. 2024-09-10T07:50:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.698+0000 7fe9da0a8740 1 -- start start 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.698+0000 7fe9da0a8740 1 Processor -- start 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 0x562737bbcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737c2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d00cc5c0 con 0x562737b834f0 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d02005d0 con 0x562737bb3ea0 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x562737bccc40 con 0x562737baf290 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9da0a8740 10 client.0 _ll_register_callbacks cb 0x7fe9c0005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50866/0 (socket says 172.21.15.3:50866) 2024-09-10T07:50:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 -- 172.21.15.3:0/2663015514 learned_addr learned my addr 172.21.15.3:0/2663015514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9d5200640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737c2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 0x562737bbcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 -- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 msgr2=0x562737bbcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 0x562737bbcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 -- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 msgr2=0x562737c2ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737c2ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9cfe00640 1 -- 172.21.15.3:0/2663015514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562737baa7a0 con 0x562737b834f0 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.699+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 0x562737bbcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9cfe00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0013580 tx=0x7fe9c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcea4860feee242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0005420 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fe9c001e070 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b80826b0 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00014b0 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 10 client.6489 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x562737bcc3d0 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b80826b0 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7fe9c00139b0 con 0x562737b834f0 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 10 client.6489 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 10 client.6489 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 20 client.6489 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.700+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bc2240 0x562737bc2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9da0a8740 4 client.6489 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562737ad90e0 con 0x562737bc2240 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bc2240 0x562737bc2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9bd600640 10 client.6489 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9c7e00640 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bc2240 0x562737bc2670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e23b81a3dc33d2 server_cookie=7e670a7b4428041a in_seq=0 out_seq=0 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9bd600640 1 -- 172.21.15.3:0/2663015514 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562737ad90e0 con 0x562737bc2240 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9bd600640 10 client.6489 handle_command_reply: tid=0 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.701+0000 7fe9da0a8740 1 client.6489 shutdown 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 msgr2=0x562737bb4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0013580 tx=0x7fe9c0013000 comp rx=0 tx=0).stop 2024-09-10T07:50:49.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 shutdown_connections 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x562737bc2240 0x562737bc2670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562737bb3ea0 0x562737c2ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x562737baf290 0x562737bbcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9bd600640 0 client.6489 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 --2- 172.21.15.3:0/2663015514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562737b834f0 0x562737bb4f10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 >> 172.21.15.3:0/2663015514 conn(0x562737b699e0 msgr2=0x562737bb2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 shutdown_connections 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 1 -- 172.21.15.3:0/2663015514 wait complete. 2024-09-10T07:50:49.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.702+0000 7fe9da0a8740 20 client.6489 trim_cache size 0 max 16384 2024-09-10T07:50:49.708 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:50:49.708 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-10T07:50:49.708 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-10T07:50:49.708 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 msgr2=0x7fe9ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 0x7fe9ac06e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000f60 tx=0x7fe9c0007ce0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 msgr2=0x7fe9d00b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 shutdown_connections 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9ac06be30 0x7fe9ac06e2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9d0161b70 0x7fe9d00b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9d00d45a0 0x7fe9d00b49a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 --2- 172.21.15.3:0/2671030371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d00cfe90 0x7fe9d01f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 >> 172.21.15.3:0/2671030371 conn(0x7fe9d0062630 msgr2=0x7fe9d0062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 shutdown_connections 2024-09-10T07:50:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.703+0000 7fe9d5c00640 1 -- 172.21.15.3:0/2671030371 wait complete. 2024-09-10T07:50:49.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:50:49.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2024-09-10T07:50:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.875+0000 7f7cb1200640 1 Processor -- start 2024-09-10T07:50:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.876+0000 7f7cb1200640 1 -- start start 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7cb1200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7cac0b4c90 0x7f7cac15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7cb1200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cac072c50 0x7f7cac071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7cb1200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cac15eff0 con 0x7f7cac15b230 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7cb1200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7cac15fbe0 con 0x7f7cac072c50 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7cb1200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7cac15c8d0 con 0x7f7cac0b4c90 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7caa800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cac072c50 0x7f7cac071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7caa800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cac072c50 0x7f7cac071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50874/0 (socket says 172.21.15.3:50874) 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7caa800640 1 -- 172.21.15.3:0/2035916312 learned_addr learned my addr 172.21.15.3:0/2035916312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.877+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2035916312 >> v2:172.21.15.70:3300/0 conn(0x7f7cac0b4c90 0x7f7cac15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.878+0000 7f7caa800640 1 -- 172.21.15.3:0/2035916312 >> v2:172.21.15.70:3300/0 conn(0x7f7cac0b4c90 msgr2=0x7f7cac15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.878+0000 7f7caa800640 1 --2- 172.21.15.3:0/2035916312 >> v2:172.21.15.70:3300/0 conn(0x7f7cac0b4c90 0x7f7cac15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.878+0000 7f7caa800640 1 -- 172.21.15.3:0/2035916312 >> v1:172.21.15.70:6789/0 conn(0x7f7cac15b230 legacy=0x7f7cac0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.878+0000 7f7caa800640 1 -- 172.21.15.3:0/2035916312 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac15f2b0 con 0x7f7cac072c50 2024-09-10T07:50:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.879+0000 7f7caa800640 1 --2- 172.21.15.3:0/2035916312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac072c50 0x7f7cac071010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7cac15fd60 tx=0x7f7ca402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd7dcdfe26ed3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.879+0000 7f7ca9400640 1 -- 172.21.15.3:0/2035916312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4009600 con 0x7f7cac072c50 2024-09-10T07:50:49.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.879+0000 7f7ca9400640 1 -- 172.21.15.3:0/2035916312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca4037070 con 0x7f7cac072c50 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.879+0000 7f7ca9400640 1 -- 172.21.15.3:0/2035916312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40099c0 con 0x7f7cac072c50 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.879+0000 7f7cb1200640 1 -- 172.21.15.3:0/2035916312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac072c50 msgr2=0x7f7cac071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 --2- 172.21.15.3:0/2035916312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac072c50 0x7f7cac071010 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7cac15fd60 tx=0x7f7ca402c930 comp rx=0 tx=0).stop 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 -- 172.21.15.3:0/2035916312 shutdown_connections 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 --2- 172.21.15.3:0/2035916312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac072c50 0x7f7cac071010 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 --2- 172.21.15.3:0/2035916312 >> v2:172.21.15.70:3300/0 conn(0x7f7cac0b4c90 0x7f7cac15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 -- 172.21.15.3:0/2035916312 >> 172.21.15.3:0/2035916312 conn(0x7f7cac0b5cb0 msgr2=0x7f7cac06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 -- 172.21.15.3:0/2035916312 shutdown_connections 2024-09-10T07:50:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.880+0000 7f7cb1200640 1 -- 172.21.15.3:0/2035916312 wait complete. 2024-09-10T07:50:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.881+0000 7f7cb1200640 1 Processor -- start 2024-09-10T07:50:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.881+0000 7f7cb1200640 1 -- start start 2024-09-10T07:50:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 0x7f7cac0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 0x7f7cac15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac15b610 con 0x7f7cac15b230 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac15f870 con 0x7f7cac0b4c90 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.882+0000 7f7cb1200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7cac15b790 con 0x7f7cac072c50 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50880/0 (socket says 172.21.15.3:50880) 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 -- 172.21.15.3:0/767192539 learned_addr learned my addr 172.21.15.3:0/767192539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 0x7f7cac0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 0x7f7cac15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 -- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 msgr2=0x7f7cac0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 0x7f7cac0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 -- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 msgr2=0x7f7cac15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 0x7f7cac15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 -- 172.21.15.3:0/767192539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac15d3f0 con 0x7f7cac15b230 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 0x7f7cac0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 0x7f7cac15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.883+0000 7f7caa800640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4034040 tx=0x7f7ca4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.884+0000 7f7ca2000640 1 -- 172.21.15.3:0/767192539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4009dc0 con 0x7f7cac15b230 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.884+0000 7f7ca2000640 1 -- 172.21.15.3:0/767192539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca4032420 con 0x7f7cac15b230 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.884+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac15d6d0 con 0x7f7cac15b230 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.884+0000 7f7ca2000640 1 -- 172.21.15.3:0/767192539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4032810 con 0x7f7cac15b230 2024-09-10T07:50:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.884+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac15dc90 con 0x7f7cac15b230 2024-09-10T07:50:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.886+0000 7f7ca2000640 1 -- 172.21.15.3:0/767192539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ca4043020 con 0x7f7cac15b230 2024-09-10T07:50:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.886+0000 7f7ca2000640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 0x7f7c8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.887+0000 7f7ca2000640 1 -- 172.21.15.3:0/767192539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f7ca40abb80 con 0x7f7cac15b230 2024-09-10T07:50:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.887+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 0x7f7c8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.887+0000 7f7cb4048740 1 Processor -- start 2024-09-10T07:50:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 -- start start 2024-09-10T07:50:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 0x7f7c8806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c001660 tx=0x7f7c9c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 0x559f389f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f3892e490 con 0x559f38a0f880 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559f388d6690 con 0x559f38a120c0 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.888+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f3892a450 con 0x559f38a0fce0 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50888/0 (socket says 172.21.15.3:50888) 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 -- 172.21.15.3:0/2825040553 learned_addr learned my addr 172.21.15.3:0/2825040553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7caa800640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 0x559f389f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 -- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 msgr2=0x559f389f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 0x559f389f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 -- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 msgr2=0x559f38a124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 -- 172.21.15.3:0/2825040553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f388dde00 con 0x559f38a0f880 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7caa800640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 0x559f389f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.889+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009be0 tx=0x7f7c94009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd28c2150950bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7c9aa00640 1 -- 172.21.15.3:0/2825040553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94019070 con 0x559f38a0f880 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7c9aa00640 1 -- 172.21.15.3:0/2825040553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c94011cd0 con 0x559f38a0f880 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7c9aa00640 1 -- 172.21.15.3:0/2825040553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94004260 con 0x559f38a0f880 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 -- 172.21.15.3:0/2825040553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 msgr2=0x559f38a11410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009be0 tx=0x7f7c94009190 comp rx=0 tx=0).stop 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 -- 172.21.15.3:0/2825040553 shutdown_connections 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0fce0 0x559f389f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0f880 0x559f38a11410 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2825040553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 -- 172.21.15.3:0/2825040553 >> 172.21.15.3:0/2825040553 conn(0x559f389c7d20 msgr2=0x559f38a10130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 -- 172.21.15.3:0/2825040553 shutdown_connections 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.890+0000 7f7cb4048740 1 -- 172.21.15.3:0/2825040553 wait complete. 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.891+0000 7f7cb4048740 1 -- start start 2024-09-10T07:50:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.891+0000 7f7cb4048740 1 Processor -- start 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a09e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 0x559f38a0cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f38a0e7e0 con 0x559f38a0fce0 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559f388e9950 con 0x559f38a120c0 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f38a0eac0 con 0x559f38a0ca80 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7cb4048740 10 client.0 _ll_register_callbacks cb 0x559f388b54d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50900/0 (socket says 172.21.15.3:50900) 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.892+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2140577078 learned_addr learned my addr 172.21.15.3:0/2140577078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a09e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7caa800640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 0x559f38a0cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 msgr2=0x559f38a0cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 0x559f38a0cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 msgr2=0x559f38a09e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a09e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2140577078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f388e2f40 con 0x559f38a0fce0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7caa800640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 0x559f38a0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e810 tx=0x7f7c9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f055a3134b3c848 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c027060 con 0x559f38a0fce0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f7c9c010040 con 0x559f38a0fce0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.893+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c94011cd0 con 0x559f38a0fce0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c003b70 con 0x559f38a0fce0 2024-09-10T07:50:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 10 client.6501 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x559f386413f0 con 0x559f38a0fce0 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c94011cd0 con 0x559f38a0fce0 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f7c9c022430 con 0x559f38a0fce0 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 10 client.6501 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 10 client.6501 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 20 client.6501 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f389f3b00 0x559f389f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 4 client.6501 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f3893bbc0 con 0x559f389f3b00 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.894+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f389f3b00 0x559f389f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.895+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f389f3b00 0x559f389f0570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143e76d1d0dc251f server_cookie=7ff936ab05055b97 in_seq=0 out_seq=0 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.895+0000 7f7c8f400640 10 client.6501 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7c8f400640 1 -- 172.21.15.3:0/2140577078 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f3893bbc0 con 0x559f389f3b00 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7c8f400640 10 client.6501 handle_command_reply: tid=0 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 client.6501 shutdown 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 msgr2=0x559f38a24540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e810 tx=0x7f7c9c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 shutdown_connections 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f389f3b00 0x559f389f0570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a0ca80 0x559f38a0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a120c0 0x559f38a09e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2140577078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a24540 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7c8f400640 0 client.6501 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 >> 172.21.15.3:0/2140577078 conn(0x559f38a0f880 msgr2=0x559f38a10100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 shutdown_connections 2024-09-10T07:50:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 1 -- 172.21.15.3:0/2140577078 wait complete. 2024-09-10T07:50:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.896+0000 7f7cb4048740 20 client.6501 trim_cache size 0 max 16384 2024-09-10T07:50:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.910+0000 7f7cb4048740 1 Processor -- start 2024-09-10T07:50:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.910+0000 7f7cb4048740 1 -- start start 2024-09-10T07:50:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 0x559f38a0c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac200d90 con 0x559f38a0ca80 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1fff90 con 0x559f38a0fce0 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7cac200660 con 0x559f38a120c0 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50904/0 (socket says 172.21.15.3:50904) 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2516551994 learned_addr learned my addr 172.21.15.3:0/2516551994 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 0x559f38a0c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.911+0000 7f7caa800640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 msgr2=0x559f38a220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 msgr2=0x559f38a0c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 0x559f38a0c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 -- 172.21.15.3:0/2516551994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3870b7d0 con 0x559f38a0ca80 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 0x559f38a0c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e530 tx=0x7f7c9c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0ba2c8b9aafd15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7c9a000640 1 -- 172.21.15.3:0/2516551994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c001cf0 con 0x559f38a0ca80 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7c9a000640 1 -- 172.21.15.3:0/2516551994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9c003d10 con 0x559f38a0ca80 2024-09-10T07:50:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.912+0000 7f7c9a000640 1 -- 172.21.15.3:0/2516551994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c022430 con 0x559f38a0ca80 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 -- 172.21.15.3:0/2516551994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 msgr2=0x559f38a067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e530 tx=0x7f7c9c01f000 comp rx=0 tx=0).stop 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 -- 172.21.15.3:0/2516551994 shutdown_connections 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0fce0 0x559f38a0c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 --2- 172.21.15.3:0/2516551994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0ca80 0x559f38a067b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 -- 172.21.15.3:0/2516551994 >> 172.21.15.3:0/2516551994 conn(0x559f38a0f880 msgr2=0x559f38a100c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 -- 172.21.15.3:0/2516551994 shutdown_connections 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.913+0000 7f7cb4048740 1 -- 172.21.15.3:0/2516551994 wait complete. 2024-09-10T07:50:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.914+0000 7f7cb4048740 1 -- start start 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.914+0000 7f7cb4048740 1 Processor -- start 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0ca80 0x559f38a08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559f389c75e0 con 0x559f38a0fce0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1ff0b0 con 0x559f38a0ca80 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559f388e67a0 con 0x559f38a120c0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7cb4048740 10 client.0 _ll_register_callbacks cb 0x7f7c9c001c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50906/0 (socket says 172.21.15.3:50906) 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 -- 172.21.15.3:0/4204098823 learned_addr learned my addr 172.21.15.3:0/4204098823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0ca80 0x559f38a08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7caa800640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 -- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 msgr2=0x559f38a10600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a10600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 -- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0ca80 msgr2=0x559f38a08840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0ca80 0x559f38a08840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7ca3e00640 1 -- 172.21.15.3:0/4204098823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f38a08d80 con 0x559f38a0fce0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.915+0000 7f7caa800640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a10600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7ca3e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7c94018000 tx=0x7f7c94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb0a7bd84fe0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94019040 con 0x559f38a0fce0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f7c94004250 con 0x559f38a0fce0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c9c003d10 con 0x559f38a0fce0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7c94004450 con 0x559f38a0fce0 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7cb4048740 10 client.6507 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.916+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x559f38a09480 con 0x559f38a0fce0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9c003d10 con 0x559f38a0fce0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f7c9403d080 con 0x559f38a0fce0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 10 client.6507 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 10 client.6507 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 20 client.6507 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f38a74bc0 0x559f389cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 4 client.6507 mds_command: new command op to 6114 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 --> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f38935f10 con 0x559f38a74bc0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f38a74bc0 0x559f389cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7ca9e00640 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f38a74bc0 0x559f389cb2d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61554b09480ef7af server_cookie=6210cd9a7212e959 in_seq=0 out_seq=0 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.917+0000 7f7c8f400640 10 client.6507 ms_handle_connect on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7c8f400640 1 -- 172.21.15.3:0/4204098823 <== mds.0 v2:172.21.15.3:6834/1359877653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+271 (crc 0 0 0) 0x559f38935f10 con 0x559f38a74bc0 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7c8f400640 10 client.6507 handle_command_reply: tid=0 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 client.6507 shutdown 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 msgr2=0x559f38a100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7c94018000 tx=0x7f7c94004730 comp rx=0 tx=0).stop 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 shutdown_connections 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:6834/1359877653,v1:172.21.15.3:6836/1359877653] conn(0x559f38a74bc0 0x559f389cb2d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559f38a120c0 0x559f38a10600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559f38a0fce0 0x559f38a100c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7c8f400640 0 client.6507 ms_handle_reset on v2:172.21.15.3:6834/1359877653 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 --2- 172.21.15.3:0/4204098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559f38a0ca80 0x559f38a08840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 >> 172.21.15.3:0/4204098823 conn(0x559f38a0f880 msgr2=0x559f389f4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 shutdown_connections 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 1 -- 172.21.15.3:0/4204098823 wait complete. 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.918+0000 7f7cb4048740 20 client.6507 trim_cache size 0 max 16384 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 933322128, 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627779, 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3/d4" 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:49.923 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2500780978, 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627778, 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3" 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:50:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 msgr2=0x7f7c8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 0x7f7c8806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c001660 tx=0x7f7c9c009040 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 msgr2=0x7f7cac159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4034040 tx=0x7f7ca4004a40 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 shutdown_connections 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c8806be30 0x7f7c8806e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cac15b230 0x7f7cac159c30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cac0b4c90 0x7f7cac15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 --2- 172.21.15.3:0/767192539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cac072c50 0x7f7cac0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 >> 172.21.15.3:0/767192539 conn(0x7f7cac0b5cb0 msgr2=0x7f7cac0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 shutdown_connections 2024-09-10T07:50:49.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:49.919+0000 7f7cb1200640 1 -- 172.21.15.3:0/767192539 wait complete. 2024-09-10T07:50:49.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2024-09-10T07:50:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.090+0000 7f44f0e00640 1 Processor -- start 2024-09-10T07:50:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.091+0000 7f44f0e00640 1 -- start start 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44f0e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ec15b230 0x7f44ec15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44f0e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44f0e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ec15eff0 con 0x7f44ec072c50 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44f0e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec15fbe0 con 0x7f44ec15b230 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44f0e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec15c8d0 con 0x7f44ec0b4c90 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44e9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ec15b230 0x7f44ec15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44e3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44ec072c50 0x7f44ec0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33820/0 (socket says 172.21.15.3:33820) 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44e9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44ec15b230 0x7f44ec15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50908/0 (socket says 172.21.15.3:50908) 2024-09-10T07:50:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.092+0000 7f44e3e00640 1 -- 172.21.15.3:0/247506539 learned_addr learned my addr 172.21.15.3:0/247506539 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44ea800640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44e9e00640 1 -- 172.21.15.3:0/247506539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 msgr2=0x7f44ec071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44e9e00640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44e9400640 1 -- 172.21.15.3:0/247506539 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531626912 0 0) 0x7f44ec15eff0 con 0x7f44ec072c50 2024-09-10T07:50:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44e9e00640 1 -- 172.21.15.3:0/247506539 >> v1:172.21.15.3:6789/0 conn(0x7f44ec072c50 legacy=0x7f44ec0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44e9e00640 1 -- 172.21.15.3:0/247506539 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec15f2b0 con 0x7f44ec15b230 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.093+0000 7f44ea800640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44e9e00640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec15c390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0097a0 tx=0x7f44dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66a955fa542ca97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44e9400640 1 -- 172.21.15.3:0/247506539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc037070 con 0x7f44ec15b230 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44e9400640 1 -- 172.21.15.3:0/247506539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc032440 con 0x7f44ec15b230 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44e9400640 1 -- 172.21.15.3:0/247506539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc032810 con 0x7f44ec15b230 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44f0e00640 1 -- 172.21.15.3:0/247506539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 msgr2=0x7f44ec15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.094+0000 7f44f0e00640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec15c390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0097a0 tx=0x7f44dc02a180 comp rx=0 tx=0).stop 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 -- 172.21.15.3:0/247506539 shutdown_connections 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 --2- 172.21.15.3:0/247506539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec15c390 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 -- 172.21.15.3:0/247506539 >> 172.21.15.3:0/247506539 conn(0x7f44ec0b5cb0 msgr2=0x7f44ec06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 -- 172.21.15.3:0/247506539 shutdown_connections 2024-09-10T07:50:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.095+0000 7f44f0e00640 1 -- 172.21.15.3:0/247506539 wait complete. 2024-09-10T07:50:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.096+0000 7f44f0e00640 1 Processor -- start 2024-09-10T07:50:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.096+0000 7f44f0e00640 1 -- start start 2024-09-10T07:50:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 0x7f44ec1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15b610 con 0x7f44ec15b230 2024-09-10T07:50:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15f870 con 0x7f44ec072c50 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.097+0000 7f44f0e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec15b790 con 0x7f44ec0b4c90 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50910/0 (socket says 172.21.15.3:50910) 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 -- 172.21.15.3:0/791231653 learned_addr learned my addr 172.21.15.3:0/791231653 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 -- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 msgr2=0x7f44ec0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44e9e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 0x7f44ec1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44e3e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 -- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 msgr2=0x7f44ec1356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 0x7f44ec1356b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 -- 172.21.15.3:0/791231653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec0b2d80 con 0x7f44ec15b230 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44e9e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 0x7f44ec1356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44e3e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.098+0000 7f44ea800640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cef0 tx=0x7f44e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.099+0000 7f44e2000640 1 -- 172.21.15.3:0/791231653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e401a070 con 0x7f44ec15b230 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.099+0000 7f44e2000640 1 -- 172.21.15.3:0/791231653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e400d860 con 0x7f44ec15b230 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.099+0000 7f44e2000640 1 -- 172.21.15.3:0/791231653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e400dc50 con 0x7f44ec15b230 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.099+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec15cca0 con 0x7f44ec15b230 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.099+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec14f520 con 0x7f44ec15b230 2024-09-10T07:50:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.101+0000 7f44e2000640 1 -- 172.21.15.3:0/791231653 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44e401d070 con 0x7f44ec15b230 2024-09-10T07:50:50.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.102+0000 7f44e2000640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.103+0000 7f44e2000640 1 -- 172.21.15.3:0/791231653 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f44e408b730 con 0x7f44ec15b230 2024-09-10T07:50:50.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.103+0000 7f44e9e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.103+0000 7f44f3c7f740 1 Processor -- start 2024-09-10T07:50:50.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.103+0000 7f44f3c7f740 1 -- start start 2024-09-10T07:50:50.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.103+0000 7f44e9e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 0x7f44c806e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44dc034000 tx=0x7f44dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83bb014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bae1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f83ba1d490 con 0x55f83bafe880 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f83b9c5690 con 0x55f83bafece0 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f83ba19450 con 0x55f83bb010c0 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50916/0 (socket says 172.21.15.3:50916) 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44e3e00640 1 -- 172.21.15.3:0/2945076556 learned_addr learned my addr 172.21.15.3:0/2945076556 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44ea800640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bae1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.104+0000 7f44e9e00640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83bb014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 -- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 msgr2=0x55f83bb014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83bb014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 -- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 msgr2=0x55f83bae1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bae1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 -- 172.21.15.3:0/2945076556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83b9cce00 con 0x55f83bafe880 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44ea800640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bae1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44e3e00640 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f44d4001750 tx=0x7f44d4001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62780377e078d44a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44daa00640 1 -- 172.21.15.3:0/2945076556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40074e0 con 0x55f83bafe880 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44daa00640 1 -- 172.21.15.3:0/2945076556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d40047e0 con 0x55f83bafe880 2024-09-10T07:50:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.105+0000 7f44daa00640 1 -- 172.21.15.3:0/2945076556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4004b90 con 0x55f83bafe880 2024-09-10T07:50:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 -- 172.21.15.3:0/2945076556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 msgr2=0x55f83bb00410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f44d4001750 tx=0x7f44d4001c10 comp rx=0 tx=0).stop 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 -- 172.21.15.3:0/2945076556 shutdown_connections 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bae1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bafe880 0x55f83bb00410 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/2945076556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83bb014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 -- 172.21.15.3:0/2945076556 >> 172.21.15.3:0/2945076556 conn(0x55f83bab6d20 msgr2=0x55f83baff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 -- 172.21.15.3:0/2945076556 shutdown_connections 2024-09-10T07:50:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.106+0000 7f44f3c7f740 1 -- 172.21.15.3:0/2945076556 wait complete. 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.107+0000 7f44f3c7f740 1 -- start start 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.107+0000 7f44f3c7f740 1 Processor -- start 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83badeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f83bafd7e0 con 0x55f83bae3650 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f83b9d8950 con 0x55f83bafece0 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f83bafdac0 con 0x55f83bb010c0 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44f3c7f740 10 client.0 _ll_register_callbacks cb 0x55f83b9a44d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50926/0 (socket says 172.21.15.3:50926) 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 -- 172.21.15.3:0/922055595 learned_addr learned my addr 172.21.15.3:0/922055595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 -- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 msgr2=0x55f83badeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83badeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44e9e00640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.108+0000 7f44ea800640 1 -- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 msgr2=0x55f83bab9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44ea800640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44ea800640 1 -- 172.21.15.3:0/922055595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83b9d1f40 con 0x55f83bae3650 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44e9e00640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44ea800640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f44e4089b80 tx=0x7f44e4059ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948727179814b730 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44cf400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4059bf0 con 0x55f83bae3650 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f44e4059d90 con 0x55f83bae3650 2024-09-10T07:50:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d40047e0 con 0x55f83bae3650 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f44e40620e0 con 0x55f83bae3650 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44f3c7f740 10 client.6522 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55f83b7303f0 con 0x55f83bae3650 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.109+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d40047e0 con 0x55f83bae3650 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f44e4058de0 con 0x55f83bae3650 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 10 client.6522 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 10 client.6522 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 20 client.6522 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83baba7f0 0x55f83babac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 4 client.6522 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83ba2abc0 con 0x55f83baba7f0 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44e9e00640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83baba7f0 0x55f83babac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.110+0000 7f44e9e00640 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83baba7f0 0x55f83babac30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8606122d59109d06 server_cookie=715fbc3c43011b87 in_seq=0 out_seq=0 2024-09-10T07:50:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44cf400640 10 client.6522 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44cf400640 1 -- 172.21.15.3:0/922055595 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f83ba2abc0 con 0x55f83baba7f0 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44cf400640 10 client.6522 handle_command_reply: tid=0 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44f3c7f740 1 client.6522 shutdown 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 msgr2=0x55f83baf6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.111+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f44e4089b80 tx=0x7f44e4059ad0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 shutdown_connections 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83baba7f0 0x55f83babac30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83baf6e50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bb010c0 0x55f83badeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/922055595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44cf400640 0 client.6522 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 >> 172.21.15.3:0/922055595 conn(0x55f83bafe880 msgr2=0x55f83bae1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 shutdown_connections 2024-09-10T07:50:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 1 -- 172.21.15.3:0/922055595 wait complete. 2024-09-10T07:50:50.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.112+0000 7f44f3c7f740 20 client.6522 trim_cache size 0 max 16384 2024-09-10T07:50:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.126+0000 7f44f3c7f740 1 Processor -- start 2024-09-10T07:50:50.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.126+0000 7f44f3c7f740 1 -- start start 2024-09-10T07:50:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.126+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.126+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bafece0 0x55f83bb00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.126+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bb010c0 0x55f83bb02690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec200ba0 con 0x55f83bae3650 2024-09-10T07:50:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1ffda0 con 0x55f83bb010c0 2024-09-10T07:50:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec200470 con 0x55f83bafece0 2024-09-10T07:50:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50940/0 (socket says 172.21.15.3:50940) 2024-09-10T07:50:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 -- 172.21.15.3:0/399652618 learned_addr learned my addr 172.21.15.3:0/399652618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 -- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bafece0 msgr2=0x55f83bb00650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bafece0 0x55f83bb00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 -- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bb010c0 msgr2=0x55f83bb02690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bb010c0 0x55f83bb02690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 -- 172.21.15.3:0/399652618 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83b7fa7d0 con 0x55f83bae3650 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.127+0000 7f44e9e00640 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f44dc02a1c0 tx=0x7f44dc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b7043a0c11a43d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44da000640 1 -- 172.21.15.3:0/399652618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc037040 con 0x55f83bae3650 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44da000640 1 -- 172.21.15.3:0/399652618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc03cb70 con 0x55f83bae3650 2024-09-10T07:50:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44da000640 1 -- 172.21.15.3:0/399652618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc049410 con 0x55f83bae3650 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 -- 172.21.15.3:0/399652618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 msgr2=0x55f83bab6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f44dc02a1c0 tx=0x7f44dc02c000 comp rx=0 tx=0).stop 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 -- 172.21.15.3:0/399652618 shutdown_connections 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bb010c0 0x55f83bb02690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bafece0 0x55f83bb00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/399652618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bae3650 0x55f83bab6750 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 -- 172.21.15.3:0/399652618 >> 172.21.15.3:0/399652618 conn(0x55f83bafe880 msgr2=0x55f83bababd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 -- 172.21.15.3:0/399652618 shutdown_connections 2024-09-10T07:50:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.128+0000 7f44f3c7f740 1 -- 172.21.15.3:0/399652618 wait complete. 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.129+0000 7f44f3c7f740 1 -- start start 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.129+0000 7f44f3c7f740 1 Processor -- start 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bae3650 0x55f83bb7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec14e8f0 con 0x55f83bb010c0 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f83bab4e90 con 0x55f83bafece0 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f83bb23900 con 0x55f83bae3650 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44f3c7f740 10 client.0 _ll_register_callbacks cb 0x7f44e405a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44ea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50950/0 (socket says 172.21.15.3:50950) 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44ea800640 1 -- 172.21.15.3:0/3460678557 learned_addr learned my addr 172.21.15.3:0/3460678557 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44e3e00640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.130+0000 7f44e9e00640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bae3650 0x55f83bb7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 -- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bae3650 msgr2=0x55f83bb7f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bae3650 0x55f83bb7f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 -- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 msgr2=0x55f83bab3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 -- 172.21.15.3:0/3460678557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83bb05f00 con 0x55f83bb010c0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44ea800640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f44e4059640 tx=0x7f44e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200b0fd96c4f43e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44cf400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4055c30 con 0x55f83bb010c0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f44e40558f0 con 0x55f83bb010c0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44dc03cb70 con 0x55f83bb010c0 2024-09-10T07:50:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f44e4058850 con 0x55f83bb010c0 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44f3c7f740 10 client.6528 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.131+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55f83bb061c0 con 0x55f83bb010c0 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc03cb70 con 0x55f83bb010c0 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f44e4078030 con 0x55f83bb010c0 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 10 client.6528 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 10 client.6528 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.a 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 20 client.6528 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83bab5f30 0x55f83baff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 4 client.6528 mds_command: new command op to 6111 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 --> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f83baf69c0 con 0x55f83bab5f30 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44e9e00640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83bab5f30 0x55f83baff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44e9e00640 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83bab5f30 0x55f83baff0c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7603b839845a6e6b server_cookie=3354628d84d39d5a in_seq=0 out_seq=0 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.132+0000 7f44cf400640 10 client.6528 ms_handle_connect on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44cf400640 1 -- 172.21.15.3:0/3460678557 <== mds.1 v2:172.21.15.3:6835/1603892470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+284 (crc 0 0 0) 0x55f83baf69c0 con 0x55f83bab5f30 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44cf400640 10 client.6528 handle_command_reply: tid=0 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 client.6528 shutdown 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 msgr2=0x55f83bab3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f44e4059640 tx=0x7f44e4004a40 comp rx=0 tx=0).stop 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 shutdown_connections 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:6835/1603892470,v1:172.21.15.3:6837/1603892470] conn(0x55f83bab5f30 0x55f83baff0c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f83bb010c0 0x55f83bab3f90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f83bafece0 0x55f83bab3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 --2- 172.21.15.3:0/3460678557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f83bae3650 0x55f83bb7f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44cf400640 0 client.6528 ms_handle_reset on v2:172.21.15.3:6835/1603892470 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 >> 172.21.15.3:0/3460678557 conn(0x55f83bafe880 msgr2=0x55f83bae3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 shutdown_connections 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 1 -- 172.21.15.3:0/3460678557 wait complete. 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.133+0000 7f44f3c7f740 20 client.6528 trim_cache size 0 max 16384 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:50:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2087199375, 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627780, 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3/d4/d5" 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2863212621, 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627781, 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:50:50.138 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 msgr2=0x7f44c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 0x7f44c806e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44dc034000 tx=0x7f44dc02e000 comp rx=0 tx=0).stop 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 msgr2=0x7f44ec0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f44e400cef0 tx=0x7f44e400a710 comp rx=0 tx=0).stop 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 shutdown_connections 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44ec15b230 0x7f44ec0b2760 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44ec0b4c90 0x7f44ec0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 --2- 172.21.15.3:0/791231653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44ec072c50 0x7f44ec1356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 >> 172.21.15.3:0/791231653 conn(0x7f44ec0b5cb0 msgr2=0x7f44ec0aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 shutdown_connections 2024-09-10T07:50:50.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.134+0000 7f44f0e00640 1 -- 172.21.15.3:0/791231653 wait complete. 2024-09-10T07:50:50.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2024-09-10T07:50:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.309+0000 7f21cfe00640 1 Processor -- start 2024-09-10T07:50:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.309+0000 7f21cfe00640 1 -- start start 2024-09-10T07:50:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.310+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c815b230 0x7f21c815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21c8072c50 0x7f21c80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cfe00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c815eff0 con 0x7f21c80b4ac0 2024-09-10T07:50:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cfe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f21c815fbe0 con 0x7f21c8072c50 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cfe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21c815c8d0 con 0x7f21c815b230 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21c8072c50 0x7f21c80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21c8072c50 0x7f21c80b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50964/0 (socket says 172.21.15.3:50964) 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 -- 172.21.15.3:0/3572982875 learned_addr learned my addr 172.21.15.3:0/3572982875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cd400640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c815b230 0x7f21c815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 -- 172.21.15.3:0/3572982875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c815b230 msgr2=0x7f21c815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c815b230 0x7f21c815c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 -- 172.21.15.3:0/3572982875 >> v1:172.21.15.70:6789/0 conn(0x7f21c80b4ac0 legacy=0x7f21c806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.311+0000 7f21cca00640 1 -- 172.21.15.3:0/3572982875 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c815f2b0 con 0x7f21c8072c50 2024-09-10T07:50:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.312+0000 7f21cca00640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c8072c50 0x7f21c80b4580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387748a65ca8f0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21c7e00640 1 -- 172.21.15.3:0/3572982875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4037070 con 0x7f21c8072c50 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21c7e00640 1 -- 172.21.15.3:0/3572982875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4032440 con 0x7f21c8072c50 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21c7e00640 1 -- 172.21.15.3:0/3572982875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4032810 con 0x7f21c8072c50 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 -- 172.21.15.3:0/3572982875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c8072c50 msgr2=0x7f21c80b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c8072c50 0x7f21c80b4580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b402a250 comp rx=0 tx=0).stop 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 -- 172.21.15.3:0/3572982875 shutdown_connections 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c8072c50 0x7f21c80b4580 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 --2- 172.21.15.3:0/3572982875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c815b230 0x7f21c815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 -- 172.21.15.3:0/3572982875 >> 172.21.15.3:0/3572982875 conn(0x7f21c80b5cb0 msgr2=0x7f21c806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 -- 172.21.15.3:0/3572982875 shutdown_connections 2024-09-10T07:50:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.313+0000 7f21cfe00640 1 -- 172.21.15.3:0/3572982875 wait complete. 2024-09-10T07:50:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.315+0000 7f21cfe00640 1 Processor -- start 2024-09-10T07:50:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.315+0000 7f21cfe00640 1 -- start start 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 0x7f21c815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c80b4ac0 0x7f21c80aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21c815b610 con 0x7f21c815b230 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f21c815f870 con 0x7f21c80b4ac0 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cfe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21c815b790 con 0x7f21c8072c50 2024-09-10T07:50:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cd400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 0x7f21c815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50974/0 (socket says 172.21.15.3:50974) 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cd400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 0x7f21c815a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:53484/0 (socket says 172.21.15.3:53484) 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c80b4ac0 0x7f21c80aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.316+0000 7f21cde00640 1 -- 172.21.15.3:0/943367766 learned_addr learned my addr 172.21.15.3:0/943367766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 -- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 msgr2=0x7f21c815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 0x7f21c815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 -- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c80b4ac0 msgr2=0x7f21c80aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c80b4ac0 0x7f21c80aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 -- 172.21.15.3:0/943367766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c81f7a30 con 0x7f21c815b230 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cde00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c7a0 tx=0x7f21c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21c5600640 1 -- 172.21.15.3:0/943367766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0003b70 con 0x7f21c815b230 2024-09-10T07:50:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21c5600640 1 -- 172.21.15.3:0/943367766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c0003d30 con 0x7f21c815b230 2024-09-10T07:50:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21c5600640 1 -- 172.21.15.3:0/943367766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c000aa90 con 0x7f21c815b230 2024-09-10T07:50:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.317+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c81f49b0 con 0x7f21c815b230 2024-09-10T07:50:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.318+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c81f5250 con 0x7f21c815b230 2024-09-10T07:50:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.321+0000 7f21c5600640 1 -- 172.21.15.3:0/943367766 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21c000ac30 con 0x7f21c815b230 2024-09-10T07:50:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.321+0000 7f21c5600640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 0x7f21a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.322+0000 7f21c5600640 1 -- 172.21.15.3:0/943367766 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f21c008ac20 con 0x7f21c815b230 2024-09-10T07:50:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.322+0000 7f21cd400640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 0x7f21a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.322+0000 7f21d2c76740 1 Processor -- start 2024-09-10T07:50:50.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.322+0000 7f21d2c76740 1 -- start start 2024-09-10T07:50:50.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.322+0000 7f21cd400640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 0x7f21a806e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 0x55e1f26e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 0x55e1f26ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e1f2606490 con 0x55e1f26ea0c0 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e1f25ae690 con 0x55e1f26e7ce0 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e1f2602450 con 0x55e1f26e7880 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21cd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21cd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50984/0 (socket says 172.21.15.3:50984) 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21cd400640 1 -- 172.21.15.3:0/2302789873 learned_addr learned my addr 172.21.15.3:0/2302789873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21cde00640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 0x55e1f26ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.323+0000 7f21cca00640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 0x55e1f26e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 -- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 msgr2=0x55e1f26e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 0x55e1f26e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 -- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 msgr2=0x55e1f26ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 0x55e1f26ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 -- 172.21.15.3:0/2302789873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f25b5e00 con 0x55e1f26ea0c0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cde00640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 0x55e1f26ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cca00640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 0x55e1f26e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21cd400640 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f21bc007fd0 tx=0x7f21bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17adb7033cde0b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21b9c00640 1 -- 172.21.15.3:0/2302789873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc027070 con 0x55e1f26ea0c0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21b9c00640 1 -- 172.21.15.3:0/2302789873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc003820 con 0x55e1f26ea0c0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.324+0000 7f21b9c00640 1 -- 172.21.15.3:0/2302789873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc003bd0 con 0x55e1f26ea0c0 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 -- 172.21.15.3:0/2302789873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 msgr2=0x55e1f26ea4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f21bc007fd0 tx=0x7f21bc013000 comp rx=0 tx=0).stop 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 -- 172.21.15.3:0/2302789873 shutdown_connections 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e7ce0 0x55e1f26ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7880 0x55e1f26e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 --2- 172.21.15.3:0/2302789873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26ea4a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 -- 172.21.15.3:0/2302789873 >> 172.21.15.3:0/2302789873 conn(0x55e1f269fd20 msgr2=0x55e1f26e8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 -- 172.21.15.3:0/2302789873 shutdown_connections 2024-09-10T07:50:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.325+0000 7f21d2c76740 1 -- 172.21.15.3:0/2302789873 wait complete. 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.326+0000 7f21d2c76740 1 -- start start 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.326+0000 7f21d2c76740 1 Processor -- start 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e1f26e67e0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e1f25c1950 con 0x55e1f26ea0c0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e1f26e6ac0 con 0x55e1f26e2720 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21d2c76740 10 client.0 _ll_register_callbacks cb 0x55e1f258d4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50992/0 (socket says 172.21.15.3:50992) 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 -- 172.21.15.3:0/3058905786 learned_addr learned my addr 172.21.15.3:0/3058905786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cca00640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cde00640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 -- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 msgr2=0x55e1f26e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 -- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 msgr2=0x55e1f26e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.327+0000 7f21cd400640 1 -- 172.21.15.3:0/3058905786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f25baf40 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21cd400640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f21bc01a040 tx=0x7f21bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd7a7c2d130ff42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21ae000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc027040 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f21bc0168c0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21bc003820 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f21bc016b00 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21d2c76740 10 client.6540 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.328+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55e1f23193f0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc003820 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f21bc02a070 con 0x55e1f26e7ce0 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 10 client.6540 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 10 client.6540 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-09-10T07:50:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 20 client.6540 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f26fb600 0x55e1f26fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 4 client.6540 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1f2613bc0 con 0x55e1f26fb600 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.329+0000 7f21cca00640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f26fb600 0x55e1f26fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.330+0000 7f21cca00640 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f26fb600 0x55e1f26fba40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e112a84b1b86bfaf server_cookie=285b565b679875a5 in_seq=0 out_seq=0 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.330+0000 7f21ae000640 10 client.6540 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21ae000640 1 -- 172.21.15.3:0/3058905786 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1f2613bc0 con 0x55e1f26fb600 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21ae000640 10 client.6540 handle_command_reply: tid=0 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 client.6540 shutdown 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 msgr2=0x55e1f26c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f21bc01a040 tx=0x7f21bc004730 comp rx=0 tx=0).stop 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 shutdown_connections 2024-09-10T07:50:50.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f26fb600 0x55e1f26fba40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3058905786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f26c8b40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21ae000640 0 client.6540 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 >> 172.21.15.3:0/3058905786 conn(0x55e1f26e7880 msgr2=0x55e1f26e8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 shutdown_connections 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 1 -- 172.21.15.3:0/3058905786 wait complete. 2024-09-10T07:50:50.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.331+0000 7f21d2c76740 20 client.6540 trim_cache size 0 max 16384 2024-09-10T07:50:50.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.345+0000 7f21d2c76740 1 Processor -- start 2024-09-10T07:50:50.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.345+0000 7f21d2c76740 1 -- start start 2024-09-10T07:50:50.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 0x55e1f26a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 0x55e1f269e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8200e80 con 0x55e1f26ea0c0 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8200080 con 0x55e1f26e2720 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21c8200750 con 0x55e1f26e7ce0 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21cde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21cde00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50996/0 (socket says 172.21.15.3:50996) 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21cde00640 1 -- 172.21.15.3:0/942161506 learned_addr learned my addr 172.21.15.3:0/942161506 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cca00640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 0x55e1f269e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.346+0000 7f21cd400640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 0x55e1f26a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 -- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 msgr2=0x55e1f269e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 0x55e1f269e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 -- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 msgr2=0x55e1f26a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 0x55e1f26a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 -- 172.21.15.3:0/942161506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f23e37d0 con 0x55e1f26ea0c0 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cd400640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 0x55e1f26a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cca00640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 0x55e1f269e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:50.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21cde00640 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f21c0058f60 tx=0x7f21c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9104a883504e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21b9200640 1 -- 172.21.15.3:0/942161506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0086020 con 0x55e1f26ea0c0 2024-09-10T07:50:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21b9200640 1 -- 172.21.15.3:0/942161506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c005d2d0 con 0x55e1f26ea0c0 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.347+0000 7f21b9200640 1 -- 172.21.15.3:0/942161506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0061b00 con 0x55e1f26ea0c0 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 -- 172.21.15.3:0/942161506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 msgr2=0x55e1f26e9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f21c0058f60 tx=0x7f21c00014a0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 -- 172.21.15.3:0/942161506 shutdown_connections 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26ea0c0 0x55e1f26e9420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e7ce0 0x55e1f269e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 --2- 172.21.15.3:0/942161506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26e2720 0x55e1f26a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 -- 172.21.15.3:0/942161506 >> 172.21.15.3:0/942161506 conn(0x55e1f26e7880 msgr2=0x55e1f26eaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 -- 172.21.15.3:0/942161506 shutdown_connections 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.348+0000 7f21d2c76740 1 -- 172.21.15.3:0/942161506 wait complete. 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.349+0000 7f21d2c76740 1 -- start start 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.349+0000 7f21d2c76740 1 Processor -- start 2024-09-10T07:50:50.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26fa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80ce620 con 0x55e1f26e7ce0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e1f270c730 con 0x55e1f26ea0c0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e1f26e5180 con 0x55e1f26e2720 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21d2c76740 10 client.0 _ll_register_callbacks cb 0x7f21bc025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51002/0 (socket says 172.21.15.3:51002) 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 -- 172.21.15.3:0/3330973649 learned_addr learned my addr 172.21.15.3:0/3330973649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cd400640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cde00640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 -- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 msgr2=0x55e1f26fa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26fa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 -- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 msgr2=0x55e1f26c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.350+0000 7f21cca00640 1 -- 172.21.15.3:0/3330973649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f2701ed0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21cca00640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f21b402a840 tx=0x7f21b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78ad3a758735280 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b404a060 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f21b4004380 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c005d2d0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f21b4050050 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21d2c76740 10 client.6546 fetch_fsmap learned FSMap version 59 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55e1f26e1270 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c005d2d0 con 0x55e1f26e7ce0 2024-09-10T07:50:50.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.351+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 59) ==== 3550+0+0 (secure 0 0 0) 0x7f21b4053020 con 0x55e1f26e7ce0 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 10 client.6546 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 10 client.6546 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.d 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 20 client.6546 populate_metadata read hostname 'smithi003' 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f2698e40 0x55e1f26ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 4 client.6546 mds_command: new command op to 5857 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 --> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1f260df10 con 0x55e1f2698e40 2024-09-10T07:50:50.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.352+0000 7f21cd400640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f2698e40 0x55e1f26ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21cd400640 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f2698e40 0x55e1f26ca010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=896d8e7b49967910 server_cookie=de10443c03a1e19d in_seq=0 out_seq=0 2024-09-10T07:50:50.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21ae000640 10 client.6546 ms_handle_connect on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21ae000640 1 -- 172.21.15.3:0/3330973649 <== mds.2 v2:172.21.15.70:6833/4215641179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+295 (crc 0 0 0) 0x55e1f260df10 con 0x55e1f2698e40 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21ae000640 10 client.6546 handle_command_reply: tid=0 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 client.6546 shutdown 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 msgr2=0x55e1f2765f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f21b402a840 tx=0x7f21b4004730 comp rx=0 tx=0).stop 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 shutdown_connections 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:6833/4215641179,v1:172.21.15.70:6835/4215641179] conn(0x55e1f2698e40 0x55e1f26ca010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e1f26ea0c0 0x55e1f26c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e1f26e7ce0 0x55e1f2765f20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 --2- 172.21.15.3:0/3330973649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e1f26e2720 0x55e1f26fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21ae000640 0 client.6546 ms_handle_reset on v2:172.21.15.70:6833/4215641179 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 >> 172.21.15.3:0/3330973649 conn(0x55e1f26e7880 msgr2=0x55e1f26e9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 shutdown_connections 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 1 -- 172.21.15.3:0/3330973649 wait complete. 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.353+0000 7f21d2c76740 20 client.6546 trim_cache size 0 max 16384 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 981229169, 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627782, 2024-09-10T07:50:50.358 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3614977017, 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627783, 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:50:50.359 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 msgr2=0x7f21a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 0x7f21a806e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc008040 comp rx=0 tx=0).stop 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 msgr2=0x7f21c8153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c7a0 tx=0x7f21c000cc60 comp rx=0 tx=0).stop 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 shutdown_connections 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21a806be30 0x7f21a806e2f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21c815b230 0x7f21c8153ee0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c80b4ac0 0x7f21c80aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 --2- 172.21.15.3:0/943367766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21c8072c50 0x7f21c815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 >> 172.21.15.3:0/943367766 conn(0x7f21c80b5cb0 msgr2=0x7f21c806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 shutdown_connections 2024-09-10T07:50:50.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.354+0000 7f21cfe00640 1 -- 172.21.15.3:0/943367766 wait complete. 2024-09-10T07:50:50.369 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:50:50.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:50:50.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:50:50.563 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:50:50.563 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:50:50.563 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:50:50.563 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:50.599 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:50:50.599 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:50.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:50:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.872+0000 7fe4a5000640 1 Processor -- start 2024-09-10T07:50:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.872+0000 7fe4a5000640 1 -- start start 2024-09-10T07:50:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe4a5000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0156a80 0x7fe4a0072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe4a5000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4a0160640 0x7fe4a015da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe4a5000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a00727a0 con 0x7fe4a0160b80 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe4a5000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00af1d0 con 0x7fe4a0160640 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe4a5000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00af4b0 con 0x7fe4a0156a80 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe49e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4a0160640 0x7fe4a015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe49e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4a0160640 0x7fe4a015da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51018/0 (socket says 172.21.15.3:51018) 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.873+0000 7fe49e800640 1 -- 172.21.15.3:0/3829057287 learned_addr learned my addr 172.21.15.3:0/3829057287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.874+0000 7fe49de00640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0156a80 0x7fe4a0072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.874+0000 7fe49e800640 1 -- 172.21.15.3:0/3829057287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0156a80 msgr2=0x7fe4a0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.874+0000 7fe49e800640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0156a80 0x7fe4a0072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.874+0000 7fe49e800640 1 -- 172.21.15.3:0/3829057287 >> v1:172.21.15.70:6789/0 conn(0x7fe4a0160b80 legacy=0x7fe4a015ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.874+0000 7fe49e800640 1 -- 172.21.15.3:0/3829057287 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a00af790 con 0x7fe4a0160640 2024-09-10T07:50:50.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe49e800640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a015da50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e50 tx=0x7fe49802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ef1dcd2985f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe49d400640 1 -- 172.21.15.3:0/3829057287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498036070 con 0x7fe4a0160640 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe49d400640 1 -- 172.21.15.3:0/3829057287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe498031600 con 0x7fe4a0160640 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe49d400640 1 -- 172.21.15.3:0/3829057287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980319d0 con 0x7fe4a0160640 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe4a5000640 1 -- 172.21.15.3:0/3829057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 msgr2=0x7fe4a015da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.875+0000 7fe4a5000640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a015da50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e50 tx=0x7fe49802a680 comp rx=0 tx=0).stop 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 -- 172.21.15.3:0/3829057287 shutdown_connections 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a015da50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 --2- 172.21.15.3:0/3829057287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0156a80 0x7fe4a0072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 -- 172.21.15.3:0/3829057287 >> 172.21.15.3:0/3829057287 conn(0x7fe4a006bb20 msgr2=0x7fe4a006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 -- 172.21.15.3:0/3829057287 shutdown_connections 2024-09-10T07:50:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.876+0000 7fe4a5000640 1 -- 172.21.15.3:0/3829057287 wait complete. 2024-09-10T07:50:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.877+0000 7fe4a5000640 1 Processor -- start 2024-09-10T07:50:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.877+0000 7fe4a5000640 1 -- start start 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a0156a80 0x7fe4a0148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 0x7fe4a006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00aeee0 con 0x7fe4a0160640 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00afd50 con 0x7fe4a0156a80 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe4a5000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00b0270 con 0x7fe4a0160b80 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.878+0000 7fe497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51024/0 (socket says 172.21.15.3:51024) 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 -- 172.21.15.3:0/632903657 learned_addr learned my addr 172.21.15.3:0/632903657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe49de00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a0156a80 0x7fe4a0148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe49e800640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 0x7fe4a006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 -- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 msgr2=0x7fe4a006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 0x7fe4a006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 -- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a0156a80 msgr2=0x7fe4a0148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a0156a80 0x7fe4a0148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 -- 172.21.15.3:0/632903657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a00b4410 con 0x7fe4a0160640 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe49e800640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 0x7fe4a006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:50.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.879+0000 7fe497e00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c370 tx=0x7fe48800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.880+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4880052d0 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.880+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe488005470 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.880+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a00b3680 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.880+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488007260 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.880+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a00b4960 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.881+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a0072260 con 0x7fe4a0160640 2024-09-10T07:50:50.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.882+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe488007400 con 0x7fe4a0160640 2024-09-10T07:50:50.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.882+0000 7fe496000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 0x7fe47c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:50.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.883+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fe48808a420 con 0x7fe4a0160640 2024-09-10T07:50:50.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.883+0000 7fe49de00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 0x7fe47c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:50.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.883+0000 7fe49de00640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 0x7fe47c06e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:50.887+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4a00727a0 con 0x7fe4a0160640 2024-09-10T07:50:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.083+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4a00b05d0 con 0x7fe4a0160640 2024-09-10T07:50:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.084+0000 7fe496000640 1 -- 172.21.15.3:0/632903657 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7fe488058cf0 con 0x7fe4a0160640 2024-09-10T07:50:51.083 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 msgr2=0x7fe47c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 0x7fe47c06e2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490008040 comp rx=0 tx=0).stop 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 msgr2=0x7fe4a00b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c370 tx=0x7fe48800c830 comp rx=0 tx=0).stop 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 shutdown_connections 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47c06be30 0x7fe47c06e2f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a0160b80 0x7fe4a006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4a0160640 0x7fe4a00b3140 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 --2- 172.21.15.3:0/632903657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a0156a80 0x7fe4a0148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 >> 172.21.15.3:0/632903657 conn(0x7fe4a006bb20 msgr2=0x7fe4a015e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:51.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 shutdown_connections 2024-09-10T07:50:51.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.086+0000 7fe4a5000640 1 -- 172.21.15.3:0/632903657 wait complete. 2024-09-10T07:50:51.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:50:51.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:51.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:50:51.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:50:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:50:51.167 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:50:51.203 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:50:51.203 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:50:51.203 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:51.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:50:51.246 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:50:51.246 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:50:51.246 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:51.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:50:51.282 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:50:51.282 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:50:51.282 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:50:51.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:50:51.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:51.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.482+0000 7faa4ba00640 1 Processor -- start 2024-09-10T07:50:51.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.483+0000 7faa4ba00640 1 -- start start 2024-09-10T07:50:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa4ba00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faa44160710 0x7faa440cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa4ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa4ba00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa440cfdd0 con 0x7faa44160af0 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa4ba00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7faa4415cc90 con 0x7faa44160710 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa4ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faa4415cf70 con 0x7faa440d07c0 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa49000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faa44160710 0x7faa440cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa49000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7faa44160710 0x7faa440cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51034/0 (socket says 172.21.15.3:51034) 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa49000640 1 -- 172.21.15.3:0/4173548863 learned_addr learned my addr 172.21.15.3:0/4173548863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa43e00640 1 --1- 172.21.15.3:0/4173548863 >> v1:172.21.15.3:6789/0 conn(0x7faa44160af0 0x7faa440d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33832/0 (socket says 172.21.15.3:33832) 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.484+0000 7faa49a00640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.485+0000 7faa49000640 1 -- 172.21.15.3:0/4173548863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 msgr2=0x7faa44072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.485+0000 7faa49000640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.485+0000 7faa49000640 1 -- 172.21.15.3:0/4173548863 >> v1:172.21.15.3:6789/0 conn(0x7faa44160af0 legacy=0x7faa440d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.485+0000 7faa49000640 1 -- 172.21.15.3:0/4173548863 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4415d250 con 0x7faa44160710 2024-09-10T07:50:51.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.485+0000 7faa49a00640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa49000640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440cf2c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa38009870 tx=0x7faa3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e68529c2e0408 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa43400640 1 -- 172.21.15.3:0/4173548863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38037070 con 0x7faa44160710 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa43400640 1 -- 172.21.15.3:0/4173548863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa38032440 con 0x7faa44160710 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa43400640 1 -- 172.21.15.3:0/4173548863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38032810 con 0x7faa44160710 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 -- 172.21.15.3:0/4173548863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 msgr2=0x7faa440cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440cf2c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa38009870 tx=0x7faa3802a250 comp rx=0 tx=0).stop 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 -- 172.21.15.3:0/4173548863 shutdown_connections 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 --2- 172.21.15.3:0/4173548863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440cf2c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 -- 172.21.15.3:0/4173548863 >> 172.21.15.3:0/4173548863 conn(0x7faa4406d000 msgr2=0x7faa4406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.486+0000 7faa4ba00640 1 -- 172.21.15.3:0/4173548863 shutdown_connections 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.487+0000 7faa4ba00640 1 -- 172.21.15.3:0/4173548863 wait complete. 2024-09-10T07:50:51.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.488+0000 7faa4ba00640 1 Processor -- start 2024-09-10T07:50:51.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.488+0000 7faa4ba00640 1 -- start start 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 0x7faa4415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa4415d770 con 0x7faa44160710 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faa440cf800 con 0x7faa44160af0 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa4ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faa44155fe0 con 0x7faa440d07c0 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51044/0 (socket says 172.21.15.3:51044) 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.489+0000 7faa43e00640 1 -- 172.21.15.3:0/599120052 learned_addr learned my addr 172.21.15.3:0/599120052 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa49000640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa49a00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 0x7faa4415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 -- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 msgr2=0x7faa44152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 -- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 msgr2=0x7faa4415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 0x7faa4415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 -- 172.21.15.3:0/599120052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa441553d0 con 0x7faa44160710 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa49a00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 0x7faa4415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa43e00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c3f0 tx=0x7faa3000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa300074e0 con 0x7faa44160710 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa30007680 con 0x7faa44160710 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30004a00 con 0x7faa44160710 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.490+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa441556b0 con 0x7faa44160710 2024-09-10T07:50:51.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.491+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa44155c70 con 0x7faa44160710 2024-09-10T07:50:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.492+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa440afd50 con 0x7faa44160710 2024-09-10T07:50:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.493+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa3001a020 con 0x7faa44160710 2024-09-10T07:50:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.494+0000 7faa40c00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.494+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7faa3008a730 con 0x7faa44160710 2024-09-10T07:50:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.494+0000 7faa49000640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.495+0000 7faa49000640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 0x7faa2406e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa38034000 tx=0x7faa3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.499+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faa30010300 con 0x7faa44160710 2024-09-10T07:50:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.721+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa440ccb30 con 0x7faa44160710 2024-09-10T07:50:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.722+0000 7faa40c00640 1 -- 172.21.15.3:0/599120052 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5260 (secure 0 0 0) 0x7faa30059000 con 0x7faa44160710 2024-09-10T07:50:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:51.722 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"btime":"2024-09-10T07:50:23:632761+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:23.301000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114,6111,5857]},"id":4}]} 2024-09-10T07:50:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 msgr2=0x7faa2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 0x7faa2406e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa38034000 tx=0x7faa3802e000 comp rx=0 tx=0).stop 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 msgr2=0x7faa440ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c3f0 tx=0x7faa3000c8b0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 shutdown_connections 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa44160af0 0x7faa4415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa44160710 0x7faa440ce880 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 --2- 172.21.15.3:0/599120052 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faa440d07c0 0x7faa44152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 >> 172.21.15.3:0/599120052 conn(0x7faa4406d000 msgr2=0x7faa4406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 shutdown_connections 2024-09-10T07:50:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.724+0000 7faa4ba00640 1 -- 172.21.15.3:0/599120052 wait complete. 2024-09-10T07:50:51.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:51.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.896+0000 7f7985600640 1 Processor -- start 2024-09-10T07:50:51.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.897+0000 7f7985600640 1 -- start start 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7985600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7980160af0 0x7f79800d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7985600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7985600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79800d0a80 con 0x7f7980160710 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7985600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f79800cfdd0 con 0x7f7980160af0 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7985600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f798015cc90 con 0x7f79800d05f0 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f797ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7980160710 0x7f79800cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33840/0 (socket says 172.21.15.3:33840) 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f797ea00640 1 -- 172.21.15.3:0/3699879540 learned_addr learned my addr 172.21.15.3:0/3699879540 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f7976000640 1 --2- 172.21.15.3:0/3699879540 >> v2:172.21.15.70:3300/0 conn(0x7f7980160af0 0x7f79800d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.898+0000 7f797f400640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f797e000640 1 -- 172.21.15.3:0/3699879540 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004591851 0 0) 0x7f79800d0a80 con 0x7f7980160710 2024-09-10T07:50:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f797e000640 1 -- 172.21.15.3:0/3699879540 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7964003520 con 0x7f7980160710 2024-09-10T07:50:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f7976000640 1 -- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 msgr2=0x7f7980072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f7976000640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f7976000640 1 -- 172.21.15.3:0/3699879540 >> v1:172.21.15.3:6789/0 conn(0x7f7980160710 legacy=0x7f79800cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f7976000640 1 -- 172.21.15.3:0/3699879540 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798015cf70 con 0x7f7980160af0 2024-09-10T07:50:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.899+0000 7f797f400640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:51.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.900+0000 7f7976000640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f79800d00b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f79680097a0 tx=0x7f796802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cccfd76b8f68f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.900+0000 7f797e000640 1 -- 172.21.15.3:0/3699879540 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968037070 con 0x7f7980160af0 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.900+0000 7f797e000640 1 -- 172.21.15.3:0/3699879540 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968032440 con 0x7f7980160af0 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.900+0000 7f797e000640 1 -- 172.21.15.3:0/3699879540 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968032810 con 0x7f7980160af0 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 -- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 msgr2=0x7f79800d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f79800d00b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f79680097a0 tx=0x7f796802a180 comp rx=0 tx=0).stop 2024-09-10T07:50:51.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 -- 172.21.15.3:0/3699879540 shutdown_connections 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 --2- 172.21.15.3:0/3699879540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f79800d00b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 -- 172.21.15.3:0/3699879540 >> 172.21.15.3:0/3699879540 conn(0x7f798006d000 msgr2=0x7f798006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 -- 172.21.15.3:0/3699879540 shutdown_connections 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.901+0000 7f7985600640 1 -- 172.21.15.3:0/3699879540 wait complete. 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.903+0000 7f7985600640 1 Processor -- start 2024-09-10T07:50:51.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.903+0000 7f7985600640 1 -- start start 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f798015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f798015d770 con 0x7f7980160710 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f798015c960 con 0x7f7980160af0 2024-09-10T07:50:51.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7985600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f798015c110 con 0x7f79800d05f0 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51046/0 (socket says 172.21.15.3:51046) 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 -- 172.21.15.3:0/1749825893 learned_addr learned my addr 172.21.15.3:0/1749825893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f797f400640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f798015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f797ea00640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 -- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 msgr2=0x7f7980147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 -- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 msgr2=0x7f798015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f798015bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.904+0000 7f7976000640 1 -- 172.21.15.3:0/1749825893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980157190 con 0x7f7980160710 2024-09-10T07:50:51.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7976000640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f796802a1c0 tx=0x7f79680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968037040 con 0x7f7980160710 2024-09-10T07:50:51.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7968042070 con 0x7f7980160710 2024-09-10T07:50:51.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796803b930 con 0x7f7980160710 2024-09-10T07:50:51.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7980157470 con 0x7f7980160710 2024-09-10T07:50:51.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.905+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7980157a30 con 0x7f7980160710 2024-09-10T07:50:51.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.906+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7980071060 con 0x7f7980160710 2024-09-10T07:50:51.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.907+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f796803bad0 con 0x7f7980160710 2024-09-10T07:50:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.908+0000 7f7977400640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 0x7f795c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.908+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f79680ab730 con 0x7f7980160710 2024-09-10T07:50:51.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.908+0000 7f797ea00640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 0x7f795c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:51.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.909+0000 7f797ea00640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 0x7f795c06e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7970005880 tx=0x7f7970009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:51.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:51.913+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7980071060 con 0x7f7980160710 2024-09-10T07:50:52.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:50:52.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:50:52.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.135+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79800cfab0 con 0x7f7980160710 2024-09-10T07:50:52.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.136+0000 7f7977400640 1 -- 172.21.15.3:0/1749825893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5260 (secure 0 0 0) 0x7f796807a090 con 0x7f7980160710 2024-09-10T07:50:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:52.137 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"btime":"2024-09-10T07:50:23:632761+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:23.301000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114,6111,5857]},"id":4}]} 2024-09-10T07:50:52.137 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2024-09-10T07:50:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.138+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 msgr2=0x7f795c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.138+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 0x7f795c06e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7970005880 tx=0x7f7970009040 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 msgr2=0x7f7980156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f796802a1c0 tx=0x7f79680046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 shutdown_connections 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f795c06be30 0x7f795c06e2f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7980160af0 0x7f798015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7980160710 0x7f7980156610 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 --2- 172.21.15.3:0/1749825893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79800d05f0 0x7f7980147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 >> 172.21.15.3:0/1749825893 conn(0x7f798006d000 msgr2=0x7f79800ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 shutdown_connections 2024-09-10T07:50:52.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.139+0000 7f7985600640 1 -- 172.21.15.3:0/1749825893 wait complete. 2024-09-10T07:50:52.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:50:52.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.351+0000 7f7310400640 1 Processor -- start 2024-09-10T07:50:52.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.351+0000 7f7310400640 1 -- start start 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.352+0000 7f7310400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f730806bb20 0x7f730815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.352+0000 7f7310400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73081603c0 con 0x7f730815e4f0 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.352+0000 7f7310400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f730806c0c0 con 0x7f730806d160 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.352+0000 7f7310400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f730806ccd0 con 0x7f730806bb20 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730da00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f730815e4f0 0x7f730815fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33844/0 (socket says 172.21.15.3:33844) 2024-09-10T07:50:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730da00640 1 -- 172.21.15.3:0/2534642089 learned_addr learned my addr 172.21.15.3:0/2534642089 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730d000640 1 --2- 172.21.15.3:0/2534642089 >> v2:172.21.15.70:3300/0 conn(0x7f730806bb20 0x7f730815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2802636727 0 0) 0x7f73081603c0 con 0x7f730815e4f0 2024-09-10T07:50:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72f0003520 con 0x7f730815e4f0 2024-09-10T07:50:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730d000640 1 -- 172.21.15.3:0/2534642089 >> v1:172.21.15.70:6789/0 conn(0x7f730806d160 legacy=0x7f730806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730d000640 1 -- 172.21.15.3:0/2534642089 >> v1:172.21.15.3:6789/0 conn(0x7f730815e4f0 legacy=0x7f730815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137524592 0 0) 0x7f72f0003520 con 0x7f730815e4f0 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.353+0000 7f730d000640 1 -- 172.21.15.3:0/2534642089 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730806c3a0 con 0x7f730806bb20 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.354+0000 7f730d000640 1 --2- 172.21.15.3:0/2534642089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806bb20 0x7f730815f570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097a0 tx=0x7f72f402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbeffbd9c1427b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4037070 con 0x7f730806bb20 2024-09-10T07:50:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f402ac50 con 0x7f730806bb20 2024-09-10T07:50:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7307e00640 1 -- 172.21.15.3:0/2534642089 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc001670 con 0x7f730806bb20 2024-09-10T07:50:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7310400640 1 -- 172.21.15.3:0/2534642089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806bb20 msgr2=0x7f730815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7310400640 1 --2- 172.21.15.3:0/2534642089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806bb20 0x7f730815f570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097a0 tx=0x7f72f402a1c0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7310400640 1 -- 172.21.15.3:0/2534642089 shutdown_connections 2024-09-10T07:50:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7310400640 1 --2- 172.21.15.3:0/2534642089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806bb20 0x7f730815f570 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.355+0000 7f7310400640 1 -- 172.21.15.3:0/2534642089 >> 172.21.15.3:0/2534642089 conn(0x7f730806e5b0 msgr2=0x7f730806f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.356+0000 7f7310400640 1 -- 172.21.15.3:0/2534642089 shutdown_connections 2024-09-10T07:50:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.356+0000 7f7310400640 1 -- 172.21.15.3:0/2534642089 wait complete. 2024-09-10T07:50:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.357+0000 7f7310400640 1 Processor -- start 2024-09-10T07:50:52.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.357+0000 7f7310400640 1 -- start start 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806d160 0x7f730815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 0x7f730814ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f730815e8d0 con 0x7f730806bb20 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f730806c960 con 0x7f730806d160 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f7310400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f73080b6dc0 con 0x7f730815e4f0 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f730da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f730da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51060/0 (socket says 172.21.15.3:51060) 2024-09-10T07:50:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.358+0000 7f730da00640 1 -- 172.21.15.3:0/1719270285 learned_addr learned my addr 172.21.15.3:0/1719270285 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730d000640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806d160 0x7f730815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730e400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 0x7f730814ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 -- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 msgr2=0x7f730814ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 0x7f730814ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 -- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806d160 msgr2=0x7f730815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806d160 0x7f730815b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 -- 172.21.15.3:0/1719270285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730815a130 con 0x7f730806bb20 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730e400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 0x7f730814ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f730da00640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c5b0 tx=0x7f72fc00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc017070 con 0x7f730806bb20 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc005ce0 con 0x7f730806bb20 2024-09-10T07:50:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730815a410 con 0x7f730806bb20 2024-09-10T07:50:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.359+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc007330 con 0x7f730806bb20 2024-09-10T07:50:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.360+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730815a9d0 con 0x7f730806bb20 2024-09-10T07:50:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.361+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73080693c0 con 0x7f730806bb20 2024-09-10T07:50:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.362+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72fc01a070 con 0x7f730806bb20 2024-09-10T07:50:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.363+0000 7f7305600640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.363+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f72fc08a370 con 0x7f730806bb20 2024-09-10T07:50:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.363+0000 7f730d000640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.364+0000 7f730d000640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72f4034000 tx=0x7f72f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.367+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f73080693c0 con 0x7f730806bb20 2024-09-10T07:50:52.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.554+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73080b3410 con 0x7f730806bb20 2024-09-10T07:50:52.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.555+0000 7f7305600640 1 -- 172.21.15.3:0/1719270285 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16749 (secure 0 0 0) 0x7f72fc087180 con 0x7f730806bb20 2024-09-10T07:50:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:52.555 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":66,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:50:14.604186+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:49:52.454520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T07:49:53.642181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2c","mappings":[{"from":6,"to":1}]},{"pgid":"9.35","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:50:52.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 msgr2=0x7f72e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72f4034000 tx=0x7f72f402e000 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 msgr2=0x7f7308159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c5b0 tx=0x7f72fc00ca70 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 shutdown_connections 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f730815e4f0 0x7f730814ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f730806d160 0x7f730815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.557+0000 7f7310400640 1 --2- 172.21.15.3:0/1719270285 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f730806bb20 0x7f7308159b50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.558+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 >> 172.21.15.3:0/1719270285 conn(0x7f730806e5b0 msgr2=0x7f73080b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.558+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 shutdown_connections 2024-09-10T07:50:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.558+0000 7f7310400640 1 -- 172.21.15.3:0/1719270285 wait complete. 2024-09-10T07:50:52.574 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:50:52.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:52.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 Processor -- start 2024-09-10T07:50:52.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 -- start start 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d015b110 0x7fe6d00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6d015b4f0 0x7fe6d00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d00b4250 con 0x7fe6d0073d10 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d00b44f0 con 0x7fe6d015b4f0 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6d7200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d00ccd80 con 0x7fe6d015b110 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6d015b4f0 0x7fe6d00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6d015b4f0 0x7fe6d00d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51068/0 (socket says 172.21.15.3:51068) 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.733+0000 7fe6c7400640 1 -- 172.21.15.3:0/3690175498 learned_addr learned my addr 172.21.15.3:0/3690175498 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6cfe00640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d015b110 0x7fe6d00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6c7400640 1 -- 172.21.15.3:0/3690175498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d015b110 msgr2=0x7fe6d00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6c7400640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d015b110 0x7fe6d00b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6c7400640 1 -- 172.21.15.3:0/3690175498 >> v1:172.21.15.3:6789/0 conn(0x7fe6d0073d10 legacy=0x7fe6d015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6c7400640 1 -- 172.21.15.3:0/3690175498 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d00cd060 con 0x7fe6d015b4f0 2024-09-10T07:50:52.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.734+0000 7fe6cf400640 1 -- 172.21.15.3:0/3690175498 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621498681 0 0) 0x7fe6d00b4250 con 0x7fe6d0073d10 2024-09-10T07:50:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.735+0000 7fe6c7400640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b4f0 0x7fe6d00d01e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009870 tx=0x7fe6b802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59516d647cb24f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.735+0000 7fe6cf400640 1 -- 172.21.15.3:0/3690175498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8037070 con 0x7fe6d015b4f0 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.735+0000 7fe6cf400640 1 -- 172.21.15.3:0/3690175498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b8032440 con 0x7fe6d015b4f0 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.735+0000 7fe6cf400640 1 -- 172.21.15.3:0/3690175498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8032810 con 0x7fe6d015b4f0 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 -- 172.21.15.3:0/3690175498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b4f0 msgr2=0x7fe6d00d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b4f0 0x7fe6d00d01e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009870 tx=0x7fe6b802a250 comp rx=0 tx=0).stop 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 -- 172.21.15.3:0/3690175498 shutdown_connections 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b4f0 0x7fe6d00d01e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 --2- 172.21.15.3:0/3690175498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d015b110 0x7fe6d00b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 -- 172.21.15.3:0/3690175498 >> 172.21.15.3:0/3690175498 conn(0x7fe6d006d000 msgr2=0x7fe6d006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 -- 172.21.15.3:0/3690175498 shutdown_connections 2024-09-10T07:50:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.736+0000 7fe6d7200640 1 -- 172.21.15.3:0/3690175498 wait complete. 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.739+0000 7fe6d7200640 1 Processor -- start 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.739+0000 7fe6d7200640 1 -- start start 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d0073d10 0x7fe6d0146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d015b4f0 0x7fe6d0070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d00ceb50 con 0x7fe6d015b110 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d00cd620 con 0x7fe6d015b4f0 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d00cecd0 con 0x7fe6d0073d10 2024-09-10T07:50:52.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51074/0 (socket says 172.21.15.3:51074) 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 -- 172.21.15.3:0/2667881077 learned_addr learned my addr 172.21.15.3:0/2667881077 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6cfe00640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d0073d10 0x7fe6d0146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d5200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d015b4f0 0x7fe6d0070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 -- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d0073d10 msgr2=0x7fe6d0146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d0073d10 0x7fe6d0146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 -- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d015b4f0 msgr2=0x7fe6d0070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d015b4f0 0x7fe6d0070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 -- 172.21.15.3:0/2667881077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d00728c0 con 0x7fe6d015b110 2024-09-10T07:50:52.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6c7400640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8041020 con 0x7fe6d015b110 2024-09-10T07:50:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b80306c0 con 0x7fe6d015b110 2024-09-10T07:50:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d00725e0 con 0x7fe6d015b110 2024-09-10T07:50:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8037040 con 0x7fe6d015b110 2024-09-10T07:50:52.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.740+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d0144740 con 0x7fe6d015b110 2024-09-10T07:50:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.741+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d00b3d10 con 0x7fe6d015b110 2024-09-10T07:50:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.743+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6b8044020 con 0x7fe6d015b110 2024-09-10T07:50:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.744+0000 7fe6ccc00640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 0x7fe6ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.744+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7fe6b80ab6b0 con 0x7fe6d015b110 2024-09-10T07:50:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.744+0000 7fe6cfe00640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 0x7fe6ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:52.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.744+0000 7fe6cfe00640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 0x7fe6ac06e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009a60 tx=0x7fe6c0003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:52.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.748+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe6b804b030 con 0x7fe6d015b110 2024-09-10T07:50:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.970+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d006d840 con 0x7fe6d015b110 2024-09-10T07:50:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.971+0000 7fe6ccc00640 1 -- 172.21.15.3:0/2667881077 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5260 (secure 0 0 0) 0x7fe6b807a010 con 0x7fe6d015b110 2024-09-10T07:50:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:52.971 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"btime":"2024-09-10T07:50:23:632761+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:23.301000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114,6111,5857]},"id":4}]} 2024-09-10T07:50:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2024-09-10T07:50:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.973+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 msgr2=0x7fe6ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.973+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 0x7fe6ac06e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009a60 tx=0x7fe6c0003990 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 msgr2=0x7fe6d0143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 shutdown_connections 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe6ac06be30 0x7fe6ac06e2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6d015b4f0 0x7fe6d0070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d015b110 0x7fe6d0143b70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 --2- 172.21.15.3:0/2667881077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6d0073d10 0x7fe6d0146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 >> 172.21.15.3:0/2667881077 conn(0x7fe6d006d000 msgr2=0x7fe6d00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 shutdown_connections 2024-09-10T07:50:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:52.974+0000 7fe6d7200640 1 -- 172.21.15.3:0/2667881077 wait complete. 2024-09-10T07:50:52.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:50:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.144+0000 7f4c2e000640 1 Processor -- start 2024-09-10T07:50:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.144+0000 7f4c2e000640 1 -- start start 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.145+0000 7f4c2e000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c28161b30 0x7f4c2815f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.145+0000 7f4c2e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2806c230 0x7f4c280b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c2e000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2815fbb0 con 0x7f4c2815fdb0 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c2e000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4c280b36a0 con 0x7f4c2806c230 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c2e000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280b3980 con 0x7f4c28161b30 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2806c230 0x7f4c280b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c2806c230 0x7f4c280b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51076/0 (socket says 172.21.15.3:51076) 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 -- 172.21.15.3:0/2948069618 learned_addr learned my addr 172.21.15.3:0/2948069618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27400640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c28161b30 0x7f4c2815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 -- 172.21.15.3:0/2948069618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c28161b30 msgr2=0x7f4c2815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c28161b30 0x7f4c2815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 -- 172.21.15.3:0/2948069618 >> v1:172.21.15.70:6789/0 conn(0x7f4c2815fdb0 legacy=0x7f4c2806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.146+0000 7f4c27e00640 1 -- 172.21.15.3:0/2948069618 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280b3c60 con 0x7f4c2806c230 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.147+0000 7f4c27e00640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2806c230 0x7f4c280b4740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009e50 tx=0x7f4c2002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dd59d7a4f8a586 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c26a00640 1 -- 172.21.15.3:0/2948069618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20036070 con 0x7f4c2806c230 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c26a00640 1 -- 172.21.15.3:0/2948069618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20031600 con 0x7f4c2806c230 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c26a00640 1 -- 172.21.15.3:0/2948069618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c200319d0 con 0x7f4c2806c230 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 -- 172.21.15.3:0/2948069618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2806c230 msgr2=0x7f4c280b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2806c230 0x7f4c280b4740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009e50 tx=0x7f4c2002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 -- 172.21.15.3:0/2948069618 shutdown_connections 2024-09-10T07:50:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c2806c230 0x7f4c280b4740 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2948069618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c28161b30 0x7f4c2815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 -- 172.21.15.3:0/2948069618 >> 172.21.15.3:0/2948069618 conn(0x7f4c2806d000 msgr2=0x7f4c2806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 -- 172.21.15.3:0/2948069618 shutdown_connections 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.148+0000 7f4c2e000640 1 -- 172.21.15.3:0/2948069618 wait complete. 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 Processor -- start 2024-09-10T07:50:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 -- start start 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2806c230 0x7f4c280b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 0x7f4c280cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28161f10 con 0x7f4c28161b30 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280b4220 con 0x7f4c2806c230 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c2e000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280aefa0 con 0x7f4c2815fdb0 2024-09-10T07:50:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51080/0 (socket says 172.21.15.3:51080) 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 -- 172.21.15.3:0/2733835325 learned_addr learned my addr 172.21.15.3:0/2733835325 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27400640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2806c230 0x7f4c280b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c1fe00640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 0x7f4c280cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 -- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 msgr2=0x7f4c280cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 0x7f4c280cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 -- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2806c230 msgr2=0x7f4c280b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2806c230 0x7f4c280b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.151+0000 7f4c27e00640 1 -- 172.21.15.3:0/2733835325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c280b1b40 con 0x7f4c28161b30 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c1fe00640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 0x7f4c280cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:53.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c27e00640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009e50 tx=0x7f4c20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20036050 con 0x7f4c28161b30 2024-09-10T07:50:53.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20042070 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20009510 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c280b2a40 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.152+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c280b0e00 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.153+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c280b3300 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.154+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c20007bd0 con 0x7f4c28161b30 2024-09-10T07:50:53.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.155+0000 7f4c1f400640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 0x7f4c0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.155+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f4c200abdc0 con 0x7f4c28161b30 2024-09-10T07:50:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.155+0000 7f4c27400640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 0x7f4c0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.156+0000 7f4c27400640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 0x7f4c0406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006fd0 tx=0x7f4c18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.161+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c2815fbb0 con 0x7f4c28161b30 2024-09-10T07:50:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.379+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c2815b8d0 con 0x7f4c28161b30 2024-09-10T07:50:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.379+0000 7f4c1f400640 1 -- 172.21.15.3:0/2733835325 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5260 (secure 0 0 0) 0x7f4c2007a600 con 0x7f4c28161b30 2024-09-10T07:50:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:53.379 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":59,"btime":"2024-09-10T07:50:23:632761+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:49:54.647917+0000","modified":"2024-09-10T07:50:23.301000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6114,"mds_1":6111,"mds_2":5857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5857":{"gid":5857,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.70:6835/4215641179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215641179},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215641179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6111":{"gid":6111,"name":"a","rank":1,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6837/1603892470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":1603892470},{"type":"v1","addr":"172.21.15.3:6837","nonce":1603892470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6114":{"gid":6114,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.3:6836/1359877653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1359877653},{"type":"v1","addr":"172.21.15.3:6836","nonce":1359877653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6114,"qdb_cluster":[6114,6111,5857]},"id":4}]} 2024-09-10T07:50:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 59 2024-09-10T07:50:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 msgr2=0x7f4c0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 0x7f4c0406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006fd0 tx=0x7f4c18008040 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 msgr2=0x7f4c280ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009e50 tx=0x7f4c20004a40 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 shutdown_connections 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0406be30 0x7f4c0406e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c28161b30 0x7f4c280ce160 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c2815fdb0 0x7f4c280cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 --2- 172.21.15.3:0/2733835325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2806c230 0x7f4c280b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 >> 172.21.15.3:0/2733835325 conn(0x7f4c2806d000 msgr2=0x7f4c280af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 shutdown_connections 2024-09-10T07:50:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.382+0000 7f4c2e000640 1 -- 172.21.15.3:0/2733835325 wait complete. 2024-09-10T07:50:53.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:50:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.556+0000 7f05aa200640 1 Processor -- start 2024-09-10T07:50:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.556+0000 7f05aa200640 1 -- start start 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 0x7f05a40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05a40cf800 0x7f05a40d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f05a40cfdd0 con 0x7f05a40cf800 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f05a415cc90 con 0x7f05a4160710 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.557+0000 7f05aa200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f05a415cf70 con 0x7f05a4160af0 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05a40cf800 0x7f05a40d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05a40cf800 0x7f05a40d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51096/0 (socket says 172.21.15.3:51096) 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 -- 172.21.15.3:0/3168758212 learned_addr learned my addr 172.21.15.3:0/3168758212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3400640 1 --2- 172.21.15.3:0/3168758212 >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 0x7f05a40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a2a00640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 -- 172.21.15.3:0/3168758212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 msgr2=0x7f05a40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 -- 172.21.15.3:0/3168758212 >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 msgr2=0x7f05a40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 --2- 172.21.15.3:0/3168758212 >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 0x7f05a40cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 -- 172.21.15.3:0/3168758212 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a415d250 con 0x7f05a40cf800 2024-09-10T07:50:53.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3400640 1 --2- 172.21.15.3:0/3168758212 >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 0x7f05a40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a3e00640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a40cf800 0x7f05a40d05f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f05a406ccc0 tx=0x7f059c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f648666e3d5bf634 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a2000640 1 -- 172.21.15.3:0/3168758212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c036070 con 0x7f05a40cf800 2024-09-10T07:50:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a2000640 1 -- 172.21.15.3:0/3168758212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c031600 con 0x7f05a40cf800 2024-09-10T07:50:53.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.558+0000 7f05a2000640 1 -- 172.21.15.3:0/3168758212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0319d0 con 0x7f05a40cf800 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 -- 172.21.15.3:0/3168758212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a40cf800 msgr2=0x7f05a40d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a40cf800 0x7f05a40d05f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f05a406ccc0 tx=0x7f059c02a6a0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 -- 172.21.15.3:0/3168758212 shutdown_connections 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a40cf800 0x7f05a40d05f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 --2- 172.21.15.3:0/3168758212 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 --2- 172.21.15.3:0/3168758212 >> v2:172.21.15.70:3300/0 conn(0x7f05a4160710 0x7f05a40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 -- 172.21.15.3:0/3168758212 >> 172.21.15.3:0/3168758212 conn(0x7f05a406d000 msgr2=0x7f05a406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 -- 172.21.15.3:0/3168758212 shutdown_connections 2024-09-10T07:50:53.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.559+0000 7f05aa200640 1 -- 172.21.15.3:0/3168758212 wait complete. 2024-09-10T07:50:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.562+0000 7f05aa200640 1 Processor -- start 2024-09-10T07:50:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.562+0000 7f05aa200640 1 -- start start 2024-09-10T07:50:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a40cf800 0x7f05a414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a414c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f05a415d770 con 0x7f05a4160710 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f05a406bb20 con 0x7f05a40cf800 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f05a414d410 con 0x7f05a4160af0 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51102/0 (socket says 172.21.15.3:51102) 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 -- 172.21.15.3:0/1388846934 learned_addr learned my addr 172.21.15.3:0/1388846934 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a3400640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a40cf800 0x7f05a414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a3e00640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a414c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 -- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 msgr2=0x7f05a414c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a414c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 -- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a40cf800 msgr2=0x7f05a414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a40cf800 0x7f05a414cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 -- 172.21.15.3:0/1388846934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a415ee70 con 0x7f05a4160710 2024-09-10T07:50:53.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05a2a00640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f059000d5f0 tx=0x7f059000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900125f0 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590019070 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900129d0 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a415f150 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.563+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a415f710 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.564+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a40afd50 con 0x7f05a4160710 2024-09-10T07:50:53.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.565+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0590012b70 con 0x7f05a4160710 2024-09-10T07:50:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.566+0000 7f059b400640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 0x7f058406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.566+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f059008b170 con 0x7f05a4160710 2024-09-10T07:50:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.566+0000 7f05a3400640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 0x7f058406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.567+0000 7f05a3400640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 0x7f058406e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05a41f7f40 tx=0x7f0594008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.572+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0590059480 con 0x7f05a4160710 2024-09-10T07:50:53.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.759+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05a4152f30 con 0x7f05a4160710 2024-09-10T07:50:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.760+0000 7f059b400640 1 -- 172.21.15.3:0/1388846934 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16749 (secure 0 0 0) 0x7f059005d780 con 0x7f05a4160710 2024-09-10T07:50:53.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:53.760 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":66,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:50:14.604186+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:49:52.454520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T07:49:53.642181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2c","mappings":[{"from":6,"to":1}]},{"pgid":"9.35","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 msgr2=0x7f058406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 0x7f058406e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05a41f7f40 tx=0x7f0594008040 comp rx=0 tx=0).stop 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 msgr2=0x7f05a41518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f059000d5f0 tx=0x7f059000dab0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 shutdown_connections 2024-09-10T07:50:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f058406be30 0x7f058406e2f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f05a4160af0 0x7f05a414c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05a4160710 0x7f05a41518f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 --2- 172.21.15.3:0/1388846934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05a40cf800 0x7f05a414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 >> 172.21.15.3:0/1388846934 conn(0x7f05a406d000 msgr2=0x7f05a40b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 shutdown_connections 2024-09-10T07:50:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.762+0000 7f05aa200640 1 -- 172.21.15.3:0/1388846934 wait complete. 2024-09-10T07:50:53.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:50:53.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.937+0000 7f7e9e200640 1 Processor -- start 2024-09-10T07:50:53.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.937+0000 7f7e9e200640 1 -- start start 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9e200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e980d3c10 0x7f7e98073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9e200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e980cfdc0 con 0x7f7e980d4050 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9e200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e980cee40 con 0x7f7e9815e860 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9e200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7e980d3780 con 0x7f7e980d3c10 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9d800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7e980d4050 0x7f7e980b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.938+0000 7f7e9d800640 1 -- 172.21.15.3:0/3427984308 learned_addr learned my addr 172.21.15.3:0/3427984308 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97e00640 1 --2- 172.21.15.3:0/3427984308 >> v2:172.21.15.70:3300/0 conn(0x7f7e980d3c10 0x7f7e98073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131614306 0 0) 0x7f7e980cfdc0 con 0x7f7e980d4050 2024-09-10T07:50:53.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e80003520 con 0x7f7e980d4050 2024-09-10T07:50:53.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97e00640 1 -- 172.21.15.3:0/3427984308 >> v1:172.21.15.70:6789/0 conn(0x7f7e9815e860 legacy=0x7f7e980cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680214113 0 0) 0x7f7e980cee40 con 0x7f7e9815e860 2024-09-10T07:50:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97e00640 1 -- 172.21.15.3:0/3427984308 >> v1:172.21.15.3:6789/0 conn(0x7f7e980d4050 legacy=0x7f7e980b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.939+0000 7f7e97e00640 1 -- 172.21.15.3:0/3427984308 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e980cf120 con 0x7f7e980d3c10 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.940+0000 7f7e97e00640 1 --2- 172.21.15.3:0/3427984308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e98073150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097a0 tx=0x7f7e8402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85faea15a052ffb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84037070 con 0x7f7e980d3c10 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8402ac30 con 0x7f7e980d3c10 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e97400640 1 -- 172.21.15.3:0/3427984308 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900019f0 con 0x7f7e980d3c10 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 -- 172.21.15.3:0/3427984308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 msgr2=0x7f7e98073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 --2- 172.21.15.3:0/3427984308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e98073150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097a0 tx=0x7f7e8402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 -- 172.21.15.3:0/3427984308 shutdown_connections 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 --2- 172.21.15.3:0/3427984308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e98073150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 -- 172.21.15.3:0/3427984308 >> 172.21.15.3:0/3427984308 conn(0x7f7e980d4650 msgr2=0x7f7e980d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 -- 172.21.15.3:0/3427984308 shutdown_connections 2024-09-10T07:50:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.941+0000 7f7e9e200640 1 -- 172.21.15.3:0/3427984308 wait complete. 2024-09-10T07:50:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.943+0000 7f7e9e200640 1 Processor -- start 2024-09-10T07:50:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.943+0000 7f7e9e200640 1 -- start start 2024-09-10T07:50:53.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e980ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 0x7f7e9815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980d2460 con 0x7f7e980d4050 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980cf400 con 0x7f7e980d3c10 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7e980b7f90 con 0x7f7e9815e860 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36502/0 (socket says 172.21.15.3:36502) 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e97e00640 1 -- 172.21.15.3:0/556288576 learned_addr learned my addr 172.21.15.3:0/556288576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9ce00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e980ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.944+0000 7f7e9d800640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 0x7f7e9815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 -- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 msgr2=0x7f7e9815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 0x7f7e9815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 -- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 msgr2=0x7f7e980ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e980ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 -- 172.21.15.3:0/556288576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98160980 con 0x7f7e980d4050 2024-09-10T07:50:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e9d800640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 0x7f7e9815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e97e00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e84035040 tx=0x7f7e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84037040 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84004030 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.945+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e98160c60 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.946+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84004400 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.946+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e98161220 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.947+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e980b7880 con 0x7f7e980d4050 2024-09-10T07:50:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.948+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e8403a040 con 0x7f7e980d4050 2024-09-10T07:50:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.949+0000 7f7e94c00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.949+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5767+0+0 (secure 0 0 0) 0x7f7e840ab5a0 con 0x7f7e980d4050 2024-09-10T07:50:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.949+0000 7f7e9ce00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.950+0000 7f7e9ce00640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 0x7f7e7806e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000f60 tx=0x7f7e8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:53.953+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e84042020 con 0x7f7e980d4050 2024-09-10T07:50:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.172+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7e980d4da0 con 0x7f7e980d4050 2024-09-10T07:50:54.171 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:50:54.173+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:50:54.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.175+0000 7f7e94c00640 1 -- 172.21.15.3:0/556288576 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f7e84079940 con 0x7f7e980d4050 2024-09-10T07:50:54.173 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:50:54.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 msgr2=0x7f7e7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 0x7f7e7806e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000f60 tx=0x7f7e8c009040 comp rx=0 tx=0).stop 2024-09-10T07:50:54.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 msgr2=0x7f7e980ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e84035040 tx=0x7f7e840046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 shutdown_connections 2024-09-10T07:50:54.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e7806be30 0x7f7e7806e2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e9815e860 0x7f7e9815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e980d4050 0x7f7e980ae950 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 --2- 172.21.15.3:0/556288576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e980d3c10 0x7f7e980ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.179+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 >> 172.21.15.3:0/556288576 conn(0x7f7e980d4650 msgr2=0x7f7e980d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.180+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 shutdown_connections 2024-09-10T07:50:54.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.180+0000 7f7e9e200640 1 -- 172.21.15.3:0/556288576 wait complete. 2024-09-10T07:50:54.187 INFO:tasks.ceph.mds.d.smithi070.stderr: -1441> 2024-09-10T07:50:19.244+0000 7f64bbc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1) n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562725276000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.115721+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.188 INFO:tasks.ceph.mds.d.smithi070.stderr: -1427> 2024-09-10T07:50:19.244+0000 7f64bbc00640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/AsLsXsFsx@3},l=5591 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562725276580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.189 INFO:tasks.ceph.mds.a.smithi003.stderr: -1536> 2024-09-10T07:50:19.244+0000 7f0cf8600640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1) n(v2 rc2024-09-10T07:50:07.429717+0000 4=0+4) (inest lock) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55dbf7cae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:50:07.429717+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.189 INFO:tasks.ceph.mds.a.smithi003.stderr: -1398> 2024-09-10T07:50:19.246+0000 7f0d00800640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1) n(v1 rc2024-09-10T07:50:07.429717+0000 3=0+3) (inest sync->mix g=0,2) caps={5591=pAsLsXsFs/pAsLsXsFsxcral@3} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 export_pin=2 0x55dbf8cc6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.115721+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.114721+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:50:07.429717+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:50:54.197 INFO:tasks.ceph.mds.c.smithi003.stderr: -3382> 2024-09-10T07:50:19.243+0000 7fe5e5a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1) n(v3 rc2024-09-10T07:50:07.429717+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5591=pAsLsXsFsx/pAsLsXsFsxcral@4},l=5591 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56487ae49b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.112721+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:50:07.429717+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.197 INFO:tasks.ceph.mds.c.smithi003.stderr: -3260> 2024-09-10T07:50:19.245+0000 7fe5edc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1) n(v4 rc2024-09-10T07:50:07.429717+0000 5=0+5) (inest sync->mix g=1,2) caps={5591=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x56487ae4a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:50:07.429717+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:50:07.113721+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:50:07.429717+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:50:54.234 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:50:54.245 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:50:54.256 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:50:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.361+0000 7fe695400640 1 Processor -- start 2024-09-10T07:50:54.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.361+0000 7fe695400640 1 -- start start 2024-09-10T07:50:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe695400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6900aefb0 0x7fe690072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe695400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe69006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe695400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69015faf0 con 0x7fe690147ce0 2024-09-10T07:50:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe695400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe69015eaf0 con 0x7fe6900aefb0 2024-09-10T07:50:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe695400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6900730c0 con 0x7fe6900728d0 2024-09-10T07:50:54.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe687e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6900aefb0 0x7fe690072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe68f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe69006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.362+0000 7fe68e800640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe690147ce0 0x7fe6900ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe68e800640 1 -- 172.21.15.3:0/1654629982 learned_addr learned my addr 172.21.15.3:0/1654629982 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe687e00640 1 -- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 msgr2=0x7fe69006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe687e00640 1 --2- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe69006e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe687e00640 1 -- 172.21.15.3:0/1654629982 >> v1:172.21.15.70:6789/0 conn(0x7fe690147ce0 legacy=0x7fe6900ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe68de00640 1 -- 172.21.15.3:0/1654629982 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656770717 0 0) 0x7fe69015faf0 con 0x7fe690147ce0 2024-09-10T07:50:54.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.363+0000 7fe687e00640 1 -- 172.21.15.3:0/1654629982 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69015ed30 con 0x7fe6900aefb0 2024-09-10T07:50:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.364+0000 7fe687e00640 1 --2- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe690072390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097a0 tx=0x7fe67802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e00b085ae87a719b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.364+0000 7fe68de00640 1 -- 172.21.15.3:0/1654629982 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678037070 con 0x7fe6900aefb0 2024-09-10T07:50:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe68de00640 1 -- 172.21.15.3:0/1654629982 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678032440 con 0x7fe6900aefb0 2024-09-10T07:50:54.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe68de00640 1 -- 172.21.15.3:0/1654629982 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678032810 con 0x7fe6900aefb0 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 -- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 msgr2=0x7fe690072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 --2- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe690072390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097a0 tx=0x7fe67802a180 comp rx=0 tx=0).stop 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 -- 172.21.15.3:0/1654629982 shutdown_connections 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 --2- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe69006e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 --2- 172.21.15.3:0/1654629982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe690072390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 -- 172.21.15.3:0/1654629982 >> 172.21.15.3:0/1654629982 conn(0x7fe6900b5cb0 msgr2=0x7fe69006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:54.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 -- 172.21.15.3:0/1654629982 shutdown_connections 2024-09-10T07:50:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.365+0000 7fe695400640 1 -- 172.21.15.3:0/1654629982 wait complete. 2024-09-10T07:50:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.367+0000 7fe695400640 1 Processor -- start 2024-09-10T07:50:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.367+0000 7fe695400640 1 -- start start 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe6901417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe690147ce0 0x7fe690146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe69015f2f0 con 0x7fe690147ce0 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe69015f660 con 0x7fe6900aefb0 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe695400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6901480c0 con 0x7fe6900728d0 2024-09-10T07:50:54.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe68f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe690147ce0 0x7fe690146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe687e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe68e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe6901417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe687e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:55062/0 (socket says 172.21.15.3:55062) 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.368+0000 7fe687e00640 1 -- 172.21.15.3:0/829106622 learned_addr learned my addr 172.21.15.3:0/829106622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 -- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 msgr2=0x7fe6901417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe6901417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 -- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe690147ce0 msgr2=0x7fe690146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe690147ce0 0x7fe690146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 -- 172.21.15.3:0/829106622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6900ce0c0 con 0x7fe6900aefb0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe687e00640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097a0 tx=0x7fe6780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678037050 con 0x7fe6900aefb0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678038040 con 0x7fe6900aefb0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6900ce3a0 con 0x7fe6900aefb0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678004490 con 0x7fe6900aefb0 2024-09-10T07:50:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.369+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6900ce960 con 0x7fe6900aefb0 2024-09-10T07:50:54.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.370+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690072d30 con 0x7fe6900aefb0 2024-09-10T07:50:54.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.372+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe67803c6d0 con 0x7fe6900aefb0 2024-09-10T07:50:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.372+0000 7fe686a00640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 0x7fe66c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.372+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6025+0+0 (secure 0 0 0) 0x7fe6780ab8e0 con 0x7fe6900aefb0 2024-09-10T07:50:54.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.373+0000 7fe68e800640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 0x7fe66c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.373+0000 7fe68e800640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 0x7fe66c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe680007fd0 tx=0x7fe680007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.378+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe678079a60 con 0x7fe6900aefb0 2024-09-10T07:50:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:54.599+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe69015e5a0 con 0x7fe6900aefb0 2024-09-10T07:50:55.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.180+0000 7fe686a00640 1 -- 172.21.15.3:0/829106622 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7fe67807dd60 con 0x7fe6900aefb0 2024-09-10T07:50:55.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 msgr2=0x7fe66c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 0x7fe66c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe680007fd0 tx=0x7fe680007de0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 msgr2=0x7fe6901467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6780097a0 tx=0x7fe6780046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 shutdown_connections 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe66c06be30 0x7fe66c06e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe690147ce0 0x7fe690146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6900aefb0 0x7fe6901467e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 --2- 172.21.15.3:0/829106622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6900728d0 0x7fe6901417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.183+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 >> 172.21.15.3:0/829106622 conn(0x7fe6900b5cb0 msgr2=0x7fe69006ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.184+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 shutdown_connections 2024-09-10T07:50:55.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.184+0000 7fe695400640 1 -- 172.21.15.3:0/829106622 wait complete. 2024-09-10T07:50:55.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:50:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.369+0000 7fbe56800640 1 Processor -- start 2024-09-10T07:50:55.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.369+0000 7fbe56800640 1 -- start start 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe56800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe500cead0 0x7fbe500d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe56800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe500cf090 con 0x7fbe5015e7c0 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe56800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe500cf370 con 0x7fbe5006bb20 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe56800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbe500cfca0 con 0x7fbe500cead0 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe4f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe5015e7c0 0x7fbe50072d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56842/0 (socket says 172.21.15.3:56842) 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe4f400640 1 -- 172.21.15.3:0/2768769189 learned_addr learned my addr 172.21.15.3:0/2768769189 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:55.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.370+0000 7fbe47e00640 1 --2- 172.21.15.3:0/2768769189 >> v2:172.21.15.3:3300/0 conn(0x7fbe500cead0 0x7fbe500d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.371+0000 7fbe47e00640 1 -- 172.21.15.3:0/2768769189 >> v1:172.21.15.70:6789/0 conn(0x7fbe5006bb20 legacy=0x7fbe5006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.371+0000 7fbe47e00640 1 -- 172.21.15.3:0/2768769189 >> v1:172.21.15.3:6789/0 conn(0x7fbe5015e7c0 legacy=0x7fbe50072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.371+0000 7fbe47e00640 1 -- 172.21.15.3:0/2768769189 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe500cf650 con 0x7fbe500cead0 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.372+0000 7fbe47e00640 1 --2- 172.21.15.3:0/2768769189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe500d0130 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbe500cfe20 tx=0x7fbe3802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436cc3ac2593e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.372+0000 7fbe4ea00640 1 -- 172.21.15.3:0/2768769189 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38037070 con 0x7fbe500cead0 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.372+0000 7fbe4ea00640 1 -- 172.21.15.3:0/2768769189 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3802ae40 con 0x7fbe500cead0 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.372+0000 7fbe4ea00640 1 -- 172.21.15.3:0/2768769189 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38032660 con 0x7fbe500cead0 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 -- 172.21.15.3:0/2768769189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 msgr2=0x7fbe500d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 --2- 172.21.15.3:0/2768769189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe500d0130 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbe500cfe20 tx=0x7fbe3802a130 comp rx=0 tx=0).stop 2024-09-10T07:50:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 -- 172.21.15.3:0/2768769189 shutdown_connections 2024-09-10T07:50:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 --2- 172.21.15.3:0/2768769189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe500d0130 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 -- 172.21.15.3:0/2768769189 >> 172.21.15.3:0/2768769189 conn(0x7fbe500b28a0 msgr2=0x7fbe500b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 -- 172.21.15.3:0/2768769189 shutdown_connections 2024-09-10T07:50:55.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.373+0000 7fbe56800640 1 -- 172.21.15.3:0/2768769189 wait complete. 2024-09-10T07:50:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 Processor -- start 2024-09-10T07:50:55.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 -- start start 2024-09-10T07:50:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 0x7fbe5015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:55.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 0x7fbe50160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe500b3ba0 con 0x7fbe500cead0 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5015eba0 con 0x7fbe5015e7c0 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.375+0000 7fbe56800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe500b3d20 con 0x7fbe5006bb20 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36538/0 (socket says 172.21.15.3:36538) 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 -- 172.21.15.3:0/3349111009 learned_addr learned my addr 172.21.15.3:0/3349111009 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe4fe00640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 0x7fbe50160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 -- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 msgr2=0x7fbe5015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe4f400640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 0x7fbe5015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 0x7fbe5015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 -- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 msgr2=0x7fbe50160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:55.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 0x7fbe50160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe47e00640 1 -- 172.21.15.3:0/3349111009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe501564b0 con 0x7fbe500cead0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe4fe00640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 0x7fbe50160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.376+0000 7fbe4f400640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 0x7fbe5015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000c00 tx=0x7fbe380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38041070 con 0x7fbe500cead0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3803c670 con 0x7fbe500cead0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38037040 con 0x7fbe500cead0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe50156790 con 0x7fbe500cead0 2024-09-10T07:50:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.377+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501542a0 con 0x7fbe500cead0 2024-09-10T07:50:55.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.378+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe500d0130 con 0x7fbe500cead0 2024-09-10T07:50:55.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.379+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe38044070 con 0x7fbe500cead0 2024-09-10T07:50:55.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.380+0000 7fbe47400640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 0x7fbe2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:55.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.380+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6025+0+0 (secure 0 0 0) 0x7fbe380ab7e0 con 0x7fbe500cead0 2024-09-10T07:50:55.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.380+0000 7fbe4f400640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 0x7fbe2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:55.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.381+0000 7fbe4f400640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 0x7fbe2c06e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000f60 tx=0x7fbe40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:55.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.384+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe3807a160 con 0x7fbe500cead0 2024-09-10T07:50:55.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:55.568+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe5006e4a0 con 0x7fbe500cead0 2024-09-10T07:50:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.182+0000 7fbe47400640 1 -- 172.21.15.3:0/3349111009 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) ==== 181+0+0 (secure 0 0 0) 0x7fbe38079a60 con 0x7fbe500cead0 2024-09-10T07:50:56.181 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:50:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.184+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 msgr2=0x7fbe2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.184+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 0x7fbe2c06e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000f60 tx=0x7fbe40009040 comp rx=0 tx=0).stop 2024-09-10T07:50:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.184+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 msgr2=0x7fbe50153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.184+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000c00 tx=0x7fbe380046c0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 shutdown_connections 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe2c06be30 0x7fbe2c06e2f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe5015e7c0 0x7fbe50160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe500cead0 0x7fbe50153d60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 --2- 172.21.15.3:0/3349111009 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe5006bb20 0x7fbe5015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 >> 172.21.15.3:0/3349111009 conn(0x7fbe500b28a0 msgr2=0x7fbe5015d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 shutdown_connections 2024-09-10T07:50:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.186+0000 7fbe56800640 1 -- 172.21.15.3:0/3349111009 wait complete. 2024-09-10T07:50:56.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:50:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.369+0000 7f8176400640 1 Processor -- start 2024-09-10T07:50:56.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.370+0000 7f8176400640 1 -- start start 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8176400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81701591b0 0x7f817015c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8176400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8176400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81700cd8d0 con 0x7f81700ccc50 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8176400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f817015b900 con 0x7f81701591b0 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8176400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81700b47b0 con 0x7f81700b3e10 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8167e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f816f400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81701591b0 0x7f817015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8167e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:50878/0 (socket says 172.21.15.3:50878) 2024-09-10T07:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.371+0000 7f8167e00640 1 -- 172.21.15.3:0/3247899953 learned_addr learned my addr 172.21.15.3:0/3247899953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.372+0000 7f816f400640 1 -- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 msgr2=0x7f81700b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.372+0000 7f816f400640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.372+0000 7f816f400640 1 -- 172.21.15.3:0/3247899953 >> v1:172.21.15.70:6789/0 conn(0x7f81700ccc50 legacy=0x7f81700cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.372+0000 7f816f400640 1 -- 172.21.15.3:0/3247899953 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817015bb20 con 0x7f81701591b0 2024-09-10T07:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.372+0000 7f8167e00640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:50:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.373+0000 7f816f400640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701591b0 0x7f817015c600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f81600097a0 tx=0x7f816002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d17250960a64964 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.373+0000 7f816ea00640 1 -- 172.21.15.3:0/3247899953 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160037070 con 0x7f81701591b0 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.373+0000 7f816ea00640 1 -- 172.21.15.3:0/3247899953 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8160032440 con 0x7f81701591b0 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.373+0000 7f816ea00640 1 -- 172.21.15.3:0/3247899953 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160032810 con 0x7f81701591b0 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 -- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701591b0 msgr2=0x7f817015c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701591b0 0x7f817015c600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f81600097a0 tx=0x7f816002a180 comp rx=0 tx=0).stop 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 -- 172.21.15.3:0/3247899953 shutdown_connections 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700b3e10 0x7f81700b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 --2- 172.21.15.3:0/3247899953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81701591b0 0x7f817015c600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 -- 172.21.15.3:0/3247899953 >> 172.21.15.3:0/3247899953 conn(0x7f8170062cc0 msgr2=0x7f817006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 -- 172.21.15.3:0/3247899953 shutdown_connections 2024-09-10T07:50:56.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.374+0000 7f8176400640 1 -- 172.21.15.3:0/3247899953 wait complete. 2024-09-10T07:50:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.376+0000 7f8176400640 1 Processor -- start 2024-09-10T07:50:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.376+0000 7f8176400640 1 -- start start 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700b3e10 0x7f81700d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700722c0 0x7f81701528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81700ce290 con 0x7f81700ccc50 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81700cdbb0 con 0x7f81700b3e10 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8176400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81700ce410 con 0x7f81700722c0 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36552/0 (socket says 172.21.15.3:36552) 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 -- 172.21.15.3:0/2527550264 learned_addr learned my addr 172.21.15.3:0/2527550264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f816fe00640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700722c0 0x7f81701528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f816f400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700b3e10 0x7f81700d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 -- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700722c0 msgr2=0x7f81701528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700722c0 0x7f81701528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 -- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700b3e10 msgr2=0x7f81700d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700b3e10 0x7f81700d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.377+0000 7f8167e00640 1 -- 172.21.15.3:0/2527550264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8170072c10 con 0x7f81700ccc50 2024-09-10T07:50:56.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8167e00640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f815800e9e0 tx=0x7f815800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158017070 con 0x7f81700ccc50 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815800ca30 con 0x7f81700ccc50 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815800ce20 con 0x7f81700ccc50 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8170072ef0 con 0x7f81700ccc50 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.378+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81700734b0 con 0x7f81700ccc50 2024-09-10T07:50:56.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.379+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817015d670 con 0x7f81700ccc50 2024-09-10T07:50:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.380+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f815801a070 con 0x7f81700ccc50 2024-09-10T07:50:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.381+0000 7f8167400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 0x7f814c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:56.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.381+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5587+0+0 (secure 0 0 0) 0x7f815808b0c0 con 0x7f81700ccc50 2024-09-10T07:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.381+0000 7f816f400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 0x7f814c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:56.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.382+0000 7f816f400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 0x7f814c06e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8160034000 tx=0x7f816002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:56.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.385+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81580593f0 con 0x7f81700ccc50 2024-09-10T07:50:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:56.569+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f817006d290 con 0x7f81700ccc50 2024-09-10T07:50:57.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.184+0000 7f8167400640 1 -- 172.21.15.3:0/2527550264 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) ==== 169+0+0 (secure 0 0 0) 0x7f815805d6f0 con 0x7f81700ccc50 2024-09-10T07:50:57.183 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:50:57.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 msgr2=0x7f814c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 0x7f814c06e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8160034000 tx=0x7f816002e000 comp rx=0 tx=0).stop 2024-09-10T07:50:57.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 msgr2=0x7f81701510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f815800e9e0 tx=0x7f815800eea0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 shutdown_connections 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f814c06be60 0x7f814c06e320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81700722c0 0x7f81701528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81700ccc50 0x7f81701510c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.186+0000 7f8176400640 1 --2- 172.21.15.3:0/2527550264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81700b3e10 0x7f81700d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.187+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 >> 172.21.15.3:0/2527550264 conn(0x7f8170062cc0 msgr2=0x7f81700b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.187+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 shutdown_connections 2024-09-10T07:50:57.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.187+0000 7f8176400640 1 -- 172.21.15.3:0/2527550264 wait complete. 2024-09-10T07:50:57.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:50:57.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.367+0000 7f441fc00640 1 Processor -- start 2024-09-10T07:50:57.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.367+0000 7f441fc00640 1 -- start start 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441fc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f441806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441fc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44180720b0 con 0x7f441815e530 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441fc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4418072390 con 0x7f4418151870 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441fc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4418072670 con 0x7f4418151c50 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441dc00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f441815e530 0x7f44180b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56858/0 (socket says 172.21.15.3:56858) 2024-09-10T07:50:57.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.368+0000 7f441dc00640 1 -- 172.21.15.3:0/1275752935 learned_addr learned my addr 172.21.15.3:0/1275752935 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:57.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829593243 0 0) 0x7f44180720b0 con 0x7f441815e530 2024-09-10T07:50:57.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417e00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f441806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4400003520 con 0x7f441815e530 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2369459903 0 0) 0x7f4400003520 con 0x7f441815e530 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 msgr2=0x7f441806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f441806e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 >> v1:172.21.15.70:6789/0 conn(0x7f4418151870 legacy=0x7f44180718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4418072950 con 0x7f441815e530 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f44100019f0 con 0x7f441815e530 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 >> v1:172.21.15.3:6789/0 conn(0x7f441815e530 legacy=0x7f44180b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f44000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4400004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4400004d40 0x7f44000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4418072670 con 0x7f4400003f60 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f440000bb50 con 0x7f4400004d40 2024-09-10T07:50:57.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f440000df60 con 0x7f4418151c50 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.369+0000 7f4417e00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4400004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f441d200640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f44000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f441dc00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4400004d40 0x7f44000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f4417e00640 1 -- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 msgr2=0x7f44000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f4417e00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418151c50 0x7f44000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f4417e00640 1 -- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4400004d40 msgr2=0x7f44000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f4417e00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4400004d40 0x7f44000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f4417e00640 1 -- 172.21.15.3:0/1275752935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4400003520 con 0x7f4400003f60 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.370+0000 7f441dc00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4400004d40 0x7f44000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.371+0000 7f4417e00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4400004800 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4404009a90 tx=0x7f4400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d6c3a20233cc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.371+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404036070 con 0x7f4400003f60 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.371+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4404031400 con 0x7f4400003f60 2024-09-10T07:50:57.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.371+0000 7f4417400640 1 -- 172.21.15.3:0/1275752935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404009690 con 0x7f4400003f60 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 -- 172.21.15.3:0/1275752935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 msgr2=0x7f4400004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 --2- 172.21.15.3:0/1275752935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4400004800 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4404009a90 tx=0x7f4400005830 comp rx=0 tx=0).stop 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441dc00640 1 -- 172.21.15.3:0/1275752935 reap_dead start 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 -- 172.21.15.3:0/1275752935 shutdown_connections 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 -- 172.21.15.3:0/1275752935 >> 172.21.15.3:0/1275752935 conn(0x7f44180b5cb0 msgr2=0x7f441806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 -- 172.21.15.3:0/1275752935 shutdown_connections 2024-09-10T07:50:57.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.372+0000 7f441fc00640 1 -- 172.21.15.3:0/1275752935 wait complete. 2024-09-10T07:50:57.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.373+0000 7f441fc00640 1 Processor -- start 2024-09-10T07:50:57.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 -- start start 2024-09-10T07:50:57.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418146d10 0x7f44180b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 0x7f44180ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4418073150 con 0x7f4400003f60 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.374+0000 7f441fc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f441806e6b0 con 0x7f44181548c0 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441fc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4418154430 con 0x7f4418146d10 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36564/0 (socket says 172.21.15.3:36564) 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 -- 172.21.15.3:0/3444162766 learned_addr learned my addr 172.21.15.3:0/3444162766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f4417e00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418146d10 0x7f44180b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441dc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 0x7f44180ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 -- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418146d10 msgr2=0x7f44180b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418146d10 0x7f44180b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 -- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 msgr2=0x7f44180ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 0x7f44180ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 -- 172.21.15.3:0/3444162766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4418147190 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441dc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 0x7f44180ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.375+0000 7f441d200640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f440c00f820 tx=0x7f440c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.376+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0050d0 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.376+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c018070 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.376+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c005490 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.376+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44180b0f60 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.376+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4418153b00 con 0x7f4400003f60 2024-09-10T07:50:57.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.377+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441806ea50 con 0x7f4400003f60 2024-09-10T07:50:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.378+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f440c00d730 con 0x7f4400003f60 2024-09-10T07:50:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.379+0000 7f4414c00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 0x7f43f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.379+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7f440c08aa30 con 0x7f4400003f60 2024-09-10T07:50:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.379+0000 7f4417e00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 0x7f43f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.379+0000 7f4417e00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 0x7f43f806e320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4404009a90 tx=0x7f4404008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.383+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f440c059620 con 0x7f4400003f60 2024-09-10T07:50:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.573+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f441815ca70 con 0x7f4400003f60 2024-09-10T07:50:57.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.576+0000 7f4414c00640 1 -- 172.21.15.3:0/3444162766 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f440c058f20 con 0x7f4400003f60 2024-09-10T07:50:57.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.578+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 msgr2=0x7f43f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.578+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 0x7f43f806e320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4404009a90 tx=0x7f4404008030 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.578+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 msgr2=0x7f4418153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.578+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f440c00f820 tx=0x7f440c00fce0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 shutdown_connections 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f43f806be60 0x7f43f806e320 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44181548c0 0x7f44180ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4418146d10 0x7f44180b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 --2- 172.21.15.3:0/3444162766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4400003f60 0x7f4418153120 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 >> 172.21.15.3:0/3444162766 conn(0x7f44180b5cb0 msgr2=0x7f4418150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 shutdown_connections 2024-09-10T07:50:57.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.579+0000 7f441fc00640 1 -- 172.21.15.3:0/3444162766 wait complete. 2024-09-10T07:50:57.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-10T07:50:57.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.751+0000 7fafd8e00640 1 Processor -- start 2024-09-10T07:50:57.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.751+0000 7fafd8e00640 1 -- start start 2024-09-10T07:50:57.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd8e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fafd40d05f0 0x7fafd4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd8e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd40cfdd0 con 0x7fafd4160710 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd8e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fafd415cc90 con 0x7fafd40d05f0 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd8e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fafd415cf70 con 0x7fafd4160af0 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd2800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fafd40d05f0 0x7fafd4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd1e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fafd4160710 0x7fafd40cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56860/0 (socket says 172.21.15.3:56860) 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafd1e00640 1 -- 172.21.15.3:0/3248462528 learned_addr learned my addr 172.21.15.3:0/3248462528 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.752+0000 7fafcbe00640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.753+0000 7fafd1400640 1 -- 172.21.15.3:0/3248462528 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612286027 0 0) 0x7fafd40cfdd0 con 0x7fafd4160710 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.753+0000 7fafd2800640 1 -- 172.21.15.3:0/3248462528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 msgr2=0x7fafd40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.753+0000 7fafd2800640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.753+0000 7fafd2800640 1 -- 172.21.15.3:0/3248462528 >> v1:172.21.15.3:6789/0 conn(0x7fafd4160710 legacy=0x7fafd40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.753+0000 7fafd2800640 1 -- 172.21.15.3:0/3248462528 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd415d250 con 0x7fafd40d05f0 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.754+0000 7fafd2800640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd4072070 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789bb0d3087ed9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.754+0000 7fafd1400640 1 -- 172.21.15.3:0/3248462528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc007740 con 0x7fafd40d05f0 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.754+0000 7fafd1400640 1 -- 172.21.15.3:0/3248462528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc0078e0 con 0x7fafd40d05f0 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.754+0000 7fafd1400640 1 -- 172.21.15.3:0/3248462528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc007cb0 con 0x7fafd40d05f0 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 -- 172.21.15.3:0/3248462528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 msgr2=0x7fafd4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd4072070 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc02a660 comp rx=0 tx=0).stop 2024-09-10T07:50:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 -- 172.21.15.3:0/3248462528 shutdown_connections 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd4072070 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 --2- 172.21.15.3:0/3248462528 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 -- 172.21.15.3:0/3248462528 >> 172.21.15.3:0/3248462528 conn(0x7fafd406d000 msgr2=0x7fafd406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 -- 172.21.15.3:0/3248462528 shutdown_connections 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.755+0000 7fafd8e00640 1 -- 172.21.15.3:0/3248462528 wait complete. 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.756+0000 7fafd8e00640 1 Processor -- start 2024-09-10T07:50:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.756+0000 7fafd8e00640 1 -- start start 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafd4160710 0x7fafd40ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fafd415d770 con 0x7fafd40d05f0 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fafd40d0a10 con 0x7fafd4160710 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.757+0000 7fafd8e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fafd414b3c0 con 0x7fafd4160af0 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36592/0 (socket says 172.21.15.3:36592) 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 -- 172.21.15.3:0/1185150967 learned_addr learned my addr 172.21.15.3:0/1185150967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd2800640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafcbe00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafd4160710 0x7fafd40ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 -- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 msgr2=0x7fafd40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 -- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafd4160710 msgr2=0x7fafd40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:57.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafd4160710 0x7fafd40ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:57.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.758+0000 7fafd1e00640 1 -- 172.21.15.3:0/1185150967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd4155a00 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafd1e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c500 tx=0x7fafc400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4018070 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4007580 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4004a20 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd4155ce0 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.759+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41562a0 con 0x7fafd40d05f0 2024-09-10T07:50:57.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.760+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd40cfa60 con 0x7fafd40d05f0 2024-09-10T07:50:57.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.761+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafc4004bc0 con 0x7fafd40d05f0 2024-09-10T07:50:57.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.762+0000 7fafca000640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 0x7fafb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:57.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.762+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7fafc4089dc0 con 0x7fafd40d05f0 2024-09-10T07:50:57.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.762+0000 7fafcbe00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 0x7fafb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:57.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.763+0000 7fafcbe00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 0x7fafb006e320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafd415c330 tx=0x7fafbc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:57.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:57.766+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fafc4058360 con 0x7fafd40d05f0 2024-09-10T07:50:58.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.023+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7fafd40b3400 con 0x7fafd40d05f0 2024-09-10T07:50:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.189+0000 7fafca000640 1 -- 172.21.15.3:0/1185150967 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v235) ==== 142+0+0 (secure 0 0 0) 0x7fafc405c660 con 0x7fafd40d05f0 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 msgr2=0x7fafb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 0x7fafb006e320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafd415c330 tx=0x7fafbc009190 comp rx=0 tx=0).stop 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 msgr2=0x7fafd414ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c500 tx=0x7fafc400c9c0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 shutdown_connections 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fafb006be60 0x7fafb006e320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fafd4160af0 0x7fafd40aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafd4160710 0x7fafd40ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 --2- 172.21.15.3:0/1185150967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafd40d05f0 0x7fafd414ae80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 >> 172.21.15.3:0/1185150967 conn(0x7fafd406d000 msgr2=0x7fafd406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 shutdown_connections 2024-09-10T07:50:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.192+0000 7fafd8e00640 1 -- 172.21.15.3:0/1185150967 wait complete. 2024-09-10T07:50:58.208 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:50:58.208 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:50:58.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:50:58.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.366+0000 7f78e7a00640 1 Processor -- start 2024-09-10T07:50:58.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.366+0000 7f78e7a00640 1 -- start start 2024-09-10T07:50:58.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78e0159ea0 0x7f78e0073140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78e00b6310 0x7f78e0073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e0071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f78e015a280 con 0x7f78e0159ea0 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f78e00719e0 con 0x7f78e00b6310 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e7a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f78e00b5550 con 0x7f78e0070c70 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.367+0000 7f78e5000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78e0159ea0 0x7f78e0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78e0159ea0 0x7f78e0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36594/0 (socket says 172.21.15.3:36594) 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 -- 172.21.15.3:0/820452056 learned_addr learned my addr 172.21.15.3:0/820452056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78dfe00640 1 --2- 172.21.15.3:0/820452056 >> v2:172.21.15.70:3300/0 conn(0x7f78e00b6310 0x7f78e0073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5a00640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e0071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 -- 172.21.15.3:0/820452056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 msgr2=0x7f78e0071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:58.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e0071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 -- 172.21.15.3:0/820452056 >> v2:172.21.15.70:3300/0 conn(0x7f78e00b6310 msgr2=0x7f78e0073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:50:58.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 --2- 172.21.15.3:0/820452056 >> v2:172.21.15.70:3300/0 conn(0x7f78e00b6310 0x7f78e0073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.368+0000 7f78e5000640 1 -- 172.21.15.3:0/820452056 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e00b5830 con 0x7f78e0159ea0 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.369+0000 7f78e5000640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e0159ea0 0x7f78e0073140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f78e0071bf0 tx=0x7f78d402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810ce6c267d249fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.369+0000 7f78df400640 1 -- 172.21.15.3:0/820452056 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4037070 con 0x7f78e0159ea0 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.369+0000 7f78df400640 1 -- 172.21.15.3:0/820452056 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d4032440 con 0x7f78e0159ea0 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.369+0000 7f78df400640 1 -- 172.21.15.3:0/820452056 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4032810 con 0x7f78e0159ea0 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 -- 172.21.15.3:0/820452056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e0159ea0 msgr2=0x7f78e0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e0159ea0 0x7f78e0073140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f78e0071bf0 tx=0x7f78d402a1c0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 -- 172.21.15.3:0/820452056 shutdown_connections 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e0071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 --2- 172.21.15.3:0/820452056 >> v2:172.21.15.70:3300/0 conn(0x7f78e00b6310 0x7f78e0073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 --2- 172.21.15.3:0/820452056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e0159ea0 0x7f78e0073140 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 -- 172.21.15.3:0/820452056 >> 172.21.15.3:0/820452056 conn(0x7f78e006e5b0 msgr2=0x7f78e006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 -- 172.21.15.3:0/820452056 shutdown_connections 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.370+0000 7f78e7a00640 1 -- 172.21.15.3:0/820452056 wait complete. 2024-09-10T07:50:58.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.372+0000 7f78e7a00640 1 Processor -- start 2024-09-10T07:50:58.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.372+0000 7f78e7a00640 1 -- start start 2024-09-10T07:50:58.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e00b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e0159ea0 0x7f78e00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78e00715c0 con 0x7f78e00b6310 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f78e00b5df0 con 0x7f78e0159ea0 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e7a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f78e0071740 con 0x7f78e0070c70 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36608/0 (socket says 172.21.15.3:36608) 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78dfe00640 1 -- 172.21.15.3:0/3405534354 learned_addr learned my addr 172.21.15.3:0/3405534354 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e5a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e0159ea0 0x7f78e00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.373+0000 7f78e5000640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e00b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 -- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 msgr2=0x7f78e00b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e00b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 -- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e0159ea0 msgr2=0x7f78e00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e0159ea0 0x7f78e00ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 -- 172.21.15.3:0/3405534354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0158f10 con 0x7f78e00b6310 2024-09-10T07:50:58.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dfe00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00d7d0 tx=0x7f78cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc004020 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc0041e0 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01f4a40 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0043d0 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.374+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01f5000 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.375+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e0071080 con 0x7f78e00b6310 2024-09-10T07:50:58.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.376+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78cc01b020 con 0x7f78e00b6310 2024-09-10T07:50:58.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.377+0000 7f78dcc00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 0x7f78c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.377+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7f78cc08a860 con 0x7f78e00b6310 2024-09-10T07:50:58.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.377+0000 7f78e5000640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 0x7f78c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.378+0000 7f78e5000640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 0x7f78c006e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78d4034000 tx=0x7f78d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.381+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f78cc058e00 con 0x7f78e00b6310 2024-09-10T07:50:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.637+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78e015c3d0 con 0x7f78e00b6310 2024-09-10T07:50:58.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.638+0000 7f78dcc00640 1 -- 172.21.15.3:0/3405534354 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+13216 (secure 0 0 0) 0x7f78cc05d100 con 0x7f78e00b6310 2024-09-10T07:50:58.637 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:50:58.637 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:50:57.182192+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/1603892470":"2024-09-11T07:50:54.173087+0000","172.21.15.70:6835/4215641179":"2024-09-11T07:50:54.173075+0000","172.21.15.70:6833/4215641179":"2024-09-11T07:50:54.173075+0000","172.21.15.3:6837/1603892470":"2024-09-11T07:50:54.173087+0000","172.21.15.3:6836/1359877653":"2024-09-11T07:50:54.173091+0000","172.21.15.3:6834/1359877653":"2024-09-11T07:50:54.173091+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:50:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.640+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 msgr2=0x7f78c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 0x7f78c006e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78d4034000 tx=0x7f78d402e000 comp rx=0 tx=0).stop 2024-09-10T07:50:58.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 msgr2=0x7f78e01589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00d7d0 tx=0x7f78cc00dc90 comp rx=0 tx=0).stop 2024-09-10T07:50:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 shutdown_connections 2024-09-10T07:50:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78c006be30 0x7f78c006e2f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e0159ea0 0x7f78e00ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78e00b6310 0x7f78e01589d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 --2- 172.21.15.3:0/3405534354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f78e0070c70 0x7f78e00b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 >> 172.21.15.3:0/3405534354 conn(0x7f78e006e5b0 msgr2=0x7f78e015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:58.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 shutdown_connections 2024-09-10T07:50:58.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.641+0000 7f78e7a00640 1 -- 172.21.15.3:0/3405534354 wait complete. 2024-09-10T07:50:58.657 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:50:58.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:50:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.818+0000 7fb99f200640 1 Processor -- start 2024-09-10T07:50:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.818+0000 7fb99f200640 1 -- start start 2024-09-10T07:50:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.819+0000 7fb99f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9a0162960 0x7fb9a00d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.819+0000 7fb99f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a015ec70 con 0x7fb9a00d2b70 2024-09-10T07:50:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.819+0000 7fb99f200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a015ef50 con 0x7fb9a0162580 2024-09-10T07:50:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.819+0000 7fb99f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a015fa30 con 0x7fb9a0162960 2024-09-10T07:50:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99e800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9a00d2b70 0x7fb9a006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56864/0 (socket says 172.21.15.3:56864) 2024-09-10T07:50:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99e800640 1 -- 172.21.15.3:0/2504624559 learned_addr learned my addr 172.21.15.3:0/2504624559 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb995400640 1 --2- 172.21.15.3:0/2504624559 >> v2:172.21.15.70:3300/0 conn(0x7fb9a0162960 0x7fb9a00d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3972296174 0 0) 0x7fb9a015ec70 con 0x7fb9a00d2b70 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb984003520 con 0x7fb9a00d2b70 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354121587 0 0) 0x7fb9a015ef50 con 0x7fb9a0162580 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9a015ec70 con 0x7fb9a0162580 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb995400640 1 -- 172.21.15.3:0/2504624559 >> v1:172.21.15.70:6789/0 conn(0x7fb9a0162580 legacy=0x7fb9a00d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb995400640 1 -- 172.21.15.3:0/2504624559 >> v1:172.21.15.3:6789/0 conn(0x7fb9a00d2b70 legacy=0x7fb9a006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.820+0000 7fb995400640 1 -- 172.21.15.3:0/2504624559 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a015f230 con 0x7fb9a0162960 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.821+0000 7fb995400640 1 --2- 172.21.15.3:0/2504624559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0162960 0x7fb9a00d2630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097a0 tx=0x7fb98802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1859e5358a46fc98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988037070 con 0x7fb9a0162960 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988032440 con 0x7fb9a0162960 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99d400640 1 -- 172.21.15.3:0/2504624559 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988032810 con 0x7fb9a0162960 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 -- 172.21.15.3:0/2504624559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0162960 msgr2=0x7fb9a00d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 --2- 172.21.15.3:0/2504624559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0162960 0x7fb9a00d2630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097a0 tx=0x7fb98802a230 comp rx=0 tx=0).stop 2024-09-10T07:50:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 -- 172.21.15.3:0/2504624559 shutdown_connections 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 --2- 172.21.15.3:0/2504624559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0162960 0x7fb9a00d2630 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 -- 172.21.15.3:0/2504624559 >> 172.21.15.3:0/2504624559 conn(0x7fb9a006df10 msgr2=0x7fb9a006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 -- 172.21.15.3:0/2504624559 shutdown_connections 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.822+0000 7fb99f200640 1 -- 172.21.15.3:0/2504624559 wait complete. 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.824+0000 7fb99f200640 1 Processor -- start 2024-09-10T07:50:58.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.824+0000 7fb99f200640 1 -- start start 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 0x7fb9a00cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 0x7fb9a0149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0162dd0 con 0x7fb9a0162960 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00d2fe0 con 0x7fb9a00d2b70 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a0162f50 con 0x7fb9a0162580 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36618/0 (socket says 172.21.15.3:36618) 2024-09-10T07:50:58.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.825+0000 7fb99e800640 1 -- 172.21.15.3:0/3321530521 learned_addr learned my addr 172.21.15.3:0/3321530521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb995400640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 0x7fb9a0149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99de00640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 0x7fb9a00cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 -- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 msgr2=0x7fb9a0149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 0x7fb9a0149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 -- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 msgr2=0x7fb9a00cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 0x7fb9a00cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 -- 172.21.15.3:0/3321530521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a0156090 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb995400640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 0x7fb9a0149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99de00640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 0x7fb9a00cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99e800640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb99800cb40 tx=0x7fb99800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998017070 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99800ac40 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998012400 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a00b6bc0 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.826+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a00b7180 con 0x7fb9a0162960 2024-09-10T07:50:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.827+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a00d2630 con 0x7fb9a0162960 2024-09-10T07:50:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.828+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb99801a070 con 0x7fb9a0162960 2024-09-10T07:50:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.829+0000 7fb996a00640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 0x7fb97c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.829+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7fb99808a8d0 con 0x7fb9a0162960 2024-09-10T07:50:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.829+0000 7fb99de00640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 0x7fb97c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.830+0000 7fb99de00640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 0x7fb97c06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb990001010 tx=0x7fb990009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:58.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:58.833+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb998058df0 con 0x7fb9a0162960 2024-09-10T07:50:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.015+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9a00d2130 con 0x7fb9a0162960 2024-09-10T07:50:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.016+0000 7fb996a00640 1 -- 172.21.15.3:0/3321530521 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fb99805d0f0 con 0x7fb9a0162960 2024-09-10T07:50:59.015 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:50:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 msgr2=0x7fb97c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 0x7fb97c06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb990001010 tx=0x7fb990009040 comp rx=0 tx=0).stop 2024-09-10T07:50:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 msgr2=0x7fb9a01f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb99800cb40 tx=0x7fb99800a060 comp rx=0 tx=0).stop 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 shutdown_connections 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be30 0x7fb97c06e2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a0162960 0x7fb9a01f4630 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0162580 0x7fb9a0149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 --2- 172.21.15.3:0/3321530521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a00d2b70 0x7fb9a00cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.018+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 >> 172.21.15.3:0/3321530521 conn(0x7fb9a006df10 msgr2=0x7fb9a00b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.019+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 shutdown_connections 2024-09-10T07:50:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.019+0000 7fb99f200640 1 -- 172.21.15.3:0/3321530521 wait complete. 2024-09-10T07:50:59.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-10T07:50:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.191+0000 7fa305600640 1 Processor -- start 2024-09-10T07:50:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.191+0000 7fa305600640 1 -- start start 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa305600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3000d16e0 0x7fa3000d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa305600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3000d1b10 con 0x7fa3000b6a70 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa305600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3000d1d90 con 0x7fa3000b6690 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa305600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa3000cf540 con 0x7fa3000d16e0 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa2fe000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa3000b6a70 0x7fa3000d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56880/0 (socket says 172.21.15.3:56880) 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa2fe000640 1 -- 172.21.15.3:0/1313841747 learned_addr learned my addr 172.21.15.3:0/1313841747 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:50:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.192+0000 7fa2ff400640 1 --2- 172.21.15.3:0/1313841747 >> v2:172.21.15.70:3300/0 conn(0x7fa3000d16e0 0x7fa3000d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.193+0000 7fa2ff400640 1 -- 172.21.15.3:0/1313841747 >> v1:172.21.15.70:6789/0 conn(0x7fa3000b6690 legacy=0x7fa3000cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.193+0000 7fa2ff400640 1 -- 172.21.15.3:0/1313841747 >> v1:172.21.15.3:6789/0 conn(0x7fa3000b6a70 legacy=0x7fa3000d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.193+0000 7fa2ff400640 1 -- 172.21.15.3:0/1313841747 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3000ceef0 con 0x7fa3000d16e0 2024-09-10T07:50:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.194+0000 7fa2ff400640 1 --2- 172.21.15.3:0/1313841747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000d16e0 0x7fa3000d2b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009d90 tx=0x7fa2f802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a3a4905483ea81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:59.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa2fd600640 1 -- 172.21.15.3:0/1313841747 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007740 con 0x7fa3000d16e0 2024-09-10T07:50:59.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa2fd600640 1 -- 172.21.15.3:0/1313841747 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f80078e0 con 0x7fa3000d16e0 2024-09-10T07:50:59.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa2fd600640 1 -- 172.21.15.3:0/1313841747 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007cb0 con 0x7fa3000d16e0 2024-09-10T07:50:59.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 -- 172.21.15.3:0/1313841747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000d16e0 msgr2=0x7fa3000d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 --2- 172.21.15.3:0/1313841747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000d16e0 0x7fa3000d2b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009d90 tx=0x7fa2f802a6f0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 -- 172.21.15.3:0/1313841747 shutdown_connections 2024-09-10T07:50:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 --2- 172.21.15.3:0/1313841747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000d16e0 0x7fa3000d2b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 -- 172.21.15.3:0/1313841747 >> 172.21.15.3:0/1313841747 conn(0x7fa300072560 msgr2=0x7fa300072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:50:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.195+0000 7fa305600640 1 -- 172.21.15.3:0/1313841747 shutdown_connections 2024-09-10T07:50:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.196+0000 7fa305600640 1 -- 172.21.15.3:0/1313841747 wait complete. 2024-09-10T07:50:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.197+0000 7fa305600640 1 Processor -- start 2024-09-10T07:50:59.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.197+0000 7fa305600640 1 -- start start 2024-09-10T07:50:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000b6690 0x7fa3000ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3000d16e0 0x7fa30015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa30015c530 con 0x7fa3000b6a70 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000b6f00 con 0x7fa3000b6690 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa305600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa30015c6b0 con 0x7fa3000d16e0 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa2fe000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa2fe000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36624/0 (socket says 172.21.15.3:36624) 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa2fe000640 1 -- 172.21.15.3:0/4293178675 learned_addr learned my addr 172.21.15.3:0/4293178675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.198+0000 7fa2fea00640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000b6690 0x7fa3000ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2ff400640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3000d16e0 0x7fa30015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 -- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3000d16e0 msgr2=0x7fa30015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3000d16e0 0x7fa30015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 -- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000b6690 msgr2=0x7fa3000ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000b6690 0x7fa3000ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 -- 172.21.15.3:0/4293178675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300159190 con 0x7fa3000b6a70 2024-09-10T07:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2fe000640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00c4a0 tx=0x7fa2ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec017070 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec005ce0 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.199+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec007270 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.200+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa300159470 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.200+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa300159f30 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.201+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30015a130 con 0x7fa3000b6a70 2024-09-10T07:50:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.202+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2ec01a070 con 0x7fa3000b6a70 2024-09-10T07:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.202+0000 7fa2f6a00640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 0x7fa2e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.203+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7fa2ec089f10 con 0x7fa3000b6a70 2024-09-10T07:50:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.203+0000 7fa2fea00640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 0x7fa2e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:50:59.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.203+0000 7fa2fea00640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 0x7fa2e006e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000f60 tx=0x7fa2f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:50:59.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.207+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa30015a130 con 0x7fa3000b6a70 2024-09-10T07:50:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:50:59.458+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7fa3000cf9d0 con 0x7fa3000b6a70 2024-09-10T07:51:00.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.198+0000 7fa2f6a00640 1 -- 172.21.15.3:0/4293178675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v237) ==== 144+0+0 (secure 0 0 0) 0x7fa3000d1b10 con 0x7fa3000b6a70 2024-09-10T07:51:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 msgr2=0x7fa2e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 0x7fa2e006e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000f60 tx=0x7fa2f0009040 comp rx=0 tx=0).stop 2024-09-10T07:51:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 msgr2=0x7fa300158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00c4a0 tx=0x7fa2ec00c960 comp rx=0 tx=0).stop 2024-09-10T07:51:00.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 shutdown_connections 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa2e006be30 0x7fa2e006e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3000d16e0 0x7fa30015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3000b6a70 0x7fa300158740 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 --2- 172.21.15.3:0/4293178675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3000b6690 0x7fa3000ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 >> 172.21.15.3:0/4293178675 conn(0x7fa300072560 msgr2=0x7fa3000cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.202+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 shutdown_connections 2024-09-10T07:51:00.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.203+0000 7fa305600640 1 -- 172.21.15.3:0/4293178675 wait complete. 2024-09-10T07:51:00.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:51:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.388+0000 7fa5a3400640 1 Processor -- start 2024-09-10T07:51:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.388+0000 7fa5a3400640 1 -- start start 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa5a3400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa59c15e7e0 0x7fa59c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa5a3400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c0d4f50 con 0x7fa59c0b7780 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa5a3400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c0b6f90 con 0x7fa59c0d4b40 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa5a3400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0cfc90 con 0x7fa59c15e7e0 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa59be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa59c0b7780 0x7fa59c0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56882/0 (socket says 172.21.15.3:56882) 2024-09-10T07:51:00.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.389+0000 7fa59be00640 1 -- 172.21.15.3:0/3917055694 learned_addr learned my addr 172.21.15.3:0/3917055694 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa5a0a00640 1 --2- 172.21.15.3:0/3917055694 >> v2:172.21.15.70:3300/0 conn(0x7fa59c15e7e0 0x7fa59c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271707809 0 0) 0x7fa59c0d4f50 con 0x7fa59c0b7780 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa584003520 con 0x7fa59c0b7780 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa5a0a00640 1 -- 172.21.15.3:0/3917055694 >> v1:172.21.15.70:6789/0 conn(0x7fa59c0d4b40 legacy=0x7fa59c0b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa5a0a00640 1 -- 172.21.15.3:0/3917055694 >> v1:172.21.15.3:6789/0 conn(0x7fa59c0b7780 legacy=0x7fa59c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2877602462 0 0) 0x7fa584003520 con 0x7fa59c0b7780 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.390+0000 7fa5a0a00640 1 -- 172.21.15.3:0/3917055694 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0d2a90 con 0x7fa59c15e7e0 2024-09-10T07:51:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.391+0000 7fa5a0a00640 1 --2- 172.21.15.3:0/3917055694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c15e7e0 0x7fa59c0cf750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5900097a0 tx=0x7fa59002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9129038df2af117d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590037070 con 0x7fa59c15e7e0 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59002ac50 con 0x7fa59c15e7e0 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa59b400640 1 -- 172.21.15.3:0/3917055694 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588001670 con 0x7fa59c15e7e0 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 -- 172.21.15.3:0/3917055694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c15e7e0 msgr2=0x7fa59c0cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 --2- 172.21.15.3:0/3917055694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c15e7e0 0x7fa59c0cf750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5900097a0 tx=0x7fa59002a1c0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 -- 172.21.15.3:0/3917055694 shutdown_connections 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 --2- 172.21.15.3:0/3917055694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c15e7e0 0x7fa59c0cf750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 -- 172.21.15.3:0/3917055694 >> 172.21.15.3:0/3917055694 conn(0x7fa59c072560 msgr2=0x7fa59c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 -- 172.21.15.3:0/3917055694 shutdown_connections 2024-09-10T07:51:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.392+0000 7fa5a3400640 1 -- 172.21.15.3:0/3917055694 wait complete. 2024-09-10T07:51:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.394+0000 7fa5a3400640 1 Processor -- start 2024-09-10T07:51:00.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.394+0000 7fa5a3400640 1 -- start start 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 0x7fa59c161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 0x7fa59c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d3330 con 0x7fa59c0b7780 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d3050 con 0x7fa59c0d4b40 2024-09-10T07:51:00.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a3400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0d3610 con 0x7fa59c15e7e0 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36628/0 (socket says 172.21.15.3:36628) 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 -- 172.21.15.3:0/1403371161 learned_addr learned my addr 172.21.15.3:0/1403371161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa59be00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 0x7fa59c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a1400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 0x7fa59c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 -- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 msgr2=0x7fa59c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 0x7fa59c15b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 -- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 msgr2=0x7fa59c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 0x7fa59c161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a0a00640 1 -- 172.21.15.3:0/1403371161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c15a400 con 0x7fa59c0b7780 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa5a1400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 0x7fa59c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.395+0000 7fa59be00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 0x7fa59c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa5a0a00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa590035040 tx=0x7fa5900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590041070 con 0x7fa59c0b7780 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59003c460 con 0x7fa59c0b7780 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590037040 con 0x7fa59c0b7780 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c15a6e0 con 0x7fa59c0b7780 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.396+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c15aca0 con 0x7fa59c0b7780 2024-09-10T07:51:00.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.397+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c0b3940 con 0x7fa59c0b7780 2024-09-10T07:51:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.398+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa590044070 con 0x7fa59c0b7780 2024-09-10T07:51:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.399+0000 7fa598c00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 0x7fa57c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:00.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.399+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7fa5900ab270 con 0x7fa59c0b7780 2024-09-10T07:51:00.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.399+0000 7fa59be00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 0x7fa57c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:00.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.400+0000 7fa59be00640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 0x7fa57c06e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b5e20 tx=0x7fa588009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:00.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.404+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa590079e60 con 0x7fa59c0b7780 2024-09-10T07:51:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.586+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa59c0b8cc0 con 0x7fa59c0b7780 2024-09-10T07:51:00.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.590+0000 7fa598c00640 1 -- 172.21.15.3:0/1403371161 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fa590079760 con 0x7fa59c0b7780 2024-09-10T07:51:00.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 msgr2=0x7fa57c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 0x7fa57c06e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b5e20 tx=0x7fa588009040 comp rx=0 tx=0).stop 2024-09-10T07:51:00.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 msgr2=0x7fa59c14fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:00.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa590035040 tx=0x7fa5900046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 shutdown_connections 2024-09-10T07:51:00.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa57c06be30 0x7fa57c06e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa59c15e7e0 0x7fa59c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59c0d4b40 0x7fa59c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 --2- 172.21.15.3:0/1403371161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c0b7780 0x7fa59c14fbb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 >> 172.21.15.3:0/1403371161 conn(0x7fa59c072560 msgr2=0x7fa59c0d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 shutdown_connections 2024-09-10T07:51:00.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:00.592+0000 7fa5a3400640 1 -- 172.21.15.3:0/1403371161 wait complete. 2024-09-10T07:51:00.607 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:51:00.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:00.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:00.804 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:51:00.804 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:00.804 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:00.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:51:00.840 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:00.840 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:00.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:00.878 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:00.878 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:00.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:51:00.915 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:00.915 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:00.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:00.955 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:51:00.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.112+0000 7f2923200640 1 Processor -- start 2024-09-10T07:51:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.112+0000 7f2923200640 1 -- start start 2024-09-10T07:51:01.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2923200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f29240b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2923200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29240cfdc0 con 0x7f29240d3c10 2024-09-10T07:51:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2923200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29240cee40 con 0x7f292415e860 2024-09-10T07:51:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2923200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29240d3780 con 0x7f2924073860 2024-09-10T07:51:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2921400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f29240d3c10 0x7f2924073150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56892/0 (socket says 172.21.15.3:56892) 2024-09-10T07:51:01.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2921400640 1 -- 172.21.15.3:0/686904280 learned_addr learned my addr 172.21.15.3:0/686904280 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.113+0000 7f2922800640 1 --2- 172.21.15.3:0/686904280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f29240b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155038521 0 0) 0x7f29240cfdc0 con 0x7f29240d3c10 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2908003520 con 0x7f29240d3c10 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72055329 0 0) 0x7f29240cee40 con 0x7f292415e860 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29240cfdc0 con 0x7f292415e860 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2922800640 1 -- 172.21.15.3:0/686904280 >> v1:172.21.15.70:6789/0 conn(0x7f292415e860 legacy=0x7f29240cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2922800640 1 -- 172.21.15.3:0/686904280 >> v1:172.21.15.3:6789/0 conn(0x7f29240d3c10 legacy=0x7f2924073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.114+0000 7f2922800640 1 -- 172.21.15.3:0/686904280 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29240cf120 con 0x7f2924073860 2024-09-10T07:51:01.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.115+0000 7f2922800640 1 --2- 172.21.15.3:0/686904280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f29240b7880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f291c009dc0 tx=0x7f291c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740c16a63b177ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.115+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c007740 con 0x7f2924073860 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.115+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c0078e0 con 0x7f2924073860 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.115+0000 7f2920a00640 1 -- 172.21.15.3:0/686904280 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c007cb0 con 0x7f2924073860 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 -- 172.21.15.3:0/686904280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 msgr2=0x7f29240b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 --2- 172.21.15.3:0/686904280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f29240b7880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f291c009dc0 tx=0x7f291c02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 -- 172.21.15.3:0/686904280 shutdown_connections 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 --2- 172.21.15.3:0/686904280 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f29240b7880 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 -- 172.21.15.3:0/686904280 >> 172.21.15.3:0/686904280 conn(0x7f29240d4650 msgr2=0x7f29240d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 -- 172.21.15.3:0/686904280 shutdown_connections 2024-09-10T07:51:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.116+0000 7f2923200640 1 -- 172.21.15.3:0/686904280 wait complete. 2024-09-10T07:51:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.118+0000 7f2923200640 1 Processor -- start 2024-09-10T07:51:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.118+0000 7f2923200640 1 -- start start 2024-09-10T07:51:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.118+0000 7f2923200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f2924155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.118+0000 7f2923200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2923200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f292415e860 0x7f29241f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2923200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29240d2460 con 0x7f29240d3c10 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2923200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29240cf400 con 0x7f292415e860 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2923200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29240d4050 con 0x7f2924073860 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36634/0 (socket says 172.21.15.3:36634) 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 -- 172.21.15.3:0/109397106 learned_addr learned my addr 172.21.15.3:0/109397106 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921e00640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f2924155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2922800640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f292415e860 0x7f29241f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 -- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 msgr2=0x7f2924155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f2924155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 -- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f292415e860 msgr2=0x7f29241f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f292415e860 0x7f29241f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.119+0000 7f2921400640 1 -- 172.21.15.3:0/109397106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2924160f60 con 0x7f29240d3c10 2024-09-10T07:51:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f2921400640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f291000c500 tx=0x7f291000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910017070 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2910005d00 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910007330 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2924161240 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.120+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2924161800 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.121+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29241598d0 con 0x7f29240d3c10 2024-09-10T07:51:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.122+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f291001a070 con 0x7f29240d3c10 2024-09-10T07:51:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.123+0000 7f291a000640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.123+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7f2910089fd0 con 0x7f29240d3c10 2024-09-10T07:51:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.123+0000 7f2921e00640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.124+0000 7f2921e00640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 0x7f290406e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2914007fd0 tx=0x7f2914009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.127+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2910058630 con 0x7f29240d3c10 2024-09-10T07:51:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.349+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29240cf6b0 con 0x7f29240d3c10 2024-09-10T07:51:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.350+0000 7f291a000640 1 -- 172.21.15.3:0/109397106 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3899 (secure 0 0 0) 0x7f291005c930 con 0x7f29240d3c10 2024-09-10T07:51:01.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:01.349 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":61,"btime":"2024-09-10T07:50:55:175261+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/3568799823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3568799823},{"type":"v1","addr":"172.21.15.70:6834","nonce":3568799823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/749507767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":749507767},{"type":"v1","addr":"172.21.15.70:6835","nonce":749507767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3248531009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3248531009},{"type":"v1","addr":"172.21.15.3:6835","nonce":3248531009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6618,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/133815097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":133815097},{"type":"v1","addr":"172.21.15.3:6837","nonce":133815097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[]} 2024-09-10T07:51:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 61 2024-09-10T07:51:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 msgr2=0x7f290406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 0x7f290406e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2914007fd0 tx=0x7f2914009040 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 msgr2=0x7f29241f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f291000c500 tx=0x7f291000c9c0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 shutdown_connections 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f290406be30 0x7f290406e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f292415e860 0x7f29241f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29240d3c10 0x7f29241f77b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 --2- 172.21.15.3:0/109397106 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2924073860 0x7f2924155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 >> 172.21.15.3:0/109397106 conn(0x7f29240d4650 msgr2=0x7f29240d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 shutdown_connections 2024-09-10T07:51:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:01.352+0000 7f2923200640 1 -- 172.21.15.3:0/109397106 wait complete. 2024-09-10T07:51:01.366 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:51:01.367 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:51:01.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:51:01.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:01.367 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:51:01.367 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:51:01.367 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:51:01.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:01.367 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:51:01.368 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:51:01.368 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:51:01.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:01.368 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:51:01.368 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:51:01.368 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:51:01.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:01.384 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:51:01.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:51:01.385 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:51:01.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:51:01.387 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:51:01.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:51:01.430 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:51:01.430 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:01.431 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:51:01.432 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:01.432 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:51:01.432 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:01.559 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:51:01.561 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:51:01.561 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:51:04.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-10T07:51:07.387 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:51:07.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:51:07.392 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:51:07.392 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:07.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:51:07.531 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:51:07.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.561+0000 7f644a600640 1 Processor -- start 2024-09-10T07:51:07.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.561+0000 7f644a600640 1 -- start start 2024-09-10T07:51:07.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.562+0000 7f644a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 0x7f64441698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.562+0000 7f644a600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64440b47d0 con 0x7f6444161dd0 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.562+0000 7f644a600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644406c870 con 0x7f64440b5ea0 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.562+0000 7f644a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f644406cb50 con 0x7f64441674d0 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6444161dd0 0x7f644406f8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56144/0 (socket says 172.21.15.3:56144) 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443400640 1 -- 172.21.15.3:0/3357787854 learned_addr learned my addr 172.21.15.3:0/3357787854 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443e00640 1 --2- 172.21.15.3:0/3357787854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 0x7f64441698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3479494185 0 0) 0x7f64440b47d0 con 0x7f6444161dd0 2024-09-10T07:51:07.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6428003520 con 0x7f6444161dd0 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445141831 0 0) 0x7f644406c870 con 0x7f64440b5ea0 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64440b47d0 con 0x7f64440b5ea0 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443e00640 1 -- 172.21.15.3:0/3357787854 >> v1:172.21.15.70:6789/0 conn(0x7f64440b5ea0 legacy=0x7f64440b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443e00640 1 -- 172.21.15.3:0/3357787854 >> v1:172.21.15.3:6789/0 conn(0x7f6444161dd0 legacy=0x7f644406f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.563+0000 7f6443e00640 1 -- 172.21.15.3:0/3357787854 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64441621b0 con 0x7f64441674d0 2024-09-10T07:51:07.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f6443e00640 1 --2- 172.21.15.3:0/3357787854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 0x7f64441698c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f643c009dc0 tx=0x7f643c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6c851255794968 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c007740 con 0x7f64441674d0 2024-09-10T07:51:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c0078e0 con 0x7f64441674d0 2024-09-10T07:51:07.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f6442000640 1 -- 172.21.15.3:0/3357787854 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c007cb0 con 0x7f64441674d0 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f644a600640 1 -- 172.21.15.3:0/3357787854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 msgr2=0x7f64441698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f644a600640 1 --2- 172.21.15.3:0/3357787854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 0x7f64441698c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f643c009dc0 tx=0x7f643c02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f644a600640 1 -- 172.21.15.3:0/3357787854 shutdown_connections 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.565+0000 7f644a600640 1 --2- 172.21.15.3:0/3357787854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64441674d0 0x7f64441698c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.566+0000 7f644a600640 1 -- 172.21.15.3:0/3357787854 >> 172.21.15.3:0/3357787854 conn(0x7f64440b3540 msgr2=0x7f64440b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.566+0000 7f644a600640 1 -- 172.21.15.3:0/3357787854 shutdown_connections 2024-09-10T07:51:07.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.566+0000 7f644a600640 1 -- 172.21.15.3:0/3357787854 wait complete. 2024-09-10T07:51:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.567+0000 7f644a600640 1 Processor -- start 2024-09-10T07:51:07.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.567+0000 7f644a600640 1 -- start start 2024-09-10T07:51:07.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440b5ea0 0x7f644415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64441674d0 0x7f6444143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f644406f520 con 0x7f6444161dd0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f644406f3a0 con 0x7f64441674d0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f644a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f64440b43f0 con 0x7f64440b5ea0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.568+0000 7f6442a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58508/0 (socket says 172.21.15.3:58508) 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 -- 172.21.15.3:0/2040451695 learned_addr learned my addr 172.21.15.3:0/2040451695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6443e00640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64441674d0 0x7f6444143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6443400640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440b5ea0 0x7f644415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 -- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440b5ea0 msgr2=0x7f644415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440b5ea0 0x7f644415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 -- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64441674d0 msgr2=0x7f6444143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64441674d0 0x7f6444143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:07.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 -- 172.21.15.3:0/2040451695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644415dfd0 con 0x7f6444161dd0 2024-09-10T07:51:07.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f6442a00640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f643000c490 tx=0x7f643000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.569+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300052d0 con 0x7f6444161dd0 2024-09-10T07:51:07.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.570+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6430005470 con 0x7f6444161dd0 2024-09-10T07:51:07.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.570+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644415a930 con 0x7f6444161dd0 2024-09-10T07:51:07.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.570+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430012870 con 0x7f6444161dd0 2024-09-10T07:51:07.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.570+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644415aef0 con 0x7f6444161dd0 2024-09-10T07:51:07.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.571+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64441587d0 con 0x7f6444161dd0 2024-09-10T07:51:07.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.572+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6430012a10 con 0x7f6444161dd0 2024-09-10T07:51:07.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.572+0000 7f643b400640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 0x7f642406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:07.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.573+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5115+0+0 (secure 0 0 0) 0x7f643008a7f0 con 0x7f6444161dd0 2024-09-10T07:51:07.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.573+0000 7f6443400640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 0x7f642406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:07.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.573+0000 7f6443400640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 0x7f642406e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6434007fd0 tx=0x7f6434009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:07.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.577+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6430059480 con 0x7f6444161dd0 2024-09-10T07:51:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:07.773+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f64441584f0 con 0x7f6444161dd0 2024-09-10T07:51:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.614+0000 7f643b400640 1 -- 172.21.15.3:0/2040451695 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7f6430058d80 con 0x7f6444161dd0 2024-09-10T07:51:08.614 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:51:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 msgr2=0x7f642406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 0x7f642406e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6434007fd0 tx=0x7f6434009040 comp rx=0 tx=0).stop 2024-09-10T07:51:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 msgr2=0x7f644415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f643000c490 tx=0x7f643000c950 comp rx=0 tx=0).stop 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 shutdown_connections 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f642406be30 0x7f642406e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64441674d0 0x7f6444143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6444161dd0 0x7f644415d860 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 --2- 172.21.15.3:0/2040451695 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f64440b5ea0 0x7f644415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.617+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 >> 172.21.15.3:0/2040451695 conn(0x7f64440b3540 msgr2=0x7f64440b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.618+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 shutdown_connections 2024-09-10T07:51:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.618+0000 7f644a600640 1 -- 172.21.15.3:0/2040451695 wait complete. 2024-09-10T07:51:08.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:51:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.797+0000 7feeb0800640 1 Processor -- start 2024-09-10T07:51:08.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.797+0000 7feeb0800640 1 -- start start 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feeb0800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feea80b6310 0x7feea8071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feeb0800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea815a280 con 0x7feea8073850 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feeb0800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea80719e0 con 0x7feea8159ea0 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feeb0800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7feea80b5550 con 0x7feea80b6310 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feead400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feea8073850 0x7feea8070c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56152/0 (socket says 172.21.15.3:56152) 2024-09-10T07:51:08.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.798+0000 7feead400640 1 -- 172.21.15.3:0/118966429 learned_addr learned my addr 172.21.15.3:0/118966429 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeae800640 1 --2- 172.21.15.3:0/118966429 >> v2:172.21.15.70:3300/0 conn(0x7feea80b6310 0x7feea8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386726358 0 0) 0x7feea815a280 con 0x7feea8073850 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee90003520 con 0x7feea8073850 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4288423210 0 0) 0x7fee90003520 con 0x7feea8073850 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 >> v2:172.21.15.70:3300/0 conn(0x7feea80b6310 msgr2=0x7feea8071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 --2- 172.21.15.3:0/118966429 >> v2:172.21.15.70:3300/0 conn(0x7feea80b6310 0x7feea8071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 >> v1:172.21.15.70:6789/0 conn(0x7feea8159ea0 legacy=0x7feea8073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea80b5830 con 0x7feea8073850 2024-09-10T07:51:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fee98001670 con 0x7feea8073850 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 >> v1:172.21.15.3:6789/0 conn(0x7feea8073850 legacy=0x7feea8070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 0x7fee90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feea80b5550 con 0x7fee90003a00 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fee9000c0c0 con 0x7fee90004800 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.799+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fee9000c2c0 con 0x7fee900051c0 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 0x7fee90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feead400640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeae800640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 -- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee900051c0 msgr2=0x7fee9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fee900051c0 0x7fee9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 -- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90004800 msgr2=0x7fee90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:08.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90004800 0x7fee90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeade00640 1 -- 172.21.15.3:0/118966429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee90003520 con 0x7fee90003a00 2024-09-10T07:51:08.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.800+0000 7feeae800640 1 -- 172.21.15.3:0/118966429 reap_dead start 2024-09-10T07:51:08.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.801+0000 7feeade00640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 0x7fee90003e20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fee9c001750 tx=0x7fee90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9fc61a3c904b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.801+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c001370 con 0x7fee90003a00 2024-09-10T07:51:08.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.801+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c0052d0 con 0x7fee90003a00 2024-09-10T07:51:08.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.801+0000 7feeaca00640 1 -- 172.21.15.3:0/118966429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02abe0 con 0x7fee90003a00 2024-09-10T07:51:08.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 -- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 msgr2=0x7fee90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 0x7fee90003e20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fee9c001750 tx=0x7fee90005830 comp rx=0 tx=0).stop 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 -- 172.21.15.3:0/118966429 shutdown_connections 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 --2- 172.21.15.3:0/118966429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee90003a00 0x7fee90003e20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 -- 172.21.15.3:0/118966429 >> 172.21.15.3:0/118966429 conn(0x7feea806e5b0 msgr2=0x7feea806f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 -- 172.21.15.3:0/118966429 shutdown_connections 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.802+0000 7feeb0800640 1 -- 172.21.15.3:0/118966429 wait complete. 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.804+0000 7feeb0800640 1 Processor -- start 2024-09-10T07:51:08.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.804+0000 7feeb0800640 1 -- start start 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90003a00 0x7feea81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feea81f2fa0 0x7feea80d2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feea8073140 con 0x7feea80d2570 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feea80b5df0 con 0x7fee90003a00 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeb0800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feea80732c0 con 0x7feea81f2fa0 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58530/0 (socket says 172.21.15.3:58530) 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 -- 172.21.15.3:0/3014480579 learned_addr learned my addr 172.21.15.3:0/3014480579 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:08.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeade00640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90003a00 0x7feea81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feead400640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feea81f2fa0 0x7feea80d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 -- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feea81f2fa0 msgr2=0x7feea80d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feea81f2fa0 0x7feea80d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 -- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90003a00 msgr2=0x7feea81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90003a00 0x7feea81590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.805+0000 7feeae800640 1 -- 172.21.15.3:0/3014480579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea80cc440 con 0x7feea80d2570 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feeae800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7feea80718f0 tx=0x7feea400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea400fc80 con 0x7feea80d2570 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea401d070 con 0x7feea80d2570 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4015430 con 0x7feea80d2570 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea80c93e0 con 0x7feea80d2570 2024-09-10T07:51:08.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.806+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea80c99a0 con 0x7feea80d2570 2024-09-10T07:51:08.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.807+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea80c9ba0 con 0x7feea80d2570 2024-09-10T07:51:08.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.808+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feea40155d0 con 0x7feea80d2570 2024-09-10T07:51:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.809+0000 7feea2000640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 0x7fee8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.810+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 4857+0+0 (secure 0 0 0) 0x7feea408b330 con 0x7feea80d2570 2024-09-10T07:51:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.810+0000 7feeade00640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 0x7fee8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:08.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.810+0000 7feeade00640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 0x7fee8c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee9c032000 tx=0x7fee9c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:08.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:08.814+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feea80c9ba0 con 0x7feea80d2570 2024-09-10T07:51:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.066+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feea8071430 con 0x7feea80d2570 2024-09-10T07:51:09.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.067+0000 7feea2000640 1 -- 172.21.15.3:0/3014480579 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7feea4010040 con 0x7feea80d2570 2024-09-10T07:51:09.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:09.067 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:09.067 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:51:09.067 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.068 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:51:09.068 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:51:09.068 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.068 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.068 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:09.069 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:09.069 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:51:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:51:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:51:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:51:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:51:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:51:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:51:09.083 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:09.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.085 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:51:09.086 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.069+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 msgr2=0x7fee8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.069+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 0x7fee8c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee9c032000 tx=0x7fee9c02e000 comp rx=0 tx=0).stop 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.069+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 msgr2=0x7feea80cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.069+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7feea80718f0 tx=0x7feea400f1b0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 shutdown_connections 2024-09-10T07:51:09.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fee8c06be60 0x7fee8c06e320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea80d2570 0x7feea80cb6f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feea81f2fa0 0x7feea80d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 --2- 172.21.15.3:0/3014480579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee90003a00 0x7feea81590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 >> 172.21.15.3:0/3014480579 conn(0x7feea806e5b0 msgr2=0x7feea815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 shutdown_connections 2024-09-10T07:51:09.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.070+0000 7feeb0800640 1 -- 172.21.15.3:0/3014480579 wait complete. 2024-09-10T07:51:09.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:51:09.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.244+0000 7fe382000640 1 Processor -- start 2024-09-10T07:51:09.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.244+0000 7fe382000640 1 -- start start 2024-09-10T07:51:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.245+0000 7fe382000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe37c070f30 0x7fe37c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.245+0000 7fe382000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0cfdc0 0x7fe37c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.245+0000 7fe382000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0cd7f0 con 0x7fe37c15c720 2024-09-10T07:51:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.245+0000 7fe382000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c072210 con 0x7fe37c070f30 2024-09-10T07:51:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.245+0000 7fe382000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c0cf880 con 0x7fe37c0cfdc0 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe37c070f30 0x7fe37c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe37c070f30 0x7fe37c071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58542/0 (socket says 172.21.15.3:58542) 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 -- 172.21.15.3:0/229742134 learned_addr learned my addr 172.21.15.3:0/229742134 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe37be00640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0cfdc0 0x7fe37c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 -- 172.21.15.3:0/229742134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0cfdc0 msgr2=0x7fe37c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0cfdc0 0x7fe37c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 -- 172.21.15.3:0/229742134 >> v1:172.21.15.70:6789/0 conn(0x7fe37c15c720 legacy=0x7fe37c073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.246+0000 7fe373e00640 1 -- 172.21.15.3:0/229742134 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0cda90 con 0x7fe37c070f30 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.247+0000 7fe373e00640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c070f30 0x7fe37c071cd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe364009870 tx=0x7fe36402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e754da8d4a75557 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe37aa00640 1 -- 172.21.15.3:0/229742134 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364037070 con 0x7fe37c070f30 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe37aa00640 1 -- 172.21.15.3:0/229742134 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364032440 con 0x7fe37c070f30 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe37aa00640 1 -- 172.21.15.3:0/229742134 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364032810 con 0x7fe37c070f30 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 -- 172.21.15.3:0/229742134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c070f30 msgr2=0x7fe37c071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c070f30 0x7fe37c071cd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe364009870 tx=0x7fe36402a250 comp rx=0 tx=0).stop 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 -- 172.21.15.3:0/229742134 shutdown_connections 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c0cfdc0 0x7fe37c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 --2- 172.21.15.3:0/229742134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c070f30 0x7fe37c071cd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 -- 172.21.15.3:0/229742134 >> 172.21.15.3:0/229742134 conn(0x7fe37c06cfc0 msgr2=0x7fe37c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 -- 172.21.15.3:0/229742134 shutdown_connections 2024-09-10T07:51:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.248+0000 7fe382000640 1 -- 172.21.15.3:0/229742134 wait complete. 2024-09-10T07:51:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.250+0000 7fe382000640 1 Processor -- start 2024-09-10T07:51:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.250+0000 7fe382000640 1 -- start start 2024-09-10T07:51:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c070f30 0x7fe37c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0cfdc0 0x7fe37c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c073d20 con 0x7fe37c15c720 2024-09-10T07:51:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c06ed10 con 0x7fe37c0cfdc0 2024-09-10T07:51:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe382000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c073ea0 con 0x7fe37c070f30 2024-09-10T07:51:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe37be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe37be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58558/0 (socket says 172.21.15.3:58558) 2024-09-10T07:51:09.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe37be00640 1 -- 172.21.15.3:0/1346024026 learned_addr learned my addr 172.21.15.3:0/1346024026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:09.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe37b400640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c070f30 0x7fe37c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.251+0000 7fe373e00640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0cfdc0 0x7fe37c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 -- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c070f30 msgr2=0x7fe37c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c070f30 0x7fe37c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 -- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0cfdc0 msgr2=0x7fe37c14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0cfdc0 0x7fe37c14fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 -- 172.21.15.3:0/1346024026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0b0160 con 0x7fe37c15c720 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe37be00640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe37400efc0 tx=0x7fe37400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374018020 con 0x7fe37c15c720 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe374014480 con 0x7fe37c15c720 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374014870 con 0x7fe37c15c720 2024-09-10T07:51:09.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c0b0400 con 0x7fe37c15c720 2024-09-10T07:51:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.252+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c0b09c0 con 0x7fe37c15c720 2024-09-10T07:51:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.253+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c0cf2b0 con 0x7fe37c15c720 2024-09-10T07:51:09.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.254+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe374005d10 con 0x7fe37c15c720 2024-09-10T07:51:09.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.256+0000 7fe373400640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 0x7fe35806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.256+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 4857+0+0 (secure 0 0 0) 0x7fe37408b420 con 0x7fe37c15c720 2024-09-10T07:51:09.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.256+0000 7fe37b400640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 0x7fe35806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.257+0000 7fe37b400640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 0x7fe35806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe36c001780 tx=0x7fe36c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.260+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe374022070 con 0x7fe37c15c720 2024-09-10T07:51:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.442+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe37c0737b0 con 0x7fe37c15c720 2024-09-10T07:51:09.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.443+0000 7fe373400640 1 -- 172.21.15.3:0/1346024026 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fe374010040 con 0x7fe37c15c720 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:09.443 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 msgr2=0x7fe35806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 0x7fe35806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe36c001780 tx=0x7fe36c009040 comp rx=0 tx=0).stop 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 msgr2=0x7fe37c150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe37400efc0 tx=0x7fe37400c490 comp rx=0 tx=0).stop 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 shutdown_connections 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe35806be30 0x7fe35806e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe37c15c720 0x7fe37c150110 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe37c0cfdc0 0x7fe37c14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 --2- 172.21.15.3:0/1346024026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe37c070f30 0x7fe37c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 >> 172.21.15.3:0/1346024026 conn(0x7fe37c06cfc0 msgr2=0x7fe37c15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 shutdown_connections 2024-09-10T07:51:09.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.445+0000 7fe382000640 1 -- 172.21.15.3:0/1346024026 wait complete. 2024-09-10T07:51:09.461 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:51:09.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:51:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.625+0000 7f4d29800640 1 Processor -- start 2024-09-10T07:51:09.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.625+0000 7f4d29800640 1 -- start start 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.626+0000 7f4d29800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d24141310 0x7f4d24141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d29800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d24141fb0 con 0x7f4d2415f0d0 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d29800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2415f680 con 0x7f4d240d4490 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d29800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4d240d3d50 con 0x7f4d24141310 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d24141310 0x7f4d24141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d24141310 0x7f4d24141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58564/0 (socket says 172.21.15.3:58564) 2024-09-10T07:51:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d28e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d2415f0d0 0x7f4d240b6450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56158/0 (socket says 172.21.15.3:56158) 2024-09-10T07:51:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 -- 172.21.15.3:0/2947827659 learned_addr learned my addr 172.21.15.3:0/2947827659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 -- 172.21.15.3:0/2947827659 >> v1:172.21.15.70:6789/0 conn(0x7f4d240d4490 legacy=0x7f4d2415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 -- 172.21.15.3:0/2947827659 >> v1:172.21.15.3:6789/0 conn(0x7f4d2415f0d0 legacy=0x7f4d240b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.627+0000 7f4d23e00640 1 -- 172.21.15.3:0/2947827659 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d240b5970 con 0x7f4d24141310 2024-09-10T07:51:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.628+0000 7f4d23e00640 1 --2- 172.21.15.3:0/2947827659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d24141310 0x7f4d24141a70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c280aab910915c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.628+0000 7f4d23400640 1 -- 172.21.15.3:0/2947827659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14037070 con 0x7f4d24141310 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d23400640 1 -- 172.21.15.3:0/2947827659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d14032440 con 0x7f4d24141310 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d23400640 1 -- 172.21.15.3:0/2947827659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14032810 con 0x7f4d24141310 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 -- 172.21.15.3:0/2947827659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d24141310 msgr2=0x7f4d24141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 --2- 172.21.15.3:0/2947827659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d24141310 0x7f4d24141a70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1402a210 comp rx=0 tx=0).stop 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 -- 172.21.15.3:0/2947827659 shutdown_connections 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 --2- 172.21.15.3:0/2947827659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d24141310 0x7f4d24141a70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 -- 172.21.15.3:0/2947827659 >> 172.21.15.3:0/2947827659 conn(0x7f4d24062a50 msgr2=0x7f4d24072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 -- 172.21.15.3:0/2947827659 shutdown_connections 2024-09-10T07:51:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.629+0000 7f4d29800640 1 -- 172.21.15.3:0/2947827659 wait complete. 2024-09-10T07:51:09.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.631+0000 7f4d29800640 1 Processor -- start 2024-09-10T07:51:09.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.631+0000 7f4d29800640 1 -- start start 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d24141310 0x7f4d240c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2415f0d0 0x7f4d2415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4d240b5f30 con 0x7f4d240d4490 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2415f960 con 0x7f4d2415f0d0 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d29800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4d241416f0 con 0x7f4d24141310 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58566/0 (socket says 172.21.15.3:58566) 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d23e00640 1 -- 172.21.15.3:0/3313384520 learned_addr learned my addr 172.21.15.3:0/3313384520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:09.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d28e00640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2415f0d0 0x7f4d2415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.632+0000 7f4d1b400640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d24141310 0x7f4d240c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 -- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d24141310 msgr2=0x7f4d240c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d24141310 0x7f4d240c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 -- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2415f0d0 msgr2=0x7f4d2415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2415f0d0 0x7f4d2415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 -- 172.21.15.3:0/3313384520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d240cb2f0 con 0x7f4d240d4490 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d23e00640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14041020 con 0x7f4d240d4490 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d14030970 con 0x7f4d240d4490 2024-09-10T07:51:09.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14037040 con 0x7f4d240d4490 2024-09-10T07:51:09.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d240c9d10 con 0x7f4d240d4490 2024-09-10T07:51:09.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.633+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d240ca2d0 con 0x7f4d240d4490 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.634+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d24141a70 con 0x7f4d240d4490 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.635+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d14044020 con 0x7f4d240d4490 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.636+0000 7f4d20c00640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 0x7f4d0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.636+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 4857+0+0 (secure 0 0 0) 0x7f4d140ab260 con 0x7f4d240d4490 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.637+0000 7f4d1b400640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 0x7f4d0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:09.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.637+0000 7f4d1b400640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 0x7f4d0006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:09.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.641+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4d1404b070 con 0x7f4d240d4490 2024-09-10T07:51:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:09.825+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4d240b74f0 con 0x7f4d240d4490 2024-09-10T07:51:10.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.627+0000 7f4d20c00640 1 -- 172.21.15.3:0/3313384520 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f4d14079ec0 con 0x7f4d240d4490 2024-09-10T07:51:10.627 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:51:10.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 msgr2=0x7f4d0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:10.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 0x7f4d0006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 msgr2=0x7f4d240cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d140046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 shutdown_connections 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4d0006be30 0x7f4d0006e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2415f0d0 0x7f4d2415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4d24141310 0x7f4d240c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 --2- 172.21.15.3:0/3313384520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d240d4490 0x7f4d240cadb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.630+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 >> 172.21.15.3:0/3313384520 conn(0x7f4d24062a50 msgr2=0x7f4d24160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:10.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.631+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 shutdown_connections 2024-09-10T07:51:10.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.631+0000 7f4d29800640 1 -- 172.21.15.3:0/3313384520 wait complete. 2024-09-10T07:51:10.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:51:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.822+0000 7f26c6400640 1 Processor -- start 2024-09-10T07:51:10.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.823+0000 7f26c6400640 1 -- start start 2024-09-10T07:51:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26c6400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c015e970 0x7f26c0070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26c6400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26c006fa30 0x7f26c00afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26c6400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c00aeb40 con 0x7f26c0062630 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26c6400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f26c006f5a0 con 0x7f26c006fa30 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26c6400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26c006ec70 con 0x7f26c015e970 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26b7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26c006fa30 0x7f26c00afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26b7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26c006fa30 0x7f26c00afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58582/0 (socket says 172.21.15.3:58582) 2024-09-10T07:51:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26b7e00640 1 -- 172.21.15.3:0/2783092011 learned_addr learned my addr 172.21.15.3:0/2783092011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.824+0000 7f26bf400640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c015e970 0x7f26c0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.825+0000 7f26b7e00640 1 -- 172.21.15.3:0/2783092011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c015e970 msgr2=0x7f26c0070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.825+0000 7f26b7e00640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c015e970 0x7f26c0070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.825+0000 7f26b7e00640 1 -- 172.21.15.3:0/2783092011 >> v1:172.21.15.70:6789/0 conn(0x7f26c0062630 legacy=0x7f26c00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.825+0000 7f26b7e00640 1 -- 172.21.15.3:0/2783092011 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c006ef50 con 0x7f26c006fa30 2024-09-10T07:51:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26b7e00640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c006fa30 0x7f26c00afea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009870 tx=0x7f26a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c115ebfa08d194 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26bea00640 1 -- 172.21.15.3:0/2783092011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8037070 con 0x7f26c006fa30 2024-09-10T07:51:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26bea00640 1 -- 172.21.15.3:0/2783092011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8032440 con 0x7f26c006fa30 2024-09-10T07:51:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26bea00640 1 -- 172.21.15.3:0/2783092011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8032810 con 0x7f26c006fa30 2024-09-10T07:51:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 -- 172.21.15.3:0/2783092011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c006fa30 msgr2=0x7f26c00afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:10.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c006fa30 0x7f26c00afea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009870 tx=0x7f26a802a180 comp rx=0 tx=0).stop 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 -- 172.21.15.3:0/2783092011 shutdown_connections 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c006fa30 0x7f26c00afea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 --2- 172.21.15.3:0/2783092011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c015e970 0x7f26c0070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.826+0000 7f26c6400640 1 -- 172.21.15.3:0/2783092011 >> 172.21.15.3:0/2783092011 conn(0x7f26c0073060 msgr2=0x7f26c006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.827+0000 7f26c6400640 1 -- 172.21.15.3:0/2783092011 shutdown_connections 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.827+0000 7f26c6400640 1 -- 172.21.15.3:0/2783092011 wait complete. 2024-09-10T07:51:10.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.828+0000 7f26c6400640 1 Processor -- start 2024-09-10T07:51:10.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.828+0000 7f26c6400640 1 -- start start 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c0062630 0x7f26c01580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26c006fa30 0x7f26c00b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0062ab0 con 0x7f26c015e970 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26c006e920 con 0x7f26c006fa30 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26c6400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f26c0062c30 con 0x7f26c0062630 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26bf400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c0062630 0x7f26c01580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58588/0 (socket says 172.21.15.3:58588) 2024-09-10T07:51:10.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.829+0000 7f26bfe00640 1 -- 172.21.15.3:0/769699844 learned_addr learned my addr 172.21.15.3:0/769699844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:10.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26b7e00640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26c006fa30 0x7f26c00b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 -- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c0062630 msgr2=0x7f26c01580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c0062630 0x7f26c01580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 -- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26c006fa30 msgr2=0x7f26c00b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26c006fa30 0x7f26c00b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 -- 172.21.15.3:0/769699844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01e9950 con 0x7f26c015e970 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26bfe00640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c780 tx=0x7f26b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004b60 con 0x7f26c015e970 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8004d00 con 0x7f26c015e970 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b800aa90 con 0x7f26c015e970 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.830+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01528e0 con 0x7f26c015e970 2024-09-10T07:51:10.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.831+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c0152ea0 con 0x7f26c015e970 2024-09-10T07:51:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.832+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c0156240 con 0x7f26c015e970 2024-09-10T07:51:10.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.833+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26b800ac30 con 0x7f26c015e970 2024-09-10T07:51:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.835+0000 7f26b7400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 0x7f269c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.835+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5227+0+0 (secure 0 0 0) 0x7f26b808b450 con 0x7f26c015e970 2024-09-10T07:51:10.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.835+0000 7f26bf400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 0x7f269c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:10.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.835+0000 7f26bf400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 0x7f269c06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26b00079a0 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:10.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:10.839+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26b8020070 con 0x7f26c015e970 2024-09-10T07:51:11.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.022+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f26c0070190 con 0x7f26c015e970 2024-09-10T07:51:11.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.634+0000 7f26b7400640 1 -- 172.21.15.3:0/769699844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f26c01e9950 con 0x7f26c015e970 2024-09-10T07:51:11.634 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:51:11.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.640+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 msgr2=0x7f269c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.640+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 0x7f269c06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26b00079a0 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.640+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 msgr2=0x7f26c00b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.640+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c780 tx=0x7f26b800cc40 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 shutdown_connections 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f269c06be30 0x7f269c06e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26c015e970 0x7f26c00b1bf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26c006fa30 0x7f26c00b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 --2- 172.21.15.3:0/769699844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26c0062630 0x7f26c01580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 >> 172.21.15.3:0/769699844 conn(0x7f26c0073060 msgr2=0x7f26c00b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.641+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 shutdown_connections 2024-09-10T07:51:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.642+0000 7f26c6400640 1 -- 172.21.15.3:0/769699844 wait complete. 2024-09-10T07:51:11.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:51:11.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-10T07:51:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.832+0000 7f51eac00640 1 Processor -- start 2024-09-10T07:51:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.832+0000 7f51eac00640 1 -- start start 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.833+0000 7f51eac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.833+0000 7f51eac00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51e40d4cb0 0x7f51e40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.833+0000 7f51eac00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e40d3de0 con 0x7f51e40d3860 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.833+0000 7f51eac00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f51e4161f50 con 0x7f51e40d4cb0 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.833+0000 7f51eac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51e40d40c0 con 0x7f51e4161b70 2024-09-10T07:51:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51ea200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51e40d4cb0 0x7f51e40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:49636/0 (socket says 172.21.15.3:49636) 2024-09-10T07:51:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 -- 172.21.15.3:0/3303328911 learned_addr learned my addr 172.21.15.3:0/3303328911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 -- 172.21.15.3:0/3303328911 >> v2:172.21.15.70:3300/0 conn(0x7f51e40d4cb0 msgr2=0x7f51e40cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:11.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 --2- 172.21.15.3:0/3303328911 >> v2:172.21.15.70:3300/0 conn(0x7f51e40d4cb0 0x7f51e40cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 -- 172.21.15.3:0/3303328911 >> v1:172.21.15.70:6789/0 conn(0x7f51e40d3860 legacy=0x7f51e40d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.834+0000 7f51e9800640 1 -- 172.21.15.3:0/3303328911 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e40cf690 con 0x7f51e4161b70 2024-09-10T07:51:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.835+0000 7f51e9800640 1 --2- 172.21.15.3:0/3303328911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009870 tx=0x7f51d802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917a5c662aa94840 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.835+0000 7f51e3e00640 1 -- 172.21.15.3:0/3303328911 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8037070 con 0x7f51e4161b70 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.835+0000 7f51e3e00640 1 -- 172.21.15.3:0/3303328911 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d8032440 con 0x7f51e4161b70 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.835+0000 7f51e3e00640 1 -- 172.21.15.3:0/3303328911 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8032810 con 0x7f51e4161b70 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 -- 172.21.15.3:0/3303328911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 msgr2=0x7f51e40d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 --2- 172.21.15.3:0/3303328911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009870 tx=0x7f51d802a250 comp rx=0 tx=0).stop 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 -- 172.21.15.3:0/3303328911 shutdown_connections 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 --2- 172.21.15.3:0/3303328911 >> v2:172.21.15.70:3300/0 conn(0x7f51e40d4cb0 0x7f51e40cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 --2- 172.21.15.3:0/3303328911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e40d3320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 -- 172.21.15.3:0/3303328911 >> 172.21.15.3:0/3303328911 conn(0x7f51e4062630 msgr2=0x7f51e4062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 -- 172.21.15.3:0/3303328911 shutdown_connections 2024-09-10T07:51:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.836+0000 7f51eac00640 1 -- 172.21.15.3:0/3303328911 wait complete. 2024-09-10T07:51:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.838+0000 7f51eac00640 1 Processor -- start 2024-09-10T07:51:11.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.838+0000 7f51eac00640 1 -- start start 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 0x7f51e41f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 0x7f51e415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40cf2d0 con 0x7f51e40d4cb0 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40c9fc0 con 0x7f51e40d3860 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51eac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f51e40cf450 con 0x7f51e4161b70 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51e8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 0x7f51e415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51e9800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 0x7f51e41f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51ea200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51e9800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 0x7f51e41f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:36438/0 (socket says 172.21.15.3:36438) 2024-09-10T07:51:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.839+0000 7f51e9800640 1 -- 172.21.15.3:0/144913624 learned_addr learned my addr 172.21.15.3:0/144913624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 -- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 msgr2=0x7f51e41f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 0x7f51e41f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 -- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 msgr2=0x7f51e415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 0x7f51e415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 -- 172.21.15.3:0/144913624 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41f5600 con 0x7f51e4161b70 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51e8e00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 0x7f51e415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51ea200640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e414b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc018020 con 0x7f51e4161b70 2024-09-10T07:51:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc014480 con 0x7f51e4161b70 2024-09-10T07:51:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc014870 con 0x7f51e4161b70 2024-09-10T07:51:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.840+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e4159ed0 con 0x7f51e4161b70 2024-09-10T07:51:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.841+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e415a490 con 0x7f51e4161b70 2024-09-10T07:51:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.841+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e40b75d0 con 0x7f51e4161b70 2024-09-10T07:51:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.843+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51dc005d10 con 0x7f51e4161b70 2024-09-10T07:51:11.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.844+0000 7f51e1600640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 0x7f51c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:11.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.844+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5609+0+0 (secure 0 0 0) 0x7f51dc08bdb0 con 0x7f51e4161b70 2024-09-10T07:51:11.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.844+0000 7f51e9800640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 0x7f51c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:11.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.845+0000 7f51e9800640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 0x7f51c406e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51d80090b0 tx=0x7f51d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:11.848+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51dc022030 con 0x7f51e4161b70 2024-09-10T07:51:12.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.068+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f51e40b6350 con 0x7f51e4161b70 2024-09-10T07:51:12.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.641+0000 7f51e1600640 1 -- 172.21.15.3:0/144913624 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v65) ==== 174+0+0 (secure 0 0 0) 0x7f51e41f5600 con 0x7f51e4161b70 2024-09-10T07:51:12.641 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T07:51:12.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.652+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 msgr2=0x7f51c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.652+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 0x7f51c406e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51d80090b0 tx=0x7f51d802e000 comp rx=0 tx=0).stop 2024-09-10T07:51:12.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.652+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 msgr2=0x7f51e414b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.652+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e414b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc00c490 comp rx=0 tx=0).stop 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 shutdown_connections 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51c406be30 0x7f51c406e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f51e4161b70 0x7f51e414b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51e40d4cb0 0x7f51e415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 --2- 172.21.15.3:0/144913624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e40d3860 0x7f51e41f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.653+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 >> 172.21.15.3:0/144913624 conn(0x7f51e4062630 msgr2=0x7f51e40b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.654+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 shutdown_connections 2024-09-10T07:51:12.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.655+0000 7f51eac00640 1 -- 172.21.15.3:0/144913624 wait complete. 2024-09-10T07:51:12.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:12.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.850+0000 7f4c03e00640 1 Processor -- start 2024-09-10T07:51:12.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.850+0000 7f4c03e00640 1 -- start start 2024-09-10T07:51:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.851+0000 7f4c03e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bfc161b30 0x7f4bfc15f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.851+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 0x7f4bfc06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.851+0000 7f4c03e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bfc15fbb0 con 0x7f4bfc1601f0 2024-09-10T07:51:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.851+0000 7f4c03e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4bfc0b36a0 con 0x7f4bfc161b30 2024-09-10T07:51:12.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.851+0000 7f4c03e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4bfc0b3980 con 0x7f4bfc15fdb0 2024-09-10T07:51:12.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4c01e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4bfc1601f0 0x7f4bfc0b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56170/0 (socket says 172.21.15.3:56170) 2024-09-10T07:51:12.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4c01e00640 1 -- 172.21.15.3:0/2751367586 learned_addr learned my addr 172.21.15.3:0/2751367586 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:12.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4c01400640 1 --2- 172.21.15.3:0/2751367586 >> v2:172.21.15.70:3300/0 conn(0x7f4bfc161b30 0x7f4bfc15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4c00a00640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 0x7f4bfc06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714043235 0 0) 0x7f4bf4001490 con 0x7f4bfc1601f0 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.852+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4be4003520 con 0x7f4bfc1601f0 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4c01400640 1 -- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 msgr2=0x7f4bfc06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4c01400640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 0x7f4bfc06bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4c01400640 1 -- 172.21.15.3:0/2751367586 >> v1:172.21.15.3:6789/0 conn(0x7f4bfc1601f0 legacy=0x7f4bfc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 672821172 0 0) 0x7f4be4003520 con 0x7f4bfc1601f0 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4c01400640 1 -- 172.21.15.3:0/2751367586 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc0b3c60 con 0x7f4bfc161b30 2024-09-10T07:51:12.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.853+0000 7f4c00a00640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 0x7f4bfc06bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.854+0000 7f4c01400640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc161b30 0x7f4bfc15f670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00097a0 tx=0x7f4bf002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6c3f99c7d7f656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.854+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0037070 con 0x7f4bfc161b30 2024-09-10T07:51:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.854+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf002aba0 con 0x7f4bfc161b30 2024-09-10T07:51:12.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.854+0000 7f4bfbe00640 1 -- 172.21.15.3:0/2751367586 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4001bc0 con 0x7f4bfc161b30 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 -- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc161b30 msgr2=0x7f4bfc15f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc161b30 0x7f4bfc15f670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00097a0 tx=0x7f4bf002a110 comp rx=0 tx=0).stop 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 -- 172.21.15.3:0/2751367586 shutdown_connections 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc15fdb0 0x7f4bfc06bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 --2- 172.21.15.3:0/2751367586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc161b30 0x7f4bfc15f670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 -- 172.21.15.3:0/2751367586 >> 172.21.15.3:0/2751367586 conn(0x7f4bfc06d000 msgr2=0x7f4bfc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 -- 172.21.15.3:0/2751367586 shutdown_connections 2024-09-10T07:51:12.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.855+0000 7f4c03e00640 1 -- 172.21.15.3:0/2751367586 wait complete. 2024-09-10T07:51:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.857+0000 7f4c03e00640 1 Processor -- start 2024-09-10T07:51:12.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.857+0000 7f4c03e00640 1 -- start start 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 0x7f4bfc14e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc1601f0 0x7f4bfc14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 0x7f4bfc159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc161f10 con 0x7f4bfc161b30 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc0b4220 con 0x7f4bfc15fdb0 2024-09-10T07:51:12.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c03e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4bfc0aefa0 con 0x7f4bfc1601f0 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 0x7f4bfc159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 0x7f4bfc14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c00a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc1601f0 0x7f4bfc14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 0x7f4bfc14e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:36462/0 (socket says 172.21.15.3:36462) 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01400640 1 -- 172.21.15.3:0/3833881404 learned_addr learned my addr 172.21.15.3:0/3833881404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01e00640 1 -- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc1601f0 msgr2=0x7f4bfc14edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.858+0000 7f4c01e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc1601f0 0x7f4bfc14edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c01e00640 1 -- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 msgr2=0x7f4bfc14e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c01e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 0x7f4bfc14e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c01e00640 1 -- 172.21.15.3:0/3833881404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc0d0470 con 0x7f4bfc161b30 2024-09-10T07:51:12.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c01e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 0x7f4bfc159520 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400cb10 tx=0x7f4bf400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400ac70 con 0x7f4bfc161b30 2024-09-10T07:51:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf400ae10 con 0x7f4bfc161b30 2024-09-10T07:51:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bfc0d0750 con 0x7f4bfc161b30 2024-09-10T07:51:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4012640 con 0x7f4bfc161b30 2024-09-10T07:51:12.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.859+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bfc15a580 con 0x7f4bfc161b30 2024-09-10T07:51:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.860+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bfc0d0c70 con 0x7f4bfc161b30 2024-09-10T07:51:12.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.861+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bf40127e0 con 0x7f4bfc161b30 2024-09-10T07:51:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.862+0000 7f4bf9600640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 0x7f4bdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.862+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f4bf408b450 con 0x7f4bfc161b30 2024-09-10T07:51:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.862+0000 7f4c01400640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 0x7f4bdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:12.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.863+0000 7f4c01400640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 0x7f4bdc06e320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:12.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:12.867+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4bfc0d0c70 con 0x7f4bfc161b30 2024-09-10T07:51:13.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.055+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bfc0b3300 con 0x7f4bfc161b30 2024-09-10T07:51:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.056+0000 7f4bf9600640 1 -- 172.21.15.3:0/3833881404 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16609 (secure 0 0 0) 0x7f4bfc15fbb0 con 0x7f4bfc161b30 2024-09-10T07:51:13.056 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:13.057 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:12.071234+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:09.867983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T07:51:11.065097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 msgr2=0x7f4bdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 0x7f4bdc06e320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 msgr2=0x7f4bfc159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 0x7f4bfc159520 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400cb10 tx=0x7f4bf400a060 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 shutdown_connections 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4bdc06be60 0x7f4bdc06e320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bfc161b30 0x7f4bfc159520 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bfc1601f0 0x7f4bfc14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 --2- 172.21.15.3:0/3833881404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc15fdb0 0x7f4bfc14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 >> 172.21.15.3:0/3833881404 conn(0x7f4bfc06d000 msgr2=0x7f4bfc160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 shutdown_connections 2024-09-10T07:51:13.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.059+0000 7f4c03e00640 1 -- 172.21.15.3:0/3833881404 wait complete. 2024-09-10T07:51:13.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:13.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.231+0000 7f2f8da00640 1 Processor -- start 2024-09-10T07:51:13.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.231+0000 7f2f8da00640 1 -- start start 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f8da00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f8815e860 0x7f2f880cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f8da00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f880cee40 con 0x7f2f880cfbf0 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f8da00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f880cf120 con 0x7f2f88073150 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f8da00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2f880d3780 con 0x7f2f8815e860 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f87400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f880cfbf0 0x7f2f880d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56178/0 (socket says 172.21.15.3:56178) 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.232+0000 7f2f87400640 1 -- 172.21.15.3:0/2049244639 learned_addr learned my addr 172.21.15.3:0/2049244639 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:13.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2049244639 >> v2:172.21.15.70:3300/0 conn(0x7f2f8815e860 0x7f2f880cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f86a00640 1 -- 172.21.15.3:0/2049244639 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3803624548 0 0) 0x7f2f880cee40 con 0x7f2f880cfbf0 2024-09-10T07:51:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f86a00640 1 -- 172.21.15.3:0/2049244639 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f6c003520 con 0x7f2f880cfbf0 2024-09-10T07:51:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f87e00640 1 -- 172.21.15.3:0/2049244639 >> v1:172.21.15.70:6789/0 conn(0x7f2f88073150 legacy=0x7f2f880b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f87e00640 1 -- 172.21.15.3:0/2049244639 >> v1:172.21.15.3:6789/0 conn(0x7f2f880cfbf0 legacy=0x7f2f880d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.233+0000 7f2f87e00640 1 -- 172.21.15.3:0/2049244639 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f880d3410 con 0x7f2f8815e860 2024-09-10T07:51:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.234+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2049244639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8815e860 0x7f2f880cf6b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2f780097a0 tx=0x7f2f7802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318e744738aea7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f86a00640 1 -- 172.21.15.3:0/2049244639 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78037070 con 0x7f2f8815e860 2024-09-10T07:51:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f86a00640 1 -- 172.21.15.3:0/2049244639 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f78032440 con 0x7f2f8815e860 2024-09-10T07:51:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f86a00640 1 -- 172.21.15.3:0/2049244639 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78032810 con 0x7f2f8815e860 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 -- 172.21.15.3:0/2049244639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8815e860 msgr2=0x7f2f880cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2049244639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8815e860 0x7f2f880cf6b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2f780097a0 tx=0x7f2f7802a230 comp rx=0 tx=0).stop 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 -- 172.21.15.3:0/2049244639 shutdown_connections 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2049244639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8815e860 0x7f2f880cf6b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 -- 172.21.15.3:0/2049244639 >> 172.21.15.3:0/2049244639 conn(0x7f2f880d4650 msgr2=0x7f2f880d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 -- 172.21.15.3:0/2049244639 shutdown_connections 2024-09-10T07:51:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.235+0000 7f2f8da00640 1 -- 172.21.15.3:0/2049244639 wait complete. 2024-09-10T07:51:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.237+0000 7f2f8da00640 1 Processor -- start 2024-09-10T07:51:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.237+0000 7f2f8da00640 1 -- start start 2024-09-10T07:51:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 0x7f2f880ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 0x7f2f8815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f880d2460 con 0x7f2f880cfbf0 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f880cf400 con 0x7f2f88073150 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8da00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f880b7f90 con 0x7f2f8815e860 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f87400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f87400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58610/0 (socket says 172.21.15.3:58610) 2024-09-10T07:51:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f87400640 1 -- 172.21.15.3:0/2570170522 learned_addr learned my addr 172.21.15.3:0/2570170522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 0x7f2f880ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.238+0000 7f2f8d000640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 0x7f2f8815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 -- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 msgr2=0x7f2f8815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 0x7f2f8815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 -- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 msgr2=0x7f2f880ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 0x7f2f880ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 -- 172.21.15.3:0/2570170522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f88159680 con 0x7f2f880cfbf0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 0x7f2f880ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f8d000640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 0x7f2f8815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f87400640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c500 tx=0x7f2f7400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74017070 con 0x7f2f880cfbf0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f74005ce0 con 0x7f2f880cfbf0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74007330 con 0x7f2f880cfbf0 2024-09-10T07:51:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f88159960 con 0x7f2f880cfbf0 2024-09-10T07:51:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.239+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f88159f20 con 0x7f2f880cfbf0 2024-09-10T07:51:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.240+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f880d4a30 con 0x7f2f880cfbf0 2024-09-10T07:51:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.241+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f7401a070 con 0x7f2f880cfbf0 2024-09-10T07:51:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.242+0000 7f2f7fe00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.242+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7408a320 con 0x7f2f880cfbf0 2024-09-10T07:51:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.242+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.243+0000 7f2f87e00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 0x7f2f6806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f78034000 tx=0x7f2f7802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.247+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f880d4a30 con 0x7f2f880cfbf0 2024-09-10T07:51:13.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.432+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f880b5000 con 0x7f2f880cfbf0 2024-09-10T07:51:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.433+0000 7f2f7fe00640 1 -- 172.21.15.3:0/2570170522 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16609 (secure 0 0 0) 0x7f2f74058d80 con 0x7f2f880cfbf0 2024-09-10T07:51:13.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:13.433 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:12.071234+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:09.867983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T07:51:11.065097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 msgr2=0x7f2f6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 0x7f2f6806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f78034000 tx=0x7f2f7802e000 comp rx=0 tx=0).stop 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 msgr2=0x7f2f880cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c500 tx=0x7f2f7400c9c0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 shutdown_connections 2024-09-10T07:51:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8815e860 0x7f2f8815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f880cfbf0 0x7f2f880cb720 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 --2- 172.21.15.3:0/2570170522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f88073150 0x7f2f880ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 >> 172.21.15.3:0/2570170522 conn(0x7f2f880d4650 msgr2=0x7f2f880b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 shutdown_connections 2024-09-10T07:51:13.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.435+0000 7f2f8da00640 1 -- 172.21.15.3:0/2570170522 wait complete. 2024-09-10T07:51:13.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:51:13.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.611+0000 7f7531200640 1 Processor -- start 2024-09-10T07:51:13.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.611+0000 7f7531200640 1 -- start start 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.612+0000 7f7531200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f752c072aa0 0x7f752c1692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.612+0000 7f7531200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c15faf0 con 0x7f752c0aefb0 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.612+0000 7f7531200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c15ed30 con 0x7f752c147ce0 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.612+0000 7f7531200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f752c15eaf0 con 0x7f752c072aa0 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f752c072aa0 0x7f752c1692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f752c072aa0 0x7f752c1692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58622/0 (socket says 172.21.15.3:58622) 2024-09-10T07:51:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 -- 172.21.15.3:0/271479521 learned_addr learned my addr 172.21.15.3:0/271479521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 -- 172.21.15.3:0/271479521 >> v1:172.21.15.70:6789/0 conn(0x7f752c147ce0 legacy=0x7f752c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 -- 172.21.15.3:0/271479521 >> v1:172.21.15.3:6789/0 conn(0x7f752c0aefb0 legacy=0x7f752c072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.613+0000 7f752a800640 1 -- 172.21.15.3:0/271479521 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c15f010 con 0x7f752c072aa0 2024-09-10T07:51:13.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.614+0000 7f752a800640 1 --2- 172.21.15.3:0/271479521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c072aa0 0x7f752c1692b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e50 tx=0x7f752402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4c303ffeba897b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.614+0000 7f7528a00640 1 -- 172.21.15.3:0/271479521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524036070 con 0x7f752c072aa0 2024-09-10T07:51:13.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.614+0000 7f7528a00640 1 -- 172.21.15.3:0/271479521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524031600 con 0x7f752c072aa0 2024-09-10T07:51:13.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.614+0000 7f7528a00640 1 -- 172.21.15.3:0/271479521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240319d0 con 0x7f752c072aa0 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 -- 172.21.15.3:0/271479521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c072aa0 msgr2=0x7f752c1692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 --2- 172.21.15.3:0/271479521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c072aa0 0x7f752c1692b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e50 tx=0x7f752402a6c0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 -- 172.21.15.3:0/271479521 shutdown_connections 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 --2- 172.21.15.3:0/271479521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c072aa0 0x7f752c1692b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 -- 172.21.15.3:0/271479521 >> 172.21.15.3:0/271479521 conn(0x7f752c0b5cb0 msgr2=0x7f752c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 -- 172.21.15.3:0/271479521 shutdown_connections 2024-09-10T07:51:13.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.615+0000 7f7531200640 1 -- 172.21.15.3:0/271479521 wait complete. 2024-09-10T07:51:13.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.618+0000 7f7531200640 1 Processor -- start 2024-09-10T07:51:13.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.618+0000 7f7531200640 1 -- start start 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f752c072aa0 0x7f752c0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f752c147ce0 0x7f752c1412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f752c15f660 con 0x7f752c0aefb0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f752c0730c0 con 0x7f752c147ce0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.619+0000 7f7531200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f752c1480c0 con 0x7f752c072aa0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58628/0 (socket says 172.21.15.3:58628) 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 -- 172.21.15.3:0/3056615799 learned_addr learned my addr 172.21.15.3:0/3056615799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529e00640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f752c072aa0 0x7f752c0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f752a800640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f752c147ce0 0x7f752c1412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 -- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f752c072aa0 msgr2=0x7f752c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f752c072aa0 0x7f752c0b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 -- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f752c147ce0 msgr2=0x7f752c1412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f752c147ce0 0x7f752c1412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 -- 172.21.15.3:0/3056615799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0b1680 con 0x7f752c0aefb0 2024-09-10T07:51:13.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.620+0000 7f7529400640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f751800c780 tx=0x7f751800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.621+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518017070 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.621+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7518005270 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.621+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751800a640 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.621+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c0b1960 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.621+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c0b1f20 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.622+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c06f130 con 0x7f752c0aefb0 2024-09-10T07:51:13.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.623+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f751801a070 con 0x7f752c0aefb0 2024-09-10T07:51:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.623+0000 7f7522000640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 0x7f750c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.624+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f751808a3e0 con 0x7f752c0aefb0 2024-09-10T07:51:13.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.624+0000 7f7529e00640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 0x7f750c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:13.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.624+0000 7f7529e00640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 0x7f750c06e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f751c001640 tx=0x7f751c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:13.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.628+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7518058cf0 con 0x7f752c0aefb0 2024-09-10T07:51:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:13.851+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f752c161330 con 0x7f752c0aefb0 2024-09-10T07:51:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.658+0000 7f7522000640 1 -- 172.21.15.3:0/3056615799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) ==== 118+0+0 (secure 0 0 0) 0x7f75180585f0 con 0x7f752c0aefb0 2024-09-10T07:51:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.661+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 msgr2=0x7f750c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.661+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 0x7f750c06e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f751c001640 tx=0x7f751c009040 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.661+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 msgr2=0x7f752c13f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.661+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f751800c780 tx=0x7f751800cc40 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 shutdown_connections 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f750c06be60 0x7f750c06e320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f752c147ce0 0x7f752c1412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f752c0aefb0 0x7f752c13f610 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 --2- 172.21.15.3:0/3056615799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f752c072aa0 0x7f752c0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 >> 172.21.15.3:0/3056615799 conn(0x7f752c0b5cb0 msgr2=0x7f752c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 shutdown_connections 2024-09-10T07:51:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.662+0000 7f7531200640 1 -- 172.21.15.3:0/3056615799 wait complete. 2024-09-10T07:51:14.680 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:51:14.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:51:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.844+0000 7f4a50c00640 1 Processor -- start 2024-09-10T07:51:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.844+0000 7f4a50c00640 1 -- start start 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a50c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 0x7f4a4c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a50c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 0x7f4a4c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a50c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c15eff0 con 0x7f4a4c15b230 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a50c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c15fbe0 con 0x7f4a4c0b4c90 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a50c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c15c8d0 con 0x7f4a4c072c50 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.845+0000 7f4a49e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4a4c15b230 0x7f4a4c0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35116/0 (socket says 172.21.15.3:35116) 2024-09-10T07:51:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a49e00640 1 -- 172.21.15.3:0/1977192166 learned_addr learned my addr 172.21.15.3:0/1977192166 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:14.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a4a800640 1 --2- 172.21.15.3:0/1977192166 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 0x7f4a4c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a49400640 1 --2- 172.21.15.3:0/1977192166 >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 0x7f4a4c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a48a00640 1 -- 172.21.15.3:0/1977192166 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801901142 0 0) 0x7f4a4c15eff0 con 0x7f4a4c15b230 2024-09-10T07:51:14.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a48a00640 1 -- 172.21.15.3:0/1977192166 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a30003520 con 0x7f4a4c15b230 2024-09-10T07:51:14.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a4a800640 1 -- 172.21.15.3:0/1977192166 >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 msgr2=0x7f4a4c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:14.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a4a800640 1 --2- 172.21.15.3:0/1977192166 >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 0x7f4a4c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a4a800640 1 -- 172.21.15.3:0/1977192166 >> v1:172.21.15.3:6789/0 conn(0x7f4a4c15b230 legacy=0x7f4a4c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a4a800640 1 -- 172.21.15.3:0/1977192166 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c15f2b0 con 0x7f4a4c072c50 2024-09-10T07:51:14.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.846+0000 7f4a49400640 1 --2- 172.21.15.3:0/1977192166 >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 0x7f4a4c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.847+0000 7f4a4a800640 1 --2- 172.21.15.3:0/1977192166 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 0x7f4a4c071010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009d90 tx=0x7f4a4402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f593048937f6d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a48a00640 1 -- 172.21.15.3:0/1977192166 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44007740 con 0x7f4a4c072c50 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a48a00640 1 -- 172.21.15.3:0/1977192166 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a440078e0 con 0x7f4a4c072c50 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a48a00640 1 -- 172.21.15.3:0/1977192166 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44007cb0 con 0x7f4a4c072c50 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 -- 172.21.15.3:0/1977192166 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 msgr2=0x7f4a4c071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 --2- 172.21.15.3:0/1977192166 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 0x7f4a4c071010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009d90 tx=0x7f4a4402a660 comp rx=0 tx=0).stop 2024-09-10T07:51:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 -- 172.21.15.3:0/1977192166 shutdown_connections 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 --2- 172.21.15.3:0/1977192166 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c072c50 0x7f4a4c071010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 --2- 172.21.15.3:0/1977192166 >> v2:172.21.15.70:3300/0 conn(0x7f4a4c0b4c90 0x7f4a4c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 -- 172.21.15.3:0/1977192166 >> 172.21.15.3:0/1977192166 conn(0x7f4a4c0b5cb0 msgr2=0x7f4a4c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 -- 172.21.15.3:0/1977192166 shutdown_connections 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.848+0000 7f4a50c00640 1 -- 172.21.15.3:0/1977192166 wait complete. 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.850+0000 7f4a50c00640 1 Processor -- start 2024-09-10T07:51:14.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.850+0000 7f4a50c00640 1 -- start start 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a4c0b4c90 0x7f4a4c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c15b230 0x7f4a4c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15b610 con 0x7f4a4c072c50 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15f870 con 0x7f4a4c0b4c90 2024-09-10T07:51:14.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a50c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c15b790 con 0x7f4a4c15b230 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a49e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a49e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55442/0 (socket says 172.21.15.3:55442) 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a49e00640 1 -- 172.21.15.3:0/2358567737 learned_addr learned my addr 172.21.15.3:0/2358567737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a4a800640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c15b230 0x7f4a4c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.851+0000 7f4a49400640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a4c0b4c90 0x7f4a4c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 -- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c15b230 msgr2=0x7f4a4c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c15b230 0x7f4a4c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 -- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a4c0b4c90 msgr2=0x7f4a4c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a4c0b4c90 0x7f4a4c0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 -- 172.21.15.3:0/2358567737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c0b0460 con 0x7f4a4c072c50 2024-09-10T07:51:14.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a49e00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c390 tx=0x7f4a3c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:14.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c017070 con 0x7f4a4c072c50 2024-09-10T07:51:14.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3c005ce0 con 0x7f4a4c072c50 2024-09-10T07:51:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0072f0 con 0x7f4a4c072c50 2024-09-10T07:51:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c0b0740 con 0x7f4a4c072c50 2024-09-10T07:51:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.852+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c0b0d00 con 0x7f4a4c072c50 2024-09-10T07:51:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.853+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c06e1e0 con 0x7f4a4c072c50 2024-09-10T07:51:14.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.854+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a3c01a070 con 0x7f4a4c072c50 2024-09-10T07:51:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.855+0000 7f4a42000640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.855+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f4a3c08a250 con 0x7f4a4c072c50 2024-09-10T07:51:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.856+0000 7f4a49400640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:14.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.856+0000 7f4a49400640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c1524e0 tx=0x7f4a38009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:14.860+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a3c058510 con 0x7f4a4c072c50 2024-09-10T07:51:15.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.079+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4a4c155640 con 0x7f4a4c072c50 2024-09-10T07:51:15.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.664+0000 7f4a42000640 1 -- 172.21.15.3:0/2358567737 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) ==== 122+0+0 (secure 0 0 0) 0x7f4a3c05c810 con 0x7f4a4c072c50 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 msgr2=0x7f4a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c1524e0 tx=0x7f4a38009190 comp rx=0 tx=0).stop 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 msgr2=0x7f4a4c1ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c390 tx=0x7f4a3c00c850 comp rx=0 tx=0).stop 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 shutdown_connections 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a4c15b230 0x7f4a4c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a4c0b4c90 0x7f4a4c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 --2- 172.21.15.3:0/2358567737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c072c50 0x7f4a4c1ee220 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 >> 172.21.15.3:0/2358567737 conn(0x7f4a4c0b5cb0 msgr2=0x7f4a4c15e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.666+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 shutdown_connections 2024-09-10T07:51:15.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.667+0000 7f4a50c00640 1 -- 172.21.15.3:0/2358567737 wait complete. 2024-09-10T07:51:15.682 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:51:15.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:15.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.841+0000 7f7f5aa00640 1 Processor -- start 2024-09-10T07:51:15.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.842+0000 7f7f5aa00640 1 -- start start 2024-09-10T07:51:15.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f5aa00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f5415b4f0 0x7f7f540d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f5aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f5aa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f540ccd80 con 0x7f7f5415b110 2024-09-10T07:51:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f5aa00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7f540b44f0 con 0x7f7f5415b4f0 2024-09-10T07:51:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f5aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7f540cd060 con 0x7f7f54073d10 2024-09-10T07:51:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f53e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7f5415b110 0x7f7f540b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35124/0 (socket says 172.21.15.3:35124) 2024-09-10T07:51:15.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f53e00640 1 -- 172.21.15.3:0/914610337 learned_addr learned my addr 172.21.15.3:0/914610337 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f4b400640 1 --2- 172.21.15.3:0/914610337 >> v2:172.21.15.70:3300/0 conn(0x7f7f5415b4f0 0x7f7f540d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.843+0000 7f7f58a00640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f53400640 1 -- 172.21.15.3:0/914610337 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814047769 0 0) 0x7f7f540ccd80 con 0x7f7f5415b110 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f53400640 1 -- 172.21.15.3:0/914610337 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f38003520 con 0x7f7f5415b110 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f4b400640 1 -- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 msgr2=0x7f7f540d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f4b400640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f4b400640 1 -- 172.21.15.3:0/914610337 >> v1:172.21.15.3:6789/0 conn(0x7f7f5415b110 legacy=0x7f7f540b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f4b400640 1 -- 172.21.15.3:0/914610337 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f540cd340 con 0x7f7f5415b4f0 2024-09-10T07:51:15.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.844+0000 7f7f58a00640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:15.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.845+0000 7f7f4b400640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f540d01e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097a0 tx=0x7f7f3c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc72c4adb3ef9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.845+0000 7f7f53400640 1 -- 172.21.15.3:0/914610337 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c037070 con 0x7f7f5415b4f0 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.845+0000 7f7f53400640 1 -- 172.21.15.3:0/914610337 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c02ae50 con 0x7f7f5415b4f0 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.845+0000 7f7f53400640 1 -- 172.21.15.3:0/914610337 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c032660 con 0x7f7f5415b4f0 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 -- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 msgr2=0x7f7f540d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f540d01e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097a0 tx=0x7f7f3c02a100 comp rx=0 tx=0).stop 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 -- 172.21.15.3:0/914610337 shutdown_connections 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/914610337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f540d01e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 -- 172.21.15.3:0/914610337 >> 172.21.15.3:0/914610337 conn(0x7f7f5406d000 msgr2=0x7f7f5406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 -- 172.21.15.3:0/914610337 shutdown_connections 2024-09-10T07:51:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.846+0000 7f7f5aa00640 1 -- 172.21.15.3:0/914610337 wait complete. 2024-09-10T07:51:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.848+0000 7f7f5aa00640 1 Processor -- start 2024-09-10T07:51:15.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.848+0000 7f7f5aa00640 1 -- start start 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f5415df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5406bca0 con 0x7f7f5415b110 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5406bb20 con 0x7f7f5415b4f0 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f5aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5406be20 con 0x7f7f54073d10 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55452/0 (socket says 172.21.15.3:55452) 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 -- 172.21.15.3:0/3422587100 learned_addr learned my addr 172.21.15.3:0/3422587100 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f53e00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f58a00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f5415df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 -- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 msgr2=0x7f7f540b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 -- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 msgr2=0x7f7f5415df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f5415df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.849+0000 7f7f4b400640 1 -- 172.21.15.3:0/3422587100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5415ba70 con 0x7f7f5415b110 2024-09-10T07:51:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f4b400640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c035040 tx=0x7f7f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c041070 con 0x7f7f5415b110 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c03c460 con 0x7f7f5415b110 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c037040 con 0x7f7f5415b110 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5415bd50 con 0x7f7f5415b110 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.850+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5415c310 con 0x7f7f5415b110 2024-09-10T07:51:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.851+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f540b3d10 con 0x7f7f5415b110 2024-09-10T07:51:15.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.852+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f3c03a070 con 0x7f7f5415b110 2024-09-10T07:51:15.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.853+0000 7f7f50c00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 0x7f7f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.853+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7f3c0aae50 con 0x7f7f5415b110 2024-09-10T07:51:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.853+0000 7f7f53e00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 0x7f7f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:15.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.854+0000 7f7f53e00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 0x7f7f3006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005880 tx=0x7f7f44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:15.857+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f3c0791a0 con 0x7f7f5415b110 2024-09-10T07:51:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.079+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f540b3490 con 0x7f7f5415b110 2024-09-10T07:51:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.080+0000 7f7f50c00640 1 -- 172.21.15.3:0/3422587100 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5219 (secure 0 0 0) 0x7f7f3c07d4a0 con 0x7f7f5415b110 2024-09-10T07:51:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:16.080 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"btime":"2024-09-10T07:51:15:660073+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6355,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:15.081436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702]},"id":5}]} 2024-09-10T07:51:16.080 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2024-09-10T07:51:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 msgr2=0x7f7f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 0x7f7f3006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005880 tx=0x7f7f44009040 comp rx=0 tx=0).stop 2024-09-10T07:51:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 msgr2=0x7f7f5415da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c035040 tx=0x7f7f3c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 shutdown_connections 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7f3006be30 0x7f7f3006e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5415b4f0 0x7f7f5415df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f5415b110 0x7f7f5415da10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 --2- 172.21.15.3:0/3422587100 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f54073d10 0x7f7f540b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.082+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 >> 172.21.15.3:0/3422587100 conn(0x7f7f5406d000 msgr2=0x7f7f5406c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.083+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 shutdown_connections 2024-09-10T07:51:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.083+0000 7f7f5aa00640 1 -- 172.21.15.3:0/3422587100 wait complete. 2024-09-10T07:51:16.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.254+0000 7f0e32a00640 1 Processor -- start 2024-09-10T07:51:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.254+0000 7f0e32a00640 1 -- start start 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.255+0000 7f0e32a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e2c15b4f0 0x7f0e2c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.255+0000 7f0e32a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e2c073d10 0x7f0e2c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.255+0000 7f0e32a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2c0ccd80 con 0x7f0e2c15b110 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.255+0000 7f0e32a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2c0b44f0 con 0x7f0e2c073d10 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e32a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2c0cd060 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e2c073d10 0x7f0e2c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e2c073d10 0x7f0e2c0d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55458/0 (socket says 172.21.15.3:55458) 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 -- 172.21.15.3:0/2856183428 learned_addr learned my addr 172.21.15.3:0/2856183428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e23400640 1 --2- 172.21.15.3:0/2856183428 >> v2:172.21.15.70:3300/0 conn(0x7f0e2c15b4f0 0x7f0e2c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 -- 172.21.15.3:0/2856183428 >> v2:172.21.15.70:3300/0 conn(0x7f0e2c15b4f0 msgr2=0x7f0e2c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 --2- 172.21.15.3:0/2856183428 >> v2:172.21.15.70:3300/0 conn(0x7f0e2c15b4f0 0x7f0e2c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 -- 172.21.15.3:0/2856183428 >> v1:172.21.15.70:6789/0 conn(0x7f0e2c15b110 legacy=0x7f0e2c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.256+0000 7f0e30a00640 1 -- 172.21.15.3:0/2856183428 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c0cd340 con 0x7f0e2c073d10 2024-09-10T07:51:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.257+0000 7f0e30a00640 1 --2- 172.21.15.3:0/2856183428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c073d10 0x7f0e2c0d0720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0d1310 tx=0x7f0e2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5915c65de50b583 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.257+0000 7f0e2b400640 1 -- 172.21.15.3:0/2856183428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24036070 con 0x7f0e2c073d10 2024-09-10T07:51:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e2b400640 1 -- 172.21.15.3:0/2856183428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24031600 con 0x7f0e2c073d10 2024-09-10T07:51:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e2b400640 1 -- 172.21.15.3:0/2856183428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240319d0 con 0x7f0e2c073d10 2024-09-10T07:51:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 -- 172.21.15.3:0/2856183428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c073d10 msgr2=0x7f0e2c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 --2- 172.21.15.3:0/2856183428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c073d10 0x7f0e2c0d0720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0d1310 tx=0x7f0e2402a6f0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 -- 172.21.15.3:0/2856183428 shutdown_connections 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 --2- 172.21.15.3:0/2856183428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c073d10 0x7f0e2c0d0720 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 --2- 172.21.15.3:0/2856183428 >> v2:172.21.15.70:3300/0 conn(0x7f0e2c15b4f0 0x7f0e2c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 -- 172.21.15.3:0/2856183428 >> 172.21.15.3:0/2856183428 conn(0x7f0e2c06d000 msgr2=0x7f0e2c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 -- 172.21.15.3:0/2856183428 shutdown_connections 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.258+0000 7f0e32a00640 1 -- 172.21.15.3:0/2856183428 wait complete. 2024-09-10T07:51:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.260+0000 7f0e32a00640 1 Processor -- start 2024-09-10T07:51:16.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.260+0000 7f0e32a00640 1 -- start start 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e2c073d10 0x7f0e2c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c15b110 0x7f0e2c0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c06bca0 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c06bb20 con 0x7f0e2c15b110 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e32a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2c06be20 con 0x7f0e2c073d10 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e30a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e30a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55462/0 (socket says 172.21.15.3:55462) 2024-09-10T07:51:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e30a00640 1 -- 172.21.15.3:0/1889093307 learned_addr learned my addr 172.21.15.3:0/1889093307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e23400640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c15b110 0x7f0e2c0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e2be00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e2c073d10 0x7f0e2c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e30a00640 1 -- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e2c073d10 msgr2=0x7f0e2c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.261+0000 7f0e30a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e2c073d10 0x7f0e2c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e30a00640 1 -- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c15b110 msgr2=0x7f0e2c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e30a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c15b110 0x7f0e2c0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e30a00640 1 -- 172.21.15.3:0/1889093307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c072c10 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e30a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402ace0 tx=0x7f0e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24036050 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24042070 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c15c970 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24009510 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.262+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c15cf30 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.263+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2c0b3d10 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.264+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e24007bd0 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.265+0000 7f0e28c00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 0x7f0e0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.265+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f0e240abc00 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.265+0000 7f0e2be00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 0x7f0e0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.266+0000 7f0e2be00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 0x7f0e0806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c153010 tx=0x7f0e1c007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.269+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e24079f50 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.454+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e2c1562a0 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.455+0000 7f0e28c00640 1 -- 172.21.15.3:0/1889093307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16596 (secure 0 0 0) 0x7f0e2407e250 con 0x7f0e2c15b4f0 2024-09-10T07:51:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:16.455 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":78,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:13.646589+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:09.867983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T07:51:11.065097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:16.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.457+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 msgr2=0x7f0e0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.457+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 0x7f0e0806e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c153010 tx=0x7f0e1c007e30 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.457+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 msgr2=0x7f0e2c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.457+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402ace0 tx=0x7f0e24004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 shutdown_connections 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0e0806be30 0x7f0e0806e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e2c15b4f0 0x7f0e2c1526e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e2c15b110 0x7f0e2c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 --2- 172.21.15.3:0/1889093307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e2c073d10 0x7f0e2c15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 >> 172.21.15.3:0/1889093307 conn(0x7f0e2c06d000 msgr2=0x7f0e2c0b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 shutdown_connections 2024-09-10T07:51:16.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.458+0000 7f0e32a00640 1 -- 172.21.15.3:0/1889093307 wait complete. 2024-09-10T07:51:16.473 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:51:16.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:51:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.630+0000 7f316ea00640 1 Processor -- start 2024-09-10T07:51:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.630+0000 7f316ea00640 1 -- start start 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.631+0000 7f316ea00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31680b37a0 0x7f3168162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.631+0000 7f316ea00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31680b70b0 con 0x7f31680d1c10 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.631+0000 7f316ea00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31680b78b0 con 0x7f316815e970 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.631+0000 7f316ea00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3168163210 con 0x7f31680b37a0 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f315f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f31680d1c10 0x7f31680ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35128/0 (socket says 172.21.15.3:35128) 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f315f400640 1 -- 172.21.15.3:0/698242124 learned_addr learned my addr 172.21.15.3:0/698242124 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f316ca00640 1 --2- 172.21.15.3:0/698242124 >> v2:172.21.15.70:3300/0 conn(0x7f31680b37a0 0x7f3168162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f3167400640 1 -- 172.21.15.3:0/698242124 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045006408 0 0) 0x7f3150001110 con 0x7f31680d1c10 2024-09-10T07:51:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f3167400640 1 -- 172.21.15.3:0/698242124 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f314c003520 con 0x7f31680d1c10 2024-09-10T07:51:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f316ca00640 1 -- 172.21.15.3:0/698242124 >> v1:172.21.15.70:6789/0 conn(0x7f316815e970 legacy=0x7f31680d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f316ca00640 1 -- 172.21.15.3:0/698242124 >> v1:172.21.15.3:6789/0 conn(0x7f31680d1c10 legacy=0x7f31680ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.632+0000 7f316ca00640 1 -- 172.21.15.3:0/698242124 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31680b62f0 con 0x7f31680b37a0 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.633+0000 7f316ca00640 1 --2- 172.21.15.3:0/698242124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f3168162cd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be811a2524b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f3167400640 1 -- 172.21.15.3:0/698242124 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160007740 con 0x7f31680b37a0 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f3167400640 1 -- 172.21.15.3:0/698242124 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31600078e0 con 0x7f31680b37a0 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f3167400640 1 -- 172.21.15.3:0/698242124 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160007c90 con 0x7f31680b37a0 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 -- 172.21.15.3:0/698242124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 msgr2=0x7f3168162cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 --2- 172.21.15.3:0/698242124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f3168162cd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002a6f0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 -- 172.21.15.3:0/698242124 shutdown_connections 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 --2- 172.21.15.3:0/698242124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f3168162cd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 -- 172.21.15.3:0/698242124 >> 172.21.15.3:0/698242124 conn(0x7f316806bd00 msgr2=0x7f316806c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 -- 172.21.15.3:0/698242124 shutdown_connections 2024-09-10T07:51:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.634+0000 7f316ea00640 1 -- 172.21.15.3:0/698242124 wait complete. 2024-09-10T07:51:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.636+0000 7f316ea00640 1 Processor -- start 2024-09-10T07:51:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.636+0000 7f316ea00640 1 -- start start 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f316815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f316815e970 0x7f31680ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31680b68b0 con 0x7f31680d1c10 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f31680b3bb0 con 0x7f31680b37a0 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f316ea00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f31680b6b90 con 0x7f316815e970 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f315f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f315f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55468/0 (socket says 172.21.15.3:55468) 2024-09-10T07:51:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.637+0000 7f315f400640 1 -- 172.21.15.3:0/596831819 learned_addr learned my addr 172.21.15.3:0/596831819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f3167e00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f316815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f316ca00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f316815e970 0x7f31680ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 -- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f316815e970 msgr2=0x7f31680ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f316815e970 0x7f31680ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 -- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 msgr2=0x7f316815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f316815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 -- 172.21.15.3:0/596831819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316815b9f0 con 0x7f31680d1c10 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f315f400640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f315000c730 tx=0x7f315000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150017070 con 0x7f31680d1c10 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150005ab0 con 0x7f31680d1c10 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150007070 con 0x7f31680d1c10 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.638+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3168160450 con 0x7f31680d1c10 2024-09-10T07:51:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.639+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31681609d0 con 0x7f31680d1c10 2024-09-10T07:51:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.640+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3168162cd0 con 0x7f31680d1c10 2024-09-10T07:51:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.640+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f315001a070 con 0x7f31680d1c10 2024-09-10T07:51:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.641+0000 7f3164c00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.641+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f315008a2c0 con 0x7f31680d1c10 2024-09-10T07:51:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.642+0000 7f3167e00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.642+0000 7f3167e00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 0x7f314406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3158000f60 tx=0x7f3158009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.646+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f316815b9f0 con 0x7f31680d1c10 2024-09-10T07:51:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.826+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31680b81d0 con 0x7f314406be30 2024-09-10T07:51:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.833+0000 7f3164c00640 1 -- 172.21.15.3:0/596831819 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245007 (secure 0 0 0) 0x7f31680b81d0 con 0x7f314406be30 2024-09-10T07:51:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:16.837 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 msgr2=0x7f314406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 0x7f314406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3158000f60 tx=0x7f3158009040 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 msgr2=0x7f31680ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f315000c730 tx=0x7f315000cbf0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 shutdown_connections 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f316815e970 0x7f31680ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31680d1c10 0x7f31680ca4a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 --2- 172.21.15.3:0/596831819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680b37a0 0x7f316815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:16.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 >> 172.21.15.3:0/596831819 conn(0x7f316806bd00 msgr2=0x7f31680b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 shutdown_connections 2024-09-10T07:51:16.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:16.837+0000 7f316ea00640 1 -- 172.21.15.3:0/596831819 wait complete. 2024-09-10T07:51:16.853 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T07:51:15.553366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":261860,"kb_used_data":8176,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712860,"statfs":{"total":773094113280,"available":772825968640,"internally_reserved":0,"allocated":8372224,"data_stored":4231792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.924334"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639913+0000","last_change":"2024-09-10T07:51:11.639913+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639913+0000","last_undegraded":"2024-09-10T07:51:11.639913+0000","last_fullsized":"2024-09-10T07:51:11.639913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651835+0000","last_change":"2024-09-10T07:51:11.651835+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651835+0000","last_undegraded":"2024-09-10T07:51:11.651835+0000","last_fullsized":"2024-09-10T07:51:11.651835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.646868+0000","last_change":"2024-09-10T07:51:11.646868+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.646868+0000","last_undegraded":"2024-09-10T07:51:11.646868+0000","last_fullsized":"2024-09-10T07:51:11.646868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.654231+0000","last_change":"2024-09-10T07:51:11.654231+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.654231+0000","last_undegraded":"2024-09-10T07:51:11.654231+0000","last_fullsized":"2024-09-10T07:51:11.654231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639753+0000","last_change":"2024-09-10T07:51:11.639753+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639753+0000","last_undegraded":"2024-09-10T07:51:11.639753+0000","last_fullsized":"2024-09-10T07:51:11.639753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653980+0000","last_change":"2024-09-10T07:51:11.653980+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653980+0000","last_undegraded":"2024-09-10T07:51:11.653980+0000","last_fullsized":"2024-09-10T07:51:11.653980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640234+0000","last_change":"2024-09-10T07:51:11.640234+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640234+0000","last_undegraded":"2024-09-10T07:51:11.640234+0000","last_fullsized":"2024-09-10T07:51:11.640234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.654324+0000","last_change":"2024-09-10T07:51:12.647897+0000","last_active":"2024-09-10T07:51:12.654324+0000","last_peered":"2024-09-10T07:51:12.654324+0000","last_clean":"2024-09-10T07:51:12.654324+0000","last_became_active":"2024-09-10T07:51:12.647493+0000","last_became_peered":"2024-09-10T07:51:12.647493+0000","last_unstale":"2024-09-10T07:51:12.654324+0000","last_undegraded":"2024-09-10T07:51:12.654324+0000","last_fullsized":"2024-09-10T07:51:12.654324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:54.143299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.677787+0000","last_change":"2024-09-10T07:51:12.676376+0000","last_active":"2024-09-10T07:51:12.677787+0000","last_peered":"2024-09-10T07:51:12.677787+0000","last_clean":"2024-09-10T07:51:12.677787+0000","last_became_active":"2024-09-10T07:51:12.676068+0000","last_became_peered":"2024-09-10T07:51:12.676068+0000","last_unstale":"2024-09-10T07:51:12.677787+0000","last_undegraded":"2024-09-10T07:51:12.677787+0000","last_fullsized":"2024-09-10T07:51:12.677787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:03.847359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.664304+0000","last_change":"2024-09-10T07:51:12.659238+0000","last_active":"2024-09-10T07:51:12.664304+0000","last_peered":"2024-09-10T07:51:12.664304+0000","last_clean":"2024-09-10T07:51:12.664304+0000","last_became_active":"2024-09-10T07:51:12.658975+0000","last_became_peered":"2024-09-10T07:51:12.658975+0000","last_unstale":"2024-09-10T07:51:12.664304+0000","last_undegraded":"2024-09-10T07:51:12.664304+0000","last_fullsized":"2024-09-10T07:51:12.664304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:48.409488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639350+0000","last_change":"2024-09-10T07:51:11.639350+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639350+0000","last_undegraded":"2024-09-10T07:51:11.639350+0000","last_fullsized":"2024-09-10T07:51:11.639350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640694+0000","last_change":"2024-09-10T07:51:11.640694+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640694+0000","last_undegraded":"2024-09-10T07:51:11.640694+0000","last_fullsized":"2024-09-10T07:51:11.640694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653111+0000","last_change":"2024-09-10T07:51:11.653111+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653111+0000","last_undegraded":"2024-09-10T07:51:11.653111+0000","last_fullsized":"2024-09-10T07:51:11.653111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.646399+0000","last_change":"2024-09-10T07:51:11.646399+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.646399+0000","last_undegraded":"2024-09-10T07:51:11.646399+0000","last_fullsized":"2024-09-10T07:51:11.646399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651744+0000","last_change":"2024-09-10T07:51:11.651744+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651744+0000","last_undegraded":"2024-09-10T07:51:11.651744+0000","last_fullsized":"2024-09-10T07:51:11.651744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.638687+0000","last_change":"2024-09-10T07:51:11.638687+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.638687+0000","last_undegraded":"2024-09-10T07:51:11.638687+0000","last_fullsized":"2024-09-10T07:51:11.638687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650992+0000","last_change":"2024-09-10T07:51:11.650992+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650992+0000","last_undegraded":"2024-09-10T07:51:11.650992+0000","last_fullsized":"2024-09-10T07:51:11.650992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640344+0000","last_change":"2024-09-10T07:51:11.640344+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640344+0000","last_undegraded":"2024-09-10T07:51:11.640344+0000","last_fullsized":"2024-09-10T07:51:11.640344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648866+0000","last_change":"2024-09-10T07:51:11.648866+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648866+0000","last_undegraded":"2024-09-10T07:51:11.648866+0000","last_fullsized":"2024-09-10T07:51:11.648866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.642085+0000","last_change":"2024-09-10T07:51:11.642085+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.642085+0000","last_undegraded":"2024-09-10T07:51:11.642085+0000","last_fullsized":"2024-09-10T07:51:11.642085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648951+0000","last_change":"2024-09-10T07:51:11.648951+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648951+0000","last_undegraded":"2024-09-10T07:51:11.648951+0000","last_fullsized":"2024-09-10T07:51:11.648951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650214+0000","last_change":"2024-09-10T07:51:11.650214+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650214+0000","last_undegraded":"2024-09-10T07:51:11.650214+0000","last_fullsized":"2024-09-10T07:51:11.650214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639878+0000","last_change":"2024-09-10T07:51:11.639878+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639878+0000","last_undegraded":"2024-09-10T07:51:11.639878+0000","last_fullsized":"2024-09-10T07:51:11.639878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.655388+0000","last_change":"2024-09-10T07:51:11.647694+0000","last_active":"2024-09-10T07:51:11.655388+0000","last_peered":"2024-09-10T07:51:11.655388+0000","last_clean":"2024-09-10T07:51:11.655388+0000","last_became_active":"2024-09-10T07:51:11.647419+0000","last_became_peered":"2024-09-10T07:51:11.647419+0000","last_unstale":"2024-09-10T07:51:11.655388+0000","last_undegraded":"2024-09-10T07:51:11.655388+0000","last_fullsized":"2024-09-10T07:51:11.655388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:36.594271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638653+0000","last_change":"2024-09-10T07:51:10.638653+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638653+0000","last_undegraded":"2024-09-10T07:51:10.638653+0000","last_fullsized":"2024-09-10T07:51:10.638653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.645311+0000","last_change":"2024-09-10T07:51:11.645311+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.645311+0000","last_undegraded":"2024-09-10T07:51:11.645311+0000","last_fullsized":"2024-09-10T07:51:11.645311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.652725+0000","last_change":"2024-09-10T07:51:11.648632+0000","last_active":"2024-09-10T07:51:11.652725+0000","last_peered":"2024-09-10T07:51:11.652725+0000","last_clean":"2024-09-10T07:51:11.652725+0000","last_became_active":"2024-09-10T07:51:11.648267+0000","last_became_peered":"2024-09-10T07:51:11.648267+0000","last_unstale":"2024-09-10T07:51:11.652725+0000","last_undegraded":"2024-09-10T07:51:11.652725+0000","last_fullsized":"2024-09-10T07:51:11.652725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:44.793056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653487+0000","last_change":"2024-09-10T07:51:11.653487+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653487+0000","last_undegraded":"2024-09-10T07:51:11.653487+0000","last_fullsized":"2024-09-10T07:51:11.653487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665473+0000","last_change":"2024-09-10T07:51:11.661609+0000","last_active":"2024-09-10T07:51:11.665473+0000","last_peered":"2024-09-10T07:51:11.665473+0000","last_clean":"2024-09-10T07:51:11.665473+0000","last_became_active":"2024-09-10T07:51:11.660016+0000","last_became_peered":"2024-09-10T07:51:11.660016+0000","last_unstale":"2024-09-10T07:51:11.665473+0000","last_undegraded":"2024-09-10T07:51:11.665473+0000","last_fullsized":"2024-09-10T07:51:11.665473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:57.530612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.641553+0000","last_change":"2024-09-10T07:51:11.641553+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.641553+0000","last_undegraded":"2024-09-10T07:51:11.641553+0000","last_fullsized":"2024-09-10T07:51:11.641553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.671056+0000","last_change":"2024-09-10T07:51:11.654006+0000","last_active":"2024-09-10T07:51:12.671056+0000","last_peered":"2024-09-10T07:51:12.671056+0000","last_clean":"2024-09-10T07:51:12.671056+0000","last_became_active":"2024-09-10T07:51:11.651332+0000","last_became_peered":"2024-09-10T07:51:11.651332+0000","last_unstale":"2024-09-10T07:51:12.671056+0000","last_undegraded":"2024-09-10T07:51:12.671056+0000","last_fullsized":"2024-09-10T07:51:12.671056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:41.047040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648338+0000","last_change":"2024-09-10T07:51:11.648338+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648338+0000","last_undegraded":"2024-09-10T07:51:11.648338+0000","last_fullsized":"2024-09-10T07:51:11.648338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.670809+0000","last_change":"2024-09-10T07:51:11.666134+0000","last_active":"2024-09-10T07:51:11.670809+0000","last_peered":"2024-09-10T07:51:11.670809+0000","last_clean":"2024-09-10T07:51:11.670809+0000","last_became_active":"2024-09-10T07:51:11.665826+0000","last_became_peered":"2024-09-10T07:51:11.665826+0000","last_unstale":"2024-09-10T07:51:11.670809+0000","last_undegraded":"2024-09-10T07:51:11.670809+0000","last_fullsized":"2024-09-10T07:51:11.670809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:56.418338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632063+0000","last_change":"2024-09-10T07:51:10.632063+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632063+0000","last_undegraded":"2024-09-10T07:51:10.632063+0000","last_fullsized":"2024-09-10T07:51:10.632063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.669674+0000","last_change":"2024-09-10T07:51:12.668160+0000","last_active":"2024-09-10T07:51:12.669674+0000","last_peered":"2024-09-10T07:51:12.669674+0000","last_clean":"2024-09-10T07:51:12.669674+0000","last_became_active":"2024-09-10T07:51:12.667845+0000","last_became_peered":"2024-09-10T07:51:12.667845+0000","last_unstale":"2024-09-10T07:51:12.669674+0000","last_undegraded":"2024-09-10T07:51:12.669674+0000","last_fullsized":"2024-09-10T07:51:12.669674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:57.084096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642604+0000","last_change":"2024-09-10T07:51:11.647781+0000","last_active":"2024-09-10T07:51:12.642604+0000","last_peered":"2024-09-10T07:51:12.642604+0000","last_clean":"2024-09-10T07:51:12.642604+0000","last_became_active":"2024-09-10T07:51:11.647502+0000","last_became_peered":"2024-09-10T07:51:11.647502+0000","last_unstale":"2024-09-10T07:51:12.642604+0000","last_undegraded":"2024-09-10T07:51:12.642604+0000","last_fullsized":"2024-09-10T07:51:12.642604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:38.219907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651338+0000","last_change":"2024-09-10T07:51:11.651338+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651338+0000","last_undegraded":"2024-09-10T07:51:11.651338+0000","last_fullsized":"2024-09-10T07:51:11.651338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.655795+0000","last_change":"2024-09-10T07:51:11.652611+0000","last_active":"2024-09-10T07:51:11.655795+0000","last_peered":"2024-09-10T07:51:11.655795+0000","last_clean":"2024-09-10T07:51:11.655795+0000","last_became_active":"2024-09-10T07:51:11.652322+0000","last_became_peered":"2024-09-10T07:51:11.652322+0000","last_unstale":"2024-09-10T07:51:11.655795+0000","last_undegraded":"2024-09-10T07:51:11.655795+0000","last_fullsized":"2024-09-10T07:51:11.655795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:35.591184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650915+0000","last_change":"2024-09-10T07:51:11.650915+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650915+0000","last_undegraded":"2024-09-10T07:51:11.650915+0000","last_fullsized":"2024-09-10T07:51:11.650915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665507+0000","last_change":"2024-09-10T07:51:11.661453+0000","last_active":"2024-09-10T07:51:11.665507+0000","last_peered":"2024-09-10T07:51:11.665507+0000","last_clean":"2024-09-10T07:51:11.665507+0000","last_became_active":"2024-09-10T07:51:11.661202+0000","last_became_peered":"2024-09-10T07:51:11.661202+0000","last_unstale":"2024-09-10T07:51:11.665507+0000","last_undegraded":"2024-09-10T07:51:11.665507+0000","last_fullsized":"2024-09-10T07:51:11.665507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:30.240343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.642581+0000","last_change":"2024-09-10T07:51:11.642581+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.642581+0000","last_undegraded":"2024-09-10T07:51:11.642581+0000","last_fullsized":"2024-09-10T07:51:11.642581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.665387+0000","last_change":"2024-09-10T07:51:11.651919+0000","last_active":"2024-09-10T07:51:12.665387+0000","last_peered":"2024-09-10T07:51:12.665387+0000","last_clean":"2024-09-10T07:51:12.665387+0000","last_became_active":"2024-09-10T07:51:11.651676+0000","last_became_peered":"2024-09-10T07:51:11.651676+0000","last_unstale":"2024-09-10T07:51:12.665387+0000","last_undegraded":"2024-09-10T07:51:12.665387+0000","last_fullsized":"2024-09-10T07:51:12.665387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632070+0000","last_change":"2024-09-10T07:51:10.632070+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632070+0000","last_undegraded":"2024-09-10T07:51:10.632070+0000","last_fullsized":"2024-09-10T07:51:10.632070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.644693+0000","last_change":"2024-09-10T07:51:11.644693+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.644693+0000","last_undegraded":"2024-09-10T07:51:11.644693+0000","last_fullsized":"2024-09-10T07:51:11.644693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632072+0000","last_change":"2024-09-10T07:51:10.632072+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632072+0000","last_undegraded":"2024-09-10T07:51:10.632072+0000","last_fullsized":"2024-09-10T07:51:10.632072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648634+0000","last_change":"2024-09-10T07:51:11.648634+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648634+0000","last_undegraded":"2024-09-10T07:51:11.648634+0000","last_fullsized":"2024-09-10T07:51:11.648634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665379+0000","last_change":"2024-09-10T07:51:11.661553+0000","last_active":"2024-09-10T07:51:11.665379+0000","last_peered":"2024-09-10T07:51:11.665379+0000","last_clean":"2024-09-10T07:51:11.665379+0000","last_became_active":"2024-09-10T07:51:11.661263+0000","last_became_peered":"2024-09-10T07:51:11.661263+0000","last_unstale":"2024-09-10T07:51:11.665379+0000","last_undegraded":"2024-09-10T07:51:11.665379+0000","last_fullsized":"2024-09-10T07:51:11.665379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:31.557854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662904+0000","last_change":"2024-09-10T07:51:11.657359+0000","last_active":"2024-09-10T07:51:11.662904+0000","last_peered":"2024-09-10T07:51:11.662904+0000","last_clean":"2024-09-10T07:51:11.662904+0000","last_became_active":"2024-09-10T07:51:11.657038+0000","last_became_peered":"2024-09-10T07:51:11.657038+0000","last_unstale":"2024-09-10T07:51:11.662904+0000","last_undegraded":"2024-09-10T07:51:11.662904+0000","last_fullsized":"2024-09-10T07:51:11.662904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:52.917432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.667641+0000","last_change":"2024-09-10T07:51:11.664261+0000","last_active":"2024-09-10T07:51:11.667641+0000","last_peered":"2024-09-10T07:51:11.667641+0000","last_clean":"2024-09-10T07:51:11.667641+0000","last_became_active":"2024-09-10T07:51:11.663940+0000","last_became_peered":"2024-09-10T07:51:11.663940+0000","last_unstale":"2024-09-10T07:51:11.667641+0000","last_undegraded":"2024-09-10T07:51:11.667641+0000","last_fullsized":"2024-09-10T07:51:11.667641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:36.181473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.649850+0000","last_change":"2024-09-10T07:51:11.649850+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.649850+0000","last_undegraded":"2024-09-10T07:51:11.649850+0000","last_fullsized":"2024-09-10T07:51:11.649850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.656352+0000","last_change":"2024-09-10T07:51:11.654332+0000","last_active":"2024-09-10T07:51:11.656352+0000","last_peered":"2024-09-10T07:51:11.656352+0000","last_clean":"2024-09-10T07:51:11.656352+0000","last_became_active":"2024-09-10T07:51:11.654042+0000","last_became_peered":"2024-09-10T07:51:11.654042+0000","last_unstale":"2024-09-10T07:51:11.656352+0000","last_undegraded":"2024-09-10T07:51:11.656352+0000","last_fullsized":"2024-09-10T07:51:11.656352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:31.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650374+0000","last_change":"2024-09-10T07:51:11.650374+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650374+0000","last_undegraded":"2024-09-10T07:51:11.650374+0000","last_fullsized":"2024-09-10T07:51:11.650374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662530+0000","last_change":"2024-09-10T07:51:11.655876+0000","last_active":"2024-09-10T07:51:11.662530+0000","last_peered":"2024-09-10T07:51:11.662530+0000","last_clean":"2024-09-10T07:51:11.662530+0000","last_became_active":"2024-09-10T07:51:11.655606+0000","last_became_peered":"2024-09-10T07:51:11.655606+0000","last_unstale":"2024-09-10T07:51:11.662530+0000","last_undegraded":"2024-09-10T07:51:11.662530+0000","last_fullsized":"2024-09-10T07:51:11.662530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:50.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.660902+0000","last_change":"2024-09-10T07:51:11.654686+0000","last_active":"2024-09-10T07:51:11.660902+0000","last_peered":"2024-09-10T07:51:11.660902+0000","last_clean":"2024-09-10T07:51:11.660902+0000","last_became_active":"2024-09-10T07:51:11.653599+0000","last_became_peered":"2024-09-10T07:51:11.653599+0000","last_unstale":"2024-09-10T07:51:11.660902+0000","last_undegraded":"2024-09-10T07:51:11.660902+0000","last_fullsized":"2024-09-10T07:51:11.660902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:59.230645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662605+0000","last_change":"2024-09-10T07:51:11.652720+0000","last_active":"2024-09-10T07:51:11.662605+0000","last_peered":"2024-09-10T07:51:11.662605+0000","last_clean":"2024-09-10T07:51:11.662605+0000","last_became_active":"2024-09-10T07:51:11.652391+0000","last_became_peered":"2024-09-10T07:51:11.652391+0000","last_unstale":"2024-09-10T07:51:11.662605+0000","last_undegraded":"2024-09-10T07:51:11.662605+0000","last_fullsized":"2024-09-10T07:51:11.662605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:20.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.651971+0000","last_change":"2024-09-10T07:51:11.644261+0000","last_active":"2024-09-10T07:51:11.651971+0000","last_peered":"2024-09-10T07:51:11.651971+0000","last_clean":"2024-09-10T07:51:11.651971+0000","last_became_active":"2024-09-10T07:51:11.643967+0000","last_became_peered":"2024-09-10T07:51:11.643967+0000","last_unstale":"2024-09-10T07:51:11.651971+0000","last_undegraded":"2024-09-10T07:51:11.651971+0000","last_fullsized":"2024-09-10T07:51:11.651971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:29.448862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665001+0000","last_change":"2024-09-10T07:51:11.659869+0000","last_active":"2024-09-10T07:51:11.665001+0000","last_peered":"2024-09-10T07:51:11.665001+0000","last_clean":"2024-09-10T07:51:11.665001+0000","last_became_active":"2024-09-10T07:51:11.659392+0000","last_became_peered":"2024-09-10T07:51:11.659392+0000","last_unstale":"2024-09-10T07:51:11.665001+0000","last_undegraded":"2024-09-10T07:51:11.665001+0000","last_fullsized":"2024-09-10T07:51:11.665001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:39.691824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.657374+0000","last_change":"2024-09-10T07:51:12.655133+0000","last_active":"2024-09-10T07:51:12.657374+0000","last_peered":"2024-09-10T07:51:12.657374+0000","last_clean":"2024-09-10T07:51:12.657374+0000","last_became_active":"2024-09-10T07:51:12.654855+0000","last_became_peered":"2024-09-10T07:51:12.654855+0000","last_unstale":"2024-09-10T07:51:12.657374+0000","last_undegraded":"2024-09-10T07:51:12.657374+0000","last_fullsized":"2024-09-10T07:51:12.657374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:18.714900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638492+0000","last_change":"2024-09-10T07:51:10.638492+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638492+0000","last_undegraded":"2024-09-10T07:51:10.638492+0000","last_fullsized":"2024-09-10T07:51:10.638492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665342+0000","last_change":"2024-09-10T07:51:11.658990+0000","last_active":"2024-09-10T07:51:11.665342+0000","last_peered":"2024-09-10T07:51:11.665342+0000","last_clean":"2024-09-10T07:51:11.665342+0000","last_became_active":"2024-09-10T07:51:11.658451+0000","last_became_peered":"2024-09-10T07:51:11.658451+0000","last_unstale":"2024-09-10T07:51:11.665342+0000","last_undegraded":"2024-09-10T07:51:11.665342+0000","last_fullsized":"2024-09-10T07:51:11.665342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:01.563963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650083+0000","last_change":"2024-09-10T07:51:11.650083+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650083+0000","last_undegraded":"2024-09-10T07:51:11.650083+0000","last_fullsized":"2024-09-10T07:51:11.650083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662062+0000","last_change":"2024-09-10T07:51:11.655978+0000","last_active":"2024-09-10T07:51:11.662062+0000","last_peered":"2024-09-10T07:51:11.662062+0000","last_clean":"2024-09-10T07:51:11.662062+0000","last_became_active":"2024-09-10T07:51:11.655686+0000","last_became_peered":"2024-09-10T07:51:11.655686+0000","last_unstale":"2024-09-10T07:51:11.662062+0000","last_undegraded":"2024-09-10T07:51:11.662062+0000","last_fullsized":"2024-09-10T07:51:11.662062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:03.620426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648423+0000","last_change":"2024-09-10T07:51:11.648423+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648423+0000","last_undegraded":"2024-09-10T07:51:11.648423+0000","last_fullsized":"2024-09-10T07:51:11.648423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643075+0000","last_change":"2024-09-10T07:51:11.653391+0000","last_active":"2024-09-10T07:51:12.643075+0000","last_peered":"2024-09-10T07:51:12.643075+0000","last_clean":"2024-09-10T07:51:12.643075+0000","last_became_active":"2024-09-10T07:51:11.652803+0000","last_became_peered":"2024-09-10T07:51:11.652803+0000","last_unstale":"2024-09-10T07:51:12.643075+0000","last_undegraded":"2024-09-10T07:51:12.643075+0000","last_fullsized":"2024-09-10T07:51:12.643075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:38.179483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.649643+0000","last_change":"2024-09-10T07:51:11.649643+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.649643+0000","last_undegraded":"2024-09-10T07:51:11.649643+0000","last_fullsized":"2024-09-10T07:51:11.649643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642361+0000","last_change":"2024-09-10T07:51:11.653518+0000","last_active":"2024-09-10T07:51:12.642361+0000","last_peered":"2024-09-10T07:51:12.642361+0000","last_clean":"2024-09-10T07:51:12.642361+0000","last_became_active":"2024-09-10T07:51:11.653198+0000","last_became_peered":"2024-09-10T07:51:11.653198+0000","last_unstale":"2024-09-10T07:51:12.642361+0000","last_undegraded":"2024-09-10T07:51:12.642361+0000","last_fullsized":"2024-09-10T07:51:12.642361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.482793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643007+0000","last_change":"2024-09-10T07:51:11.653782+0000","last_active":"2024-09-10T07:51:12.643007+0000","last_peered":"2024-09-10T07:51:12.643007+0000","last_clean":"2024-09-10T07:51:12.643007+0000","last_became_active":"2024-09-10T07:51:11.650110+0000","last_became_peered":"2024-09-10T07:51:11.650110+0000","last_unstale":"2024-09-10T07:51:12.643007+0000","last_undegraded":"2024-09-10T07:51:12.643007+0000","last_fullsized":"2024-09-10T07:51:12.643007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:16.299195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.638699+0000","last_change":"2024-09-10T07:51:11.638699+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.638699+0000","last_undegraded":"2024-09-10T07:51:11.638699+0000","last_fullsized":"2024-09-10T07:51:11.638699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638526+0000","last_change":"2024-09-10T07:51:10.638526+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638526+0000","last_undegraded":"2024-09-10T07:51:10.638526+0000","last_fullsized":"2024-09-10T07:51:10.638526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.660737+0000","last_change":"2024-09-10T07:51:11.655026+0000","last_active":"2024-09-10T07:51:11.660737+0000","last_peered":"2024-09-10T07:51:11.660737+0000","last_clean":"2024-09-10T07:51:11.660737+0000","last_became_active":"2024-09-10T07:51:11.654747+0000","last_became_peered":"2024-09-10T07:51:11.654747+0000","last_unstale":"2024-09-10T07:51:11.660737+0000","last_undegraded":"2024-09-10T07:51:11.660737+0000","last_fullsized":"2024-09-10T07:51:11.660737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:02.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.647215+0000","last_change":"2024-09-10T07:51:11.647215+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.647215+0000","last_undegraded":"2024-09-10T07:51:11.647215+0000","last_fullsized":"2024-09-10T07:51:11.647215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665355+0000","last_change":"2024-09-10T07:51:11.659975+0000","last_active":"2024-09-10T07:51:11.665355+0000","last_peered":"2024-09-10T07:51:11.665355+0000","last_clean":"2024-09-10T07:51:11.665355+0000","last_became_active":"2024-09-10T07:51:11.657387+0000","last_became_peered":"2024-09-10T07:51:11.657387+0000","last_unstale":"2024-09-10T07:51:11.665355+0000","last_undegraded":"2024-09-10T07:51:11.665355+0000","last_fullsized":"2024-09-10T07:51:11.665355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:21.534312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.663234+0000","last_change":"2024-09-10T07:51:11.660202+0000","last_active":"2024-09-10T07:51:11.663234+0000","last_peered":"2024-09-10T07:51:11.663234+0000","last_clean":"2024-09-10T07:51:11.663234+0000","last_became_active":"2024-09-10T07:51:11.659089+0000","last_became_peered":"2024-09-10T07:51:11.659089+0000","last_unstale":"2024-09-10T07:51:11.663234+0000","last_undegraded":"2024-09-10T07:51:11.663234+0000","last_fullsized":"2024-09-10T07:51:11.663234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:41:02.706942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":269,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.645481+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:51:11.645481+0000","last_peered":"2024-09-10T07:51:11.645481+0000","last_clean":"2024-09-10T07:51:11.645481+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:51:11.645481+0000","last_undegraded":"2024-09-10T07:51:11.645481+0000","last_fullsized":"2024-09-10T07:51:11.645481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476796,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1495040,"data_stored":973148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476797,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":376963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":376963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":967331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":913408,"data_stored":389047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:51:16.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:51:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.049+0000 7f3fd6800640 1 Processor -- start 2024-09-10T07:51:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.049+0000 7f3fd6800640 1 -- start start 2024-09-10T07:51:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.050+0000 7f3fd6800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.050+0000 7f3fd6800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3fd00b6a70 0x7f3fd00d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.050+0000 7f3fd6800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd00cff10 con 0x7f3fd00d1510 2024-09-10T07:51:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fd6800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd00cf540 con 0x7f3fd00b6a70 2024-09-10T07:51:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fd6800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00d1d90 con 0x7f3fd00b6690 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcfe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3fd00d1510 0x7f3fd00d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35132/0 (socket says 172.21.15.3:35132) 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcfe00640 1 -- 172.21.15.3:0/1225211182 learned_addr learned my addr 172.21.15.3:0/1225211182 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcf400640 1 --2- 172.21.15.3:0/1225211182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fc7e00640 1 --2- 172.21.15.3:0/1225211182 >> v2:172.21.15.70:3300/0 conn(0x7f3fd00b6a70 0x7f3fd00d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcea00640 1 -- 172.21.15.3:0/1225211182 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855403859 0 0) 0x7f3fd00cff10 con 0x7f3fd00d1510 2024-09-10T07:51:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcea00640 1 -- 172.21.15.3:0/1225211182 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fb4003520 con 0x7f3fd00d1510 2024-09-10T07:51:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcf400640 1 -- 172.21.15.3:0/1225211182 >> v2:172.21.15.70:3300/0 conn(0x7f3fd00b6a70 msgr2=0x7f3fd00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcf400640 1 --2- 172.21.15.3:0/1225211182 >> v2:172.21.15.70:3300/0 conn(0x7f3fd00b6a70 0x7f3fd00d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcf400640 1 -- 172.21.15.3:0/1225211182 >> v1:172.21.15.3:6789/0 conn(0x7f3fd00d1510 legacy=0x7f3fd00d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.051+0000 7f3fcf400640 1 -- 172.21.15.3:0/1225211182 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd00ceef0 con 0x7f3fd00b6690 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fcf400640 1 --2- 172.21.15.3:0/1225211182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd00cf9d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68fd847014121e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fcea00640 1 -- 172.21.15.3:0/1225211182 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0037070 con 0x7f3fd00b6690 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fcea00640 1 -- 172.21.15.3:0/1225211182 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc0032440 con 0x7f3fd00b6690 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fcea00640 1 -- 172.21.15.3:0/1225211182 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0032810 con 0x7f3fd00b6690 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 -- 172.21.15.3:0/1225211182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 msgr2=0x7f3fd00cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 --2- 172.21.15.3:0/1225211182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd00cf9d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002a180 comp rx=0 tx=0).stop 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 -- 172.21.15.3:0/1225211182 shutdown_connections 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 --2- 172.21.15.3:0/1225211182 >> v2:172.21.15.70:3300/0 conn(0x7f3fd00b6a70 0x7f3fd00d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 --2- 172.21.15.3:0/1225211182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd00cf9d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 -- 172.21.15.3:0/1225211182 >> 172.21.15.3:0/1225211182 conn(0x7f3fd0072560 msgr2=0x7f3fd0072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.053+0000 7f3fd6800640 1 -- 172.21.15.3:0/1225211182 shutdown_connections 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.054+0000 7f3fd6800640 1 -- 172.21.15.3:0/1225211182 wait complete. 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.055+0000 7f3fd6800640 1 Processor -- start 2024-09-10T07:51:17.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.055+0000 7f3fd6800640 1 -- start start 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fd00d1510 0x7f3fd00d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00d19a0 con 0x7f3fd00b6a70 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00b6ed0 con 0x7f3fd00d1510 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fd6800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00d1b20 con 0x7f3fd00b6690 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fc7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fc7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55478/0 (socket says 172.21.15.3:55478) 2024-09-10T07:51:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.056+0000 7f3fc7e00640 1 -- 172.21.15.3:0/3286644804 learned_addr learned my addr 172.21.15.3:0/3286644804 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 -- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 msgr2=0x7f3fd0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 -- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fd00d1510 msgr2=0x7f3fd00d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fd00d1510 0x7f3fd00d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 -- 172.21.15.3:0/3286644804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd00cc6e0 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7e00640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9b0 tx=0x7f3fb800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8017070 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb800c9a0 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd00cc400 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800cd70 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.057+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd00cd130 con 0x7f3fd00b6a70 2024-09-10T07:51:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.058+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd015c530 con 0x7f3fd00b6a70 2024-09-10T07:51:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.059+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fb801a070 con 0x7f3fd00b6a70 2024-09-10T07:51:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.060+0000 7f3fc7400640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 0x7f3fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.060+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f3fb808af20 con 0x7f3fd00b6a70 2024-09-10T07:51:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.061+0000 7f3fcf400640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 0x7f3fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.061+0000 7f3fcf400640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 0x7f3fac06e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0034000 tx=0x7f3fc002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.065+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3fb8059840 con 0x7f3fd00b6a70 2024-09-10T07:51:17.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.244+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3fd0069140 con 0x7f3fac06be30 2024-09-10T07:51:17.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.250+0000 7f3fc7400640 1 -- 172.21.15.3:0/3286644804 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245007 (secure 0 0 0) 0x7f3fd0069140 con 0x7f3fac06be30 2024-09-10T07:51:17.253 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 msgr2=0x7f3fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 0x7f3fac06e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0034000 tx=0x7f3fc002c000 comp rx=0 tx=0).stop 2024-09-10T07:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 msgr2=0x7f3fd00cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9b0 tx=0x7f3fb800ee70 comp rx=0 tx=0).stop 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 shutdown_connections 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3fac06be30 0x7f3fac06e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fd00d1510 0x7f3fd00d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd00b6a70 0x7f3fd00cac90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 --2- 172.21.15.3:0/3286644804 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3fd00b6690 0x7f3fd0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 >> 172.21.15.3:0/3286644804 conn(0x7f3fd0072560 msgr2=0x7f3fd015fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 shutdown_connections 2024-09-10T07:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.254+0000 7f3fd6800640 1 -- 172.21.15.3:0/3286644804 wait complete. 2024-09-10T07:51:17.269 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T07:51:15.553366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":261860,"kb_used_data":8176,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712860,"statfs":{"total":773094113280,"available":772825968640,"internally_reserved":0,"allocated":8372224,"data_stored":4231792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.924334"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639913+0000","last_change":"2024-09-10T07:51:11.639913+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639913+0000","last_undegraded":"2024-09-10T07:51:11.639913+0000","last_fullsized":"2024-09-10T07:51:11.639913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651835+0000","last_change":"2024-09-10T07:51:11.651835+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651835+0000","last_undegraded":"2024-09-10T07:51:11.651835+0000","last_fullsized":"2024-09-10T07:51:11.651835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.646868+0000","last_change":"2024-09-10T07:51:11.646868+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.646868+0000","last_undegraded":"2024-09-10T07:51:11.646868+0000","last_fullsized":"2024-09-10T07:51:11.646868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.654231+0000","last_change":"2024-09-10T07:51:11.654231+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.654231+0000","last_undegraded":"2024-09-10T07:51:11.654231+0000","last_fullsized":"2024-09-10T07:51:11.654231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639753+0000","last_change":"2024-09-10T07:51:11.639753+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639753+0000","last_undegraded":"2024-09-10T07:51:11.639753+0000","last_fullsized":"2024-09-10T07:51:11.639753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653980+0000","last_change":"2024-09-10T07:51:11.653980+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653980+0000","last_undegraded":"2024-09-10T07:51:11.653980+0000","last_fullsized":"2024-09-10T07:51:11.653980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640234+0000","last_change":"2024-09-10T07:51:11.640234+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640234+0000","last_undegraded":"2024-09-10T07:51:11.640234+0000","last_fullsized":"2024-09-10T07:51:11.640234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.654324+0000","last_change":"2024-09-10T07:51:12.647897+0000","last_active":"2024-09-10T07:51:12.654324+0000","last_peered":"2024-09-10T07:51:12.654324+0000","last_clean":"2024-09-10T07:51:12.654324+0000","last_became_active":"2024-09-10T07:51:12.647493+0000","last_became_peered":"2024-09-10T07:51:12.647493+0000","last_unstale":"2024-09-10T07:51:12.654324+0000","last_undegraded":"2024-09-10T07:51:12.654324+0000","last_fullsized":"2024-09-10T07:51:12.654324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:54.143299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.677787+0000","last_change":"2024-09-10T07:51:12.676376+0000","last_active":"2024-09-10T07:51:12.677787+0000","last_peered":"2024-09-10T07:51:12.677787+0000","last_clean":"2024-09-10T07:51:12.677787+0000","last_became_active":"2024-09-10T07:51:12.676068+0000","last_became_peered":"2024-09-10T07:51:12.676068+0000","last_unstale":"2024-09-10T07:51:12.677787+0000","last_undegraded":"2024-09-10T07:51:12.677787+0000","last_fullsized":"2024-09-10T07:51:12.677787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:03.847359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.664304+0000","last_change":"2024-09-10T07:51:12.659238+0000","last_active":"2024-09-10T07:51:12.664304+0000","last_peered":"2024-09-10T07:51:12.664304+0000","last_clean":"2024-09-10T07:51:12.664304+0000","last_became_active":"2024-09-10T07:51:12.658975+0000","last_became_peered":"2024-09-10T07:51:12.658975+0000","last_unstale":"2024-09-10T07:51:12.664304+0000","last_undegraded":"2024-09-10T07:51:12.664304+0000","last_fullsized":"2024-09-10T07:51:12.664304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:48.409488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639350+0000","last_change":"2024-09-10T07:51:11.639350+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639350+0000","last_undegraded":"2024-09-10T07:51:11.639350+0000","last_fullsized":"2024-09-10T07:51:11.639350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640694+0000","last_change":"2024-09-10T07:51:11.640694+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640694+0000","last_undegraded":"2024-09-10T07:51:11.640694+0000","last_fullsized":"2024-09-10T07:51:11.640694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653111+0000","last_change":"2024-09-10T07:51:11.653111+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653111+0000","last_undegraded":"2024-09-10T07:51:11.653111+0000","last_fullsized":"2024-09-10T07:51:11.653111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.646399+0000","last_change":"2024-09-10T07:51:11.646399+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.646399+0000","last_undegraded":"2024-09-10T07:51:11.646399+0000","last_fullsized":"2024-09-10T07:51:11.646399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651744+0000","last_change":"2024-09-10T07:51:11.651744+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651744+0000","last_undegraded":"2024-09-10T07:51:11.651744+0000","last_fullsized":"2024-09-10T07:51:11.651744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.638687+0000","last_change":"2024-09-10T07:51:11.638687+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.638687+0000","last_undegraded":"2024-09-10T07:51:11.638687+0000","last_fullsized":"2024-09-10T07:51:11.638687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650992+0000","last_change":"2024-09-10T07:51:11.650992+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650992+0000","last_undegraded":"2024-09-10T07:51:11.650992+0000","last_fullsized":"2024-09-10T07:51:11.650992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.640344+0000","last_change":"2024-09-10T07:51:11.640344+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.640344+0000","last_undegraded":"2024-09-10T07:51:11.640344+0000","last_fullsized":"2024-09-10T07:51:11.640344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648866+0000","last_change":"2024-09-10T07:51:11.648866+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648866+0000","last_undegraded":"2024-09-10T07:51:11.648866+0000","last_fullsized":"2024-09-10T07:51:11.648866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.642085+0000","last_change":"2024-09-10T07:51:11.642085+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.642085+0000","last_undegraded":"2024-09-10T07:51:11.642085+0000","last_fullsized":"2024-09-10T07:51:11.642085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648951+0000","last_change":"2024-09-10T07:51:11.648951+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648951+0000","last_undegraded":"2024-09-10T07:51:11.648951+0000","last_fullsized":"2024-09-10T07:51:11.648951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650214+0000","last_change":"2024-09-10T07:51:11.650214+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650214+0000","last_undegraded":"2024-09-10T07:51:11.650214+0000","last_fullsized":"2024-09-10T07:51:11.650214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.639878+0000","last_change":"2024-09-10T07:51:11.639878+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.639878+0000","last_undegraded":"2024-09-10T07:51:11.639878+0000","last_fullsized":"2024-09-10T07:51:11.639878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.655388+0000","last_change":"2024-09-10T07:51:11.647694+0000","last_active":"2024-09-10T07:51:11.655388+0000","last_peered":"2024-09-10T07:51:11.655388+0000","last_clean":"2024-09-10T07:51:11.655388+0000","last_became_active":"2024-09-10T07:51:11.647419+0000","last_became_peered":"2024-09-10T07:51:11.647419+0000","last_unstale":"2024-09-10T07:51:11.655388+0000","last_undegraded":"2024-09-10T07:51:11.655388+0000","last_fullsized":"2024-09-10T07:51:11.655388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:36.594271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638653+0000","last_change":"2024-09-10T07:51:10.638653+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638653+0000","last_undegraded":"2024-09-10T07:51:10.638653+0000","last_fullsized":"2024-09-10T07:51:10.638653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.645311+0000","last_change":"2024-09-10T07:51:11.645311+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.645311+0000","last_undegraded":"2024-09-10T07:51:11.645311+0000","last_fullsized":"2024-09-10T07:51:11.645311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.652725+0000","last_change":"2024-09-10T07:51:11.648632+0000","last_active":"2024-09-10T07:51:11.652725+0000","last_peered":"2024-09-10T07:51:11.652725+0000","last_clean":"2024-09-10T07:51:11.652725+0000","last_became_active":"2024-09-10T07:51:11.648267+0000","last_became_peered":"2024-09-10T07:51:11.648267+0000","last_unstale":"2024-09-10T07:51:11.652725+0000","last_undegraded":"2024-09-10T07:51:11.652725+0000","last_fullsized":"2024-09-10T07:51:11.652725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:44.793056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.653487+0000","last_change":"2024-09-10T07:51:11.653487+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.653487+0000","last_undegraded":"2024-09-10T07:51:11.653487+0000","last_fullsized":"2024-09-10T07:51:11.653487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665473+0000","last_change":"2024-09-10T07:51:11.661609+0000","last_active":"2024-09-10T07:51:11.665473+0000","last_peered":"2024-09-10T07:51:11.665473+0000","last_clean":"2024-09-10T07:51:11.665473+0000","last_became_active":"2024-09-10T07:51:11.660016+0000","last_became_peered":"2024-09-10T07:51:11.660016+0000","last_unstale":"2024-09-10T07:51:11.665473+0000","last_undegraded":"2024-09-10T07:51:11.665473+0000","last_fullsized":"2024-09-10T07:51:11.665473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:57.530612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.641553+0000","last_change":"2024-09-10T07:51:11.641553+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.641553+0000","last_undegraded":"2024-09-10T07:51:11.641553+0000","last_fullsized":"2024-09-10T07:51:11.641553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.671056+0000","last_change":"2024-09-10T07:51:11.654006+0000","last_active":"2024-09-10T07:51:12.671056+0000","last_peered":"2024-09-10T07:51:12.671056+0000","last_clean":"2024-09-10T07:51:12.671056+0000","last_became_active":"2024-09-10T07:51:11.651332+0000","last_became_peered":"2024-09-10T07:51:11.651332+0000","last_unstale":"2024-09-10T07:51:12.671056+0000","last_undegraded":"2024-09-10T07:51:12.671056+0000","last_fullsized":"2024-09-10T07:51:12.671056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:41.047040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648338+0000","last_change":"2024-09-10T07:51:11.648338+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648338+0000","last_undegraded":"2024-09-10T07:51:11.648338+0000","last_fullsized":"2024-09-10T07:51:11.648338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.670809+0000","last_change":"2024-09-10T07:51:11.666134+0000","last_active":"2024-09-10T07:51:11.670809+0000","last_peered":"2024-09-10T07:51:11.670809+0000","last_clean":"2024-09-10T07:51:11.670809+0000","last_became_active":"2024-09-10T07:51:11.665826+0000","last_became_peered":"2024-09-10T07:51:11.665826+0000","last_unstale":"2024-09-10T07:51:11.670809+0000","last_undegraded":"2024-09-10T07:51:11.670809+0000","last_fullsized":"2024-09-10T07:51:11.670809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:56.418338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632063+0000","last_change":"2024-09-10T07:51:10.632063+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632063+0000","last_undegraded":"2024-09-10T07:51:10.632063+0000","last_fullsized":"2024-09-10T07:51:10.632063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.669674+0000","last_change":"2024-09-10T07:51:12.668160+0000","last_active":"2024-09-10T07:51:12.669674+0000","last_peered":"2024-09-10T07:51:12.669674+0000","last_clean":"2024-09-10T07:51:12.669674+0000","last_became_active":"2024-09-10T07:51:12.667845+0000","last_became_peered":"2024-09-10T07:51:12.667845+0000","last_unstale":"2024-09-10T07:51:12.669674+0000","last_undegraded":"2024-09-10T07:51:12.669674+0000","last_fullsized":"2024-09-10T07:51:12.669674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:57.084096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642604+0000","last_change":"2024-09-10T07:51:11.647781+0000","last_active":"2024-09-10T07:51:12.642604+0000","last_peered":"2024-09-10T07:51:12.642604+0000","last_clean":"2024-09-10T07:51:12.642604+0000","last_became_active":"2024-09-10T07:51:11.647502+0000","last_became_peered":"2024-09-10T07:51:11.647502+0000","last_unstale":"2024-09-10T07:51:12.642604+0000","last_undegraded":"2024-09-10T07:51:12.642604+0000","last_fullsized":"2024-09-10T07:51:12.642604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:38.219907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.651338+0000","last_change":"2024-09-10T07:51:11.651338+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.651338+0000","last_undegraded":"2024-09-10T07:51:11.651338+0000","last_fullsized":"2024-09-10T07:51:11.651338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.655795+0000","last_change":"2024-09-10T07:51:11.652611+0000","last_active":"2024-09-10T07:51:11.655795+0000","last_peered":"2024-09-10T07:51:11.655795+0000","last_clean":"2024-09-10T07:51:11.655795+0000","last_became_active":"2024-09-10T07:51:11.652322+0000","last_became_peered":"2024-09-10T07:51:11.652322+0000","last_unstale":"2024-09-10T07:51:11.655795+0000","last_undegraded":"2024-09-10T07:51:11.655795+0000","last_fullsized":"2024-09-10T07:51:11.655795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:35.591184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650915+0000","last_change":"2024-09-10T07:51:11.650915+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650915+0000","last_undegraded":"2024-09-10T07:51:11.650915+0000","last_fullsized":"2024-09-10T07:51:11.650915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665507+0000","last_change":"2024-09-10T07:51:11.661453+0000","last_active":"2024-09-10T07:51:11.665507+0000","last_peered":"2024-09-10T07:51:11.665507+0000","last_clean":"2024-09-10T07:51:11.665507+0000","last_became_active":"2024-09-10T07:51:11.661202+0000","last_became_peered":"2024-09-10T07:51:11.661202+0000","last_unstale":"2024-09-10T07:51:11.665507+0000","last_undegraded":"2024-09-10T07:51:11.665507+0000","last_fullsized":"2024-09-10T07:51:11.665507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:30.240343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.642581+0000","last_change":"2024-09-10T07:51:11.642581+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.642581+0000","last_undegraded":"2024-09-10T07:51:11.642581+0000","last_fullsized":"2024-09-10T07:51:11.642581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.665387+0000","last_change":"2024-09-10T07:51:11.651919+0000","last_active":"2024-09-10T07:51:12.665387+0000","last_peered":"2024-09-10T07:51:12.665387+0000","last_clean":"2024-09-10T07:51:12.665387+0000","last_became_active":"2024-09-10T07:51:11.651676+0000","last_became_peered":"2024-09-10T07:51:11.651676+0000","last_unstale":"2024-09-10T07:51:12.665387+0000","last_undegraded":"2024-09-10T07:51:12.665387+0000","last_fullsized":"2024-09-10T07:51:12.665387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632070+0000","last_change":"2024-09-10T07:51:10.632070+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632070+0000","last_undegraded":"2024-09-10T07:51:10.632070+0000","last_fullsized":"2024-09-10T07:51:10.632070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.644693+0000","last_change":"2024-09-10T07:51:11.644693+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.644693+0000","last_undegraded":"2024-09-10T07:51:11.644693+0000","last_fullsized":"2024-09-10T07:51:11.644693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.632072+0000","last_change":"2024-09-10T07:51:10.632072+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.632072+0000","last_undegraded":"2024-09-10T07:51:10.632072+0000","last_fullsized":"2024-09-10T07:51:10.632072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648634+0000","last_change":"2024-09-10T07:51:11.648634+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648634+0000","last_undegraded":"2024-09-10T07:51:11.648634+0000","last_fullsized":"2024-09-10T07:51:11.648634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665379+0000","last_change":"2024-09-10T07:51:11.661553+0000","last_active":"2024-09-10T07:51:11.665379+0000","last_peered":"2024-09-10T07:51:11.665379+0000","last_clean":"2024-09-10T07:51:11.665379+0000","last_became_active":"2024-09-10T07:51:11.661263+0000","last_became_peered":"2024-09-10T07:51:11.661263+0000","last_unstale":"2024-09-10T07:51:11.665379+0000","last_undegraded":"2024-09-10T07:51:11.665379+0000","last_fullsized":"2024-09-10T07:51:11.665379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:31.557854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662904+0000","last_change":"2024-09-10T07:51:11.657359+0000","last_active":"2024-09-10T07:51:11.662904+0000","last_peered":"2024-09-10T07:51:11.662904+0000","last_clean":"2024-09-10T07:51:11.662904+0000","last_became_active":"2024-09-10T07:51:11.657038+0000","last_became_peered":"2024-09-10T07:51:11.657038+0000","last_unstale":"2024-09-10T07:51:11.662904+0000","last_undegraded":"2024-09-10T07:51:11.662904+0000","last_fullsized":"2024-09-10T07:51:11.662904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:52.917432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.667641+0000","last_change":"2024-09-10T07:51:11.664261+0000","last_active":"2024-09-10T07:51:11.667641+0000","last_peered":"2024-09-10T07:51:11.667641+0000","last_clean":"2024-09-10T07:51:11.667641+0000","last_became_active":"2024-09-10T07:51:11.663940+0000","last_became_peered":"2024-09-10T07:51:11.663940+0000","last_unstale":"2024-09-10T07:51:11.667641+0000","last_undegraded":"2024-09-10T07:51:11.667641+0000","last_fullsized":"2024-09-10T07:51:11.667641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:36.181473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.649850+0000","last_change":"2024-09-10T07:51:11.649850+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.649850+0000","last_undegraded":"2024-09-10T07:51:11.649850+0000","last_fullsized":"2024-09-10T07:51:11.649850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.656352+0000","last_change":"2024-09-10T07:51:11.654332+0000","last_active":"2024-09-10T07:51:11.656352+0000","last_peered":"2024-09-10T07:51:11.656352+0000","last_clean":"2024-09-10T07:51:11.656352+0000","last_became_active":"2024-09-10T07:51:11.654042+0000","last_became_peered":"2024-09-10T07:51:11.654042+0000","last_unstale":"2024-09-10T07:51:11.656352+0000","last_undegraded":"2024-09-10T07:51:11.656352+0000","last_fullsized":"2024-09-10T07:51:11.656352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:31.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650374+0000","last_change":"2024-09-10T07:51:11.650374+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650374+0000","last_undegraded":"2024-09-10T07:51:11.650374+0000","last_fullsized":"2024-09-10T07:51:11.650374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662530+0000","last_change":"2024-09-10T07:51:11.655876+0000","last_active":"2024-09-10T07:51:11.662530+0000","last_peered":"2024-09-10T07:51:11.662530+0000","last_clean":"2024-09-10T07:51:11.662530+0000","last_became_active":"2024-09-10T07:51:11.655606+0000","last_became_peered":"2024-09-10T07:51:11.655606+0000","last_unstale":"2024-09-10T07:51:11.662530+0000","last_undegraded":"2024-09-10T07:51:11.662530+0000","last_fullsized":"2024-09-10T07:51:11.662530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:50.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.660902+0000","last_change":"2024-09-10T07:51:11.654686+0000","last_active":"2024-09-10T07:51:11.660902+0000","last_peered":"2024-09-10T07:51:11.660902+0000","last_clean":"2024-09-10T07:51:11.660902+0000","last_became_active":"2024-09-10T07:51:11.653599+0000","last_became_peered":"2024-09-10T07:51:11.653599+0000","last_unstale":"2024-09-10T07:51:11.660902+0000","last_undegraded":"2024-09-10T07:51:11.660902+0000","last_fullsized":"2024-09-10T07:51:11.660902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:59.230645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662605+0000","last_change":"2024-09-10T07:51:11.652720+0000","last_active":"2024-09-10T07:51:11.662605+0000","last_peered":"2024-09-10T07:51:11.662605+0000","last_clean":"2024-09-10T07:51:11.662605+0000","last_became_active":"2024-09-10T07:51:11.652391+0000","last_became_peered":"2024-09-10T07:51:11.652391+0000","last_unstale":"2024-09-10T07:51:11.662605+0000","last_undegraded":"2024-09-10T07:51:11.662605+0000","last_fullsized":"2024-09-10T07:51:11.662605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:20.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.651971+0000","last_change":"2024-09-10T07:51:11.644261+0000","last_active":"2024-09-10T07:51:11.651971+0000","last_peered":"2024-09-10T07:51:11.651971+0000","last_clean":"2024-09-10T07:51:11.651971+0000","last_became_active":"2024-09-10T07:51:11.643967+0000","last_became_peered":"2024-09-10T07:51:11.643967+0000","last_unstale":"2024-09-10T07:51:11.651971+0000","last_undegraded":"2024-09-10T07:51:11.651971+0000","last_fullsized":"2024-09-10T07:51:11.651971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:29.448862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665001+0000","last_change":"2024-09-10T07:51:11.659869+0000","last_active":"2024-09-10T07:51:11.665001+0000","last_peered":"2024-09-10T07:51:11.665001+0000","last_clean":"2024-09-10T07:51:11.665001+0000","last_became_active":"2024-09-10T07:51:11.659392+0000","last_became_peered":"2024-09-10T07:51:11.659392+0000","last_unstale":"2024-09-10T07:51:11.665001+0000","last_undegraded":"2024-09-10T07:51:11.665001+0000","last_fullsized":"2024-09-10T07:51:11.665001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:39.691824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.657374+0000","last_change":"2024-09-10T07:51:12.655133+0000","last_active":"2024-09-10T07:51:12.657374+0000","last_peered":"2024-09-10T07:51:12.657374+0000","last_clean":"2024-09-10T07:51:12.657374+0000","last_became_active":"2024-09-10T07:51:12.654855+0000","last_became_peered":"2024-09-10T07:51:12.654855+0000","last_unstale":"2024-09-10T07:51:12.657374+0000","last_undegraded":"2024-09-10T07:51:12.657374+0000","last_fullsized":"2024-09-10T07:51:12.657374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:18.714900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638492+0000","last_change":"2024-09-10T07:51:10.638492+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638492+0000","last_undegraded":"2024-09-10T07:51:10.638492+0000","last_fullsized":"2024-09-10T07:51:10.638492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665342+0000","last_change":"2024-09-10T07:51:11.658990+0000","last_active":"2024-09-10T07:51:11.665342+0000","last_peered":"2024-09-10T07:51:11.665342+0000","last_clean":"2024-09-10T07:51:11.665342+0000","last_became_active":"2024-09-10T07:51:11.658451+0000","last_became_peered":"2024-09-10T07:51:11.658451+0000","last_unstale":"2024-09-10T07:51:11.665342+0000","last_undegraded":"2024-09-10T07:51:11.665342+0000","last_fullsized":"2024-09-10T07:51:11.665342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:01.563963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.650083+0000","last_change":"2024-09-10T07:51:11.650083+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.650083+0000","last_undegraded":"2024-09-10T07:51:11.650083+0000","last_fullsized":"2024-09-10T07:51:11.650083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.662062+0000","last_change":"2024-09-10T07:51:11.655978+0000","last_active":"2024-09-10T07:51:11.662062+0000","last_peered":"2024-09-10T07:51:11.662062+0000","last_clean":"2024-09-10T07:51:11.662062+0000","last_became_active":"2024-09-10T07:51:11.655686+0000","last_became_peered":"2024-09-10T07:51:11.655686+0000","last_unstale":"2024-09-10T07:51:11.662062+0000","last_undegraded":"2024-09-10T07:51:11.662062+0000","last_fullsized":"2024-09-10T07:51:11.662062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:03.620426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.648423+0000","last_change":"2024-09-10T07:51:11.648423+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.648423+0000","last_undegraded":"2024-09-10T07:51:11.648423+0000","last_fullsized":"2024-09-10T07:51:11.648423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643075+0000","last_change":"2024-09-10T07:51:11.653391+0000","last_active":"2024-09-10T07:51:12.643075+0000","last_peered":"2024-09-10T07:51:12.643075+0000","last_clean":"2024-09-10T07:51:12.643075+0000","last_became_active":"2024-09-10T07:51:11.652803+0000","last_became_peered":"2024-09-10T07:51:11.652803+0000","last_unstale":"2024-09-10T07:51:12.643075+0000","last_undegraded":"2024-09-10T07:51:12.643075+0000","last_fullsized":"2024-09-10T07:51:12.643075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:38.179483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.649643+0000","last_change":"2024-09-10T07:51:11.649643+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.649643+0000","last_undegraded":"2024-09-10T07:51:11.649643+0000","last_fullsized":"2024-09-10T07:51:11.649643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642361+0000","last_change":"2024-09-10T07:51:11.653518+0000","last_active":"2024-09-10T07:51:12.642361+0000","last_peered":"2024-09-10T07:51:12.642361+0000","last_clean":"2024-09-10T07:51:12.642361+0000","last_became_active":"2024-09-10T07:51:11.653198+0000","last_became_peered":"2024-09-10T07:51:11.653198+0000","last_unstale":"2024-09-10T07:51:12.642361+0000","last_undegraded":"2024-09-10T07:51:12.642361+0000","last_fullsized":"2024-09-10T07:51:12.642361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.482793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643007+0000","last_change":"2024-09-10T07:51:11.653782+0000","last_active":"2024-09-10T07:51:12.643007+0000","last_peered":"2024-09-10T07:51:12.643007+0000","last_clean":"2024-09-10T07:51:12.643007+0000","last_became_active":"2024-09-10T07:51:11.650110+0000","last_became_peered":"2024-09-10T07:51:11.650110+0000","last_unstale":"2024-09-10T07:51:12.643007+0000","last_undegraded":"2024-09-10T07:51:12.643007+0000","last_fullsized":"2024-09-10T07:51:12.643007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:16.299195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.638699+0000","last_change":"2024-09-10T07:51:11.638699+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.638699+0000","last_undegraded":"2024-09-10T07:51:11.638699+0000","last_fullsized":"2024-09-10T07:51:11.638699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-09-10T07:51:10.638526+0000","last_change":"2024-09-10T07:51:10.638526+0000","last_active":"2024-09-10T07:51:10.621492+0000","last_peered":"2024-09-10T07:51:10.621492+0000","last_clean":"2024-09-10T07:51:10.621492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:10.638526+0000","last_undegraded":"2024-09-10T07:51:10.638526+0000","last_fullsized":"2024-09-10T07:51:10.638526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.660737+0000","last_change":"2024-09-10T07:51:11.655026+0000","last_active":"2024-09-10T07:51:11.660737+0000","last_peered":"2024-09-10T07:51:11.660737+0000","last_clean":"2024-09-10T07:51:11.660737+0000","last_became_active":"2024-09-10T07:51:11.654747+0000","last_became_peered":"2024-09-10T07:51:11.654747+0000","last_unstale":"2024-09-10T07:51:11.660737+0000","last_undegraded":"2024-09-10T07:51:11.660737+0000","last_fullsized":"2024-09-10T07:51:11.660737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:02.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":76,"state":"creating+peering","last_fresh":"2024-09-10T07:51:11.647215+0000","last_change":"2024-09-10T07:51:11.647215+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.647215+0000","last_undegraded":"2024-09-10T07:51:11.647215+0000","last_fullsized":"2024-09-10T07:51:11.647215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.665355+0000","last_change":"2024-09-10T07:51:11.659975+0000","last_active":"2024-09-10T07:51:11.665355+0000","last_peered":"2024-09-10T07:51:11.665355+0000","last_clean":"2024-09-10T07:51:11.665355+0000","last_became_active":"2024-09-10T07:51:11.657387+0000","last_became_peered":"2024-09-10T07:51:11.657387+0000","last_unstale":"2024-09-10T07:51:11.665355+0000","last_undegraded":"2024-09-10T07:51:11.665355+0000","last_fullsized":"2024-09-10T07:51:11.665355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:21.534312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T07:51:11.628863+0000","last_change":"2024-09-10T07:51:11.628863+0000","last_active":"2024-09-10T07:51:11.628863+0000","last_peered":"2024-09-10T07:51:11.628863+0000","last_clean":"2024-09-10T07:51:11.628863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:51:11.628863+0000","last_undegraded":"2024-09-10T07:51:11.628863+0000","last_fullsized":"2024-09-10T07:51:11.628863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.663234+0000","last_change":"2024-09-10T07:51:11.660202+0000","last_active":"2024-09-10T07:51:11.663234+0000","last_peered":"2024-09-10T07:51:11.663234+0000","last_clean":"2024-09-10T07:51:11.663234+0000","last_became_active":"2024-09-10T07:51:11.659089+0000","last_became_peered":"2024-09-10T07:51:11.659089+0000","last_unstale":"2024-09-10T07:51:11.663234+0000","last_undegraded":"2024-09-10T07:51:11.663234+0000","last_fullsized":"2024-09-10T07:51:11.663234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:41:02.706942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":269,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-10T07:51:11.645481+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:51:11.645481+0000","last_peered":"2024-09-10T07:51:11.645481+0000","last_clean":"2024-09-10T07:51:11.645481+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:51:11.645481+0000","last_undegraded":"2024-09-10T07:51:11.645481+0000","last_fullsized":"2024-09-10T07:51:11.645481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476796,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1495040,"data_stored":973148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476797,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":382780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":376963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":888832,"data_stored":376963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":967331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":913408,"data_stored":389047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:51:17.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:51:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.467+0000 7f4427200640 1 Processor -- start 2024-09-10T07:51:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.467+0000 7f4427200640 1 -- start start 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.468+0000 7f4427200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.468+0000 7f4427200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44200d07c0 0x7f4420072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.468+0000 7f4427200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44200cfdd0 con 0x7f4420160af0 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.468+0000 7f4427200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f442015cc90 con 0x7f44200d07c0 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.468+0000 7f4427200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f442015cf70 con 0x7f4420160710 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:52176/0 (socket says 172.21.15.3:52176) 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 -- 172.21.15.3:0/3597321891 learned_addr learned my addr 172.21.15.3:0/3597321891 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f4425200640 1 --2- 172.21.15.3:0/3597321891 >> v2:172.21.15.70:3300/0 conn(0x7f44200d07c0 0x7f4420072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 -- 172.21.15.3:0/3597321891 >> v2:172.21.15.70:3300/0 conn(0x7f44200d07c0 msgr2=0x7f4420072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 --2- 172.21.15.3:0/3597321891 >> v2:172.21.15.70:3300/0 conn(0x7f44200d07c0 0x7f4420072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 -- 172.21.15.3:0/3597321891 >> v1:172.21.15.70:6789/0 conn(0x7f4420160af0 legacy=0x7f44200d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.469+0000 7f441fe00640 1 -- 172.21.15.3:0/3597321891 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442015d250 con 0x7f4420160710 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.470+0000 7f441fe00640 1 --2- 172.21.15.3:0/3597321891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d106241a90647ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.470+0000 7f441ea00640 1 -- 172.21.15.3:0/3597321891 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f4420160710 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f441ea00640 1 -- 172.21.15.3:0/3597321891 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410032440 con 0x7f4420160710 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f441ea00640 1 -- 172.21.15.3:0/3597321891 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410032810 con 0x7f4420160710 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 -- 172.21.15.3:0/3597321891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 msgr2=0x7f44200cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 --2- 172.21.15.3:0/3597321891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a270 comp rx=0 tx=0).stop 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 -- 172.21.15.3:0/3597321891 shutdown_connections 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 --2- 172.21.15.3:0/3597321891 >> v2:172.21.15.70:3300/0 conn(0x7f44200d07c0 0x7f4420072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 --2- 172.21.15.3:0/3597321891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4420160710 0x7f44200cf2c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 -- 172.21.15.3:0/3597321891 >> 172.21.15.3:0/3597321891 conn(0x7f442006d000 msgr2=0x7f442006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 -- 172.21.15.3:0/3597321891 shutdown_connections 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.471+0000 7f4427200640 1 -- 172.21.15.3:0/3597321891 wait complete. 2024-09-10T07:51:17.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.473+0000 7f4427200640 1 Processor -- start 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.473+0000 7f4427200640 1 -- start start 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44200d07c0 0x7f44201f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4420160af0 0x7f442015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f442015d770 con 0x7f4420160710 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44200cf800 con 0x7f4420160af0 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4427200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4420152070 con 0x7f44200d07c0 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f441f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f441f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55486/0 (socket says 172.21.15.3:55486) 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f441f400640 1 -- 172.21.15.3:0/1000469497 learned_addr learned my addr 172.21.15.3:0/1000469497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f441fe00640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44200d07c0 0x7f44201f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.474+0000 7f4425200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4420160af0 0x7f442015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 -- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44200d07c0 msgr2=0x7f44201f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44200d07c0 0x7f44201f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 -- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4420160af0 msgr2=0x7f442015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4420160af0 0x7f442015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 -- 172.21.15.3:0/1000469497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442015f240 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f441f400640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c3f0 tx=0x7f440c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0052d0 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c005490 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44201f5050 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c007260 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.475+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44201f5610 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.476+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44200afd50 con 0x7f4420160710 2024-09-10T07:51:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.477+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f440c007400 con 0x7f4420160710 2024-09-10T07:51:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.478+0000 7f4417e00640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 0x7f440006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.478+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f440c08a200 con 0x7f4420160710 2024-09-10T07:51:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.478+0000 7f441fe00640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 0x7f440006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.479+0000 7f441fe00640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 0x7f440006e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.483+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f440c058550 con 0x7f4420160710 2024-09-10T07:51:17.618 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:51:17.619+0000 7f2ad8a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:51:17.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.666+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44200ce7a0 con 0x7f440006be30 2024-09-10T07:51:17.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.673+0000 7f4417e00640 1 -- 172.21.15.3:0/1000469497 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252065 (secure 0 0 0) 0x7f44200ce7a0 con 0x7f440006be30 2024-09-10T07:51:17.675 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:17.677 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.676+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 msgr2=0x7f440006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 0x7f440006e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 msgr2=0x7f4420151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c3f0 tx=0x7f440c00c8b0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 shutdown_connections 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f440006be30 0x7f440006e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4420160af0 0x7f442015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4420160710 0x7f4420151b30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 --2- 172.21.15.3:0/1000469497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44200d07c0 0x7f44201f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 >> 172.21.15.3:0/1000469497 conn(0x7f442006d000 msgr2=0x7f44200b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 shutdown_connections 2024-09-10T07:51:17.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.677+0000 7f4427200640 1 -- 172.21.15.3:0/1000469497 wait complete. 2024-09-10T07:51:17.691 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T07:51:17.553692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262212,"kb_used_data":8516,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712508,"statfs":{"total":773094113280,"available":772825608192,"internally_reserved":0,"allocated":8720384,"data_stored":4388763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001075"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828738+0000","last_change":"2024-09-10T07:51:12.676690+0000","last_active":"2024-09-10T07:51:13.828738+0000","last_peered":"2024-09-10T07:51:13.828738+0000","last_clean":"2024-09-10T07:51:13.828738+0000","last_became_active":"2024-09-10T07:51:12.676418+0000","last_became_peered":"2024-09-10T07:51:12.676418+0000","last_unstale":"2024-09-10T07:51:13.828738+0000","last_undegraded":"2024-09-10T07:51:13.828738+0000","last_fullsized":"2024-09-10T07:51:13.828738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:00.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827421+0000","last_change":"2024-09-10T07:51:12.676886+0000","last_active":"2024-09-10T07:51:13.827421+0000","last_peered":"2024-09-10T07:51:13.827421+0000","last_clean":"2024-09-10T07:51:13.827421+0000","last_became_active":"2024-09-10T07:51:12.676680+0000","last_became_peered":"2024-09-10T07:51:12.676680+0000","last_unstale":"2024-09-10T07:51:13.827421+0000","last_undegraded":"2024-09-10T07:51:13.827421+0000","last_fullsized":"2024-09-10T07:51:13.827421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656352+0000","last_change":"2024-09-10T07:51:12.677258+0000","last_active":"2024-09-10T07:51:13.656352+0000","last_peered":"2024-09-10T07:51:13.656352+0000","last_clean":"2024-09-10T07:51:13.656352+0000","last_became_active":"2024-09-10T07:51:12.676957+0000","last_became_peered":"2024-09-10T07:51:12.676957+0000","last_unstale":"2024-09-10T07:51:13.656352+0000","last_undegraded":"2024-09-10T07:51:13.656352+0000","last_fullsized":"2024-09-10T07:51:13.656352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655764+0000","last_change":"2024-09-10T07:51:12.675735+0000","last_active":"2024-09-10T07:51:13.655764+0000","last_peered":"2024-09-10T07:51:13.655764+0000","last_clean":"2024-09-10T07:51:13.655764+0000","last_became_active":"2024-09-10T07:51:12.675442+0000","last_became_peered":"2024-09-10T07:51:12.675442+0000","last_unstale":"2024-09-10T07:51:13.655764+0000","last_undegraded":"2024-09-10T07:51:13.655764+0000","last_fullsized":"2024-09-10T07:51:13.655764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:00.169360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827458+0000","last_change":"2024-09-10T07:51:12.670954+0000","last_active":"2024-09-10T07:51:13.827458+0000","last_peered":"2024-09-10T07:51:13.827458+0000","last_clean":"2024-09-10T07:51:13.827458+0000","last_became_active":"2024-09-10T07:51:12.670673+0000","last_became_peered":"2024-09-10T07:51:12.670673+0000","last_unstale":"2024-09-10T07:51:13.827458+0000","last_undegraded":"2024-09-10T07:51:13.827458+0000","last_fullsized":"2024-09-10T07:51:13.827458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:52.871411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826972+0000","last_change":"2024-09-10T07:51:12.681827+0000","last_active":"2024-09-10T07:51:13.826972+0000","last_peered":"2024-09-10T07:51:13.826972+0000","last_clean":"2024-09-10T07:51:13.826972+0000","last_became_active":"2024-09-10T07:51:12.681558+0000","last_became_peered":"2024-09-10T07:51:12.681558+0000","last_unstale":"2024-09-10T07:51:13.826972+0000","last_undegraded":"2024-09-10T07:51:13.826972+0000","last_fullsized":"2024-09-10T07:51:13.826972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654188+0000","last_change":"2024-09-10T07:51:12.667538+0000","last_active":"2024-09-10T07:51:13.654188+0000","last_peered":"2024-09-10T07:51:13.654188+0000","last_clean":"2024-09-10T07:51:13.654188+0000","last_became_active":"2024-09-10T07:51:12.667277+0000","last_became_peered":"2024-09-10T07:51:12.667277+0000","last_unstale":"2024-09-10T07:51:13.654188+0000","last_undegraded":"2024-09-10T07:51:13.654188+0000","last_fullsized":"2024-09-10T07:51:13.654188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826644+0000","last_change":"2024-09-10T07:51:12.668124+0000","last_active":"2024-09-10T07:51:13.826644+0000","last_peered":"2024-09-10T07:51:13.826644+0000","last_clean":"2024-09-10T07:51:13.826644+0000","last_became_active":"2024-09-10T07:51:12.667798+0000","last_became_peered":"2024-09-10T07:51:12.667798+0000","last_unstale":"2024-09-10T07:51:13.826644+0000","last_undegraded":"2024-09-10T07:51:13.826644+0000","last_fullsized":"2024-09-10T07:51:13.826644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:27.002731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656452+0000","last_change":"2024-09-10T07:51:12.663829+0000","last_active":"2024-09-10T07:51:13.656452+0000","last_peered":"2024-09-10T07:51:13.656452+0000","last_clean":"2024-09-10T07:51:13.656452+0000","last_became_active":"2024-09-10T07:51:12.663536+0000","last_became_peered":"2024-09-10T07:51:12.663536+0000","last_unstale":"2024-09-10T07:51:13.656452+0000","last_undegraded":"2024-09-10T07:51:13.656452+0000","last_fullsized":"2024-09-10T07:51:13.656452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654749+0000","last_change":"2024-09-10T07:51:12.667036+0000","last_active":"2024-09-10T07:51:13.654749+0000","last_peered":"2024-09-10T07:51:13.654749+0000","last_clean":"2024-09-10T07:51:13.654749+0000","last_became_active":"2024-09-10T07:51:12.666744+0000","last_became_peered":"2024-09-10T07:51:12.666744+0000","last_unstale":"2024-09-10T07:51:13.654749+0000","last_undegraded":"2024-09-10T07:51:13.654749+0000","last_fullsized":"2024-09-10T07:51:13.654749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828107+0000","last_change":"2024-09-10T07:51:12.663072+0000","last_active":"2024-09-10T07:51:13.828107+0000","last_peered":"2024-09-10T07:51:13.828107+0000","last_clean":"2024-09-10T07:51:13.828107+0000","last_became_active":"2024-09-10T07:51:12.662799+0000","last_became_peered":"2024-09-10T07:51:12.662799+0000","last_unstale":"2024-09-10T07:51:13.828107+0000","last_undegraded":"2024-09-10T07:51:13.828107+0000","last_fullsized":"2024-09-10T07:51:13.828107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:40.034143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.654324+0000","last_change":"2024-09-10T07:51:12.647897+0000","last_active":"2024-09-10T07:51:12.654324+0000","last_peered":"2024-09-10T07:51:12.654324+0000","last_clean":"2024-09-10T07:51:12.654324+0000","last_became_active":"2024-09-10T07:51:12.647493+0000","last_became_peered":"2024-09-10T07:51:12.647493+0000","last_unstale":"2024-09-10T07:51:12.654324+0000","last_undegraded":"2024-09-10T07:51:12.654324+0000","last_fullsized":"2024-09-10T07:51:12.654324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:54.143299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655874+0000","last_change":"2024-09-10T07:51:12.675728+0000","last_active":"2024-09-10T07:51:13.655874+0000","last_peered":"2024-09-10T07:51:13.655874+0000","last_clean":"2024-09-10T07:51:13.655874+0000","last_became_active":"2024-09-10T07:51:12.675422+0000","last_became_peered":"2024-09-10T07:51:12.675422+0000","last_unstale":"2024-09-10T07:51:13.655874+0000","last_undegraded":"2024-09-10T07:51:13.655874+0000","last_fullsized":"2024-09-10T07:51:13.655874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:59.255796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.677787+0000","last_change":"2024-09-10T07:51:12.676376+0000","last_active":"2024-09-10T07:51:12.677787+0000","last_peered":"2024-09-10T07:51:12.677787+0000","last_clean":"2024-09-10T07:51:12.677787+0000","last_became_active":"2024-09-10T07:51:12.676068+0000","last_became_peered":"2024-09-10T07:51:12.676068+0000","last_unstale":"2024-09-10T07:51:12.677787+0000","last_undegraded":"2024-09-10T07:51:12.677787+0000","last_fullsized":"2024-09-10T07:51:12.677787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:03.847359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.664304+0000","last_change":"2024-09-10T07:51:12.659238+0000","last_active":"2024-09-10T07:51:12.664304+0000","last_peered":"2024-09-10T07:51:12.664304+0000","last_clean":"2024-09-10T07:51:12.664304+0000","last_became_active":"2024-09-10T07:51:12.658975+0000","last_became_peered":"2024-09-10T07:51:12.658975+0000","last_unstale":"2024-09-10T07:51:12.664304+0000","last_undegraded":"2024-09-10T07:51:12.664304+0000","last_fullsized":"2024-09-10T07:51:12.664304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:48.409488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827585+0000","last_change":"2024-09-10T07:51:12.675441+0000","last_active":"2024-09-10T07:51:13.827585+0000","last_peered":"2024-09-10T07:51:13.827585+0000","last_clean":"2024-09-10T07:51:13.827585+0000","last_became_active":"2024-09-10T07:51:12.673010+0000","last_became_peered":"2024-09-10T07:51:12.673010+0000","last_unstale":"2024-09-10T07:51:13.827585+0000","last_undegraded":"2024-09-10T07:51:13.827585+0000","last_fullsized":"2024-09-10T07:51:13.827585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:20.220354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827527+0000","last_change":"2024-09-10T07:51:12.657787+0000","last_active":"2024-09-10T07:51:13.827527+0000","last_peered":"2024-09-10T07:51:13.827527+0000","last_clean":"2024-09-10T07:51:13.827527+0000","last_became_active":"2024-09-10T07:51:12.657501+0000","last_became_peered":"2024-09-10T07:51:12.657501+0000","last_unstale":"2024-09-10T07:51:13.827527+0000","last_undegraded":"2024-09-10T07:51:13.827527+0000","last_fullsized":"2024-09-10T07:51:13.827527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:21.758327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827383+0000","last_change":"2024-09-10T07:51:12.674036+0000","last_active":"2024-09-10T07:51:13.827383+0000","last_peered":"2024-09-10T07:51:13.827383+0000","last_clean":"2024-09-10T07:51:13.827383+0000","last_became_active":"2024-09-10T07:51:12.673722+0000","last_became_peered":"2024-09-10T07:51:12.673722+0000","last_unstale":"2024-09-10T07:51:13.827383+0000","last_undegraded":"2024-09-10T07:51:13.827383+0000","last_fullsized":"2024-09-10T07:51:13.827383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:17.696759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827952+0000","last_change":"2024-09-10T07:51:12.663140+0000","last_active":"2024-09-10T07:51:13.827952+0000","last_peered":"2024-09-10T07:51:13.827952+0000","last_clean":"2024-09-10T07:51:13.827952+0000","last_became_active":"2024-09-10T07:51:12.662830+0000","last_became_peered":"2024-09-10T07:51:12.662830+0000","last_unstale":"2024-09-10T07:51:13.827952+0000","last_undegraded":"2024-09-10T07:51:13.827952+0000","last_fullsized":"2024-09-10T07:51:13.827952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:52.000639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827123+0000","last_change":"2024-09-10T07:51:12.663045+0000","last_active":"2024-09-10T07:51:13.827123+0000","last_peered":"2024-09-10T07:51:13.827123+0000","last_clean":"2024-09-10T07:51:13.827123+0000","last_became_active":"2024-09-10T07:51:12.662741+0000","last_became_peered":"2024-09-10T07:51:12.662741+0000","last_unstale":"2024-09-10T07:51:13.827123+0000","last_undegraded":"2024-09-10T07:51:13.827123+0000","last_fullsized":"2024-09-10T07:51:13.827123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:49.480454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826731+0000","last_change":"2024-09-10T07:51:12.660520+0000","last_active":"2024-09-10T07:51:13.826731+0000","last_peered":"2024-09-10T07:51:13.826731+0000","last_clean":"2024-09-10T07:51:13.826731+0000","last_became_active":"2024-09-10T07:51:12.660258+0000","last_became_peered":"2024-09-10T07:51:12.660258+0000","last_unstale":"2024-09-10T07:51:13.826731+0000","last_undegraded":"2024-09-10T07:51:13.826731+0000","last_fullsized":"2024-09-10T07:51:13.826731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:10.975319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827016+0000","last_change":"2024-09-10T07:51:12.657447+0000","last_active":"2024-09-10T07:51:13.827016+0000","last_peered":"2024-09-10T07:51:13.827016+0000","last_clean":"2024-09-10T07:51:13.827016+0000","last_became_active":"2024-09-10T07:51:12.657127+0000","last_became_peered":"2024-09-10T07:51:12.657127+0000","last_unstale":"2024-09-10T07:51:13.827016+0000","last_undegraded":"2024-09-10T07:51:13.827016+0000","last_fullsized":"2024-09-10T07:51:13.827016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654357+0000","last_change":"2024-09-10T07:51:12.679669+0000","last_active":"2024-09-10T07:51:13.654357+0000","last_peered":"2024-09-10T07:51:13.654357+0000","last_clean":"2024-09-10T07:51:13.654357+0000","last_became_active":"2024-09-10T07:51:12.679361+0000","last_became_peered":"2024-09-10T07:51:12.679361+0000","last_unstale":"2024-09-10T07:51:13.654357+0000","last_undegraded":"2024-09-10T07:51:13.654357+0000","last_fullsized":"2024-09-10T07:51:13.654357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827729+0000","last_change":"2024-09-10T07:51:12.667532+0000","last_active":"2024-09-10T07:51:13.827729+0000","last_peered":"2024-09-10T07:51:13.827729+0000","last_clean":"2024-09-10T07:51:13.827729+0000","last_became_active":"2024-09-10T07:51:12.666659+0000","last_became_peered":"2024-09-10T07:51:12.666659+0000","last_unstale":"2024-09-10T07:51:13.827729+0000","last_undegraded":"2024-09-10T07:51:13.827729+0000","last_fullsized":"2024-09-10T07:51:13.827729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:33.240184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654775+0000","last_change":"2024-09-10T07:51:12.679672+0000","last_active":"2024-09-10T07:51:13.654775+0000","last_peered":"2024-09-10T07:51:13.654775+0000","last_clean":"2024-09-10T07:51:13.654775+0000","last_became_active":"2024-09-10T07:51:12.679422+0000","last_became_peered":"2024-09-10T07:51:12.679422+0000","last_unstale":"2024-09-10T07:51:13.654775+0000","last_undegraded":"2024-09-10T07:51:13.654775+0000","last_fullsized":"2024-09-10T07:51:13.654775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827692+0000","last_change":"2024-09-10T07:51:12.681734+0000","last_active":"2024-09-10T07:51:13.827692+0000","last_peered":"2024-09-10T07:51:13.827692+0000","last_clean":"2024-09-10T07:51:13.827692+0000","last_became_active":"2024-09-10T07:51:12.681454+0000","last_became_peered":"2024-09-10T07:51:12.681454+0000","last_unstale":"2024-09-10T07:51:13.827692+0000","last_undegraded":"2024-09-10T07:51:13.827692+0000","last_fullsized":"2024-09-10T07:51:13.827692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:59.692249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827301+0000","last_change":"2024-09-10T07:51:12.661140+0000","last_active":"2024-09-10T07:51:13.827301+0000","last_peered":"2024-09-10T07:51:13.827301+0000","last_clean":"2024-09-10T07:51:13.827301+0000","last_became_active":"2024-09-10T07:51:12.660758+0000","last_became_peered":"2024-09-10T07:51:12.660758+0000","last_unstale":"2024-09-10T07:51:13.827301+0000","last_undegraded":"2024-09-10T07:51:13.827301+0000","last_fullsized":"2024-09-10T07:51:13.827301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:12.250282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827897+0000","last_change":"2024-09-10T07:51:12.670722+0000","last_active":"2024-09-10T07:51:13.827897+0000","last_peered":"2024-09-10T07:51:13.827897+0000","last_clean":"2024-09-10T07:51:13.827897+0000","last_became_active":"2024-09-10T07:51:12.667899+0000","last_became_peered":"2024-09-10T07:51:12.667899+0000","last_unstale":"2024-09-10T07:51:13.827897+0000","last_undegraded":"2024-09-10T07:51:13.827897+0000","last_fullsized":"2024-09-10T07:51:13.827897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656066+0000","last_change":"2024-09-10T07:51:12.680797+0000","last_active":"2024-09-10T07:51:13.656066+0000","last_peered":"2024-09-10T07:51:13.656066+0000","last_clean":"2024-09-10T07:51:13.656066+0000","last_became_active":"2024-09-10T07:51:12.680521+0000","last_became_peered":"2024-09-10T07:51:12.680521+0000","last_unstale":"2024-09-10T07:51:13.656066+0000","last_undegraded":"2024-09-10T07:51:13.656066+0000","last_fullsized":"2024-09-10T07:51:13.656066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828525+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.828525+0000","last_peered":"2024-09-10T07:51:13.828525+0000","last_clean":"2024-09-10T07:51:13.828525+0000","last_became_active":"2024-09-10T07:51:12.666221+0000","last_became_peered":"2024-09-10T07:51:12.666221+0000","last_unstale":"2024-09-10T07:51:13.828525+0000","last_undegraded":"2024-09-10T07:51:13.828525+0000","last_fullsized":"2024-09-10T07:51:13.828525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:20.727768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826628+0000","last_change":"2024-09-10T07:51:12.667382+0000","last_active":"2024-09-10T07:51:13.826628+0000","last_peered":"2024-09-10T07:51:13.826628+0000","last_clean":"2024-09-10T07:51:13.826628+0000","last_became_active":"2024-09-10T07:51:12.667121+0000","last_became_peered":"2024-09-10T07:51:12.667121+0000","last_unstale":"2024-09-10T07:51:13.826628+0000","last_undegraded":"2024-09-10T07:51:13.826628+0000","last_fullsized":"2024-09-10T07:51:13.826628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826696+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.826696+0000","last_peered":"2024-09-10T07:51:13.826696+0000","last_clean":"2024-09-10T07:51:13.826696+0000","last_became_active":"2024-09-10T07:51:12.668445+0000","last_became_peered":"2024-09-10T07:51:12.668445+0000","last_unstale":"2024-09-10T07:51:13.826696+0000","last_undegraded":"2024-09-10T07:51:13.826696+0000","last_fullsized":"2024-09-10T07:51:13.826696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:56.232909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827833+0000","last_change":"2024-09-10T07:51:12.679812+0000","last_active":"2024-09-10T07:51:13.827833+0000","last_peered":"2024-09-10T07:51:13.827833+0000","last_clean":"2024-09-10T07:51:13.827833+0000","last_became_active":"2024-09-10T07:51:12.669963+0000","last_became_peered":"2024-09-10T07:51:12.669963+0000","last_unstale":"2024-09-10T07:51:13.827833+0000","last_undegraded":"2024-09-10T07:51:13.827833+0000","last_fullsized":"2024-09-10T07:51:13.827833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:47.211095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828342+0000","last_change":"2024-09-10T07:51:11.647694+0000","last_active":"2024-09-10T07:51:13.828342+0000","last_peered":"2024-09-10T07:51:13.828342+0000","last_clean":"2024-09-10T07:51:13.828342+0000","last_became_active":"2024-09-10T07:51:11.647419+0000","last_became_peered":"2024-09-10T07:51:11.647419+0000","last_unstale":"2024-09-10T07:51:13.828342+0000","last_undegraded":"2024-09-10T07:51:13.828342+0000","last_fullsized":"2024-09-10T07:51:13.828342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:07.450590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656051+0000","last_change":"2024-09-10T07:51:12.660668+0000","last_active":"2024-09-10T07:51:13.656051+0000","last_peered":"2024-09-10T07:51:13.656051+0000","last_clean":"2024-09-10T07:51:13.656051+0000","last_became_active":"2024-09-10T07:51:12.660364+0000","last_became_peered":"2024-09-10T07:51:12.660364+0000","last_unstale":"2024-09-10T07:51:13.656051+0000","last_undegraded":"2024-09-10T07:51:13.656051+0000","last_fullsized":"2024-09-10T07:51:13.656051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827698+0000","last_change":"2024-09-10T07:51:11.657105+0000","last_active":"2024-09-10T07:51:13.827698+0000","last_peered":"2024-09-10T07:51:13.827698+0000","last_clean":"2024-09-10T07:51:13.827698+0000","last_became_active":"2024-09-10T07:51:11.656523+0000","last_became_peered":"2024-09-10T07:51:11.656523+0000","last_unstale":"2024-09-10T07:51:13.827698+0000","last_undegraded":"2024-09-10T07:51:13.827698+0000","last_fullsized":"2024-09-10T07:51:13.827698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:08.541922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827892+0000","last_change":"2024-09-10T07:51:12.669879+0000","last_active":"2024-09-10T07:51:13.827892+0000","last_peered":"2024-09-10T07:51:13.827892+0000","last_clean":"2024-09-10T07:51:13.827892+0000","last_became_active":"2024-09-10T07:51:12.668574+0000","last_became_peered":"2024-09-10T07:51:12.668574+0000","last_unstale":"2024-09-10T07:51:13.827892+0000","last_undegraded":"2024-09-10T07:51:13.827892+0000","last_fullsized":"2024-09-10T07:51:13.827892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828196+0000","last_change":"2024-09-10T07:51:11.648632+0000","last_active":"2024-09-10T07:51:13.828196+0000","last_peered":"2024-09-10T07:51:13.828196+0000","last_clean":"2024-09-10T07:51:13.828196+0000","last_became_active":"2024-09-10T07:51:11.648267+0000","last_became_peered":"2024-09-10T07:51:11.648267+0000","last_unstale":"2024-09-10T07:51:13.828196+0000","last_undegraded":"2024-09-10T07:51:13.828196+0000","last_fullsized":"2024-09-10T07:51:13.828196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:41.341276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654539+0000","last_change":"2024-09-10T07:51:12.670265+0000","last_active":"2024-09-10T07:51:13.654539+0000","last_peered":"2024-09-10T07:51:13.654539+0000","last_clean":"2024-09-10T07:51:13.654539+0000","last_became_active":"2024-09-10T07:51:12.669524+0000","last_became_peered":"2024-09-10T07:51:12.669524+0000","last_unstale":"2024-09-10T07:51:13.654539+0000","last_undegraded":"2024-09-10T07:51:13.654539+0000","last_fullsized":"2024-09-10T07:51:13.654539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827979+0000","last_change":"2024-09-10T07:51:11.661609+0000","last_active":"2024-09-10T07:51:13.827979+0000","last_peered":"2024-09-10T07:51:13.827979+0000","last_clean":"2024-09-10T07:51:13.827979+0000","last_became_active":"2024-09-10T07:51:11.660016+0000","last_became_peered":"2024-09-10T07:51:11.660016+0000","last_unstale":"2024-09-10T07:51:13.827979+0000","last_undegraded":"2024-09-10T07:51:13.827979+0000","last_fullsized":"2024-09-10T07:51:13.827979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:46.277801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828438+0000","last_change":"2024-09-10T07:51:12.667278+0000","last_active":"2024-09-10T07:51:13.828438+0000","last_peered":"2024-09-10T07:51:13.828438+0000","last_clean":"2024-09-10T07:51:13.828438+0000","last_became_active":"2024-09-10T07:51:12.666209+0000","last_became_peered":"2024-09-10T07:51:12.666209+0000","last_unstale":"2024-09-10T07:51:13.828438+0000","last_undegraded":"2024-09-10T07:51:13.828438+0000","last_fullsized":"2024-09-10T07:51:13.828438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:26.172183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.671056+0000","last_change":"2024-09-10T07:51:11.654006+0000","last_active":"2024-09-10T07:51:12.671056+0000","last_peered":"2024-09-10T07:51:12.671056+0000","last_clean":"2024-09-10T07:51:12.671056+0000","last_became_active":"2024-09-10T07:51:11.651332+0000","last_became_peered":"2024-09-10T07:51:11.651332+0000","last_unstale":"2024-09-10T07:51:12.671056+0000","last_undegraded":"2024-09-10T07:51:12.671056+0000","last_fullsized":"2024-09-10T07:51:12.671056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:41.047040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828245+0000","last_change":"2024-09-10T07:51:12.666838+0000","last_active":"2024-09-10T07:51:13.828245+0000","last_peered":"2024-09-10T07:51:13.828245+0000","last_clean":"2024-09-10T07:51:13.828245+0000","last_became_active":"2024-09-10T07:51:12.666551+0000","last_became_peered":"2024-09-10T07:51:12.666551+0000","last_unstale":"2024-09-10T07:51:13.828245+0000","last_undegraded":"2024-09-10T07:51:13.828245+0000","last_fullsized":"2024-09-10T07:51:13.828245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654240+0000","last_change":"2024-09-10T07:51:11.666134+0000","last_active":"2024-09-10T07:51:13.654240+0000","last_peered":"2024-09-10T07:51:13.654240+0000","last_clean":"2024-09-10T07:51:13.654240+0000","last_became_active":"2024-09-10T07:51:11.665826+0000","last_became_peered":"2024-09-10T07:51:11.665826+0000","last_unstale":"2024-09-10T07:51:13.654240+0000","last_undegraded":"2024-09-10T07:51:13.654240+0000","last_fullsized":"2024-09-10T07:51:13.654240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:49.241190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656243+0000","last_change":"2024-09-10T07:51:12.667702+0000","last_active":"2024-09-10T07:51:13.656243+0000","last_peered":"2024-09-10T07:51:13.656243+0000","last_clean":"2024-09-10T07:51:13.656243+0000","last_became_active":"2024-09-10T07:51:12.667391+0000","last_became_peered":"2024-09-10T07:51:12.667391+0000","last_unstale":"2024-09-10T07:51:13.656243+0000","last_undegraded":"2024-09-10T07:51:13.656243+0000","last_fullsized":"2024-09-10T07:51:13.656243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.456232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655436+0000","last_change":"2024-09-10T07:51:11.664356+0000","last_active":"2024-09-10T07:51:13.655436+0000","last_peered":"2024-09-10T07:51:13.655436+0000","last_clean":"2024-09-10T07:51:13.655436+0000","last_became_active":"2024-09-10T07:51:11.663984+0000","last_became_peered":"2024-09-10T07:51:11.663984+0000","last_unstale":"2024-09-10T07:51:13.655436+0000","last_undegraded":"2024-09-10T07:51:13.655436+0000","last_fullsized":"2024-09-10T07:51:13.655436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:02.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.669674+0000","last_change":"2024-09-10T07:51:12.668160+0000","last_active":"2024-09-10T07:51:12.669674+0000","last_peered":"2024-09-10T07:51:12.669674+0000","last_clean":"2024-09-10T07:51:12.669674+0000","last_became_active":"2024-09-10T07:51:12.667845+0000","last_became_peered":"2024-09-10T07:51:12.667845+0000","last_unstale":"2024-09-10T07:51:12.669674+0000","last_undegraded":"2024-09-10T07:51:12.669674+0000","last_fullsized":"2024-09-10T07:51:12.669674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:57.084096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642604+0000","last_change":"2024-09-10T07:51:11.647781+0000","last_active":"2024-09-10T07:51:12.642604+0000","last_peered":"2024-09-10T07:51:12.642604+0000","last_clean":"2024-09-10T07:51:12.642604+0000","last_became_active":"2024-09-10T07:51:11.647502+0000","last_became_peered":"2024-09-10T07:51:11.647502+0000","last_unstale":"2024-09-10T07:51:12.642604+0000","last_undegraded":"2024-09-10T07:51:12.642604+0000","last_fullsized":"2024-09-10T07:51:12.642604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:38.219907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654838+0000","last_change":"2024-09-10T07:51:12.680163+0000","last_active":"2024-09-10T07:51:13.654838+0000","last_peered":"2024-09-10T07:51:13.654838+0000","last_clean":"2024-09-10T07:51:13.654838+0000","last_became_active":"2024-09-10T07:51:12.679482+0000","last_became_peered":"2024-09-10T07:51:12.679482+0000","last_unstale":"2024-09-10T07:51:13.654838+0000","last_undegraded":"2024-09-10T07:51:13.654838+0000","last_fullsized":"2024-09-10T07:51:13.654838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828385+0000","last_change":"2024-09-10T07:51:11.652611+0000","last_active":"2024-09-10T07:51:13.828385+0000","last_peered":"2024-09-10T07:51:13.828385+0000","last_clean":"2024-09-10T07:51:13.828385+0000","last_became_active":"2024-09-10T07:51:11.652322+0000","last_became_peered":"2024-09-10T07:51:11.652322+0000","last_unstale":"2024-09-10T07:51:13.828385+0000","last_undegraded":"2024-09-10T07:51:13.828385+0000","last_fullsized":"2024-09-10T07:51:13.828385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:48.473083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827225+0000","last_change":"2024-09-10T07:51:12.681992+0000","last_active":"2024-09-10T07:51:13.827225+0000","last_peered":"2024-09-10T07:51:13.827225+0000","last_clean":"2024-09-10T07:51:13.827225+0000","last_became_active":"2024-09-10T07:51:12.680523+0000","last_became_peered":"2024-09-10T07:51:12.680523+0000","last_unstale":"2024-09-10T07:51:13.827225+0000","last_undegraded":"2024-09-10T07:51:13.827225+0000","last_fullsized":"2024-09-10T07:51:13.827225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:49.854131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827642+0000","last_change":"2024-09-10T07:51:11.661453+0000","last_active":"2024-09-10T07:51:13.827642+0000","last_peered":"2024-09-10T07:51:13.827642+0000","last_clean":"2024-09-10T07:51:13.827642+0000","last_became_active":"2024-09-10T07:51:11.661202+0000","last_became_peered":"2024-09-10T07:51:11.661202+0000","last_unstale":"2024-09-10T07:51:13.827642+0000","last_undegraded":"2024-09-10T07:51:13.827642+0000","last_fullsized":"2024-09-10T07:51:13.827642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:25.009100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828046+0000","last_change":"2024-09-10T07:51:12.677005+0000","last_active":"2024-09-10T07:51:13.828046+0000","last_peered":"2024-09-10T07:51:13.828046+0000","last_clean":"2024-09-10T07:51:13.828046+0000","last_became_active":"2024-09-10T07:51:12.676702+0000","last_became_peered":"2024-09-10T07:51:12.676702+0000","last_unstale":"2024-09-10T07:51:13.828046+0000","last_undegraded":"2024-09-10T07:51:13.828046+0000","last_fullsized":"2024-09-10T07:51:13.828046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:55.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.665387+0000","last_change":"2024-09-10T07:51:11.651919+0000","last_active":"2024-09-10T07:51:12.665387+0000","last_peered":"2024-09-10T07:51:12.665387+0000","last_clean":"2024-09-10T07:51:12.665387+0000","last_became_active":"2024-09-10T07:51:11.651676+0000","last_became_peered":"2024-09-10T07:51:11.651676+0000","last_unstale":"2024-09-10T07:51:12.665387+0000","last_undegraded":"2024-09-10T07:51:12.665387+0000","last_fullsized":"2024-09-10T07:51:12.665387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827121+0000","last_change":"2024-09-10T07:51:12.661023+0000","last_active":"2024-09-10T07:51:13.827121+0000","last_peered":"2024-09-10T07:51:13.827121+0000","last_clean":"2024-09-10T07:51:13.827121+0000","last_became_active":"2024-09-10T07:51:12.660734+0000","last_became_peered":"2024-09-10T07:51:12.660734+0000","last_unstale":"2024-09-10T07:51:13.827121+0000","last_undegraded":"2024-09-10T07:51:13.827121+0000","last_fullsized":"2024-09-10T07:51:13.827121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:18.004956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656317+0000","last_change":"2024-09-10T07:51:11.659183+0000","last_active":"2024-09-10T07:51:13.656317+0000","last_peered":"2024-09-10T07:51:13.656317+0000","last_clean":"2024-09-10T07:51:13.656317+0000","last_became_active":"2024-09-10T07:51:11.658378+0000","last_became_peered":"2024-09-10T07:51:11.658378+0000","last_unstale":"2024-09-10T07:51:13.656317+0000","last_undegraded":"2024-09-10T07:51:13.656317+0000","last_fullsized":"2024-09-10T07:51:13.656317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:59.230645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827115+0000","last_change":"2024-09-10T07:51:12.647744+0000","last_active":"2024-09-10T07:51:13.827115+0000","last_peered":"2024-09-10T07:51:13.827115+0000","last_clean":"2024-09-10T07:51:13.827115+0000","last_became_active":"2024-09-10T07:51:12.647440+0000","last_became_peered":"2024-09-10T07:51:12.647440+0000","last_unstale":"2024-09-10T07:51:13.827115+0000","last_undegraded":"2024-09-10T07:51:13.827115+0000","last_fullsized":"2024-09-10T07:51:13.827115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655557+0000","last_change":"2024-09-10T07:51:11.666220+0000","last_active":"2024-09-10T07:51:13.655557+0000","last_peered":"2024-09-10T07:51:13.655557+0000","last_clean":"2024-09-10T07:51:13.655557+0000","last_became_active":"2024-09-10T07:51:11.665886+0000","last_became_peered":"2024-09-10T07:51:11.665886+0000","last_unstale":"2024-09-10T07:51:13.655557+0000","last_undegraded":"2024-09-10T07:51:13.655557+0000","last_fullsized":"2024-09-10T07:51:13.655557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:50.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654970+0000","last_change":"2024-09-10T07:51:12.682681+0000","last_active":"2024-09-10T07:51:13.654970+0000","last_peered":"2024-09-10T07:51:13.654970+0000","last_clean":"2024-09-10T07:51:13.654970+0000","last_became_active":"2024-09-10T07:51:12.682424+0000","last_became_peered":"2024-09-10T07:51:12.682424+0000","last_unstale":"2024-09-10T07:51:13.654970+0000","last_undegraded":"2024-09-10T07:51:13.654970+0000","last_fullsized":"2024-09-10T07:51:13.654970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654264+0000","last_change":"2024-09-10T07:51:11.661553+0000","last_active":"2024-09-10T07:51:13.654264+0000","last_peered":"2024-09-10T07:51:13.654264+0000","last_clean":"2024-09-10T07:51:13.654264+0000","last_became_active":"2024-09-10T07:51:11.661263+0000","last_became_peered":"2024-09-10T07:51:11.661263+0000","last_unstale":"2024-09-10T07:51:13.654264+0000","last_undegraded":"2024-09-10T07:51:13.654264+0000","last_fullsized":"2024-09-10T07:51:13.654264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:54.532976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826983+0000","last_change":"2024-09-10T07:51:12.661693+0000","last_active":"2024-09-10T07:51:13.826983+0000","last_peered":"2024-09-10T07:51:13.826983+0000","last_clean":"2024-09-10T07:51:13.826983+0000","last_became_active":"2024-09-10T07:51:12.661427+0000","last_became_peered":"2024-09-10T07:51:12.661427+0000","last_unstale":"2024-09-10T07:51:13.826983+0000","last_undegraded":"2024-09-10T07:51:13.826983+0000","last_fullsized":"2024-09-10T07:51:13.826983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:15.540616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654447+0000","last_change":"2024-09-10T07:51:11.657359+0000","last_active":"2024-09-10T07:51:13.654447+0000","last_peered":"2024-09-10T07:51:13.654447+0000","last_clean":"2024-09-10T07:51:13.654447+0000","last_became_active":"2024-09-10T07:51:11.657038+0000","last_became_peered":"2024-09-10T07:51:11.657038+0000","last_unstale":"2024-09-10T07:51:13.654447+0000","last_undegraded":"2024-09-10T07:51:13.654447+0000","last_fullsized":"2024-09-10T07:51:13.654447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:55.533736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656447+0000","last_change":"2024-09-10T07:51:12.661323+0000","last_active":"2024-09-10T07:51:13.656447+0000","last_peered":"2024-09-10T07:51:13.656447+0000","last_clean":"2024-09-10T07:51:13.656447+0000","last_became_active":"2024-09-10T07:51:12.658712+0000","last_became_peered":"2024-09-10T07:51:12.658712+0000","last_unstale":"2024-09-10T07:51:13.656447+0000","last_undegraded":"2024-09-10T07:51:13.656447+0000","last_fullsized":"2024-09-10T07:51:13.656447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654091+0000","last_change":"2024-09-10T07:51:11.664261+0000","last_active":"2024-09-10T07:51:13.654091+0000","last_peered":"2024-09-10T07:51:13.654091+0000","last_clean":"2024-09-10T07:51:13.654091+0000","last_became_active":"2024-09-10T07:51:11.663940+0000","last_became_peered":"2024-09-10T07:51:11.663940+0000","last_unstale":"2024-09-10T07:51:13.654091+0000","last_undegraded":"2024-09-10T07:51:13.654091+0000","last_fullsized":"2024-09-10T07:51:13.654091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:20.541470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827555+0000","last_change":"2024-09-10T07:51:12.658206+0000","last_active":"2024-09-10T07:51:13.827555+0000","last_peered":"2024-09-10T07:51:13.827555+0000","last_clean":"2024-09-10T07:51:13.827555+0000","last_became_active":"2024-09-10T07:51:12.657838+0000","last_became_peered":"2024-09-10T07:51:12.657838+0000","last_unstale":"2024-09-10T07:51:13.827555+0000","last_undegraded":"2024-09-10T07:51:13.827555+0000","last_fullsized":"2024-09-10T07:51:13.827555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828648+0000","last_change":"2024-09-10T07:51:11.654332+0000","last_active":"2024-09-10T07:51:13.828648+0000","last_peered":"2024-09-10T07:51:13.828648+0000","last_clean":"2024-09-10T07:51:13.828648+0000","last_became_active":"2024-09-10T07:51:11.654042+0000","last_became_peered":"2024-09-10T07:51:11.654042+0000","last_unstale":"2024-09-10T07:51:13.828648+0000","last_undegraded":"2024-09-10T07:51:13.828648+0000","last_fullsized":"2024-09-10T07:51:13.828648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:16.997585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827284+0000","last_change":"2024-09-10T07:51:12.667032+0000","last_active":"2024-09-10T07:51:13.827284+0000","last_peered":"2024-09-10T07:51:13.827284+0000","last_clean":"2024-09-10T07:51:13.827284+0000","last_became_active":"2024-09-10T07:51:12.666726+0000","last_became_peered":"2024-09-10T07:51:12.666726+0000","last_unstale":"2024-09-10T07:51:13.827284+0000","last_undegraded":"2024-09-10T07:51:13.827284+0000","last_fullsized":"2024-09-10T07:51:13.827284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827732+0000","last_change":"2024-09-10T07:51:11.655876+0000","last_active":"2024-09-10T07:51:13.827732+0000","last_peered":"2024-09-10T07:51:13.827732+0000","last_clean":"2024-09-10T07:51:13.827732+0000","last_became_active":"2024-09-10T07:51:11.655606+0000","last_became_peered":"2024-09-10T07:51:11.655606+0000","last_unstale":"2024-09-10T07:51:13.827732+0000","last_undegraded":"2024-09-10T07:51:13.827732+0000","last_fullsized":"2024-09-10T07:51:13.827732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:36.453798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655926+0000","last_change":"2024-09-10T07:51:12.657238+0000","last_active":"2024-09-10T07:51:13.655926+0000","last_peered":"2024-09-10T07:51:13.655926+0000","last_clean":"2024-09-10T07:51:13.655926+0000","last_became_active":"2024-09-10T07:51:12.656922+0000","last_became_peered":"2024-09-10T07:51:12.656922+0000","last_unstale":"2024-09-10T07:51:13.655926+0000","last_undegraded":"2024-09-10T07:51:13.655926+0000","last_fullsized":"2024-09-10T07:51:13.655926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.556394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826896+0000","last_change":"2024-09-10T07:51:11.654686+0000","last_active":"2024-09-10T07:51:13.826896+0000","last_peered":"2024-09-10T07:51:13.826896+0000","last_clean":"2024-09-10T07:51:13.826896+0000","last_became_active":"2024-09-10T07:51:11.653599+0000","last_became_peered":"2024-09-10T07:51:11.653599+0000","last_unstale":"2024-09-10T07:51:13.826896+0000","last_undegraded":"2024-09-10T07:51:13.826896+0000","last_fullsized":"2024-09-10T07:51:13.826896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:52.484707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656395+0000","last_change":"2024-09-10T07:51:12.680792+0000","last_active":"2024-09-10T07:51:13.656395+0000","last_peered":"2024-09-10T07:51:13.656395+0000","last_clean":"2024-09-10T07:51:13.656395+0000","last_became_active":"2024-09-10T07:51:12.680491+0000","last_became_peered":"2024-09-10T07:51:12.680491+0000","last_unstale":"2024-09-10T07:51:13.656395+0000","last_undegraded":"2024-09-10T07:51:13.656395+0000","last_fullsized":"2024-09-10T07:51:13.656395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828075+0000","last_change":"2024-09-10T07:51:11.652720+0000","last_active":"2024-09-10T07:51:13.828075+0000","last_peered":"2024-09-10T07:51:13.828075+0000","last_clean":"2024-09-10T07:51:13.828075+0000","last_became_active":"2024-09-10T07:51:11.652391+0000","last_became_peered":"2024-09-10T07:51:11.652391+0000","last_unstale":"2024-09-10T07:51:13.828075+0000","last_undegraded":"2024-09-10T07:51:13.828075+0000","last_fullsized":"2024-09-10T07:51:13.828075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:51.242641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655492+0000","last_change":"2024-09-10T07:51:12.653138+0000","last_active":"2024-09-10T07:51:13.655492+0000","last_peered":"2024-09-10T07:51:13.655492+0000","last_clean":"2024-09-10T07:51:13.655492+0000","last_became_active":"2024-09-10T07:51:12.652876+0000","last_became_peered":"2024-09-10T07:51:12.652876+0000","last_unstale":"2024-09-10T07:51:13.655492+0000","last_undegraded":"2024-09-10T07:51:13.655492+0000","last_fullsized":"2024-09-10T07:51:13.655492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:39.429271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827092+0000","last_change":"2024-09-10T07:51:11.644261+0000","last_active":"2024-09-10T07:51:13.827092+0000","last_peered":"2024-09-10T07:51:13.827092+0000","last_clean":"2024-09-10T07:51:13.827092+0000","last_became_active":"2024-09-10T07:51:11.643967+0000","last_became_peered":"2024-09-10T07:51:11.643967+0000","last_unstale":"2024-09-10T07:51:13.827092+0000","last_undegraded":"2024-09-10T07:51:13.827092+0000","last_fullsized":"2024-09-10T07:51:13.827092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:10.313315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655294+0000","last_change":"2024-09-10T07:51:12.647492+0000","last_active":"2024-09-10T07:51:13.655294+0000","last_peered":"2024-09-10T07:51:13.655294+0000","last_clean":"2024-09-10T07:51:13.655294+0000","last_became_active":"2024-09-10T07:51:12.647153+0000","last_became_peered":"2024-09-10T07:51:12.647153+0000","last_unstale":"2024-09-10T07:51:13.655294+0000","last_undegraded":"2024-09-10T07:51:13.655294+0000","last_fullsized":"2024-09-10T07:51:13.655294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826984+0000","last_change":"2024-09-10T07:51:11.659869+0000","last_active":"2024-09-10T07:51:13.826984+0000","last_peered":"2024-09-10T07:51:13.826984+0000","last_clean":"2024-09-10T07:51:13.826984+0000","last_became_active":"2024-09-10T07:51:11.659392+0000","last_became_peered":"2024-09-10T07:51:11.659392+0000","last_unstale":"2024-09-10T07:51:13.826984+0000","last_undegraded":"2024-09-10T07:51:13.826984+0000","last_fullsized":"2024-09-10T07:51:13.826984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:38.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.657374+0000","last_change":"2024-09-10T07:51:12.655133+0000","last_active":"2024-09-10T07:51:12.657374+0000","last_peered":"2024-09-10T07:51:12.657374+0000","last_clean":"2024-09-10T07:51:12.657374+0000","last_became_active":"2024-09-10T07:51:12.654855+0000","last_became_peered":"2024-09-10T07:51:12.654855+0000","last_unstale":"2024-09-10T07:51:12.657374+0000","last_undegraded":"2024-09-10T07:51:12.657374+0000","last_fullsized":"2024-09-10T07:51:12.657374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:18.714900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826722+0000","last_change":"2024-09-10T07:51:11.664218+0000","last_active":"2024-09-10T07:51:13.826722+0000","last_peered":"2024-09-10T07:51:13.826722+0000","last_clean":"2024-09-10T07:51:13.826722+0000","last_became_active":"2024-09-10T07:51:11.663837+0000","last_became_peered":"2024-09-10T07:51:11.663837+0000","last_unstale":"2024-09-10T07:51:13.826722+0000","last_undegraded":"2024-09-10T07:51:13.826722+0000","last_fullsized":"2024-09-10T07:51:13.826722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:43.311010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827577+0000","last_change":"2024-09-10T07:51:12.658871+0000","last_active":"2024-09-10T07:51:13.827577+0000","last_peered":"2024-09-10T07:51:13.827577+0000","last_clean":"2024-09-10T07:51:13.827577+0000","last_became_active":"2024-09-10T07:51:12.658601+0000","last_became_peered":"2024-09-10T07:51:12.658601+0000","last_unstale":"2024-09-10T07:51:13.827577+0000","last_undegraded":"2024-09-10T07:51:13.827577+0000","last_fullsized":"2024-09-10T07:51:13.827577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:02.300806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828721+0000","last_change":"2024-09-10T07:51:11.658990+0000","last_active":"2024-09-10T07:51:13.828721+0000","last_peered":"2024-09-10T07:51:13.828721+0000","last_clean":"2024-09-10T07:51:13.828721+0000","last_became_active":"2024-09-10T07:51:11.658451+0000","last_became_peered":"2024-09-10T07:51:11.658451+0000","last_unstale":"2024-09-10T07:51:13.828721+0000","last_undegraded":"2024-09-10T07:51:13.828721+0000","last_fullsized":"2024-09-10T07:51:13.828721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.242911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654799+0000","last_change":"2024-09-10T07:51:12.663182+0000","last_active":"2024-09-10T07:51:13.654799+0000","last_peered":"2024-09-10T07:51:13.654799+0000","last_clean":"2024-09-10T07:51:13.654799+0000","last_became_active":"2024-09-10T07:51:12.662787+0000","last_became_peered":"2024-09-10T07:51:12.662787+0000","last_unstale":"2024-09-10T07:51:13.654799+0000","last_undegraded":"2024-09-10T07:51:13.654799+0000","last_fullsized":"2024-09-10T07:51:13.654799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:37.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827966+0000","last_change":"2024-09-10T07:51:11.655978+0000","last_active":"2024-09-10T07:51:13.827966+0000","last_peered":"2024-09-10T07:51:13.827966+0000","last_clean":"2024-09-10T07:51:13.827966+0000","last_became_active":"2024-09-10T07:51:11.655686+0000","last_became_peered":"2024-09-10T07:51:11.655686+0000","last_unstale":"2024-09-10T07:51:13.827966+0000","last_undegraded":"2024-09-10T07:51:13.827966+0000","last_fullsized":"2024-09-10T07:51:13.827966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:14.533246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828044+0000","last_change":"2024-09-10T07:51:12.665991+0000","last_active":"2024-09-10T07:51:13.828044+0000","last_peered":"2024-09-10T07:51:13.828044+0000","last_clean":"2024-09-10T07:51:13.828044+0000","last_became_active":"2024-09-10T07:51:12.665689+0000","last_became_peered":"2024-09-10T07:51:12.665689+0000","last_unstale":"2024-09-10T07:51:13.828044+0000","last_undegraded":"2024-09-10T07:51:13.828044+0000","last_fullsized":"2024-09-10T07:51:13.828044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:39.884571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643075+0000","last_change":"2024-09-10T07:51:11.653391+0000","last_active":"2024-09-10T07:51:12.643075+0000","last_peered":"2024-09-10T07:51:12.643075+0000","last_clean":"2024-09-10T07:51:12.643075+0000","last_became_active":"2024-09-10T07:51:11.652803+0000","last_became_peered":"2024-09-10T07:51:11.652803+0000","last_unstale":"2024-09-10T07:51:12.643075+0000","last_undegraded":"2024-09-10T07:51:12.643075+0000","last_fullsized":"2024-09-10T07:51:12.643075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:38.179483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654603+0000","last_change":"2024-09-10T07:51:12.676013+0000","last_active":"2024-09-10T07:51:13.654603+0000","last_peered":"2024-09-10T07:51:13.654603+0000","last_clean":"2024-09-10T07:51:13.654603+0000","last_became_active":"2024-09-10T07:51:12.675718+0000","last_became_peered":"2024-09-10T07:51:12.675718+0000","last_unstale":"2024-09-10T07:51:13.654603+0000","last_undegraded":"2024-09-10T07:51:13.654603+0000","last_fullsized":"2024-09-10T07:51:13.654603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642361+0000","last_change":"2024-09-10T07:51:11.653518+0000","last_active":"2024-09-10T07:51:12.642361+0000","last_peered":"2024-09-10T07:51:12.642361+0000","last_clean":"2024-09-10T07:51:12.642361+0000","last_became_active":"2024-09-10T07:51:11.653198+0000","last_became_peered":"2024-09-10T07:51:11.653198+0000","last_unstale":"2024-09-10T07:51:12.642361+0000","last_undegraded":"2024-09-10T07:51:12.642361+0000","last_fullsized":"2024-09-10T07:51:12.642361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.482793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827696+0000","last_change":"2024-09-10T07:51:12.657827+0000","last_active":"2024-09-10T07:51:13.827696+0000","last_peered":"2024-09-10T07:51:13.827696+0000","last_clean":"2024-09-10T07:51:13.827696+0000","last_became_active":"2024-09-10T07:51:12.657483+0000","last_became_peered":"2024-09-10T07:51:12.657483+0000","last_unstale":"2024-09-10T07:51:13.827696+0000","last_undegraded":"2024-09-10T07:51:13.827696+0000","last_fullsized":"2024-09-10T07:51:13.827696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.217868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643007+0000","last_change":"2024-09-10T07:51:11.653782+0000","last_active":"2024-09-10T07:51:12.643007+0000","last_peered":"2024-09-10T07:51:12.643007+0000","last_clean":"2024-09-10T07:51:12.643007+0000","last_became_active":"2024-09-10T07:51:11.650110+0000","last_became_peered":"2024-09-10T07:51:11.650110+0000","last_unstale":"2024-09-10T07:51:12.643007+0000","last_undegraded":"2024-09-10T07:51:12.643007+0000","last_fullsized":"2024-09-10T07:51:12.643007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:16.299195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827512+0000","last_change":"2024-09-10T07:51:12.677759+0000","last_active":"2024-09-10T07:51:13.827512+0000","last_peered":"2024-09-10T07:51:13.827512+0000","last_clean":"2024-09-10T07:51:13.827512+0000","last_became_active":"2024-09-10T07:51:12.677359+0000","last_became_peered":"2024-09-10T07:51:12.677359+0000","last_unstale":"2024-09-10T07:51:13.827512+0000","last_undegraded":"2024-09-10T07:51:13.827512+0000","last_fullsized":"2024-09-10T07:51:13.827512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:25.109899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827404+0000","last_change":"2024-09-10T07:51:11.662912+0000","last_active":"2024-09-10T07:51:13.827404+0000","last_peered":"2024-09-10T07:51:13.827404+0000","last_clean":"2024-09-10T07:51:13.827404+0000","last_became_active":"2024-09-10T07:51:11.662677+0000","last_became_peered":"2024-09-10T07:51:11.662677+0000","last_unstale":"2024-09-10T07:51:13.827404+0000","last_undegraded":"2024-09-10T07:51:13.827404+0000","last_fullsized":"2024-09-10T07:51:13.827404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:18.751797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655470+0000","last_change":"2024-09-10T07:51:12.672712+0000","last_active":"2024-09-10T07:51:13.655470+0000","last_peered":"2024-09-10T07:51:13.655470+0000","last_clean":"2024-09-10T07:51:13.655470+0000","last_became_active":"2024-09-10T07:51:12.670900+0000","last_became_peered":"2024-09-10T07:51:12.670900+0000","last_unstale":"2024-09-10T07:51:13.655470+0000","last_undegraded":"2024-09-10T07:51:13.655470+0000","last_fullsized":"2024-09-10T07:51:13.655470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827078+0000","last_change":"2024-09-10T07:51:11.655026+0000","last_active":"2024-09-10T07:51:13.827078+0000","last_peered":"2024-09-10T07:51:13.827078+0000","last_clean":"2024-09-10T07:51:13.827078+0000","last_became_active":"2024-09-10T07:51:11.654747+0000","last_became_peered":"2024-09-10T07:51:11.654747+0000","last_unstale":"2024-09-10T07:51:13.827078+0000","last_undegraded":"2024-09-10T07:51:13.827078+0000","last_fullsized":"2024-09-10T07:51:13.827078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:13.073312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826895+0000","last_change":"2024-09-10T07:51:12.680995+0000","last_active":"2024-09-10T07:51:13.826895+0000","last_peered":"2024-09-10T07:51:13.826895+0000","last_clean":"2024-09-10T07:51:13.826895+0000","last_became_active":"2024-09-10T07:51:12.680592+0000","last_became_peered":"2024-09-10T07:51:12.680592+0000","last_unstale":"2024-09-10T07:51:13.826895+0000","last_undegraded":"2024-09-10T07:51:13.826895+0000","last_fullsized":"2024-09-10T07:51:13.826895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:43.727455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826629+0000","last_change":"2024-09-10T07:51:11.659975+0000","last_active":"2024-09-10T07:51:13.826629+0000","last_peered":"2024-09-10T07:51:13.826629+0000","last_clean":"2024-09-10T07:51:13.826629+0000","last_became_active":"2024-09-10T07:51:11.657387+0000","last_became_peered":"2024-09-10T07:51:11.657387+0000","last_unstale":"2024-09-10T07:51:13.826629+0000","last_undegraded":"2024-09-10T07:51:13.826629+0000","last_fullsized":"2024-09-10T07:51:13.826629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:07.331178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827325+0000","last_change":"2024-09-10T07:51:12.675741+0000","last_active":"2024-09-10T07:51:13.827325+0000","last_peered":"2024-09-10T07:51:13.827325+0000","last_clean":"2024-09-10T07:51:13.827325+0000","last_became_active":"2024-09-10T07:51:12.675054+0000","last_became_peered":"2024-09-10T07:51:12.675054+0000","last_unstale":"2024-09-10T07:51:13.827325+0000","last_undegraded":"2024-09-10T07:51:13.827325+0000","last_fullsized":"2024-09-10T07:51:13.827325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:46.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654538+0000","last_change":"2024-09-10T07:51:11.660202+0000","last_active":"2024-09-10T07:51:13.654538+0000","last_peered":"2024-09-10T07:51:13.654538+0000","last_clean":"2024-09-10T07:51:13.654538+0000","last_became_active":"2024-09-10T07:51:11.659089+0000","last_became_peered":"2024-09-10T07:51:11.659089+0000","last_unstale":"2024-09-10T07:51:13.654538+0000","last_undegraded":"2024-09-10T07:51:13.654538+0000","last_fullsized":"2024-09-10T07:51:13.654538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:34.657006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":273,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654859+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:51:13.654859+0000","last_peered":"2024-09-10T07:51:13.654859+0000","last_clean":"2024-09-10T07:51:13.654859+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:51:13.654859+0000","last_undegraded":"2024-09-10T07:51:13.654859+0000","last_fullsized":"2024-09-10T07:51:13.654859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476799,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":402248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1531904,"data_stored":992490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":403734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":403768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1536000,"data_stored":992580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":913408,"data_stored":389047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:51:17.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:51:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.886+0000 7ff9e8200640 1 Processor -- start 2024-09-10T07:51:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.886+0000 7ff9e8200640 1 -- start start 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.887+0000 7ff9e8200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9e015c720 0x7ff9e0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.887+0000 7ff9e8200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e0070f30 0x7ff9e0073610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.887+0000 7ff9e8200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00cd7f0 con 0x7ff9e00cfdc0 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.887+0000 7ff9e8200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e0072210 con 0x7ff9e015c720 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.887+0000 7ff9e8200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00cf880 con 0x7ff9e0070f30 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9e015c720 0x7ff9e0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9e015c720 0x7ff9e0071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55492/0 (socket says 172.21.15.3:55492) 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 -- 172.21.15.3:0/3332119429 learned_addr learned my addr 172.21.15.3:0/3332119429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e4e00640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e0070f30 0x7ff9e0073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 -- 172.21.15.3:0/3332119429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e0070f30 msgr2=0x7ff9e0073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e0070f30 0x7ff9e0073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 -- 172.21.15.3:0/3332119429 >> v1:172.21.15.70:6789/0 conn(0x7ff9e00cfdc0 legacy=0x7ff9e00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.888+0000 7ff9e5800640 1 -- 172.21.15.3:0/3332119429 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00cda90 con 0x7ff9e015c720 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.889+0000 7ff9e5800640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e015c720 0x7ff9e0071cd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009870 tx=0x7ff9d402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6d424fca88c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.889+0000 7ff9dfe00640 1 -- 172.21.15.3:0/3332119429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4037070 con 0x7ff9e015c720 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9dfe00640 1 -- 172.21.15.3:0/3332119429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4032440 con 0x7ff9e015c720 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9dfe00640 1 -- 172.21.15.3:0/3332119429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4032810 con 0x7ff9e015c720 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 -- 172.21.15.3:0/3332119429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e015c720 msgr2=0x7ff9e0071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e015c720 0x7ff9e0071cd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009870 tx=0x7ff9d402a1c0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 -- 172.21.15.3:0/3332119429 shutdown_connections 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e0070f30 0x7ff9e0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 --2- 172.21.15.3:0/3332119429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e015c720 0x7ff9e0071cd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 -- 172.21.15.3:0/3332119429 >> 172.21.15.3:0/3332119429 conn(0x7ff9e006cfc0 msgr2=0x7ff9e006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 -- 172.21.15.3:0/3332119429 shutdown_connections 2024-09-10T07:51:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.890+0000 7ff9e8200640 1 -- 172.21.15.3:0/3332119429 wait complete. 2024-09-10T07:51:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.892+0000 7ff9e8200640 1 Processor -- start 2024-09-10T07:51:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.892+0000 7ff9e8200640 1 -- start start 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 0x7ff9e01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9e015c720 0x7ff9e0153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0073b50 con 0x7ff9e0070f30 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e006ed10 con 0x7ff9e015c720 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e8200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e0073cd0 con 0x7ff9e00cfdc0 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55504/0 (socket says 172.21.15.3:55504) 2024-09-10T07:51:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e5800640 1 -- 172.21.15.3:0/1020181821 learned_addr learned my addr 172.21.15.3:0/1020181821 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e4e00640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 0x7ff9e01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.893+0000 7ff9e6200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9e015c720 0x7ff9e0153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 -- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 msgr2=0x7ff9e01539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 0x7ff9e01539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 -- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9e015c720 msgr2=0x7ff9e0153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9e015c720 0x7ff9e0153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 -- 172.21.15.3:0/1020181821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e0152900 con 0x7ff9e0070f30 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e4e00640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 0x7ff9e01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e5800640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000c00 tx=0x7ff9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4037040 con 0x7ff9e0070f30 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4042070 con 0x7ff9e0070f30 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4030880 con 0x7ff9e0070f30 2024-09-10T07:51:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e0152be0 con 0x7ff9e0070f30 2024-09-10T07:51:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.894+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01531a0 con 0x7ff9e0070f30 2024-09-10T07:51:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.895+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0073610 con 0x7ff9e0070f30 2024-09-10T07:51:17.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.896+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9d402ad10 con 0x7ff9e0070f30 2024-09-10T07:51:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.897+0000 7ff9dd600640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 0x7ff9c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.897+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7ff9d40ab3b0 con 0x7ff9e0070f30 2024-09-10T07:51:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.898+0000 7ff9e4e00640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 0x7ff9c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:17.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.898+0000 7ff9e4e00640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 0x7ff9c006e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b2a30 tx=0x7ff9cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:17.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:17.901+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff9d4079d50 con 0x7ff9e0070f30 2024-09-10T07:51:18.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.152+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff9e014e360 con 0x7ff9e0070f30 2024-09-10T07:51:18.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.153+0000 7ff9dd600640 1 -- 172.21.15.3:0/1020181821 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1475 (secure 0 0 0) 0x7ff9d4079650 con 0x7ff9e0070f30 2024-09-10T07:51:18.152 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:18.153 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":311,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":78,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":268505088,"bytes_avail":772825608192,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":69,"btime":"2024-09-10T07:51:15:660073+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6702}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-09-10T07:50:51.549186+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:51:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 msgr2=0x7ff9c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 0x7ff9c006e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00b2a30 tx=0x7ff9cc008040 comp rx=0 tx=0).stop 2024-09-10T07:51:18.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 msgr2=0x7ff9e0154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000c00 tx=0x7ff9d40046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 shutdown_connections 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff9c006be30 0x7ff9c006e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9e015c720 0x7ff9e0153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9e00cfdc0 0x7ff9e01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 --2- 172.21.15.3:0/1020181821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0070f30 0x7ff9e0154700 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.155+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 >> 172.21.15.3:0/1020181821 conn(0x7ff9e006cfc0 msgr2=0x7ff9e015f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.156+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 shutdown_connections 2024-09-10T07:51:18.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.156+0000 7ff9e8200640 1 -- 172.21.15.3:0/1020181821 wait complete. 2024-09-10T07:51:18.171 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:51:18.172 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:51:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.326+0000 7f2e6d800640 1 Processor -- start 2024-09-10T07:51:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.326+0000 7f2e6d800640 1 -- start start 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.327+0000 7f2e6d800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e68160710 0x7f2e680cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.327+0000 7f2e6d800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e68160af0 0x7f2e680d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e6d800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e680cf800 0x7f2e680d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e6d800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2e680cfdd0 con 0x7f2e68160af0 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e6d800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6815cc90 con 0x7f2e68160710 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e6d800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6815cf70 con 0x7f2e680cf800 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e68160af0 0x7f2e680d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e68160af0 0x7f2e680d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55506/0 (socket says 172.21.15.3:55506) 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 -- 172.21.15.3:0/204364516 learned_addr learned my addr 172.21.15.3:0/204364516 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e66a00640 1 --2- 172.21.15.3:0/204364516 >> v2:172.21.15.70:3300/0 conn(0x7f2e68160710 0x7f2e680cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e67400640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e680cf800 0x7f2e680d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 -- 172.21.15.3:0/204364516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e680cf800 msgr2=0x7f2e680d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e680cf800 0x7f2e680d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 -- 172.21.15.3:0/204364516 >> v2:172.21.15.70:3300/0 conn(0x7f2e68160710 msgr2=0x7f2e680cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 --2- 172.21.15.3:0/204364516 >> v2:172.21.15.70:3300/0 conn(0x7f2e68160710 0x7f2e680cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.328+0000 7f2e5e000640 1 -- 172.21.15.3:0/204364516 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6815d250 con 0x7f2e68160af0 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.329+0000 7f2e5e000640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160af0 0x7f2e680d00b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2e6806ccc0 tx=0x7f2e5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d24160063dbd08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e66000640 1 -- 172.21.15.3:0/204364516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50037070 con 0x7f2e68160af0 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e66000640 1 -- 172.21.15.3:0/204364516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e50032440 con 0x7f2e68160af0 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e66000640 1 -- 172.21.15.3:0/204364516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50032810 con 0x7f2e68160af0 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 -- 172.21.15.3:0/204364516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160af0 msgr2=0x7f2e680d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160af0 0x7f2e680d00b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2e6806ccc0 tx=0x7f2e5002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 -- 172.21.15.3:0/204364516 shutdown_connections 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e680cf800 0x7f2e680d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 --2- 172.21.15.3:0/204364516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160af0 0x7f2e680d00b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 --2- 172.21.15.3:0/204364516 >> v2:172.21.15.70:3300/0 conn(0x7f2e68160710 0x7f2e680cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 -- 172.21.15.3:0/204364516 >> 172.21.15.3:0/204364516 conn(0x7f2e6806d000 msgr2=0x7f2e6806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 -- 172.21.15.3:0/204364516 shutdown_connections 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.330+0000 7f2e6d800640 1 -- 172.21.15.3:0/204364516 wait complete. 2024-09-10T07:51:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.332+0000 7f2e6d800640 1 Processor -- start 2024-09-10T07:51:18.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.332+0000 7f2e6d800640 1 -- start start 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 0x7f2e6814dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e68160af0 0x7f2e681552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6815d770 con 0x7f2e68160710 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6806bb20 con 0x7f2e680cf800 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e6d800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6814ef00 con 0x7f2e68160af0 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e5e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e5e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55514/0 (socket says 172.21.15.3:55514) 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.333+0000 7f2e5e000640 1 -- 172.21.15.3:0/861911476 learned_addr learned my addr 172.21.15.3:0/861911476 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e66a00640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 0x7f2e6814dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e67400640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e68160af0 0x7f2e681552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 -- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e68160af0 msgr2=0x7f2e681552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e68160af0 0x7f2e681552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 -- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 msgr2=0x7f2e6814dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 0x7f2e6814dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 -- 172.21.15.3:0/861911476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e681557f0 con 0x7f2e68160710 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e66a00640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 0x7f2e6814dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5e000640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2e5002a220 tx=0x7f2e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50037040 con 0x7f2e68160710 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e50042070 con 0x7f2e68160710 2024-09-10T07:51:18.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50030880 con 0x7f2e68160710 2024-09-10T07:51:18.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.334+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6814e2f0 con 0x7f2e68160710 2024-09-10T07:51:18.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.335+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6814e8b0 con 0x7f2e68160710 2024-09-10T07:51:18.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.336+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6814f080 con 0x7f2e68160710 2024-09-10T07:51:18.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.337+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e50030a20 con 0x7f2e68160710 2024-09-10T07:51:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.337+0000 7f2e5f400640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 0x7f2e4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.338+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f2e500ab6e0 con 0x7f2e68160710 2024-09-10T07:51:18.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.338+0000 7f2e66a00640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 0x7f2e4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:18.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.338+0000 7f2e66a00640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 0x7f2e4406e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009b10 tx=0x7f2e58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:18.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.342+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e5007a080 con 0x7f2e68160710 2024-09-10T07:51:18.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.524+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e6814c4d0 con 0x7f2e4406be30 2024-09-10T07:51:18.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.530+0000 7f2e5f400640 1 -- 172.21.15.3:0/861911476 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252065 (secure 0 0 0) 0x7f2e580119d0 con 0x7f2e4406be30 2024-09-10T07:51:18.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:18.534 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:51:18.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 msgr2=0x7f2e4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 0x7f2e4406e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009b10 tx=0x7f2e58008040 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 msgr2=0x7f2e6814bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2e5002a220 tx=0x7f2e500046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 shutdown_connections 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2e4406be30 0x7f2e4406e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2e68160af0 0x7f2e681552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e68160710 0x7f2e6814bf90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 --2- 172.21.15.3:0/861911476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680cf800 0x7f2e6814dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.534+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 >> 172.21.15.3:0/861911476 conn(0x7f2e6806d000 msgr2=0x7f2e680aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.535+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 shutdown_connections 2024-09-10T07:51:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:18.535+0000 7f2e6d800640 1 -- 172.21.15.3:0/861911476 wait complete. 2024-09-10T07:51:18.550 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T07:51:17.553692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262212,"kb_used_data":8516,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712508,"statfs":{"total":773094113280,"available":772825608192,"internally_reserved":0,"allocated":8720384,"data_stored":4388763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001075"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828738+0000","last_change":"2024-09-10T07:51:12.676690+0000","last_active":"2024-09-10T07:51:13.828738+0000","last_peered":"2024-09-10T07:51:13.828738+0000","last_clean":"2024-09-10T07:51:13.828738+0000","last_became_active":"2024-09-10T07:51:12.676418+0000","last_became_peered":"2024-09-10T07:51:12.676418+0000","last_unstale":"2024-09-10T07:51:13.828738+0000","last_undegraded":"2024-09-10T07:51:13.828738+0000","last_fullsized":"2024-09-10T07:51:13.828738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:00.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827421+0000","last_change":"2024-09-10T07:51:12.676886+0000","last_active":"2024-09-10T07:51:13.827421+0000","last_peered":"2024-09-10T07:51:13.827421+0000","last_clean":"2024-09-10T07:51:13.827421+0000","last_became_active":"2024-09-10T07:51:12.676680+0000","last_became_peered":"2024-09-10T07:51:12.676680+0000","last_unstale":"2024-09-10T07:51:13.827421+0000","last_undegraded":"2024-09-10T07:51:13.827421+0000","last_fullsized":"2024-09-10T07:51:13.827421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656352+0000","last_change":"2024-09-10T07:51:12.677258+0000","last_active":"2024-09-10T07:51:13.656352+0000","last_peered":"2024-09-10T07:51:13.656352+0000","last_clean":"2024-09-10T07:51:13.656352+0000","last_became_active":"2024-09-10T07:51:12.676957+0000","last_became_peered":"2024-09-10T07:51:12.676957+0000","last_unstale":"2024-09-10T07:51:13.656352+0000","last_undegraded":"2024-09-10T07:51:13.656352+0000","last_fullsized":"2024-09-10T07:51:13.656352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655764+0000","last_change":"2024-09-10T07:51:12.675735+0000","last_active":"2024-09-10T07:51:13.655764+0000","last_peered":"2024-09-10T07:51:13.655764+0000","last_clean":"2024-09-10T07:51:13.655764+0000","last_became_active":"2024-09-10T07:51:12.675442+0000","last_became_peered":"2024-09-10T07:51:12.675442+0000","last_unstale":"2024-09-10T07:51:13.655764+0000","last_undegraded":"2024-09-10T07:51:13.655764+0000","last_fullsized":"2024-09-10T07:51:13.655764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:00.169360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827458+0000","last_change":"2024-09-10T07:51:12.670954+0000","last_active":"2024-09-10T07:51:13.827458+0000","last_peered":"2024-09-10T07:51:13.827458+0000","last_clean":"2024-09-10T07:51:13.827458+0000","last_became_active":"2024-09-10T07:51:12.670673+0000","last_became_peered":"2024-09-10T07:51:12.670673+0000","last_unstale":"2024-09-10T07:51:13.827458+0000","last_undegraded":"2024-09-10T07:51:13.827458+0000","last_fullsized":"2024-09-10T07:51:13.827458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:52.871411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826972+0000","last_change":"2024-09-10T07:51:12.681827+0000","last_active":"2024-09-10T07:51:13.826972+0000","last_peered":"2024-09-10T07:51:13.826972+0000","last_clean":"2024-09-10T07:51:13.826972+0000","last_became_active":"2024-09-10T07:51:12.681558+0000","last_became_peered":"2024-09-10T07:51:12.681558+0000","last_unstale":"2024-09-10T07:51:13.826972+0000","last_undegraded":"2024-09-10T07:51:13.826972+0000","last_fullsized":"2024-09-10T07:51:13.826972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654188+0000","last_change":"2024-09-10T07:51:12.667538+0000","last_active":"2024-09-10T07:51:13.654188+0000","last_peered":"2024-09-10T07:51:13.654188+0000","last_clean":"2024-09-10T07:51:13.654188+0000","last_became_active":"2024-09-10T07:51:12.667277+0000","last_became_peered":"2024-09-10T07:51:12.667277+0000","last_unstale":"2024-09-10T07:51:13.654188+0000","last_undegraded":"2024-09-10T07:51:13.654188+0000","last_fullsized":"2024-09-10T07:51:13.654188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826644+0000","last_change":"2024-09-10T07:51:12.668124+0000","last_active":"2024-09-10T07:51:13.826644+0000","last_peered":"2024-09-10T07:51:13.826644+0000","last_clean":"2024-09-10T07:51:13.826644+0000","last_became_active":"2024-09-10T07:51:12.667798+0000","last_became_peered":"2024-09-10T07:51:12.667798+0000","last_unstale":"2024-09-10T07:51:13.826644+0000","last_undegraded":"2024-09-10T07:51:13.826644+0000","last_fullsized":"2024-09-10T07:51:13.826644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:27.002731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656452+0000","last_change":"2024-09-10T07:51:12.663829+0000","last_active":"2024-09-10T07:51:13.656452+0000","last_peered":"2024-09-10T07:51:13.656452+0000","last_clean":"2024-09-10T07:51:13.656452+0000","last_became_active":"2024-09-10T07:51:12.663536+0000","last_became_peered":"2024-09-10T07:51:12.663536+0000","last_unstale":"2024-09-10T07:51:13.656452+0000","last_undegraded":"2024-09-10T07:51:13.656452+0000","last_fullsized":"2024-09-10T07:51:13.656452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654749+0000","last_change":"2024-09-10T07:51:12.667036+0000","last_active":"2024-09-10T07:51:13.654749+0000","last_peered":"2024-09-10T07:51:13.654749+0000","last_clean":"2024-09-10T07:51:13.654749+0000","last_became_active":"2024-09-10T07:51:12.666744+0000","last_became_peered":"2024-09-10T07:51:12.666744+0000","last_unstale":"2024-09-10T07:51:13.654749+0000","last_undegraded":"2024-09-10T07:51:13.654749+0000","last_fullsized":"2024-09-10T07:51:13.654749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828107+0000","last_change":"2024-09-10T07:51:12.663072+0000","last_active":"2024-09-10T07:51:13.828107+0000","last_peered":"2024-09-10T07:51:13.828107+0000","last_clean":"2024-09-10T07:51:13.828107+0000","last_became_active":"2024-09-10T07:51:12.662799+0000","last_became_peered":"2024-09-10T07:51:12.662799+0000","last_unstale":"2024-09-10T07:51:13.828107+0000","last_undegraded":"2024-09-10T07:51:13.828107+0000","last_fullsized":"2024-09-10T07:51:13.828107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:40.034143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.654324+0000","last_change":"2024-09-10T07:51:12.647897+0000","last_active":"2024-09-10T07:51:12.654324+0000","last_peered":"2024-09-10T07:51:12.654324+0000","last_clean":"2024-09-10T07:51:12.654324+0000","last_became_active":"2024-09-10T07:51:12.647493+0000","last_became_peered":"2024-09-10T07:51:12.647493+0000","last_unstale":"2024-09-10T07:51:12.654324+0000","last_undegraded":"2024-09-10T07:51:12.654324+0000","last_fullsized":"2024-09-10T07:51:12.654324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:54.143299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655874+0000","last_change":"2024-09-10T07:51:12.675728+0000","last_active":"2024-09-10T07:51:13.655874+0000","last_peered":"2024-09-10T07:51:13.655874+0000","last_clean":"2024-09-10T07:51:13.655874+0000","last_became_active":"2024-09-10T07:51:12.675422+0000","last_became_peered":"2024-09-10T07:51:12.675422+0000","last_unstale":"2024-09-10T07:51:13.655874+0000","last_undegraded":"2024-09-10T07:51:13.655874+0000","last_fullsized":"2024-09-10T07:51:13.655874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:59.255796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.677787+0000","last_change":"2024-09-10T07:51:12.676376+0000","last_active":"2024-09-10T07:51:12.677787+0000","last_peered":"2024-09-10T07:51:12.677787+0000","last_clean":"2024-09-10T07:51:12.677787+0000","last_became_active":"2024-09-10T07:51:12.676068+0000","last_became_peered":"2024-09-10T07:51:12.676068+0000","last_unstale":"2024-09-10T07:51:12.677787+0000","last_undegraded":"2024-09-10T07:51:12.677787+0000","last_fullsized":"2024-09-10T07:51:12.677787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:03.847359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.664304+0000","last_change":"2024-09-10T07:51:12.659238+0000","last_active":"2024-09-10T07:51:12.664304+0000","last_peered":"2024-09-10T07:51:12.664304+0000","last_clean":"2024-09-10T07:51:12.664304+0000","last_became_active":"2024-09-10T07:51:12.658975+0000","last_became_peered":"2024-09-10T07:51:12.658975+0000","last_unstale":"2024-09-10T07:51:12.664304+0000","last_undegraded":"2024-09-10T07:51:12.664304+0000","last_fullsized":"2024-09-10T07:51:12.664304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:48.409488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827585+0000","last_change":"2024-09-10T07:51:12.675441+0000","last_active":"2024-09-10T07:51:13.827585+0000","last_peered":"2024-09-10T07:51:13.827585+0000","last_clean":"2024-09-10T07:51:13.827585+0000","last_became_active":"2024-09-10T07:51:12.673010+0000","last_became_peered":"2024-09-10T07:51:12.673010+0000","last_unstale":"2024-09-10T07:51:13.827585+0000","last_undegraded":"2024-09-10T07:51:13.827585+0000","last_fullsized":"2024-09-10T07:51:13.827585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:20.220354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827527+0000","last_change":"2024-09-10T07:51:12.657787+0000","last_active":"2024-09-10T07:51:13.827527+0000","last_peered":"2024-09-10T07:51:13.827527+0000","last_clean":"2024-09-10T07:51:13.827527+0000","last_became_active":"2024-09-10T07:51:12.657501+0000","last_became_peered":"2024-09-10T07:51:12.657501+0000","last_unstale":"2024-09-10T07:51:13.827527+0000","last_undegraded":"2024-09-10T07:51:13.827527+0000","last_fullsized":"2024-09-10T07:51:13.827527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:21.758327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827383+0000","last_change":"2024-09-10T07:51:12.674036+0000","last_active":"2024-09-10T07:51:13.827383+0000","last_peered":"2024-09-10T07:51:13.827383+0000","last_clean":"2024-09-10T07:51:13.827383+0000","last_became_active":"2024-09-10T07:51:12.673722+0000","last_became_peered":"2024-09-10T07:51:12.673722+0000","last_unstale":"2024-09-10T07:51:13.827383+0000","last_undegraded":"2024-09-10T07:51:13.827383+0000","last_fullsized":"2024-09-10T07:51:13.827383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:17.696759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827952+0000","last_change":"2024-09-10T07:51:12.663140+0000","last_active":"2024-09-10T07:51:13.827952+0000","last_peered":"2024-09-10T07:51:13.827952+0000","last_clean":"2024-09-10T07:51:13.827952+0000","last_became_active":"2024-09-10T07:51:12.662830+0000","last_became_peered":"2024-09-10T07:51:12.662830+0000","last_unstale":"2024-09-10T07:51:13.827952+0000","last_undegraded":"2024-09-10T07:51:13.827952+0000","last_fullsized":"2024-09-10T07:51:13.827952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:52.000639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827123+0000","last_change":"2024-09-10T07:51:12.663045+0000","last_active":"2024-09-10T07:51:13.827123+0000","last_peered":"2024-09-10T07:51:13.827123+0000","last_clean":"2024-09-10T07:51:13.827123+0000","last_became_active":"2024-09-10T07:51:12.662741+0000","last_became_peered":"2024-09-10T07:51:12.662741+0000","last_unstale":"2024-09-10T07:51:13.827123+0000","last_undegraded":"2024-09-10T07:51:13.827123+0000","last_fullsized":"2024-09-10T07:51:13.827123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:49.480454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826731+0000","last_change":"2024-09-10T07:51:12.660520+0000","last_active":"2024-09-10T07:51:13.826731+0000","last_peered":"2024-09-10T07:51:13.826731+0000","last_clean":"2024-09-10T07:51:13.826731+0000","last_became_active":"2024-09-10T07:51:12.660258+0000","last_became_peered":"2024-09-10T07:51:12.660258+0000","last_unstale":"2024-09-10T07:51:13.826731+0000","last_undegraded":"2024-09-10T07:51:13.826731+0000","last_fullsized":"2024-09-10T07:51:13.826731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:10.975319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827016+0000","last_change":"2024-09-10T07:51:12.657447+0000","last_active":"2024-09-10T07:51:13.827016+0000","last_peered":"2024-09-10T07:51:13.827016+0000","last_clean":"2024-09-10T07:51:13.827016+0000","last_became_active":"2024-09-10T07:51:12.657127+0000","last_became_peered":"2024-09-10T07:51:12.657127+0000","last_unstale":"2024-09-10T07:51:13.827016+0000","last_undegraded":"2024-09-10T07:51:13.827016+0000","last_fullsized":"2024-09-10T07:51:13.827016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654357+0000","last_change":"2024-09-10T07:51:12.679669+0000","last_active":"2024-09-10T07:51:13.654357+0000","last_peered":"2024-09-10T07:51:13.654357+0000","last_clean":"2024-09-10T07:51:13.654357+0000","last_became_active":"2024-09-10T07:51:12.679361+0000","last_became_peered":"2024-09-10T07:51:12.679361+0000","last_unstale":"2024-09-10T07:51:13.654357+0000","last_undegraded":"2024-09-10T07:51:13.654357+0000","last_fullsized":"2024-09-10T07:51:13.654357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827729+0000","last_change":"2024-09-10T07:51:12.667532+0000","last_active":"2024-09-10T07:51:13.827729+0000","last_peered":"2024-09-10T07:51:13.827729+0000","last_clean":"2024-09-10T07:51:13.827729+0000","last_became_active":"2024-09-10T07:51:12.666659+0000","last_became_peered":"2024-09-10T07:51:12.666659+0000","last_unstale":"2024-09-10T07:51:13.827729+0000","last_undegraded":"2024-09-10T07:51:13.827729+0000","last_fullsized":"2024-09-10T07:51:13.827729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:33.240184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654775+0000","last_change":"2024-09-10T07:51:12.679672+0000","last_active":"2024-09-10T07:51:13.654775+0000","last_peered":"2024-09-10T07:51:13.654775+0000","last_clean":"2024-09-10T07:51:13.654775+0000","last_became_active":"2024-09-10T07:51:12.679422+0000","last_became_peered":"2024-09-10T07:51:12.679422+0000","last_unstale":"2024-09-10T07:51:13.654775+0000","last_undegraded":"2024-09-10T07:51:13.654775+0000","last_fullsized":"2024-09-10T07:51:13.654775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827692+0000","last_change":"2024-09-10T07:51:12.681734+0000","last_active":"2024-09-10T07:51:13.827692+0000","last_peered":"2024-09-10T07:51:13.827692+0000","last_clean":"2024-09-10T07:51:13.827692+0000","last_became_active":"2024-09-10T07:51:12.681454+0000","last_became_peered":"2024-09-10T07:51:12.681454+0000","last_unstale":"2024-09-10T07:51:13.827692+0000","last_undegraded":"2024-09-10T07:51:13.827692+0000","last_fullsized":"2024-09-10T07:51:13.827692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:59.692249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827301+0000","last_change":"2024-09-10T07:51:12.661140+0000","last_active":"2024-09-10T07:51:13.827301+0000","last_peered":"2024-09-10T07:51:13.827301+0000","last_clean":"2024-09-10T07:51:13.827301+0000","last_became_active":"2024-09-10T07:51:12.660758+0000","last_became_peered":"2024-09-10T07:51:12.660758+0000","last_unstale":"2024-09-10T07:51:13.827301+0000","last_undegraded":"2024-09-10T07:51:13.827301+0000","last_fullsized":"2024-09-10T07:51:13.827301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:12.250282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827897+0000","last_change":"2024-09-10T07:51:12.670722+0000","last_active":"2024-09-10T07:51:13.827897+0000","last_peered":"2024-09-10T07:51:13.827897+0000","last_clean":"2024-09-10T07:51:13.827897+0000","last_became_active":"2024-09-10T07:51:12.667899+0000","last_became_peered":"2024-09-10T07:51:12.667899+0000","last_unstale":"2024-09-10T07:51:13.827897+0000","last_undegraded":"2024-09-10T07:51:13.827897+0000","last_fullsized":"2024-09-10T07:51:13.827897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656066+0000","last_change":"2024-09-10T07:51:12.680797+0000","last_active":"2024-09-10T07:51:13.656066+0000","last_peered":"2024-09-10T07:51:13.656066+0000","last_clean":"2024-09-10T07:51:13.656066+0000","last_became_active":"2024-09-10T07:51:12.680521+0000","last_became_peered":"2024-09-10T07:51:12.680521+0000","last_unstale":"2024-09-10T07:51:13.656066+0000","last_undegraded":"2024-09-10T07:51:13.656066+0000","last_fullsized":"2024-09-10T07:51:13.656066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828525+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.828525+0000","last_peered":"2024-09-10T07:51:13.828525+0000","last_clean":"2024-09-10T07:51:13.828525+0000","last_became_active":"2024-09-10T07:51:12.666221+0000","last_became_peered":"2024-09-10T07:51:12.666221+0000","last_unstale":"2024-09-10T07:51:13.828525+0000","last_undegraded":"2024-09-10T07:51:13.828525+0000","last_fullsized":"2024-09-10T07:51:13.828525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:20.727768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826628+0000","last_change":"2024-09-10T07:51:12.667382+0000","last_active":"2024-09-10T07:51:13.826628+0000","last_peered":"2024-09-10T07:51:13.826628+0000","last_clean":"2024-09-10T07:51:13.826628+0000","last_became_active":"2024-09-10T07:51:12.667121+0000","last_became_peered":"2024-09-10T07:51:12.667121+0000","last_unstale":"2024-09-10T07:51:13.826628+0000","last_undegraded":"2024-09-10T07:51:13.826628+0000","last_fullsized":"2024-09-10T07:51:13.826628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826696+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.826696+0000","last_peered":"2024-09-10T07:51:13.826696+0000","last_clean":"2024-09-10T07:51:13.826696+0000","last_became_active":"2024-09-10T07:51:12.668445+0000","last_became_peered":"2024-09-10T07:51:12.668445+0000","last_unstale":"2024-09-10T07:51:13.826696+0000","last_undegraded":"2024-09-10T07:51:13.826696+0000","last_fullsized":"2024-09-10T07:51:13.826696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:56.232909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827833+0000","last_change":"2024-09-10T07:51:12.679812+0000","last_active":"2024-09-10T07:51:13.827833+0000","last_peered":"2024-09-10T07:51:13.827833+0000","last_clean":"2024-09-10T07:51:13.827833+0000","last_became_active":"2024-09-10T07:51:12.669963+0000","last_became_peered":"2024-09-10T07:51:12.669963+0000","last_unstale":"2024-09-10T07:51:13.827833+0000","last_undegraded":"2024-09-10T07:51:13.827833+0000","last_fullsized":"2024-09-10T07:51:13.827833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:47.211095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828342+0000","last_change":"2024-09-10T07:51:11.647694+0000","last_active":"2024-09-10T07:51:13.828342+0000","last_peered":"2024-09-10T07:51:13.828342+0000","last_clean":"2024-09-10T07:51:13.828342+0000","last_became_active":"2024-09-10T07:51:11.647419+0000","last_became_peered":"2024-09-10T07:51:11.647419+0000","last_unstale":"2024-09-10T07:51:13.828342+0000","last_undegraded":"2024-09-10T07:51:13.828342+0000","last_fullsized":"2024-09-10T07:51:13.828342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:07.450590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656051+0000","last_change":"2024-09-10T07:51:12.660668+0000","last_active":"2024-09-10T07:51:13.656051+0000","last_peered":"2024-09-10T07:51:13.656051+0000","last_clean":"2024-09-10T07:51:13.656051+0000","last_became_active":"2024-09-10T07:51:12.660364+0000","last_became_peered":"2024-09-10T07:51:12.660364+0000","last_unstale":"2024-09-10T07:51:13.656051+0000","last_undegraded":"2024-09-10T07:51:13.656051+0000","last_fullsized":"2024-09-10T07:51:13.656051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827698+0000","last_change":"2024-09-10T07:51:11.657105+0000","last_active":"2024-09-10T07:51:13.827698+0000","last_peered":"2024-09-10T07:51:13.827698+0000","last_clean":"2024-09-10T07:51:13.827698+0000","last_became_active":"2024-09-10T07:51:11.656523+0000","last_became_peered":"2024-09-10T07:51:11.656523+0000","last_unstale":"2024-09-10T07:51:13.827698+0000","last_undegraded":"2024-09-10T07:51:13.827698+0000","last_fullsized":"2024-09-10T07:51:13.827698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:08.541922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827892+0000","last_change":"2024-09-10T07:51:12.669879+0000","last_active":"2024-09-10T07:51:13.827892+0000","last_peered":"2024-09-10T07:51:13.827892+0000","last_clean":"2024-09-10T07:51:13.827892+0000","last_became_active":"2024-09-10T07:51:12.668574+0000","last_became_peered":"2024-09-10T07:51:12.668574+0000","last_unstale":"2024-09-10T07:51:13.827892+0000","last_undegraded":"2024-09-10T07:51:13.827892+0000","last_fullsized":"2024-09-10T07:51:13.827892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828196+0000","last_change":"2024-09-10T07:51:11.648632+0000","last_active":"2024-09-10T07:51:13.828196+0000","last_peered":"2024-09-10T07:51:13.828196+0000","last_clean":"2024-09-10T07:51:13.828196+0000","last_became_active":"2024-09-10T07:51:11.648267+0000","last_became_peered":"2024-09-10T07:51:11.648267+0000","last_unstale":"2024-09-10T07:51:13.828196+0000","last_undegraded":"2024-09-10T07:51:13.828196+0000","last_fullsized":"2024-09-10T07:51:13.828196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:41.341276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654539+0000","last_change":"2024-09-10T07:51:12.670265+0000","last_active":"2024-09-10T07:51:13.654539+0000","last_peered":"2024-09-10T07:51:13.654539+0000","last_clean":"2024-09-10T07:51:13.654539+0000","last_became_active":"2024-09-10T07:51:12.669524+0000","last_became_peered":"2024-09-10T07:51:12.669524+0000","last_unstale":"2024-09-10T07:51:13.654539+0000","last_undegraded":"2024-09-10T07:51:13.654539+0000","last_fullsized":"2024-09-10T07:51:13.654539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827979+0000","last_change":"2024-09-10T07:51:11.661609+0000","last_active":"2024-09-10T07:51:13.827979+0000","last_peered":"2024-09-10T07:51:13.827979+0000","last_clean":"2024-09-10T07:51:13.827979+0000","last_became_active":"2024-09-10T07:51:11.660016+0000","last_became_peered":"2024-09-10T07:51:11.660016+0000","last_unstale":"2024-09-10T07:51:13.827979+0000","last_undegraded":"2024-09-10T07:51:13.827979+0000","last_fullsized":"2024-09-10T07:51:13.827979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:46.277801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828438+0000","last_change":"2024-09-10T07:51:12.667278+0000","last_active":"2024-09-10T07:51:13.828438+0000","last_peered":"2024-09-10T07:51:13.828438+0000","last_clean":"2024-09-10T07:51:13.828438+0000","last_became_active":"2024-09-10T07:51:12.666209+0000","last_became_peered":"2024-09-10T07:51:12.666209+0000","last_unstale":"2024-09-10T07:51:13.828438+0000","last_undegraded":"2024-09-10T07:51:13.828438+0000","last_fullsized":"2024-09-10T07:51:13.828438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:26.172183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.671056+0000","last_change":"2024-09-10T07:51:11.654006+0000","last_active":"2024-09-10T07:51:12.671056+0000","last_peered":"2024-09-10T07:51:12.671056+0000","last_clean":"2024-09-10T07:51:12.671056+0000","last_became_active":"2024-09-10T07:51:11.651332+0000","last_became_peered":"2024-09-10T07:51:11.651332+0000","last_unstale":"2024-09-10T07:51:12.671056+0000","last_undegraded":"2024-09-10T07:51:12.671056+0000","last_fullsized":"2024-09-10T07:51:12.671056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:41.047040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828245+0000","last_change":"2024-09-10T07:51:12.666838+0000","last_active":"2024-09-10T07:51:13.828245+0000","last_peered":"2024-09-10T07:51:13.828245+0000","last_clean":"2024-09-10T07:51:13.828245+0000","last_became_active":"2024-09-10T07:51:12.666551+0000","last_became_peered":"2024-09-10T07:51:12.666551+0000","last_unstale":"2024-09-10T07:51:13.828245+0000","last_undegraded":"2024-09-10T07:51:13.828245+0000","last_fullsized":"2024-09-10T07:51:13.828245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654240+0000","last_change":"2024-09-10T07:51:11.666134+0000","last_active":"2024-09-10T07:51:13.654240+0000","last_peered":"2024-09-10T07:51:13.654240+0000","last_clean":"2024-09-10T07:51:13.654240+0000","last_became_active":"2024-09-10T07:51:11.665826+0000","last_became_peered":"2024-09-10T07:51:11.665826+0000","last_unstale":"2024-09-10T07:51:13.654240+0000","last_undegraded":"2024-09-10T07:51:13.654240+0000","last_fullsized":"2024-09-10T07:51:13.654240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:49.241190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656243+0000","last_change":"2024-09-10T07:51:12.667702+0000","last_active":"2024-09-10T07:51:13.656243+0000","last_peered":"2024-09-10T07:51:13.656243+0000","last_clean":"2024-09-10T07:51:13.656243+0000","last_became_active":"2024-09-10T07:51:12.667391+0000","last_became_peered":"2024-09-10T07:51:12.667391+0000","last_unstale":"2024-09-10T07:51:13.656243+0000","last_undegraded":"2024-09-10T07:51:13.656243+0000","last_fullsized":"2024-09-10T07:51:13.656243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.456232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655436+0000","last_change":"2024-09-10T07:51:11.664356+0000","last_active":"2024-09-10T07:51:13.655436+0000","last_peered":"2024-09-10T07:51:13.655436+0000","last_clean":"2024-09-10T07:51:13.655436+0000","last_became_active":"2024-09-10T07:51:11.663984+0000","last_became_peered":"2024-09-10T07:51:11.663984+0000","last_unstale":"2024-09-10T07:51:13.655436+0000","last_undegraded":"2024-09-10T07:51:13.655436+0000","last_fullsized":"2024-09-10T07:51:13.655436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:02.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.669674+0000","last_change":"2024-09-10T07:51:12.668160+0000","last_active":"2024-09-10T07:51:12.669674+0000","last_peered":"2024-09-10T07:51:12.669674+0000","last_clean":"2024-09-10T07:51:12.669674+0000","last_became_active":"2024-09-10T07:51:12.667845+0000","last_became_peered":"2024-09-10T07:51:12.667845+0000","last_unstale":"2024-09-10T07:51:12.669674+0000","last_undegraded":"2024-09-10T07:51:12.669674+0000","last_fullsized":"2024-09-10T07:51:12.669674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:57.084096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642604+0000","last_change":"2024-09-10T07:51:11.647781+0000","last_active":"2024-09-10T07:51:12.642604+0000","last_peered":"2024-09-10T07:51:12.642604+0000","last_clean":"2024-09-10T07:51:12.642604+0000","last_became_active":"2024-09-10T07:51:11.647502+0000","last_became_peered":"2024-09-10T07:51:11.647502+0000","last_unstale":"2024-09-10T07:51:12.642604+0000","last_undegraded":"2024-09-10T07:51:12.642604+0000","last_fullsized":"2024-09-10T07:51:12.642604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:38.219907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654838+0000","last_change":"2024-09-10T07:51:12.680163+0000","last_active":"2024-09-10T07:51:13.654838+0000","last_peered":"2024-09-10T07:51:13.654838+0000","last_clean":"2024-09-10T07:51:13.654838+0000","last_became_active":"2024-09-10T07:51:12.679482+0000","last_became_peered":"2024-09-10T07:51:12.679482+0000","last_unstale":"2024-09-10T07:51:13.654838+0000","last_undegraded":"2024-09-10T07:51:13.654838+0000","last_fullsized":"2024-09-10T07:51:13.654838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828385+0000","last_change":"2024-09-10T07:51:11.652611+0000","last_active":"2024-09-10T07:51:13.828385+0000","last_peered":"2024-09-10T07:51:13.828385+0000","last_clean":"2024-09-10T07:51:13.828385+0000","last_became_active":"2024-09-10T07:51:11.652322+0000","last_became_peered":"2024-09-10T07:51:11.652322+0000","last_unstale":"2024-09-10T07:51:13.828385+0000","last_undegraded":"2024-09-10T07:51:13.828385+0000","last_fullsized":"2024-09-10T07:51:13.828385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:48.473083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827225+0000","last_change":"2024-09-10T07:51:12.681992+0000","last_active":"2024-09-10T07:51:13.827225+0000","last_peered":"2024-09-10T07:51:13.827225+0000","last_clean":"2024-09-10T07:51:13.827225+0000","last_became_active":"2024-09-10T07:51:12.680523+0000","last_became_peered":"2024-09-10T07:51:12.680523+0000","last_unstale":"2024-09-10T07:51:13.827225+0000","last_undegraded":"2024-09-10T07:51:13.827225+0000","last_fullsized":"2024-09-10T07:51:13.827225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:49.854131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827642+0000","last_change":"2024-09-10T07:51:11.661453+0000","last_active":"2024-09-10T07:51:13.827642+0000","last_peered":"2024-09-10T07:51:13.827642+0000","last_clean":"2024-09-10T07:51:13.827642+0000","last_became_active":"2024-09-10T07:51:11.661202+0000","last_became_peered":"2024-09-10T07:51:11.661202+0000","last_unstale":"2024-09-10T07:51:13.827642+0000","last_undegraded":"2024-09-10T07:51:13.827642+0000","last_fullsized":"2024-09-10T07:51:13.827642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:25.009100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828046+0000","last_change":"2024-09-10T07:51:12.677005+0000","last_active":"2024-09-10T07:51:13.828046+0000","last_peered":"2024-09-10T07:51:13.828046+0000","last_clean":"2024-09-10T07:51:13.828046+0000","last_became_active":"2024-09-10T07:51:12.676702+0000","last_became_peered":"2024-09-10T07:51:12.676702+0000","last_unstale":"2024-09-10T07:51:13.828046+0000","last_undegraded":"2024-09-10T07:51:13.828046+0000","last_fullsized":"2024-09-10T07:51:13.828046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:55.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.665387+0000","last_change":"2024-09-10T07:51:11.651919+0000","last_active":"2024-09-10T07:51:12.665387+0000","last_peered":"2024-09-10T07:51:12.665387+0000","last_clean":"2024-09-10T07:51:12.665387+0000","last_became_active":"2024-09-10T07:51:11.651676+0000","last_became_peered":"2024-09-10T07:51:11.651676+0000","last_unstale":"2024-09-10T07:51:12.665387+0000","last_undegraded":"2024-09-10T07:51:12.665387+0000","last_fullsized":"2024-09-10T07:51:12.665387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827121+0000","last_change":"2024-09-10T07:51:12.661023+0000","last_active":"2024-09-10T07:51:13.827121+0000","last_peered":"2024-09-10T07:51:13.827121+0000","last_clean":"2024-09-10T07:51:13.827121+0000","last_became_active":"2024-09-10T07:51:12.660734+0000","last_became_peered":"2024-09-10T07:51:12.660734+0000","last_unstale":"2024-09-10T07:51:13.827121+0000","last_undegraded":"2024-09-10T07:51:13.827121+0000","last_fullsized":"2024-09-10T07:51:13.827121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:18.004956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656317+0000","last_change":"2024-09-10T07:51:11.659183+0000","last_active":"2024-09-10T07:51:13.656317+0000","last_peered":"2024-09-10T07:51:13.656317+0000","last_clean":"2024-09-10T07:51:13.656317+0000","last_became_active":"2024-09-10T07:51:11.658378+0000","last_became_peered":"2024-09-10T07:51:11.658378+0000","last_unstale":"2024-09-10T07:51:13.656317+0000","last_undegraded":"2024-09-10T07:51:13.656317+0000","last_fullsized":"2024-09-10T07:51:13.656317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:59.230645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827115+0000","last_change":"2024-09-10T07:51:12.647744+0000","last_active":"2024-09-10T07:51:13.827115+0000","last_peered":"2024-09-10T07:51:13.827115+0000","last_clean":"2024-09-10T07:51:13.827115+0000","last_became_active":"2024-09-10T07:51:12.647440+0000","last_became_peered":"2024-09-10T07:51:12.647440+0000","last_unstale":"2024-09-10T07:51:13.827115+0000","last_undegraded":"2024-09-10T07:51:13.827115+0000","last_fullsized":"2024-09-10T07:51:13.827115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655557+0000","last_change":"2024-09-10T07:51:11.666220+0000","last_active":"2024-09-10T07:51:13.655557+0000","last_peered":"2024-09-10T07:51:13.655557+0000","last_clean":"2024-09-10T07:51:13.655557+0000","last_became_active":"2024-09-10T07:51:11.665886+0000","last_became_peered":"2024-09-10T07:51:11.665886+0000","last_unstale":"2024-09-10T07:51:13.655557+0000","last_undegraded":"2024-09-10T07:51:13.655557+0000","last_fullsized":"2024-09-10T07:51:13.655557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:50.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654970+0000","last_change":"2024-09-10T07:51:12.682681+0000","last_active":"2024-09-10T07:51:13.654970+0000","last_peered":"2024-09-10T07:51:13.654970+0000","last_clean":"2024-09-10T07:51:13.654970+0000","last_became_active":"2024-09-10T07:51:12.682424+0000","last_became_peered":"2024-09-10T07:51:12.682424+0000","last_unstale":"2024-09-10T07:51:13.654970+0000","last_undegraded":"2024-09-10T07:51:13.654970+0000","last_fullsized":"2024-09-10T07:51:13.654970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654264+0000","last_change":"2024-09-10T07:51:11.661553+0000","last_active":"2024-09-10T07:51:13.654264+0000","last_peered":"2024-09-10T07:51:13.654264+0000","last_clean":"2024-09-10T07:51:13.654264+0000","last_became_active":"2024-09-10T07:51:11.661263+0000","last_became_peered":"2024-09-10T07:51:11.661263+0000","last_unstale":"2024-09-10T07:51:13.654264+0000","last_undegraded":"2024-09-10T07:51:13.654264+0000","last_fullsized":"2024-09-10T07:51:13.654264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:54.532976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826983+0000","last_change":"2024-09-10T07:51:12.661693+0000","last_active":"2024-09-10T07:51:13.826983+0000","last_peered":"2024-09-10T07:51:13.826983+0000","last_clean":"2024-09-10T07:51:13.826983+0000","last_became_active":"2024-09-10T07:51:12.661427+0000","last_became_peered":"2024-09-10T07:51:12.661427+0000","last_unstale":"2024-09-10T07:51:13.826983+0000","last_undegraded":"2024-09-10T07:51:13.826983+0000","last_fullsized":"2024-09-10T07:51:13.826983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:15.540616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654447+0000","last_change":"2024-09-10T07:51:11.657359+0000","last_active":"2024-09-10T07:51:13.654447+0000","last_peered":"2024-09-10T07:51:13.654447+0000","last_clean":"2024-09-10T07:51:13.654447+0000","last_became_active":"2024-09-10T07:51:11.657038+0000","last_became_peered":"2024-09-10T07:51:11.657038+0000","last_unstale":"2024-09-10T07:51:13.654447+0000","last_undegraded":"2024-09-10T07:51:13.654447+0000","last_fullsized":"2024-09-10T07:51:13.654447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:55.533736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656447+0000","last_change":"2024-09-10T07:51:12.661323+0000","last_active":"2024-09-10T07:51:13.656447+0000","last_peered":"2024-09-10T07:51:13.656447+0000","last_clean":"2024-09-10T07:51:13.656447+0000","last_became_active":"2024-09-10T07:51:12.658712+0000","last_became_peered":"2024-09-10T07:51:12.658712+0000","last_unstale":"2024-09-10T07:51:13.656447+0000","last_undegraded":"2024-09-10T07:51:13.656447+0000","last_fullsized":"2024-09-10T07:51:13.656447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654091+0000","last_change":"2024-09-10T07:51:11.664261+0000","last_active":"2024-09-10T07:51:13.654091+0000","last_peered":"2024-09-10T07:51:13.654091+0000","last_clean":"2024-09-10T07:51:13.654091+0000","last_became_active":"2024-09-10T07:51:11.663940+0000","last_became_peered":"2024-09-10T07:51:11.663940+0000","last_unstale":"2024-09-10T07:51:13.654091+0000","last_undegraded":"2024-09-10T07:51:13.654091+0000","last_fullsized":"2024-09-10T07:51:13.654091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:20.541470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827555+0000","last_change":"2024-09-10T07:51:12.658206+0000","last_active":"2024-09-10T07:51:13.827555+0000","last_peered":"2024-09-10T07:51:13.827555+0000","last_clean":"2024-09-10T07:51:13.827555+0000","last_became_active":"2024-09-10T07:51:12.657838+0000","last_became_peered":"2024-09-10T07:51:12.657838+0000","last_unstale":"2024-09-10T07:51:13.827555+0000","last_undegraded":"2024-09-10T07:51:13.827555+0000","last_fullsized":"2024-09-10T07:51:13.827555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828648+0000","last_change":"2024-09-10T07:51:11.654332+0000","last_active":"2024-09-10T07:51:13.828648+0000","last_peered":"2024-09-10T07:51:13.828648+0000","last_clean":"2024-09-10T07:51:13.828648+0000","last_became_active":"2024-09-10T07:51:11.654042+0000","last_became_peered":"2024-09-10T07:51:11.654042+0000","last_unstale":"2024-09-10T07:51:13.828648+0000","last_undegraded":"2024-09-10T07:51:13.828648+0000","last_fullsized":"2024-09-10T07:51:13.828648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:16.997585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827284+0000","last_change":"2024-09-10T07:51:12.667032+0000","last_active":"2024-09-10T07:51:13.827284+0000","last_peered":"2024-09-10T07:51:13.827284+0000","last_clean":"2024-09-10T07:51:13.827284+0000","last_became_active":"2024-09-10T07:51:12.666726+0000","last_became_peered":"2024-09-10T07:51:12.666726+0000","last_unstale":"2024-09-10T07:51:13.827284+0000","last_undegraded":"2024-09-10T07:51:13.827284+0000","last_fullsized":"2024-09-10T07:51:13.827284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827732+0000","last_change":"2024-09-10T07:51:11.655876+0000","last_active":"2024-09-10T07:51:13.827732+0000","last_peered":"2024-09-10T07:51:13.827732+0000","last_clean":"2024-09-10T07:51:13.827732+0000","last_became_active":"2024-09-10T07:51:11.655606+0000","last_became_peered":"2024-09-10T07:51:11.655606+0000","last_unstale":"2024-09-10T07:51:13.827732+0000","last_undegraded":"2024-09-10T07:51:13.827732+0000","last_fullsized":"2024-09-10T07:51:13.827732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:36.453798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655926+0000","last_change":"2024-09-10T07:51:12.657238+0000","last_active":"2024-09-10T07:51:13.655926+0000","last_peered":"2024-09-10T07:51:13.655926+0000","last_clean":"2024-09-10T07:51:13.655926+0000","last_became_active":"2024-09-10T07:51:12.656922+0000","last_became_peered":"2024-09-10T07:51:12.656922+0000","last_unstale":"2024-09-10T07:51:13.655926+0000","last_undegraded":"2024-09-10T07:51:13.655926+0000","last_fullsized":"2024-09-10T07:51:13.655926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.556394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826896+0000","last_change":"2024-09-10T07:51:11.654686+0000","last_active":"2024-09-10T07:51:13.826896+0000","last_peered":"2024-09-10T07:51:13.826896+0000","last_clean":"2024-09-10T07:51:13.826896+0000","last_became_active":"2024-09-10T07:51:11.653599+0000","last_became_peered":"2024-09-10T07:51:11.653599+0000","last_unstale":"2024-09-10T07:51:13.826896+0000","last_undegraded":"2024-09-10T07:51:13.826896+0000","last_fullsized":"2024-09-10T07:51:13.826896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:52.484707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656395+0000","last_change":"2024-09-10T07:51:12.680792+0000","last_active":"2024-09-10T07:51:13.656395+0000","last_peered":"2024-09-10T07:51:13.656395+0000","last_clean":"2024-09-10T07:51:13.656395+0000","last_became_active":"2024-09-10T07:51:12.680491+0000","last_became_peered":"2024-09-10T07:51:12.680491+0000","last_unstale":"2024-09-10T07:51:13.656395+0000","last_undegraded":"2024-09-10T07:51:13.656395+0000","last_fullsized":"2024-09-10T07:51:13.656395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828075+0000","last_change":"2024-09-10T07:51:11.652720+0000","last_active":"2024-09-10T07:51:13.828075+0000","last_peered":"2024-09-10T07:51:13.828075+0000","last_clean":"2024-09-10T07:51:13.828075+0000","last_became_active":"2024-09-10T07:51:11.652391+0000","last_became_peered":"2024-09-10T07:51:11.652391+0000","last_unstale":"2024-09-10T07:51:13.828075+0000","last_undegraded":"2024-09-10T07:51:13.828075+0000","last_fullsized":"2024-09-10T07:51:13.828075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:51.242641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655492+0000","last_change":"2024-09-10T07:51:12.653138+0000","last_active":"2024-09-10T07:51:13.655492+0000","last_peered":"2024-09-10T07:51:13.655492+0000","last_clean":"2024-09-10T07:51:13.655492+0000","last_became_active":"2024-09-10T07:51:12.652876+0000","last_became_peered":"2024-09-10T07:51:12.652876+0000","last_unstale":"2024-09-10T07:51:13.655492+0000","last_undegraded":"2024-09-10T07:51:13.655492+0000","last_fullsized":"2024-09-10T07:51:13.655492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:39.429271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827092+0000","last_change":"2024-09-10T07:51:11.644261+0000","last_active":"2024-09-10T07:51:13.827092+0000","last_peered":"2024-09-10T07:51:13.827092+0000","last_clean":"2024-09-10T07:51:13.827092+0000","last_became_active":"2024-09-10T07:51:11.643967+0000","last_became_peered":"2024-09-10T07:51:11.643967+0000","last_unstale":"2024-09-10T07:51:13.827092+0000","last_undegraded":"2024-09-10T07:51:13.827092+0000","last_fullsized":"2024-09-10T07:51:13.827092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:10.313315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655294+0000","last_change":"2024-09-10T07:51:12.647492+0000","last_active":"2024-09-10T07:51:13.655294+0000","last_peered":"2024-09-10T07:51:13.655294+0000","last_clean":"2024-09-10T07:51:13.655294+0000","last_became_active":"2024-09-10T07:51:12.647153+0000","last_became_peered":"2024-09-10T07:51:12.647153+0000","last_unstale":"2024-09-10T07:51:13.655294+0000","last_undegraded":"2024-09-10T07:51:13.655294+0000","last_fullsized":"2024-09-10T07:51:13.655294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826984+0000","last_change":"2024-09-10T07:51:11.659869+0000","last_active":"2024-09-10T07:51:13.826984+0000","last_peered":"2024-09-10T07:51:13.826984+0000","last_clean":"2024-09-10T07:51:13.826984+0000","last_became_active":"2024-09-10T07:51:11.659392+0000","last_became_peered":"2024-09-10T07:51:11.659392+0000","last_unstale":"2024-09-10T07:51:13.826984+0000","last_undegraded":"2024-09-10T07:51:13.826984+0000","last_fullsized":"2024-09-10T07:51:13.826984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:38.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.657374+0000","last_change":"2024-09-10T07:51:12.655133+0000","last_active":"2024-09-10T07:51:12.657374+0000","last_peered":"2024-09-10T07:51:12.657374+0000","last_clean":"2024-09-10T07:51:12.657374+0000","last_became_active":"2024-09-10T07:51:12.654855+0000","last_became_peered":"2024-09-10T07:51:12.654855+0000","last_unstale":"2024-09-10T07:51:12.657374+0000","last_undegraded":"2024-09-10T07:51:12.657374+0000","last_fullsized":"2024-09-10T07:51:12.657374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:18.714900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826722+0000","last_change":"2024-09-10T07:51:11.664218+0000","last_active":"2024-09-10T07:51:13.826722+0000","last_peered":"2024-09-10T07:51:13.826722+0000","last_clean":"2024-09-10T07:51:13.826722+0000","last_became_active":"2024-09-10T07:51:11.663837+0000","last_became_peered":"2024-09-10T07:51:11.663837+0000","last_unstale":"2024-09-10T07:51:13.826722+0000","last_undegraded":"2024-09-10T07:51:13.826722+0000","last_fullsized":"2024-09-10T07:51:13.826722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:43.311010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827577+0000","last_change":"2024-09-10T07:51:12.658871+0000","last_active":"2024-09-10T07:51:13.827577+0000","last_peered":"2024-09-10T07:51:13.827577+0000","last_clean":"2024-09-10T07:51:13.827577+0000","last_became_active":"2024-09-10T07:51:12.658601+0000","last_became_peered":"2024-09-10T07:51:12.658601+0000","last_unstale":"2024-09-10T07:51:13.827577+0000","last_undegraded":"2024-09-10T07:51:13.827577+0000","last_fullsized":"2024-09-10T07:51:13.827577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:02.300806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828721+0000","last_change":"2024-09-10T07:51:11.658990+0000","last_active":"2024-09-10T07:51:13.828721+0000","last_peered":"2024-09-10T07:51:13.828721+0000","last_clean":"2024-09-10T07:51:13.828721+0000","last_became_active":"2024-09-10T07:51:11.658451+0000","last_became_peered":"2024-09-10T07:51:11.658451+0000","last_unstale":"2024-09-10T07:51:13.828721+0000","last_undegraded":"2024-09-10T07:51:13.828721+0000","last_fullsized":"2024-09-10T07:51:13.828721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.242911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654799+0000","last_change":"2024-09-10T07:51:12.663182+0000","last_active":"2024-09-10T07:51:13.654799+0000","last_peered":"2024-09-10T07:51:13.654799+0000","last_clean":"2024-09-10T07:51:13.654799+0000","last_became_active":"2024-09-10T07:51:12.662787+0000","last_became_peered":"2024-09-10T07:51:12.662787+0000","last_unstale":"2024-09-10T07:51:13.654799+0000","last_undegraded":"2024-09-10T07:51:13.654799+0000","last_fullsized":"2024-09-10T07:51:13.654799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:37.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827966+0000","last_change":"2024-09-10T07:51:11.655978+0000","last_active":"2024-09-10T07:51:13.827966+0000","last_peered":"2024-09-10T07:51:13.827966+0000","last_clean":"2024-09-10T07:51:13.827966+0000","last_became_active":"2024-09-10T07:51:11.655686+0000","last_became_peered":"2024-09-10T07:51:11.655686+0000","last_unstale":"2024-09-10T07:51:13.827966+0000","last_undegraded":"2024-09-10T07:51:13.827966+0000","last_fullsized":"2024-09-10T07:51:13.827966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:14.533246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828044+0000","last_change":"2024-09-10T07:51:12.665991+0000","last_active":"2024-09-10T07:51:13.828044+0000","last_peered":"2024-09-10T07:51:13.828044+0000","last_clean":"2024-09-10T07:51:13.828044+0000","last_became_active":"2024-09-10T07:51:12.665689+0000","last_became_peered":"2024-09-10T07:51:12.665689+0000","last_unstale":"2024-09-10T07:51:13.828044+0000","last_undegraded":"2024-09-10T07:51:13.828044+0000","last_fullsized":"2024-09-10T07:51:13.828044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:39.884571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643075+0000","last_change":"2024-09-10T07:51:11.653391+0000","last_active":"2024-09-10T07:51:12.643075+0000","last_peered":"2024-09-10T07:51:12.643075+0000","last_clean":"2024-09-10T07:51:12.643075+0000","last_became_active":"2024-09-10T07:51:11.652803+0000","last_became_peered":"2024-09-10T07:51:11.652803+0000","last_unstale":"2024-09-10T07:51:12.643075+0000","last_undegraded":"2024-09-10T07:51:12.643075+0000","last_fullsized":"2024-09-10T07:51:12.643075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:38.179483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654603+0000","last_change":"2024-09-10T07:51:12.676013+0000","last_active":"2024-09-10T07:51:13.654603+0000","last_peered":"2024-09-10T07:51:13.654603+0000","last_clean":"2024-09-10T07:51:13.654603+0000","last_became_active":"2024-09-10T07:51:12.675718+0000","last_became_peered":"2024-09-10T07:51:12.675718+0000","last_unstale":"2024-09-10T07:51:13.654603+0000","last_undegraded":"2024-09-10T07:51:13.654603+0000","last_fullsized":"2024-09-10T07:51:13.654603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.642361+0000","last_change":"2024-09-10T07:51:11.653518+0000","last_active":"2024-09-10T07:51:12.642361+0000","last_peered":"2024-09-10T07:51:12.642361+0000","last_clean":"2024-09-10T07:51:12.642361+0000","last_became_active":"2024-09-10T07:51:11.653198+0000","last_became_peered":"2024-09-10T07:51:11.653198+0000","last_unstale":"2024-09-10T07:51:12.642361+0000","last_undegraded":"2024-09-10T07:51:12.642361+0000","last_fullsized":"2024-09-10T07:51:12.642361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.482793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827696+0000","last_change":"2024-09-10T07:51:12.657827+0000","last_active":"2024-09-10T07:51:13.827696+0000","last_peered":"2024-09-10T07:51:13.827696+0000","last_clean":"2024-09-10T07:51:13.827696+0000","last_became_active":"2024-09-10T07:51:12.657483+0000","last_became_peered":"2024-09-10T07:51:12.657483+0000","last_unstale":"2024-09-10T07:51:13.827696+0000","last_undegraded":"2024-09-10T07:51:13.827696+0000","last_fullsized":"2024-09-10T07:51:13.827696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.217868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T07:51:12.643007+0000","last_change":"2024-09-10T07:51:11.653782+0000","last_active":"2024-09-10T07:51:12.643007+0000","last_peered":"2024-09-10T07:51:12.643007+0000","last_clean":"2024-09-10T07:51:12.643007+0000","last_became_active":"2024-09-10T07:51:11.650110+0000","last_became_peered":"2024-09-10T07:51:11.650110+0000","last_unstale":"2024-09-10T07:51:12.643007+0000","last_undegraded":"2024-09-10T07:51:12.643007+0000","last_fullsized":"2024-09-10T07:51:12.643007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:16.299195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827512+0000","last_change":"2024-09-10T07:51:12.677759+0000","last_active":"2024-09-10T07:51:13.827512+0000","last_peered":"2024-09-10T07:51:13.827512+0000","last_clean":"2024-09-10T07:51:13.827512+0000","last_became_active":"2024-09-10T07:51:12.677359+0000","last_became_peered":"2024-09-10T07:51:12.677359+0000","last_unstale":"2024-09-10T07:51:13.827512+0000","last_undegraded":"2024-09-10T07:51:13.827512+0000","last_fullsized":"2024-09-10T07:51:13.827512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:25.109899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827404+0000","last_change":"2024-09-10T07:51:11.662912+0000","last_active":"2024-09-10T07:51:13.827404+0000","last_peered":"2024-09-10T07:51:13.827404+0000","last_clean":"2024-09-10T07:51:13.827404+0000","last_became_active":"2024-09-10T07:51:11.662677+0000","last_became_peered":"2024-09-10T07:51:11.662677+0000","last_unstale":"2024-09-10T07:51:13.827404+0000","last_undegraded":"2024-09-10T07:51:13.827404+0000","last_fullsized":"2024-09-10T07:51:13.827404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:18.751797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655470+0000","last_change":"2024-09-10T07:51:12.672712+0000","last_active":"2024-09-10T07:51:13.655470+0000","last_peered":"2024-09-10T07:51:13.655470+0000","last_clean":"2024-09-10T07:51:13.655470+0000","last_became_active":"2024-09-10T07:51:12.670900+0000","last_became_peered":"2024-09-10T07:51:12.670900+0000","last_unstale":"2024-09-10T07:51:13.655470+0000","last_undegraded":"2024-09-10T07:51:13.655470+0000","last_fullsized":"2024-09-10T07:51:13.655470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827078+0000","last_change":"2024-09-10T07:51:11.655026+0000","last_active":"2024-09-10T07:51:13.827078+0000","last_peered":"2024-09-10T07:51:13.827078+0000","last_clean":"2024-09-10T07:51:13.827078+0000","last_became_active":"2024-09-10T07:51:11.654747+0000","last_became_peered":"2024-09-10T07:51:11.654747+0000","last_unstale":"2024-09-10T07:51:13.827078+0000","last_undegraded":"2024-09-10T07:51:13.827078+0000","last_fullsized":"2024-09-10T07:51:13.827078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:13.073312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826895+0000","last_change":"2024-09-10T07:51:12.680995+0000","last_active":"2024-09-10T07:51:13.826895+0000","last_peered":"2024-09-10T07:51:13.826895+0000","last_clean":"2024-09-10T07:51:13.826895+0000","last_became_active":"2024-09-10T07:51:12.680592+0000","last_became_peered":"2024-09-10T07:51:12.680592+0000","last_unstale":"2024-09-10T07:51:13.826895+0000","last_undegraded":"2024-09-10T07:51:13.826895+0000","last_fullsized":"2024-09-10T07:51:13.826895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:43.727455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826629+0000","last_change":"2024-09-10T07:51:11.659975+0000","last_active":"2024-09-10T07:51:13.826629+0000","last_peered":"2024-09-10T07:51:13.826629+0000","last_clean":"2024-09-10T07:51:13.826629+0000","last_became_active":"2024-09-10T07:51:11.657387+0000","last_became_peered":"2024-09-10T07:51:11.657387+0000","last_unstale":"2024-09-10T07:51:13.826629+0000","last_undegraded":"2024-09-10T07:51:13.826629+0000","last_fullsized":"2024-09-10T07:51:13.826629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:07.331178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827325+0000","last_change":"2024-09-10T07:51:12.675741+0000","last_active":"2024-09-10T07:51:13.827325+0000","last_peered":"2024-09-10T07:51:13.827325+0000","last_clean":"2024-09-10T07:51:13.827325+0000","last_became_active":"2024-09-10T07:51:12.675054+0000","last_became_peered":"2024-09-10T07:51:12.675054+0000","last_unstale":"2024-09-10T07:51:13.827325+0000","last_undegraded":"2024-09-10T07:51:13.827325+0000","last_fullsized":"2024-09-10T07:51:13.827325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:46.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654538+0000","last_change":"2024-09-10T07:51:11.660202+0000","last_active":"2024-09-10T07:51:13.654538+0000","last_peered":"2024-09-10T07:51:13.654538+0000","last_clean":"2024-09-10T07:51:13.654538+0000","last_became_active":"2024-09-10T07:51:11.659089+0000","last_became_peered":"2024-09-10T07:51:11.659089+0000","last_unstale":"2024-09-10T07:51:13.654538+0000","last_undegraded":"2024-09-10T07:51:13.654538+0000","last_fullsized":"2024-09-10T07:51:13.654538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:34.657006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":273,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654859+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:51:13.654859+0000","last_peered":"2024-09-10T07:51:13.654859+0000","last_clean":"2024-09-10T07:51:13.654859+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:51:13.654859+0000","last_undegraded":"2024-09-10T07:51:13.654859+0000","last_fullsized":"2024-09-10T07:51:13.654859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476799,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":402248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1531904,"data_stored":992490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":403734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":403768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1536000,"data_stored":992580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":913408,"data_stored":389047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:51:21.555 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:51:21.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.724+0000 7f186b600640 1 Processor -- start 2024-09-10T07:51:21.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.724+0000 7f186b600640 1 -- start start 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.725+0000 7f186b600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 0x7f18640cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.725+0000 7f186b600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18640cfdd0 con 0x7f1864160af0 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.725+0000 7f186b600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186415cc90 con 0x7f18640d07c0 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.725+0000 7f186b600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f186415cf70 con 0x7f1864160710 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1864160af0 0x7f18640d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35144/0 (socket says 172.21.15.3:35144) 2024-09-10T07:51:21.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863e00640 1 -- 172.21.15.3:0/3059454082 learned_addr learned my addr 172.21.15.3:0/3059454082 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1868c00640 1 --2- 172.21.15.3:0/3059454082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 0x7f18640cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944130385 0 0) 0x7f18640cfdd0 con 0x7f1864160af0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f184c003520 con 0x7f1864160af0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213302136 0 0) 0x7f186415cc90 con 0x7f18640d07c0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18640cfdd0 con 0x7f18640d07c0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1868c00640 1 -- 172.21.15.3:0/3059454082 >> v1:172.21.15.70:6789/0 conn(0x7f18640d07c0 legacy=0x7f1864072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1868c00640 1 -- 172.21.15.3:0/3059454082 >> v1:172.21.15.3:6789/0 conn(0x7f1864160af0 legacy=0x7f18640d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 346752989 0 0) 0x7f184c003520 con 0x7f1864160af0 2024-09-10T07:51:21.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.726+0000 7f1868c00640 1 -- 172.21.15.3:0/3059454082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186415d250 con 0x7f1864160710 2024-09-10T07:51:21.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f1868c00640 1 --2- 172.21.15.3:0/3059454082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 0x7f18640cf2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18580097d0 tx=0x7f185802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd37846a9aee4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858037070 con 0x7f1864160710 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185802ac30 con 0x7f1864160710 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f1863400640 1 -- 172.21.15.3:0/3059454082 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850001670 con 0x7f1864160710 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f186b600640 1 -- 172.21.15.3:0/3059454082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 msgr2=0x7f18640cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f186b600640 1 --2- 172.21.15.3:0/3059454082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 0x7f18640cf2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18580097d0 tx=0x7f185802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f186b600640 1 -- 172.21.15.3:0/3059454082 shutdown_connections 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f186b600640 1 --2- 172.21.15.3:0/3059454082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1864160710 0x7f18640cf2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.728+0000 7f186b600640 1 -- 172.21.15.3:0/3059454082 >> 172.21.15.3:0/3059454082 conn(0x7f186406d000 msgr2=0x7f186406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.729+0000 7f186b600640 1 -- 172.21.15.3:0/3059454082 shutdown_connections 2024-09-10T07:51:21.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.729+0000 7f186b600640 1 -- 172.21.15.3:0/3059454082 wait complete. 2024-09-10T07:51:21.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.730+0000 7f186b600640 1 Processor -- start 2024-09-10T07:51:21.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.730+0000 7f186b600640 1 -- start start 2024-09-10T07:51:21.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18640d07c0 0x7f186413ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1864160710 0x7f186414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f186415d770 con 0x7f1864160af0 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f18640cf800 con 0x7f1864160710 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.731+0000 7f186b600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1864150be0 con 0x7f18640d07c0 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55522/0 (socket says 172.21.15.3:55522) 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 -- 172.21.15.3:0/78253832 learned_addr learned my addr 172.21.15.3:0/78253832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1863e00640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1864160710 0x7f186414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1868c00640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18640d07c0 0x7f186413ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 -- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18640d07c0 msgr2=0x7f186413ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18640d07c0 0x7f186413ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 -- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1864160710 msgr2=0x7f186414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1864160710 0x7f186414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 -- 172.21.15.3:0/78253832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186414ffd0 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.732+0000 7f1869600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c790 tx=0x7f185c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.733+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c003b70 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.733+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c003d10 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.733+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00a9f0 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.733+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18641502b0 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.733+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1864150870 con 0x7f1864160af0 2024-09-10T07:51:21.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.734+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186415d8f0 con 0x7f1864160af0 2024-09-10T07:51:21.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.735+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f185c00ab90 con 0x7f1864160af0 2024-09-10T07:51:21.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.736+0000 7f1860c00640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 0x7f184406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:21.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.736+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f185c08ac80 con 0x7f1864160af0 2024-09-10T07:51:21.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.736+0000 7f1868c00640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 0x7f184406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:21.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.737+0000 7f1868c00640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 0x7f184406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1858034000 tx=0x7f185802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.740+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f185c058fd0 con 0x7f1864160af0 2024-09-10T07:51:21.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.920+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18640afd50 con 0x7f184406be30 2024-09-10T07:51:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.927+0000 7f1860c00640 1 -- 172.21.15.3:0/78253832 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252036 (secure 0 0 0) 0x7f18640afd50 con 0x7f184406be30 2024-09-10T07:51:21.929 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:21.931 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:51:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.930+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 msgr2=0x7f184406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.930+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 0x7f184406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1858034000 tx=0x7f185802e000 comp rx=0 tx=0).stop 2024-09-10T07:51:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.930+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 msgr2=0x7f1864151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.930+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c790 tx=0x7f185c00cc50 comp rx=0 tx=0).stop 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 shutdown_connections 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f184406be30 0x7f184406e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1864160af0 0x7f1864151070 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1864160710 0x7f186414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 --2- 172.21.15.3:0/78253832 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18640d07c0 0x7f186413ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 >> 172.21.15.3:0/78253832 conn(0x7f186406d000 msgr2=0x7f18640ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 shutdown_connections 2024-09-10T07:51:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:21.931+0000 7f186b600640 1 -- 172.21.15.3:0/78253832 wait complete. 2024-09-10T07:51:21.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-10T07:51:21.554696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262248,"kb_used_data":8552,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712472,"statfs":{"total":773094113280,"available":772825571328,"internally_reserved":0,"allocated":8757248,"data_stored":4402532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.915853"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828738+0000","last_change":"2024-09-10T07:51:12.676690+0000","last_active":"2024-09-10T07:51:13.828738+0000","last_peered":"2024-09-10T07:51:13.828738+0000","last_clean":"2024-09-10T07:51:13.828738+0000","last_became_active":"2024-09-10T07:51:12.676418+0000","last_became_peered":"2024-09-10T07:51:12.676418+0000","last_unstale":"2024-09-10T07:51:13.828738+0000","last_undegraded":"2024-09-10T07:51:13.828738+0000","last_fullsized":"2024-09-10T07:51:13.828738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:00.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827421+0000","last_change":"2024-09-10T07:51:12.676886+0000","last_active":"2024-09-10T07:51:13.827421+0000","last_peered":"2024-09-10T07:51:13.827421+0000","last_clean":"2024-09-10T07:51:13.827421+0000","last_became_active":"2024-09-10T07:51:12.676680+0000","last_became_peered":"2024-09-10T07:51:12.676680+0000","last_unstale":"2024-09-10T07:51:13.827421+0000","last_undegraded":"2024-09-10T07:51:13.827421+0000","last_fullsized":"2024-09-10T07:51:13.827421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656352+0000","last_change":"2024-09-10T07:51:12.677258+0000","last_active":"2024-09-10T07:51:13.656352+0000","last_peered":"2024-09-10T07:51:13.656352+0000","last_clean":"2024-09-10T07:51:13.656352+0000","last_became_active":"2024-09-10T07:51:12.676957+0000","last_became_peered":"2024-09-10T07:51:12.676957+0000","last_unstale":"2024-09-10T07:51:13.656352+0000","last_undegraded":"2024-09-10T07:51:13.656352+0000","last_fullsized":"2024-09-10T07:51:13.656352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655764+0000","last_change":"2024-09-10T07:51:12.675735+0000","last_active":"2024-09-10T07:51:13.655764+0000","last_peered":"2024-09-10T07:51:13.655764+0000","last_clean":"2024-09-10T07:51:13.655764+0000","last_became_active":"2024-09-10T07:51:12.675442+0000","last_became_peered":"2024-09-10T07:51:12.675442+0000","last_unstale":"2024-09-10T07:51:13.655764+0000","last_undegraded":"2024-09-10T07:51:13.655764+0000","last_fullsized":"2024-09-10T07:51:13.655764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:00.169360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827458+0000","last_change":"2024-09-10T07:51:12.670954+0000","last_active":"2024-09-10T07:51:13.827458+0000","last_peered":"2024-09-10T07:51:13.827458+0000","last_clean":"2024-09-10T07:51:13.827458+0000","last_became_active":"2024-09-10T07:51:12.670673+0000","last_became_peered":"2024-09-10T07:51:12.670673+0000","last_unstale":"2024-09-10T07:51:13.827458+0000","last_undegraded":"2024-09-10T07:51:13.827458+0000","last_fullsized":"2024-09-10T07:51:13.827458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:52.871411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826972+0000","last_change":"2024-09-10T07:51:12.681827+0000","last_active":"2024-09-10T07:51:13.826972+0000","last_peered":"2024-09-10T07:51:13.826972+0000","last_clean":"2024-09-10T07:51:13.826972+0000","last_became_active":"2024-09-10T07:51:12.681558+0000","last_became_peered":"2024-09-10T07:51:12.681558+0000","last_unstale":"2024-09-10T07:51:13.826972+0000","last_undegraded":"2024-09-10T07:51:13.826972+0000","last_fullsized":"2024-09-10T07:51:13.826972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654188+0000","last_change":"2024-09-10T07:51:12.667538+0000","last_active":"2024-09-10T07:51:13.654188+0000","last_peered":"2024-09-10T07:51:13.654188+0000","last_clean":"2024-09-10T07:51:13.654188+0000","last_became_active":"2024-09-10T07:51:12.667277+0000","last_became_peered":"2024-09-10T07:51:12.667277+0000","last_unstale":"2024-09-10T07:51:13.654188+0000","last_undegraded":"2024-09-10T07:51:13.654188+0000","last_fullsized":"2024-09-10T07:51:13.654188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826644+0000","last_change":"2024-09-10T07:51:12.668124+0000","last_active":"2024-09-10T07:51:13.826644+0000","last_peered":"2024-09-10T07:51:13.826644+0000","last_clean":"2024-09-10T07:51:13.826644+0000","last_became_active":"2024-09-10T07:51:12.667798+0000","last_became_peered":"2024-09-10T07:51:12.667798+0000","last_unstale":"2024-09-10T07:51:13.826644+0000","last_undegraded":"2024-09-10T07:51:13.826644+0000","last_fullsized":"2024-09-10T07:51:13.826644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:27.002731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656452+0000","last_change":"2024-09-10T07:51:12.663829+0000","last_active":"2024-09-10T07:51:13.656452+0000","last_peered":"2024-09-10T07:51:13.656452+0000","last_clean":"2024-09-10T07:51:13.656452+0000","last_became_active":"2024-09-10T07:51:12.663536+0000","last_became_peered":"2024-09-10T07:51:12.663536+0000","last_unstale":"2024-09-10T07:51:13.656452+0000","last_undegraded":"2024-09-10T07:51:13.656452+0000","last_fullsized":"2024-09-10T07:51:13.656452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654749+0000","last_change":"2024-09-10T07:51:12.667036+0000","last_active":"2024-09-10T07:51:13.654749+0000","last_peered":"2024-09-10T07:51:13.654749+0000","last_clean":"2024-09-10T07:51:13.654749+0000","last_became_active":"2024-09-10T07:51:12.666744+0000","last_became_peered":"2024-09-10T07:51:12.666744+0000","last_unstale":"2024-09-10T07:51:13.654749+0000","last_undegraded":"2024-09-10T07:51:13.654749+0000","last_fullsized":"2024-09-10T07:51:13.654749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828107+0000","last_change":"2024-09-10T07:51:12.663072+0000","last_active":"2024-09-10T07:51:13.828107+0000","last_peered":"2024-09-10T07:51:13.828107+0000","last_clean":"2024-09-10T07:51:13.828107+0000","last_became_active":"2024-09-10T07:51:12.662799+0000","last_became_peered":"2024-09-10T07:51:12.662799+0000","last_unstale":"2024-09-10T07:51:13.828107+0000","last_undegraded":"2024-09-10T07:51:13.828107+0000","last_fullsized":"2024-09-10T07:51:13.828107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:40.034143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827540+0000","last_change":"2024-09-10T07:51:12.647897+0000","last_active":"2024-09-10T07:51:13.827540+0000","last_peered":"2024-09-10T07:51:13.827540+0000","last_clean":"2024-09-10T07:51:13.827540+0000","last_became_active":"2024-09-10T07:51:12.647493+0000","last_became_peered":"2024-09-10T07:51:12.647493+0000","last_unstale":"2024-09-10T07:51:13.827540+0000","last_undegraded":"2024-09-10T07:51:13.827540+0000","last_fullsized":"2024-09-10T07:51:13.827540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:20:25.562681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655874+0000","last_change":"2024-09-10T07:51:12.675728+0000","last_active":"2024-09-10T07:51:13.655874+0000","last_peered":"2024-09-10T07:51:13.655874+0000","last_clean":"2024-09-10T07:51:13.655874+0000","last_became_active":"2024-09-10T07:51:12.675422+0000","last_became_peered":"2024-09-10T07:51:12.675422+0000","last_unstale":"2024-09-10T07:51:13.655874+0000","last_undegraded":"2024-09-10T07:51:13.655874+0000","last_fullsized":"2024-09-10T07:51:13.655874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:59.255796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827937+0000","last_change":"2024-09-10T07:51:12.676376+0000","last_active":"2024-09-10T07:51:13.827937+0000","last_peered":"2024-09-10T07:51:13.827937+0000","last_clean":"2024-09-10T07:51:13.827937+0000","last_became_active":"2024-09-10T07:51:12.676068+0000","last_became_peered":"2024-09-10T07:51:12.676068+0000","last_unstale":"2024-09-10T07:51:13.827937+0000","last_undegraded":"2024-09-10T07:51:13.827937+0000","last_fullsized":"2024-09-10T07:51:13.827937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:11.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827549+0000","last_change":"2024-09-10T07:51:12.659238+0000","last_active":"2024-09-10T07:51:13.827549+0000","last_peered":"2024-09-10T07:51:13.827549+0000","last_clean":"2024-09-10T07:51:13.827549+0000","last_became_active":"2024-09-10T07:51:12.658975+0000","last_became_peered":"2024-09-10T07:51:12.658975+0000","last_unstale":"2024-09-10T07:51:13.827549+0000","last_undegraded":"2024-09-10T07:51:13.827549+0000","last_fullsized":"2024-09-10T07:51:13.827549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:12:59.909617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827585+0000","last_change":"2024-09-10T07:51:12.675441+0000","last_active":"2024-09-10T07:51:13.827585+0000","last_peered":"2024-09-10T07:51:13.827585+0000","last_clean":"2024-09-10T07:51:13.827585+0000","last_became_active":"2024-09-10T07:51:12.673010+0000","last_became_peered":"2024-09-10T07:51:12.673010+0000","last_unstale":"2024-09-10T07:51:13.827585+0000","last_undegraded":"2024-09-10T07:51:13.827585+0000","last_fullsized":"2024-09-10T07:51:13.827585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:20.220354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827527+0000","last_change":"2024-09-10T07:51:12.657787+0000","last_active":"2024-09-10T07:51:13.827527+0000","last_peered":"2024-09-10T07:51:13.827527+0000","last_clean":"2024-09-10T07:51:13.827527+0000","last_became_active":"2024-09-10T07:51:12.657501+0000","last_became_peered":"2024-09-10T07:51:12.657501+0000","last_unstale":"2024-09-10T07:51:13.827527+0000","last_undegraded":"2024-09-10T07:51:13.827527+0000","last_fullsized":"2024-09-10T07:51:13.827527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:21.758327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827383+0000","last_change":"2024-09-10T07:51:12.674036+0000","last_active":"2024-09-10T07:51:13.827383+0000","last_peered":"2024-09-10T07:51:13.827383+0000","last_clean":"2024-09-10T07:51:13.827383+0000","last_became_active":"2024-09-10T07:51:12.673722+0000","last_became_peered":"2024-09-10T07:51:12.673722+0000","last_unstale":"2024-09-10T07:51:13.827383+0000","last_undegraded":"2024-09-10T07:51:13.827383+0000","last_fullsized":"2024-09-10T07:51:13.827383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:58:17.696759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827952+0000","last_change":"2024-09-10T07:51:12.663140+0000","last_active":"2024-09-10T07:51:13.827952+0000","last_peered":"2024-09-10T07:51:13.827952+0000","last_clean":"2024-09-10T07:51:13.827952+0000","last_became_active":"2024-09-10T07:51:12.662830+0000","last_became_peered":"2024-09-10T07:51:12.662830+0000","last_unstale":"2024-09-10T07:51:13.827952+0000","last_undegraded":"2024-09-10T07:51:13.827952+0000","last_fullsized":"2024-09-10T07:51:13.827952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:52.000639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827123+0000","last_change":"2024-09-10T07:51:12.663045+0000","last_active":"2024-09-10T07:51:13.827123+0000","last_peered":"2024-09-10T07:51:13.827123+0000","last_clean":"2024-09-10T07:51:13.827123+0000","last_became_active":"2024-09-10T07:51:12.662741+0000","last_became_peered":"2024-09-10T07:51:12.662741+0000","last_unstale":"2024-09-10T07:51:13.827123+0000","last_undegraded":"2024-09-10T07:51:13.827123+0000","last_fullsized":"2024-09-10T07:51:13.827123+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:49.480454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826731+0000","last_change":"2024-09-10T07:51:12.660520+0000","last_active":"2024-09-10T07:51:13.826731+0000","last_peered":"2024-09-10T07:51:13.826731+0000","last_clean":"2024-09-10T07:51:13.826731+0000","last_became_active":"2024-09-10T07:51:12.660258+0000","last_became_peered":"2024-09-10T07:51:12.660258+0000","last_unstale":"2024-09-10T07:51:13.826731+0000","last_undegraded":"2024-09-10T07:51:13.826731+0000","last_fullsized":"2024-09-10T07:51:13.826731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:10.975319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827016+0000","last_change":"2024-09-10T07:51:12.657447+0000","last_active":"2024-09-10T07:51:13.827016+0000","last_peered":"2024-09-10T07:51:13.827016+0000","last_clean":"2024-09-10T07:51:13.827016+0000","last_became_active":"2024-09-10T07:51:12.657127+0000","last_became_peered":"2024-09-10T07:51:12.657127+0000","last_unstale":"2024-09-10T07:51:13.827016+0000","last_undegraded":"2024-09-10T07:51:13.827016+0000","last_fullsized":"2024-09-10T07:51:13.827016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654357+0000","last_change":"2024-09-10T07:51:12.679669+0000","last_active":"2024-09-10T07:51:13.654357+0000","last_peered":"2024-09-10T07:51:13.654357+0000","last_clean":"2024-09-10T07:51:13.654357+0000","last_became_active":"2024-09-10T07:51:12.679361+0000","last_became_peered":"2024-09-10T07:51:12.679361+0000","last_unstale":"2024-09-10T07:51:13.654357+0000","last_undegraded":"2024-09-10T07:51:13.654357+0000","last_fullsized":"2024-09-10T07:51:13.654357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827729+0000","last_change":"2024-09-10T07:51:12.667532+0000","last_active":"2024-09-10T07:51:13.827729+0000","last_peered":"2024-09-10T07:51:13.827729+0000","last_clean":"2024-09-10T07:51:13.827729+0000","last_became_active":"2024-09-10T07:51:12.666659+0000","last_became_peered":"2024-09-10T07:51:12.666659+0000","last_unstale":"2024-09-10T07:51:13.827729+0000","last_undegraded":"2024-09-10T07:51:13.827729+0000","last_fullsized":"2024-09-10T07:51:13.827729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:33.240184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654775+0000","last_change":"2024-09-10T07:51:12.679672+0000","last_active":"2024-09-10T07:51:13.654775+0000","last_peered":"2024-09-10T07:51:13.654775+0000","last_clean":"2024-09-10T07:51:13.654775+0000","last_became_active":"2024-09-10T07:51:12.679422+0000","last_became_peered":"2024-09-10T07:51:12.679422+0000","last_unstale":"2024-09-10T07:51:13.654775+0000","last_undegraded":"2024-09-10T07:51:13.654775+0000","last_fullsized":"2024-09-10T07:51:13.654775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:26.016470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827692+0000","last_change":"2024-09-10T07:51:12.681734+0000","last_active":"2024-09-10T07:51:13.827692+0000","last_peered":"2024-09-10T07:51:13.827692+0000","last_clean":"2024-09-10T07:51:13.827692+0000","last_became_active":"2024-09-10T07:51:12.681454+0000","last_became_peered":"2024-09-10T07:51:12.681454+0000","last_unstale":"2024-09-10T07:51:13.827692+0000","last_undegraded":"2024-09-10T07:51:13.827692+0000","last_fullsized":"2024-09-10T07:51:13.827692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:59.692249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827301+0000","last_change":"2024-09-10T07:51:12.661140+0000","last_active":"2024-09-10T07:51:13.827301+0000","last_peered":"2024-09-10T07:51:13.827301+0000","last_clean":"2024-09-10T07:51:13.827301+0000","last_became_active":"2024-09-10T07:51:12.660758+0000","last_became_peered":"2024-09-10T07:51:12.660758+0000","last_unstale":"2024-09-10T07:51:13.827301+0000","last_undegraded":"2024-09-10T07:51:13.827301+0000","last_fullsized":"2024-09-10T07:51:13.827301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:12.250282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827897+0000","last_change":"2024-09-10T07:51:12.670722+0000","last_active":"2024-09-10T07:51:13.827897+0000","last_peered":"2024-09-10T07:51:13.827897+0000","last_clean":"2024-09-10T07:51:13.827897+0000","last_became_active":"2024-09-10T07:51:12.667899+0000","last_became_peered":"2024-09-10T07:51:12.667899+0000","last_unstale":"2024-09-10T07:51:13.827897+0000","last_undegraded":"2024-09-10T07:51:13.827897+0000","last_fullsized":"2024-09-10T07:51:13.827897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656066+0000","last_change":"2024-09-10T07:51:12.680797+0000","last_active":"2024-09-10T07:51:13.656066+0000","last_peered":"2024-09-10T07:51:13.656066+0000","last_clean":"2024-09-10T07:51:13.656066+0000","last_became_active":"2024-09-10T07:51:12.680521+0000","last_became_peered":"2024-09-10T07:51:12.680521+0000","last_unstale":"2024-09-10T07:51:13.656066+0000","last_undegraded":"2024-09-10T07:51:13.656066+0000","last_fullsized":"2024-09-10T07:51:13.656066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:03.932008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828525+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.828525+0000","last_peered":"2024-09-10T07:51:13.828525+0000","last_clean":"2024-09-10T07:51:13.828525+0000","last_became_active":"2024-09-10T07:51:12.666221+0000","last_became_peered":"2024-09-10T07:51:12.666221+0000","last_unstale":"2024-09-10T07:51:13.828525+0000","last_undegraded":"2024-09-10T07:51:13.828525+0000","last_fullsized":"2024-09-10T07:51:13.828525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:20.727768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826628+0000","last_change":"2024-09-10T07:51:12.667382+0000","last_active":"2024-09-10T07:51:13.826628+0000","last_peered":"2024-09-10T07:51:13.826628+0000","last_clean":"2024-09-10T07:51:13.826628+0000","last_became_active":"2024-09-10T07:51:12.667121+0000","last_became_peered":"2024-09-10T07:51:12.667121+0000","last_unstale":"2024-09-10T07:51:13.826628+0000","last_undegraded":"2024-09-10T07:51:13.826628+0000","last_fullsized":"2024-09-10T07:51:13.826628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:04.388302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826696+0000","last_change":"2024-09-10T07:51:12.668706+0000","last_active":"2024-09-10T07:51:13.826696+0000","last_peered":"2024-09-10T07:51:13.826696+0000","last_clean":"2024-09-10T07:51:13.826696+0000","last_became_active":"2024-09-10T07:51:12.668445+0000","last_became_peered":"2024-09-10T07:51:12.668445+0000","last_unstale":"2024-09-10T07:51:13.826696+0000","last_undegraded":"2024-09-10T07:51:13.826696+0000","last_fullsized":"2024-09-10T07:51:13.826696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:56.232909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827833+0000","last_change":"2024-09-10T07:51:12.679812+0000","last_active":"2024-09-10T07:51:13.827833+0000","last_peered":"2024-09-10T07:51:13.827833+0000","last_clean":"2024-09-10T07:51:13.827833+0000","last_became_active":"2024-09-10T07:51:12.669963+0000","last_became_peered":"2024-09-10T07:51:12.669963+0000","last_unstale":"2024-09-10T07:51:13.827833+0000","last_undegraded":"2024-09-10T07:51:13.827833+0000","last_fullsized":"2024-09-10T07:51:13.827833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:47.211095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"77'4","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828342+0000","last_change":"2024-09-10T07:51:11.647694+0000","last_active":"2024-09-10T07:51:13.828342+0000","last_peered":"2024-09-10T07:51:13.828342+0000","last_clean":"2024-09-10T07:51:13.828342+0000","last_became_active":"2024-09-10T07:51:11.647419+0000","last_became_peered":"2024-09-10T07:51:11.647419+0000","last_unstale":"2024-09-10T07:51:13.828342+0000","last_undegraded":"2024-09-10T07:51:13.828342+0000","last_fullsized":"2024-09-10T07:51:13.828342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:07.450590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656051+0000","last_change":"2024-09-10T07:51:12.660668+0000","last_active":"2024-09-10T07:51:13.656051+0000","last_peered":"2024-09-10T07:51:13.656051+0000","last_clean":"2024-09-10T07:51:13.656051+0000","last_became_active":"2024-09-10T07:51:12.660364+0000","last_became_peered":"2024-09-10T07:51:12.660364+0000","last_unstale":"2024-09-10T07:51:13.656051+0000","last_undegraded":"2024-09-10T07:51:13.656051+0000","last_fullsized":"2024-09-10T07:51:13.656051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:00.238016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827698+0000","last_change":"2024-09-10T07:51:11.657105+0000","last_active":"2024-09-10T07:51:13.827698+0000","last_peered":"2024-09-10T07:51:13.827698+0000","last_clean":"2024-09-10T07:51:13.827698+0000","last_became_active":"2024-09-10T07:51:11.656523+0000","last_became_peered":"2024-09-10T07:51:11.656523+0000","last_unstale":"2024-09-10T07:51:13.827698+0000","last_undegraded":"2024-09-10T07:51:13.827698+0000","last_fullsized":"2024-09-10T07:51:13.827698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:08.541922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827892+0000","last_change":"2024-09-10T07:51:12.669879+0000","last_active":"2024-09-10T07:51:13.827892+0000","last_peered":"2024-09-10T07:51:13.827892+0000","last_clean":"2024-09-10T07:51:13.827892+0000","last_became_active":"2024-09-10T07:51:12.668574+0000","last_became_peered":"2024-09-10T07:51:12.668574+0000","last_unstale":"2024-09-10T07:51:13.827892+0000","last_undegraded":"2024-09-10T07:51:13.827892+0000","last_fullsized":"2024-09-10T07:51:13.827892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828196+0000","last_change":"2024-09-10T07:51:11.648632+0000","last_active":"2024-09-10T07:51:13.828196+0000","last_peered":"2024-09-10T07:51:13.828196+0000","last_clean":"2024-09-10T07:51:13.828196+0000","last_became_active":"2024-09-10T07:51:11.648267+0000","last_became_peered":"2024-09-10T07:51:11.648267+0000","last_unstale":"2024-09-10T07:51:13.828196+0000","last_undegraded":"2024-09-10T07:51:13.828196+0000","last_fullsized":"2024-09-10T07:51:13.828196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:41.341276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654539+0000","last_change":"2024-09-10T07:51:12.670265+0000","last_active":"2024-09-10T07:51:13.654539+0000","last_peered":"2024-09-10T07:51:13.654539+0000","last_clean":"2024-09-10T07:51:13.654539+0000","last_became_active":"2024-09-10T07:51:12.669524+0000","last_became_peered":"2024-09-10T07:51:12.669524+0000","last_unstale":"2024-09-10T07:51:13.654539+0000","last_undegraded":"2024-09-10T07:51:13.654539+0000","last_fullsized":"2024-09-10T07:51:13.654539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827979+0000","last_change":"2024-09-10T07:51:11.661609+0000","last_active":"2024-09-10T07:51:13.827979+0000","last_peered":"2024-09-10T07:51:13.827979+0000","last_clean":"2024-09-10T07:51:13.827979+0000","last_became_active":"2024-09-10T07:51:11.660016+0000","last_became_peered":"2024-09-10T07:51:11.660016+0000","last_unstale":"2024-09-10T07:51:13.827979+0000","last_undegraded":"2024-09-10T07:51:13.827979+0000","last_fullsized":"2024-09-10T07:51:13.827979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:46.277801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828438+0000","last_change":"2024-09-10T07:51:12.667278+0000","last_active":"2024-09-10T07:51:13.828438+0000","last_peered":"2024-09-10T07:51:13.828438+0000","last_clean":"2024-09-10T07:51:13.828438+0000","last_became_active":"2024-09-10T07:51:12.666209+0000","last_became_peered":"2024-09-10T07:51:12.666209+0000","last_unstale":"2024-09-10T07:51:13.828438+0000","last_undegraded":"2024-09-10T07:51:13.828438+0000","last_fullsized":"2024-09-10T07:51:13.828438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:26.172183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827921+0000","last_change":"2024-09-10T07:51:11.654006+0000","last_active":"2024-09-10T07:51:13.827921+0000","last_peered":"2024-09-10T07:51:13.827921+0000","last_clean":"2024-09-10T07:51:13.827921+0000","last_became_active":"2024-09-10T07:51:11.651332+0000","last_became_peered":"2024-09-10T07:51:11.651332+0000","last_unstale":"2024-09-10T07:51:13.827921+0000","last_undegraded":"2024-09-10T07:51:13.827921+0000","last_fullsized":"2024-09-10T07:51:13.827921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:41.047040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828245+0000","last_change":"2024-09-10T07:51:12.666838+0000","last_active":"2024-09-10T07:51:13.828245+0000","last_peered":"2024-09-10T07:51:13.828245+0000","last_clean":"2024-09-10T07:51:13.828245+0000","last_became_active":"2024-09-10T07:51:12.666551+0000","last_became_peered":"2024-09-10T07:51:12.666551+0000","last_unstale":"2024-09-10T07:51:13.828245+0000","last_undegraded":"2024-09-10T07:51:13.828245+0000","last_fullsized":"2024-09-10T07:51:13.828245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:52.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654240+0000","last_change":"2024-09-10T07:51:11.666134+0000","last_active":"2024-09-10T07:51:13.654240+0000","last_peered":"2024-09-10T07:51:13.654240+0000","last_clean":"2024-09-10T07:51:13.654240+0000","last_became_active":"2024-09-10T07:51:11.665826+0000","last_became_peered":"2024-09-10T07:51:11.665826+0000","last_unstale":"2024-09-10T07:51:13.654240+0000","last_undegraded":"2024-09-10T07:51:13.654240+0000","last_fullsized":"2024-09-10T07:51:13.654240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:49.241190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656243+0000","last_change":"2024-09-10T07:51:12.667702+0000","last_active":"2024-09-10T07:51:13.656243+0000","last_peered":"2024-09-10T07:51:13.656243+0000","last_clean":"2024-09-10T07:51:13.656243+0000","last_became_active":"2024-09-10T07:51:12.667391+0000","last_became_peered":"2024-09-10T07:51:12.667391+0000","last_unstale":"2024-09-10T07:51:13.656243+0000","last_undegraded":"2024-09-10T07:51:13.656243+0000","last_fullsized":"2024-09-10T07:51:13.656243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:30.456232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655436+0000","last_change":"2024-09-10T07:51:11.664356+0000","last_active":"2024-09-10T07:51:13.655436+0000","last_peered":"2024-09-10T07:51:13.655436+0000","last_clean":"2024-09-10T07:51:13.655436+0000","last_became_active":"2024-09-10T07:51:11.663984+0000","last_became_peered":"2024-09-10T07:51:11.663984+0000","last_unstale":"2024-09-10T07:51:13.655436+0000","last_undegraded":"2024-09-10T07:51:13.655436+0000","last_fullsized":"2024-09-10T07:51:13.655436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:54:02.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827826+0000","last_change":"2024-09-10T07:51:12.668160+0000","last_active":"2024-09-10T07:51:13.827826+0000","last_peered":"2024-09-10T07:51:13.827826+0000","last_clean":"2024-09-10T07:51:13.827826+0000","last_became_active":"2024-09-10T07:51:12.667845+0000","last_became_peered":"2024-09-10T07:51:12.667845+0000","last_unstale":"2024-09-10T07:51:13.827826+0000","last_undegraded":"2024-09-10T07:51:13.827826+0000","last_fullsized":"2024-09-10T07:51:13.827826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:03.850277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827675+0000","last_change":"2024-09-10T07:51:11.647781+0000","last_active":"2024-09-10T07:51:13.827675+0000","last_peered":"2024-09-10T07:51:13.827675+0000","last_clean":"2024-09-10T07:51:13.827675+0000","last_became_active":"2024-09-10T07:51:11.647502+0000","last_became_peered":"2024-09-10T07:51:11.647502+0000","last_unstale":"2024-09-10T07:51:13.827675+0000","last_undegraded":"2024-09-10T07:51:13.827675+0000","last_fullsized":"2024-09-10T07:51:13.827675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:38.219907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654838+0000","last_change":"2024-09-10T07:51:12.680163+0000","last_active":"2024-09-10T07:51:13.654838+0000","last_peered":"2024-09-10T07:51:13.654838+0000","last_clean":"2024-09-10T07:51:13.654838+0000","last_became_active":"2024-09-10T07:51:12.679482+0000","last_became_peered":"2024-09-10T07:51:12.679482+0000","last_unstale":"2024-09-10T07:51:13.654838+0000","last_undegraded":"2024-09-10T07:51:13.654838+0000","last_fullsized":"2024-09-10T07:51:13.654838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828385+0000","last_change":"2024-09-10T07:51:11.652611+0000","last_active":"2024-09-10T07:51:13.828385+0000","last_peered":"2024-09-10T07:51:13.828385+0000","last_clean":"2024-09-10T07:51:13.828385+0000","last_became_active":"2024-09-10T07:51:11.652322+0000","last_became_peered":"2024-09-10T07:51:11.652322+0000","last_unstale":"2024-09-10T07:51:13.828385+0000","last_undegraded":"2024-09-10T07:51:13.828385+0000","last_fullsized":"2024-09-10T07:51:13.828385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:03:48.473083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827225+0000","last_change":"2024-09-10T07:51:12.681992+0000","last_active":"2024-09-10T07:51:13.827225+0000","last_peered":"2024-09-10T07:51:13.827225+0000","last_clean":"2024-09-10T07:51:13.827225+0000","last_became_active":"2024-09-10T07:51:12.680523+0000","last_became_peered":"2024-09-10T07:51:12.680523+0000","last_unstale":"2024-09-10T07:51:13.827225+0000","last_undegraded":"2024-09-10T07:51:13.827225+0000","last_fullsized":"2024-09-10T07:51:13.827225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:49.854131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827642+0000","last_change":"2024-09-10T07:51:11.661453+0000","last_active":"2024-09-10T07:51:13.827642+0000","last_peered":"2024-09-10T07:51:13.827642+0000","last_clean":"2024-09-10T07:51:13.827642+0000","last_became_active":"2024-09-10T07:51:11.661202+0000","last_became_peered":"2024-09-10T07:51:11.661202+0000","last_unstale":"2024-09-10T07:51:13.827642+0000","last_undegraded":"2024-09-10T07:51:13.827642+0000","last_fullsized":"2024-09-10T07:51:13.827642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:25.009100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828046+0000","last_change":"2024-09-10T07:51:12.677005+0000","last_active":"2024-09-10T07:51:13.828046+0000","last_peered":"2024-09-10T07:51:13.828046+0000","last_clean":"2024-09-10T07:51:13.828046+0000","last_became_active":"2024-09-10T07:51:12.676702+0000","last_became_peered":"2024-09-10T07:51:12.676702+0000","last_unstale":"2024-09-10T07:51:13.828046+0000","last_undegraded":"2024-09-10T07:51:13.828046+0000","last_fullsized":"2024-09-10T07:51:13.828046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:55.758076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"77'3","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828116+0000","last_change":"2024-09-10T07:51:11.651919+0000","last_active":"2024-09-10T07:51:13.828116+0000","last_peered":"2024-09-10T07:51:13.828116+0000","last_clean":"2024-09-10T07:51:13.828116+0000","last_became_active":"2024-09-10T07:51:11.651676+0000","last_became_peered":"2024-09-10T07:51:11.651676+0000","last_unstale":"2024-09-10T07:51:13.828116+0000","last_undegraded":"2024-09-10T07:51:13.828116+0000","last_fullsized":"2024-09-10T07:51:13.828116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827121+0000","last_change":"2024-09-10T07:51:12.661023+0000","last_active":"2024-09-10T07:51:13.827121+0000","last_peered":"2024-09-10T07:51:13.827121+0000","last_clean":"2024-09-10T07:51:13.827121+0000","last_became_active":"2024-09-10T07:51:12.660734+0000","last_became_peered":"2024-09-10T07:51:12.660734+0000","last_unstale":"2024-09-10T07:51:13.827121+0000","last_undegraded":"2024-09-10T07:51:13.827121+0000","last_fullsized":"2024-09-10T07:51:13.827121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:18.004956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"77'6","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656317+0000","last_change":"2024-09-10T07:51:11.659183+0000","last_active":"2024-09-10T07:51:13.656317+0000","last_peered":"2024-09-10T07:51:13.656317+0000","last_clean":"2024-09-10T07:51:13.656317+0000","last_became_active":"2024-09-10T07:51:11.658378+0000","last_became_peered":"2024-09-10T07:51:11.658378+0000","last_unstale":"2024-09-10T07:51:13.656317+0000","last_undegraded":"2024-09-10T07:51:13.656317+0000","last_fullsized":"2024-09-10T07:51:13.656317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:59.230645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827115+0000","last_change":"2024-09-10T07:51:12.647744+0000","last_active":"2024-09-10T07:51:13.827115+0000","last_peered":"2024-09-10T07:51:13.827115+0000","last_clean":"2024-09-10T07:51:13.827115+0000","last_became_active":"2024-09-10T07:51:12.647440+0000","last_became_peered":"2024-09-10T07:51:12.647440+0000","last_unstale":"2024-09-10T07:51:13.827115+0000","last_undegraded":"2024-09-10T07:51:13.827115+0000","last_fullsized":"2024-09-10T07:51:13.827115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:27.957020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655557+0000","last_change":"2024-09-10T07:51:11.666220+0000","last_active":"2024-09-10T07:51:13.655557+0000","last_peered":"2024-09-10T07:51:13.655557+0000","last_clean":"2024-09-10T07:51:13.655557+0000","last_became_active":"2024-09-10T07:51:11.665886+0000","last_became_peered":"2024-09-10T07:51:11.665886+0000","last_unstale":"2024-09-10T07:51:13.655557+0000","last_undegraded":"2024-09-10T07:51:13.655557+0000","last_fullsized":"2024-09-10T07:51:13.655557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:50.496109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654970+0000","last_change":"2024-09-10T07:51:12.682681+0000","last_active":"2024-09-10T07:51:13.654970+0000","last_peered":"2024-09-10T07:51:13.654970+0000","last_clean":"2024-09-10T07:51:13.654970+0000","last_became_active":"2024-09-10T07:51:12.682424+0000","last_became_peered":"2024-09-10T07:51:12.682424+0000","last_unstale":"2024-09-10T07:51:13.654970+0000","last_undegraded":"2024-09-10T07:51:13.654970+0000","last_fullsized":"2024-09-10T07:51:13.654970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654264+0000","last_change":"2024-09-10T07:51:11.661553+0000","last_active":"2024-09-10T07:51:13.654264+0000","last_peered":"2024-09-10T07:51:13.654264+0000","last_clean":"2024-09-10T07:51:13.654264+0000","last_became_active":"2024-09-10T07:51:11.661263+0000","last_became_peered":"2024-09-10T07:51:11.661263+0000","last_unstale":"2024-09-10T07:51:13.654264+0000","last_undegraded":"2024-09-10T07:51:13.654264+0000","last_fullsized":"2024-09-10T07:51:13.654264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:54.532976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826983+0000","last_change":"2024-09-10T07:51:12.661693+0000","last_active":"2024-09-10T07:51:13.826983+0000","last_peered":"2024-09-10T07:51:13.826983+0000","last_clean":"2024-09-10T07:51:13.826983+0000","last_became_active":"2024-09-10T07:51:12.661427+0000","last_became_peered":"2024-09-10T07:51:12.661427+0000","last_unstale":"2024-09-10T07:51:13.826983+0000","last_undegraded":"2024-09-10T07:51:13.826983+0000","last_fullsized":"2024-09-10T07:51:13.826983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:15.540616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654447+0000","last_change":"2024-09-10T07:51:11.657359+0000","last_active":"2024-09-10T07:51:13.654447+0000","last_peered":"2024-09-10T07:51:13.654447+0000","last_clean":"2024-09-10T07:51:13.654447+0000","last_became_active":"2024-09-10T07:51:11.657038+0000","last_became_peered":"2024-09-10T07:51:11.657038+0000","last_unstale":"2024-09-10T07:51:13.654447+0000","last_undegraded":"2024-09-10T07:51:13.654447+0000","last_fullsized":"2024-09-10T07:51:13.654447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:55.533736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656447+0000","last_change":"2024-09-10T07:51:12.661323+0000","last_active":"2024-09-10T07:51:13.656447+0000","last_peered":"2024-09-10T07:51:13.656447+0000","last_clean":"2024-09-10T07:51:13.656447+0000","last_became_active":"2024-09-10T07:51:12.658712+0000","last_became_peered":"2024-09-10T07:51:12.658712+0000","last_unstale":"2024-09-10T07:51:13.656447+0000","last_undegraded":"2024-09-10T07:51:13.656447+0000","last_fullsized":"2024-09-10T07:51:13.656447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:31.247714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"77'8","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654091+0000","last_change":"2024-09-10T07:51:11.664261+0000","last_active":"2024-09-10T07:51:13.654091+0000","last_peered":"2024-09-10T07:51:13.654091+0000","last_clean":"2024-09-10T07:51:13.654091+0000","last_became_active":"2024-09-10T07:51:11.663940+0000","last_became_peered":"2024-09-10T07:51:11.663940+0000","last_unstale":"2024-09-10T07:51:13.654091+0000","last_undegraded":"2024-09-10T07:51:13.654091+0000","last_fullsized":"2024-09-10T07:51:13.654091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:20.541470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827555+0000","last_change":"2024-09-10T07:51:12.658206+0000","last_active":"2024-09-10T07:51:13.827555+0000","last_peered":"2024-09-10T07:51:13.827555+0000","last_clean":"2024-09-10T07:51:13.827555+0000","last_became_active":"2024-09-10T07:51:12.657838+0000","last_became_peered":"2024-09-10T07:51:12.657838+0000","last_unstale":"2024-09-10T07:51:13.827555+0000","last_undegraded":"2024-09-10T07:51:13.827555+0000","last_fullsized":"2024-09-10T07:51:13.827555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:20.940139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828648+0000","last_change":"2024-09-10T07:51:11.654332+0000","last_active":"2024-09-10T07:51:13.828648+0000","last_peered":"2024-09-10T07:51:13.828648+0000","last_clean":"2024-09-10T07:51:13.828648+0000","last_became_active":"2024-09-10T07:51:11.654042+0000","last_became_peered":"2024-09-10T07:51:11.654042+0000","last_unstale":"2024-09-10T07:51:13.828648+0000","last_undegraded":"2024-09-10T07:51:13.828648+0000","last_fullsized":"2024-09-10T07:51:13.828648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:16.997585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827284+0000","last_change":"2024-09-10T07:51:12.667032+0000","last_active":"2024-09-10T07:51:13.827284+0000","last_peered":"2024-09-10T07:51:13.827284+0000","last_clean":"2024-09-10T07:51:13.827284+0000","last_became_active":"2024-09-10T07:51:12.666726+0000","last_became_peered":"2024-09-10T07:51:12.666726+0000","last_unstale":"2024-09-10T07:51:13.827284+0000","last_undegraded":"2024-09-10T07:51:13.827284+0000","last_fullsized":"2024-09-10T07:51:13.827284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:32.399662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827732+0000","last_change":"2024-09-10T07:51:11.655876+0000","last_active":"2024-09-10T07:51:13.827732+0000","last_peered":"2024-09-10T07:51:13.827732+0000","last_clean":"2024-09-10T07:51:13.827732+0000","last_became_active":"2024-09-10T07:51:11.655606+0000","last_became_peered":"2024-09-10T07:51:11.655606+0000","last_unstale":"2024-09-10T07:51:13.827732+0000","last_undegraded":"2024-09-10T07:51:13.827732+0000","last_fullsized":"2024-09-10T07:51:13.827732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:36.453798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655926+0000","last_change":"2024-09-10T07:51:12.657238+0000","last_active":"2024-09-10T07:51:13.655926+0000","last_peered":"2024-09-10T07:51:13.655926+0000","last_clean":"2024-09-10T07:51:13.655926+0000","last_became_active":"2024-09-10T07:51:12.656922+0000","last_became_peered":"2024-09-10T07:51:12.656922+0000","last_unstale":"2024-09-10T07:51:13.655926+0000","last_undegraded":"2024-09-10T07:51:13.655926+0000","last_fullsized":"2024-09-10T07:51:13.655926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.556394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"77'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826896+0000","last_change":"2024-09-10T07:51:11.654686+0000","last_active":"2024-09-10T07:51:13.826896+0000","last_peered":"2024-09-10T07:51:13.826896+0000","last_clean":"2024-09-10T07:51:13.826896+0000","last_became_active":"2024-09-10T07:51:11.653599+0000","last_became_peered":"2024-09-10T07:51:11.653599+0000","last_unstale":"2024-09-10T07:51:13.826896+0000","last_undegraded":"2024-09-10T07:51:13.826896+0000","last_fullsized":"2024-09-10T07:51:13.826896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:52.484707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.656395+0000","last_change":"2024-09-10T07:51:12.680792+0000","last_active":"2024-09-10T07:51:13.656395+0000","last_peered":"2024-09-10T07:51:13.656395+0000","last_clean":"2024-09-10T07:51:13.656395+0000","last_became_active":"2024-09-10T07:51:12.680491+0000","last_became_peered":"2024-09-10T07:51:12.680491+0000","last_unstale":"2024-09-10T07:51:13.656395+0000","last_undegraded":"2024-09-10T07:51:13.656395+0000","last_fullsized":"2024-09-10T07:51:13.656395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:51.503479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828075+0000","last_change":"2024-09-10T07:51:11.652720+0000","last_active":"2024-09-10T07:51:13.828075+0000","last_peered":"2024-09-10T07:51:13.828075+0000","last_clean":"2024-09-10T07:51:13.828075+0000","last_became_active":"2024-09-10T07:51:11.652391+0000","last_became_peered":"2024-09-10T07:51:11.652391+0000","last_unstale":"2024-09-10T07:51:13.828075+0000","last_undegraded":"2024-09-10T07:51:13.828075+0000","last_fullsized":"2024-09-10T07:51:13.828075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:51.242641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655492+0000","last_change":"2024-09-10T07:51:12.653138+0000","last_active":"2024-09-10T07:51:13.655492+0000","last_peered":"2024-09-10T07:51:13.655492+0000","last_clean":"2024-09-10T07:51:13.655492+0000","last_became_active":"2024-09-10T07:51:12.652876+0000","last_became_peered":"2024-09-10T07:51:12.652876+0000","last_unstale":"2024-09-10T07:51:13.655492+0000","last_undegraded":"2024-09-10T07:51:13.655492+0000","last_fullsized":"2024-09-10T07:51:13.655492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:39.429271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827092+0000","last_change":"2024-09-10T07:51:11.644261+0000","last_active":"2024-09-10T07:51:13.827092+0000","last_peered":"2024-09-10T07:51:13.827092+0000","last_clean":"2024-09-10T07:51:13.827092+0000","last_became_active":"2024-09-10T07:51:11.643967+0000","last_became_peered":"2024-09-10T07:51:11.643967+0000","last_unstale":"2024-09-10T07:51:13.827092+0000","last_undegraded":"2024-09-10T07:51:13.827092+0000","last_fullsized":"2024-09-10T07:51:13.827092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:10.313315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655294+0000","last_change":"2024-09-10T07:51:12.647492+0000","last_active":"2024-09-10T07:51:13.655294+0000","last_peered":"2024-09-10T07:51:13.655294+0000","last_clean":"2024-09-10T07:51:13.655294+0000","last_became_active":"2024-09-10T07:51:12.647153+0000","last_became_peered":"2024-09-10T07:51:12.647153+0000","last_unstale":"2024-09-10T07:51:13.655294+0000","last_undegraded":"2024-09-10T07:51:13.655294+0000","last_fullsized":"2024-09-10T07:51:13.655294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:53.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826984+0000","last_change":"2024-09-10T07:51:11.659869+0000","last_active":"2024-09-10T07:51:13.826984+0000","last_peered":"2024-09-10T07:51:13.826984+0000","last_clean":"2024-09-10T07:51:13.826984+0000","last_became_active":"2024-09-10T07:51:11.659392+0000","last_became_peered":"2024-09-10T07:51:11.659392+0000","last_unstale":"2024-09-10T07:51:13.826984+0000","last_undegraded":"2024-09-10T07:51:13.826984+0000","last_fullsized":"2024-09-10T07:51:13.826984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:38.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827815+0000","last_change":"2024-09-10T07:51:12.655133+0000","last_active":"2024-09-10T07:51:13.827815+0000","last_peered":"2024-09-10T07:51:13.827815+0000","last_clean":"2024-09-10T07:51:13.827815+0000","last_became_active":"2024-09-10T07:51:12.654855+0000","last_became_peered":"2024-09-10T07:51:12.654855+0000","last_unstale":"2024-09-10T07:51:13.827815+0000","last_undegraded":"2024-09-10T07:51:13.827815+0000","last_fullsized":"2024-09-10T07:51:13.827815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:07.243443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826722+0000","last_change":"2024-09-10T07:51:11.664218+0000","last_active":"2024-09-10T07:51:13.826722+0000","last_peered":"2024-09-10T07:51:13.826722+0000","last_clean":"2024-09-10T07:51:13.826722+0000","last_became_active":"2024-09-10T07:51:11.663837+0000","last_became_peered":"2024-09-10T07:51:11.663837+0000","last_unstale":"2024-09-10T07:51:13.826722+0000","last_undegraded":"2024-09-10T07:51:13.826722+0000","last_fullsized":"2024-09-10T07:51:13.826722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:43.311010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827577+0000","last_change":"2024-09-10T07:51:12.658871+0000","last_active":"2024-09-10T07:51:13.827577+0000","last_peered":"2024-09-10T07:51:13.827577+0000","last_clean":"2024-09-10T07:51:13.827577+0000","last_became_active":"2024-09-10T07:51:12.658601+0000","last_became_peered":"2024-09-10T07:51:12.658601+0000","last_unstale":"2024-09-10T07:51:13.827577+0000","last_undegraded":"2024-09-10T07:51:13.827577+0000","last_fullsized":"2024-09-10T07:51:13.827577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:02.300806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828721+0000","last_change":"2024-09-10T07:51:11.658990+0000","last_active":"2024-09-10T07:51:13.828721+0000","last_peered":"2024-09-10T07:51:13.828721+0000","last_clean":"2024-09-10T07:51:13.828721+0000","last_became_active":"2024-09-10T07:51:11.658451+0000","last_became_peered":"2024-09-10T07:51:11.658451+0000","last_unstale":"2024-09-10T07:51:13.828721+0000","last_undegraded":"2024-09-10T07:51:13.828721+0000","last_fullsized":"2024-09-10T07:51:13.828721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.242911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654799+0000","last_change":"2024-09-10T07:51:12.663182+0000","last_active":"2024-09-10T07:51:13.654799+0000","last_peered":"2024-09-10T07:51:13.654799+0000","last_clean":"2024-09-10T07:51:13.654799+0000","last_became_active":"2024-09-10T07:51:12.662787+0000","last_became_peered":"2024-09-10T07:51:12.662787+0000","last_unstale":"2024-09-10T07:51:13.654799+0000","last_undegraded":"2024-09-10T07:51:13.654799+0000","last_fullsized":"2024-09-10T07:51:13.654799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:10:37.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827966+0000","last_change":"2024-09-10T07:51:11.655978+0000","last_active":"2024-09-10T07:51:13.827966+0000","last_peered":"2024-09-10T07:51:13.827966+0000","last_clean":"2024-09-10T07:51:13.827966+0000","last_became_active":"2024-09-10T07:51:11.655686+0000","last_became_peered":"2024-09-10T07:51:11.655686+0000","last_unstale":"2024-09-10T07:51:13.827966+0000","last_undegraded":"2024-09-10T07:51:13.827966+0000","last_fullsized":"2024-09-10T07:51:13.827966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:14.533246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828044+0000","last_change":"2024-09-10T07:51:12.665991+0000","last_active":"2024-09-10T07:51:13.828044+0000","last_peered":"2024-09-10T07:51:13.828044+0000","last_clean":"2024-09-10T07:51:13.828044+0000","last_became_active":"2024-09-10T07:51:12.665689+0000","last_became_peered":"2024-09-10T07:51:12.665689+0000","last_unstale":"2024-09-10T07:51:13.828044+0000","last_undegraded":"2024-09-10T07:51:13.828044+0000","last_fullsized":"2024-09-10T07:51:13.828044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:39.884571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.828124+0000","last_change":"2024-09-10T07:51:11.653391+0000","last_active":"2024-09-10T07:51:13.828124+0000","last_peered":"2024-09-10T07:51:13.828124+0000","last_clean":"2024-09-10T07:51:13.828124+0000","last_became_active":"2024-09-10T07:51:11.652803+0000","last_became_peered":"2024-09-10T07:51:11.652803+0000","last_unstale":"2024-09-10T07:51:13.828124+0000","last_undegraded":"2024-09-10T07:51:13.828124+0000","last_fullsized":"2024-09-10T07:51:13.828124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:38.179483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654603+0000","last_change":"2024-09-10T07:51:12.676013+0000","last_active":"2024-09-10T07:51:13.654603+0000","last_peered":"2024-09-10T07:51:13.654603+0000","last_clean":"2024-09-10T07:51:13.654603+0000","last_became_active":"2024-09-10T07:51:12.675718+0000","last_became_peered":"2024-09-10T07:51:12.675718+0000","last_unstale":"2024-09-10T07:51:13.654603+0000","last_undegraded":"2024-09-10T07:51:13.654603+0000","last_fullsized":"2024-09-10T07:51:13.654603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:40.699194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827578+0000","last_change":"2024-09-10T07:51:11.653518+0000","last_active":"2024-09-10T07:51:13.827578+0000","last_peered":"2024-09-10T07:51:13.827578+0000","last_clean":"2024-09-10T07:51:13.827578+0000","last_became_active":"2024-09-10T07:51:11.653198+0000","last_became_peered":"2024-09-10T07:51:11.653198+0000","last_unstale":"2024-09-10T07:51:13.827578+0000","last_undegraded":"2024-09-10T07:51:13.827578+0000","last_fullsized":"2024-09-10T07:51:13.827578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.482793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827696+0000","last_change":"2024-09-10T07:51:12.657827+0000","last_active":"2024-09-10T07:51:13.827696+0000","last_peered":"2024-09-10T07:51:13.827696+0000","last_clean":"2024-09-10T07:51:13.827696+0000","last_became_active":"2024-09-10T07:51:12.657483+0000","last_became_peered":"2024-09-10T07:51:12.657483+0000","last_unstale":"2024-09-10T07:51:13.827696+0000","last_undegraded":"2024-09-10T07:51:13.827696+0000","last_fullsized":"2024-09-10T07:51:13.827696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.217868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827942+0000","last_change":"2024-09-10T07:51:11.653782+0000","last_active":"2024-09-10T07:51:13.827942+0000","last_peered":"2024-09-10T07:51:13.827942+0000","last_clean":"2024-09-10T07:51:13.827942+0000","last_became_active":"2024-09-10T07:51:11.650110+0000","last_became_peered":"2024-09-10T07:51:11.650110+0000","last_unstale":"2024-09-10T07:51:13.827942+0000","last_undegraded":"2024-09-10T07:51:13.827942+0000","last_fullsized":"2024-09-10T07:51:13.827942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:16.299195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827512+0000","last_change":"2024-09-10T07:51:12.677759+0000","last_active":"2024-09-10T07:51:13.827512+0000","last_peered":"2024-09-10T07:51:13.827512+0000","last_clean":"2024-09-10T07:51:13.827512+0000","last_became_active":"2024-09-10T07:51:12.677359+0000","last_became_peered":"2024-09-10T07:51:12.677359+0000","last_unstale":"2024-09-10T07:51:13.827512+0000","last_undegraded":"2024-09-10T07:51:13.827512+0000","last_fullsized":"2024-09-10T07:51:13.827512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:25.109899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827404+0000","last_change":"2024-09-10T07:51:11.662912+0000","last_active":"2024-09-10T07:51:13.827404+0000","last_peered":"2024-09-10T07:51:13.827404+0000","last_clean":"2024-09-10T07:51:13.827404+0000","last_became_active":"2024-09-10T07:51:11.662677+0000","last_became_peered":"2024-09-10T07:51:11.662677+0000","last_unstale":"2024-09-10T07:51:13.827404+0000","last_undegraded":"2024-09-10T07:51:13.827404+0000","last_fullsized":"2024-09-10T07:51:13.827404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:18.751797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.655470+0000","last_change":"2024-09-10T07:51:12.672712+0000","last_active":"2024-09-10T07:51:13.655470+0000","last_peered":"2024-09-10T07:51:13.655470+0000","last_clean":"2024-09-10T07:51:13.655470+0000","last_became_active":"2024-09-10T07:51:12.670900+0000","last_became_peered":"2024-09-10T07:51:12.670900+0000","last_unstale":"2024-09-10T07:51:13.655470+0000","last_undegraded":"2024-09-10T07:51:13.655470+0000","last_fullsized":"2024-09-10T07:51:13.655470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:14.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827078+0000","last_change":"2024-09-10T07:51:11.655026+0000","last_active":"2024-09-10T07:51:13.827078+0000","last_peered":"2024-09-10T07:51:13.827078+0000","last_clean":"2024-09-10T07:51:13.827078+0000","last_became_active":"2024-09-10T07:51:11.654747+0000","last_became_peered":"2024-09-10T07:51:11.654747+0000","last_unstale":"2024-09-10T07:51:13.827078+0000","last_undegraded":"2024-09-10T07:51:13.827078+0000","last_fullsized":"2024-09-10T07:51:13.827078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:13.073312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826895+0000","last_change":"2024-09-10T07:51:12.680995+0000","last_active":"2024-09-10T07:51:13.826895+0000","last_peered":"2024-09-10T07:51:13.826895+0000","last_clean":"2024-09-10T07:51:13.826895+0000","last_became_active":"2024-09-10T07:51:12.680592+0000","last_became_peered":"2024-09-10T07:51:12.680592+0000","last_unstale":"2024-09-10T07:51:13.826895+0000","last_undegraded":"2024-09-10T07:51:13.826895+0000","last_fullsized":"2024-09-10T07:51:13.826895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:43.727455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.826629+0000","last_change":"2024-09-10T07:51:11.659975+0000","last_active":"2024-09-10T07:51:13.826629+0000","last_peered":"2024-09-10T07:51:13.826629+0000","last_clean":"2024-09-10T07:51:13.826629+0000","last_became_active":"2024-09-10T07:51:11.657387+0000","last_became_peered":"2024-09-10T07:51:11.657387+0000","last_unstale":"2024-09-10T07:51:13.826629+0000","last_undegraded":"2024-09-10T07:51:13.826629+0000","last_fullsized":"2024-09-10T07:51:13.826629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:07.331178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.827325+0000","last_change":"2024-09-10T07:51:12.675741+0000","last_active":"2024-09-10T07:51:13.827325+0000","last_peered":"2024-09-10T07:51:13.827325+0000","last_clean":"2024-09-10T07:51:13.827325+0000","last_became_active":"2024-09-10T07:51:12.675054+0000","last_became_peered":"2024-09-10T07:51:12.675054+0000","last_unstale":"2024-09-10T07:51:13.827325+0000","last_undegraded":"2024-09-10T07:51:13.827325+0000","last_fullsized":"2024-09-10T07:51:13.827325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:11.628863+0000","last_clean_scrub_stamp":"2024-09-10T07:51:11.628863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:46.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"77'2","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654538+0000","last_change":"2024-09-10T07:51:11.660202+0000","last_active":"2024-09-10T07:51:13.654538+0000","last_peered":"2024-09-10T07:51:13.654538+0000","last_clean":"2024-09-10T07:51:13.654538+0000","last_became_active":"2024-09-10T07:51:11.659089+0000","last_became_peered":"2024-09-10T07:51:11.659089+0000","last_unstale":"2024-09-10T07:51:13.654538+0000","last_undegraded":"2024-09-10T07:51:13.654538+0000","last_fullsized":"2024-09-10T07:51:13.654538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:10.621492+0000","last_clean_scrub_stamp":"2024-09-10T07:51:10.621492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:34.657006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":273,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T07:51:13.654859+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:51:13.654859+0000","last_peered":"2024-09-10T07:51:13.654859+0000","last_clean":"2024-09-10T07:51:13.654859+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:51:13.654859+0000","last_undegraded":"2024-09-10T07:51:13.654859+0000","last_fullsized":"2024-09-10T07:51:13.654859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476799,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":402248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1531904,"data_stored":992490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":402192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":403734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":403768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1536000,"data_stored":992580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":950272,"data_stored":402816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:51:21.952 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:51:21.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:51:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.151+0000 7f72b3000640 1 Processor -- start 2024-09-10T07:51:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.151+0000 7f72b3000640 1 -- start start 2024-09-10T07:51:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72ac158950 0x7f72ac0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72ac0b6730 0x7f72ac0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0d1490 0x7f72ac162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f72ac0d3a70 con 0x7f72ac0b6730 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f72ac0b83a0 con 0x7f72ac158950 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.152+0000 7f72b3000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac0d1cc0 con 0x7f72ac0d1490 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72ac0b6730 0x7f72ac0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72ac0b6730 0x7f72ac0d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55528/0 (socket says 172.21.15.3:55528) 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 -- 172.21.15.3:0/2531137610 learned_addr learned my addr 172.21.15.3:0/2531137610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1c00640 1 --2- 172.21.15.3:0/2531137610 >> v2:172.21.15.70:3300/0 conn(0x7f72ac158950 0x7f72ac0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b2600640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0d1490 0x7f72ac162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:51:22.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 -- 172.21.15.3:0/2531137610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0d1490 msgr2=0x7f72ac162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0d1490 0x7f72ac162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 -- 172.21.15.3:0/2531137610 >> v2:172.21.15.70:3300/0 conn(0x7f72ac158950 msgr2=0x7f72ac0b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 --2- 172.21.15.3:0/2531137610 >> v2:172.21.15.70:3300/0 conn(0x7f72ac158950 0x7f72ac0b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.153+0000 7f72b1200640 1 -- 172.21.15.3:0/2531137610 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac0d1fa0 con 0x7f72ac0b6730 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.154+0000 7f72b1200640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac0b6730 0x7f72ac0d0f50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0d56b0 tx=0x7f729802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65330ca6603c087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.154+0000 7f72abe00640 1 -- 172.21.15.3:0/2531137610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298037070 con 0x7f72ac0b6730 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72abe00640 1 -- 172.21.15.3:0/2531137610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7298032440 con 0x7f72ac0b6730 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72abe00640 1 -- 172.21.15.3:0/2531137610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298032810 con 0x7f72ac0b6730 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 -- 172.21.15.3:0/2531137610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac0b6730 msgr2=0x7f72ac0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac0b6730 0x7f72ac0d0f50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0d56b0 tx=0x7f729802a140 comp rx=0 tx=0).stop 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 -- 172.21.15.3:0/2531137610 shutdown_connections 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0d1490 0x7f72ac162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 --2- 172.21.15.3:0/2531137610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac0b6730 0x7f72ac0d0f50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 --2- 172.21.15.3:0/2531137610 >> v2:172.21.15.70:3300/0 conn(0x7f72ac158950 0x7f72ac0b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 -- 172.21.15.3:0/2531137610 >> 172.21.15.3:0/2531137610 conn(0x7f72ac062810 msgr2=0x7f72ac072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 -- 172.21.15.3:0/2531137610 shutdown_connections 2024-09-10T07:51:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.155+0000 7f72b3000640 1 -- 172.21.15.3:0/2531137610 wait complete. 2024-09-10T07:51:22.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.157+0000 7f72b3000640 1 Processor -- start 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.157+0000 7f72b3000640 1 -- start start 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0b6730 0x7f72ac0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac0d1490 0x7f72ac14c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac0d18c0 con 0x7f72ac158950 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac0d2560 con 0x7f72ac0d1490 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b3000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac0d1a40 con 0x7f72ac0b6730 2024-09-10T07:51:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55536/0 (socket says 172.21.15.3:55536) 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b2600640 1 -- 172.21.15.3:0/3991466954 learned_addr learned my addr 172.21.15.3:0/3991466954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b1c00640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0b6730 0x7f72ac0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.158+0000 7f72b1200640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac0d1490 0x7f72ac14c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 -- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0b6730 msgr2=0x7f72ac0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0b6730 0x7f72ac0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 -- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac0d1490 msgr2=0x7f72ac14c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac0d1490 0x7f72ac14c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 -- 172.21.15.3:0/3991466954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac14d0b0 con 0x7f72ac158950 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b2600640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f72a400efc0 tx=0x7f72a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4018020 con 0x7f72ac158950 2024-09-10T07:51:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a4014480 con 0x7f72ac158950 2024-09-10T07:51:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4014870 con 0x7f72ac158950 2024-09-10T07:51:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac14d390 con 0x7f72ac158950 2024-09-10T07:51:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.159+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac14d950 con 0x7f72ac158950 2024-09-10T07:51:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.161+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac158d30 con 0x7f72ac158950 2024-09-10T07:51:22.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.161+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72a4005d10 con 0x7f72ac158950 2024-09-10T07:51:22.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.162+0000 7f72a9600640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 0x7f728c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.162+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f72a408b940 con 0x7f72ac158950 2024-09-10T07:51:22.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.163+0000 7f72b1c00640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 0x7f728c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.163+0000 7f72b1c00640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 0x7f728c06e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009a50 tx=0x7f72a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.167+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72a4022070 con 0x7f72ac158950 2024-09-10T07:51:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.425+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f72ac0d4cc0 con 0x7f72ac158950 2024-09-10T07:51:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.428+0000 7f72a9600640 1 -- 172.21.15.3:0/3991466954 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f72a4010040 con 0x7f72ac158950 2024-09-10T07:51:22.427 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:51:22.427 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:51:22.428 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:51:22.428 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:51:22.428 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:51:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 msgr2=0x7f728c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 0x7f728c06e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009a50 tx=0x7f72a0008040 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 msgr2=0x7f72ac14c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f72a400efc0 tx=0x7f72a400c490 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 shutdown_connections 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f728c06be30 0x7f728c06e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ac158950 0x7f72ac14c1b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac0d1490 0x7f72ac14c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 --2- 172.21.15.3:0/3991466954 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f72ac0b6730 0x7f72ac0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 >> 172.21.15.3:0/3991466954 conn(0x7f72ac062810 msgr2=0x7f72ac15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 shutdown_connections 2024-09-10T07:51:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.430+0000 7f72b3000640 1 -- 172.21.15.3:0/3991466954 wait complete. 2024-09-10T07:51:22.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.601+0000 7f2f7e800640 1 Processor -- start 2024-09-10T07:51:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.601+0000 7f2f7e800640 1 -- start start 2024-09-10T07:51:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7e800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f780b2710 0x7f2f780cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f780b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7e800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7815bac0 con 0x7f2f7815b330 2024-09-10T07:51:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7e800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7815bd30 con 0x7f2f780b2710 2024-09-10T07:51:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7e800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7815c010 con 0x7f2f780cfb00 2024-09-10T07:51:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7d400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f7815b330 0x7f2f780b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35146/0 (socket says 172.21.15.3:35146) 2024-09-10T07:51:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7d400640 1 -- 172.21.15.3:0/1970253781 learned_addr learned my addr 172.21.15.3:0/1970253781 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7de00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f780b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/1970253781 >> v2:172.21.15.70:3300/0 conn(0x7f2f780b2710 0x7f2f780cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853088634 0 0) 0x7f2f7815bac0 con 0x7f2f7815b330 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.603+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f60003520 con 0x7f2f7815b330 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 418281428 0 0) 0x7f2f60003520 con 0x7f2f7815b330 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 msgr2=0x7f2f780b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f780b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 >> v2:172.21.15.70:3300/0 conn(0x7f2f780b2710 msgr2=0x7f2f780cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 --2- 172.21.15.3:0/1970253781 >> v2:172.21.15.70:3300/0 conn(0x7f2f780b2710 0x7f2f780cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7815c2f0 con 0x7f2f7815b330 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f2f6c001670 con 0x7f2f7815b330 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 >> v1:172.21.15.3:6789/0 conn(0x7f2f7815b330 legacy=0x7f2f780b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f60003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fa0 0x7f2f60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f60004d40 0x7f2f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815bd30 con 0x7f2f60003fa0 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815c010 con 0x7f2f60004d40 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6000dc00 con 0x7f2f780cfb00 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fa0 0x7f2f60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f7de00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f60004d40 0x7f2f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f7d400640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f60003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f7ca00640 1 -- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 msgr2=0x7f2f60003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.604+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f60003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7ca00640 1 -- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f60004d40 msgr2=0x7f2f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f60004d40 0x7f2f600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7ca00640 1 -- 172.21.15.3:0/1970253781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f60003520 con 0x7f2f60003fa0 2024-09-10T07:51:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7d400640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f780cfb00 0x7f2f60003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7de00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f60004d40 0x7f2f600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.605+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fa0 0x7f2f60004800 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ac0 tx=0x7f2f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8def4ff27d4ad3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036070 con 0x7f2f60003fa0 2024-09-10T07:51:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f64031400 con 0x7f2f60003fa0 2024-09-10T07:51:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f77e00640 1 -- 172.21.15.3:0/1970253781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64009690 con 0x7f2f60003fa0 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 -- 172.21.15.3:0/1970253781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fa0 msgr2=0x7f2f60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 --2- 172.21.15.3:0/1970253781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f60003fa0 0x7f2f60004800 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ac0 tx=0x7f2f60005830 comp rx=0 tx=0).stop 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7de00640 1 -- 172.21.15.3:0/1970253781 reap_dead start 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 -- 172.21.15.3:0/1970253781 shutdown_connections 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 -- 172.21.15.3:0/1970253781 >> 172.21.15.3:0/1970253781 conn(0x7f2f7806bb20 msgr2=0x7f2f7806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 -- 172.21.15.3:0/1970253781 shutdown_connections 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.606+0000 7f2f7e800640 1 -- 172.21.15.3:0/1970253781 wait complete. 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.608+0000 7f2f7e800640 1 Processor -- start 2024-09-10T07:51:22.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.608+0000 7f2f7e800640 1 -- start start 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f60003fa0 0x7f2f781f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f7814f8c0 0x7f2f78155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815b710 con 0x7f2f780b1410 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780ce1c0 con 0x7f2f7814f8c0 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.609+0000 7f2f7e800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7814fd40 con 0x7f2f60003fa0 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55560/0 (socket says 172.21.15.3:55560) 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 -- 172.21.15.3:0/3948613815 learned_addr learned my addr 172.21.15.3:0/3948613815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7ca00640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f7814f8c0 0x7f2f78155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7d400640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f60003fa0 0x7f2f781f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 -- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f60003fa0 msgr2=0x7f2f781f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f60003fa0 0x7f2f781f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 -- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f7814f8c0 msgr2=0x7f2f78155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f7814f8c0 0x7f2f78155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 -- 172.21.15.3:0/3948613815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78155e50 con 0x7f2f780b1410 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.610+0000 7f2f7de00640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b2e50 tx=0x7f2f70010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.611+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7001a030 con 0x7f2f780b1410 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.611+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70010d80 con 0x7f2f780b1410 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.611+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f78155b70 con 0x7f2f780b1410 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.611+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007ea0 con 0x7f2f780b1410 2024-09-10T07:51:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.611+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f78150510 con 0x7f2f780b1410 2024-09-10T07:51:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.612+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f78063610 con 0x7f2f780b1410 2024-09-10T07:51:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.613+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f7001d070 con 0x7f2f780b1410 2024-09-10T07:51:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.613+0000 7f2f75600640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 0x7f2f5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.614+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7008b1e0 con 0x7f2f780b1410 2024-09-10T07:51:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.614+0000 7f2f7d400640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 0x7f2f5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.614+0000 7f2f7d400640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 0x7f2f5806e320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005bf0 tx=0x7f2f6c007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.618+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f70059c20 con 0x7f2f780b1410 2024-09-10T07:51:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.839+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f7814fec0 con 0x7f2f780b1410 2024-09-10T07:51:22.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.840+0000 7f2f75600640 1 -- 172.21.15.3:0/3948613815 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5219 (secure 0 0 0) 0x7f2f70059520 con 0x7f2f780b1410 2024-09-10T07:51:22.840 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:22.840 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"btime":"2024-09-10T07:51:15:660073+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6355,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:15.081436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702]},"id":5}]} 2024-09-10T07:51:22.840 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2024-09-10T07:51:22.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 msgr2=0x7f2f5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 0x7f2f5806e320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005bf0 tx=0x7f2f6c007040 comp rx=0 tx=0).stop 2024-09-10T07:51:22.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 msgr2=0x7f2f780b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:22.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b2e50 tx=0x7f2f70010590 comp rx=0 tx=0).stop 2024-09-10T07:51:22.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 shutdown_connections 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f5806be60 0x7f2f5806e320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f780b1410 0x7f2f780b1890 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f7814f8c0 0x7f2f78155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 --2- 172.21.15.3:0/3948613815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f60003fa0 0x7f2f781f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 >> 172.21.15.3:0/3948613815 conn(0x7f2f7806bb20 msgr2=0x7f2f780ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 shutdown_connections 2024-09-10T07:51:22.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:22.842+0000 7f2f7e800640 1 -- 172.21.15.3:0/3948613815 wait complete. 2024-09-10T07:51:23.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:51:12.071594+0000', 'modified': '2024-09-10T07:51:15.081436+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6702}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6702': {'gid': 6702, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2373315703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6836', 'nonce': 2373315703}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2373315703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6702, 'qdb_cluster': [6702]} 2024-09-10T07:51:23.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:51:23.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:51:23.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.420+0000 7f77e6800640 1 Processor -- start 2024-09-10T07:51:23.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.421+0000 7f77e6800640 1 -- start start 2024-09-10T07:51:23.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77e6800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77e0161b30 0x7f77e015f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77e6800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77e6800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e015fbb0 con 0x7f77e01601f0 2024-09-10T07:51:23.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77e6800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f77e00b36a0 con 0x7f77e0161b30 2024-09-10T07:51:23.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77e6800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f77e00b3980 con 0x7f77e015fdb0 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77dfe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f77e01601f0 0x7f77e00b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35160/0 (socket says 172.21.15.3:35160) 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.422+0000 7f77dfe00640 1 -- 172.21.15.3:0/490267472 learned_addr learned my addr 172.21.15.3:0/490267472 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77d7e00640 1 --2- 172.21.15.3:0/490267472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77df400640 1 --2- 172.21.15.3:0/490267472 >> v2:172.21.15.70:3300/0 conn(0x7f77e0161b30 0x7f77e015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731632318 0 0) 0x7f77d8001490 con 0x7f77e01601f0 2024-09-10T07:51:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c4003520 con 0x7f77e01601f0 2024-09-10T07:51:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1161191938 0 0) 0x7f77c4003520 con 0x7f77e01601f0 2024-09-10T07:51:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77d7e00640 1 -- 172.21.15.3:0/490267472 >> v2:172.21.15.70:3300/0 conn(0x7f77e0161b30 msgr2=0x7f77e015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77d7e00640 1 --2- 172.21.15.3:0/490267472 >> v2:172.21.15.70:3300/0 conn(0x7f77e0161b30 0x7f77e015f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77d7e00640 1 -- 172.21.15.3:0/490267472 >> v1:172.21.15.3:6789/0 conn(0x7f77e01601f0 legacy=0x7f77e00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.423+0000 7f77d7e00640 1 -- 172.21.15.3:0/490267472 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00b3c60 con 0x7f77e015fdb0 2024-09-10T07:51:23.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.424+0000 7f77d7e00640 1 --2- 172.21.15.3:0/490267472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e006bb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1f02796b3b4011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.424+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77e015fdb0 2024-09-10T07:51:23.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.424+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c802aba0 con 0x7f77e015fdb0 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.424+0000 7f77dea00640 1 -- 172.21.15.3:0/490267472 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8001bc0 con 0x7f77e015fdb0 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 -- 172.21.15.3:0/490267472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 msgr2=0x7f77e006bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 --2- 172.21.15.3:0/490267472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e006bb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802a110 comp rx=0 tx=0).stop 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 -- 172.21.15.3:0/490267472 shutdown_connections 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 --2- 172.21.15.3:0/490267472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e006bb20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 --2- 172.21.15.3:0/490267472 >> v2:172.21.15.70:3300/0 conn(0x7f77e0161b30 0x7f77e015f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 -- 172.21.15.3:0/490267472 >> 172.21.15.3:0/490267472 conn(0x7f77e006d000 msgr2=0x7f77e006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 -- 172.21.15.3:0/490267472 shutdown_connections 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.425+0000 7f77e6800640 1 -- 172.21.15.3:0/490267472 wait complete. 2024-09-10T07:51:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.427+0000 7f77e6800640 1 Processor -- start 2024-09-10T07:51:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.427+0000 7f77e6800640 1 -- start start 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e01592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 0x7f77e0150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0161f10 con 0x7f77e0161b30 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00b4220 con 0x7f77e01601f0 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77e6800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f77e00aefa0 con 0x7f77e015fdb0 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55570/0 (socket says 172.21.15.3:55570) 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 -- 172.21.15.3:0/2977913672 learned_addr learned my addr 172.21.15.3:0/2977913672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77df400640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e01592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 0x7f77e0150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 -- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 msgr2=0x7f77e01592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e01592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 -- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 msgr2=0x7f77e0150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 0x7f77e0150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77dfe00640 1 -- 172.21.15.3:0/2977913672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e015a680 con 0x7f77e0161b30 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.428+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 0x7f77e0150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77dfe00640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001d30 tx=0x7f77d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77d7400640 1 -- 172.21.15.3:0/2977913672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8017070 con 0x7f77e0161b30 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77d7400640 1 -- 172.21.15.3:0/2977913672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8012400 con 0x7f77e0161b30 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77d7400640 1 -- 172.21.15.3:0/2977913672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8012830 con 0x7f77e0161b30 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e0159b00 con 0x7f77e0161b30 2024-09-10T07:51:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.429+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e015a0c0 con 0x7f77e0161b30 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.431+0000 7f77d7400640 1 -- 172.21.15.3:0/2977913672 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77d801a070 con 0x7f77e0161b30 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.432+0000 7f77d7400640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.432+0000 7f77d7400640 1 -- 172.21.15.3:0/2977913672 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f77d808adf0 con 0x7f77e0161b30 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.432+0000 7f77df400640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.433+0000 7f77e986e740 1 Processor -- start 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.433+0000 7f77e986e740 1 -- start start 2024-09-10T07:51:23.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.433+0000 7f77df400640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 0x7f77bc06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f77e0076300 tx=0x7f77d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81859490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818114b0 0x55fd818140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fd817723b0 con 0x55fd81815480 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fd81777410 con 0x55fd818114b0 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fd8171f620 con 0x55fd81813cf0 2024-09-10T07:51:23.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55576/0 (socket says 172.21.15.3:55576) 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 -- 172.21.15.3:0/1418423179 learned_addr learned my addr 172.21.15.3:0/1418423179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77df400640 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81859490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77d7e00640 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818114b0 0x55fd818140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 -- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 msgr2=0x55fd81859490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81859490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 -- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818114b0 msgr2=0x55fd818140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818114b0 0x55fd818140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.434+0000 7f77dfe00640 1 -- 172.21.15.3:0/1418423179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8148a2d0 con 0x55fd81815480 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77dfe00640 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f77d8089340 tx=0x7f77d80598e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd1261370f35719 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77cfe00640 1 -- 172.21.15.3:0/1418423179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8061c80 con 0x55fd81815480 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77cfe00640 1 -- 172.21.15.3:0/1418423179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d80573a0 con 0x55fd81815480 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77cfe00640 1 -- 172.21.15.3:0/1418423179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8055710 con 0x55fd81815480 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77e986e740 1 -- 172.21.15.3:0/1418423179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 msgr2=0x55fd8180e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.435+0000 7f77e986e740 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f77d8089340 tx=0x7f77d80598e0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 -- 172.21.15.3:0/1418423179 shutdown_connections 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81815480 0x55fd8180e570 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818114b0 0x55fd818140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 --2- 172.21.15.3:0/1418423179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81859490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 -- 172.21.15.3:0/1418423179 >> 172.21.15.3:0/1418423179 conn(0x55fd81810740 msgr2=0x55fd81858960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 -- 172.21.15.3:0/1418423179 shutdown_connections 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 -- 172.21.15.3:0/1418423179 wait complete. 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.436+0000 7f77e986e740 1 -- start start 2024-09-10T07:51:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.437+0000 7f77e986e740 1 Processor -- start 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.437+0000 7f77e986e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.437+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd818134e0 0x55fd8186b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77e986e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fd81732780 con 0x55fd81813cf0 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fd817733d0 con 0x55fd81815480 2024-09-10T07:51:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fd8180fbe0 con 0x55fd818134e0 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77e986e740 10 client.0 _ll_register_callbacks cb 0x55fd817705d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55590/0 (socket says 172.21.15.3:55590) 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 -- 172.21.15.3:0/2358958122 learned_addr learned my addr 172.21.15.3:0/2358958122 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77dfe00640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd818134e0 0x55fd8186b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 -- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd818134e0 msgr2=0x55fd8186b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd818134e0 0x55fd8186b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 -- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 msgr2=0x55fd81812fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 -- 172.21.15.3:0/2358958122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd815546b0 con 0x55fd81813cf0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77df400640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f77d00156f0 tx=0x7f77d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0be57757dca318b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.438+0000 7f77ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0014620 con 0x55fd81813cf0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d80573a0 con 0x55fd81813cf0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f77d00147c0 con 0x55fd81813cf0 2024-09-10T07:51:23.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f77d0032050 con 0x55fd81813cf0 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 10 client.6834 fetch_fsmap learned FSMap version 69 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55fd8183fb70 con 0x55fd81813cf0 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d80573a0 con 0x55fd81813cf0 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f77d00149c0 con 0x55fd81813cf0 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 10 client.6834 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 10 client.6834 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:51:23.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 20 client.6834 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.439+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd8183bcc0 0x55fd81869350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.440+0000 7f77e986e740 4 client.6834 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.440+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd81784b40 con 0x55fd8183bcc0 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.440+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd8183bcc0 0x55fd81869350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.440+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd8183bcc0 0x55fd81869350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcea01eedaa4ab0e server_cookie=87ff87c1125757d5 in_seq=0 out_seq=0 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.440+0000 7f77ccc00640 10 client.6834 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77ccc00640 1 -- 172.21.15.3:0/2358958122 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd81784b40 con 0x55fd8183bcc0 2024-09-10T07:51:23.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77ccc00640 10 client.6834 handle_command_reply: tid=0 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 client.6834 shutdown 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 msgr2=0x55fd8183df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f77d00156f0 tx=0x7f77d00046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 shutdown_connections 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd8183bcc0 0x55fd81869350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd818134e0 0x55fd8186b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77ccc00640 0 client.6834 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 --2- 172.21.15.3:0/2358958122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd8183df60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.441+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 >> 172.21.15.3:0/2358958122 conn(0x55fd818114b0 msgr2=0x55fd81857920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.442+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 shutdown_connections 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.442+0000 7f77e986e740 1 -- 172.21.15.3:0/2358958122 wait complete. 2024-09-10T07:51:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.442+0000 7f77e986e740 20 client.6834 trim_cache size 0 max 16384 2024-09-10T07:51:23.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 Processor -- start 2024-09-10T07:51:23.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 -- start start 2024-09-10T07:51:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81855e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0157810 con 0x55fd818134e0 2024-09-10T07:51:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00719b0 con 0x55fd81815480 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.461+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f77e01571c0 con 0x55fd81813cf0 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55594/0 (socket says 172.21.15.3:55594) 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 -- 172.21.15.3:0/2807168843 learned_addr learned my addr 172.21.15.3:0/2807168843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81855e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77dfe00640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 -- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 msgr2=0x55fd81855e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81855e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 -- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 msgr2=0x55fd81812110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 -- 172.21.15.3:0/2807168843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd81815860 con 0x55fd818134e0 2024-09-10T07:51:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77d7e00640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81855e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.462+0000 7f77df400640 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009350 tx=0x7f77d0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4463c92354c659dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77cf400640 1 -- 172.21.15.3:0/2807168843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0014620 con 0x55fd818134e0 2024-09-10T07:51:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77cf400640 1 -- 172.21.15.3:0/2807168843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d001a030 con 0x55fd818134e0 2024-09-10T07:51:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77cf400640 1 -- 172.21.15.3:0/2807168843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00149c0 con 0x55fd818134e0 2024-09-10T07:51:23.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 -- 172.21.15.3:0/2807168843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 msgr2=0x55fd81812cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009350 tx=0x7f77d0023000 comp rx=0 tx=0).stop 2024-09-10T07:51:23.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 -- 172.21.15.3:0/2807168843 shutdown_connections 2024-09-10T07:51:23.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81813cf0 0x55fd81855e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 --2- 172.21.15.3:0/2807168843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 -- 172.21.15.3:0/2807168843 >> 172.21.15.3:0/2807168843 conn(0x55fd818114b0 msgr2=0x55fd8183dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 -- 172.21.15.3:0/2807168843 shutdown_connections 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.463+0000 7f77e986e740 1 -- 172.21.15.3:0/2807168843 wait complete. 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.464+0000 7f77e986e740 1 -- start start 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.464+0000 7f77e986e740 1 Processor -- start 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fd8183e630 con 0x55fd81813cf0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55fd81869d70 con 0x55fd818134e0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55fd8183c130 con 0x55fd81815480 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77e986e740 10 client.0 _ll_register_callbacks cb 0x7f77d00098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55596/0 (socket says 172.21.15.3:55596) 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.465+0000 7f77d7e00640 1 -- 172.21.15.3:0/3301201441 learned_addr learned my addr 172.21.15.3:0/3301201441 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77df400640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77dfe00640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 -- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81815480 msgr2=0x55fd81812110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 -- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818134e0 msgr2=0x55fd81812cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 -- 172.21.15.3:0/3301201441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8187b460 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77d7e00640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2589c222582a2da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c804b060 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f77c8032640 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.466+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d001a030 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f77c8037040 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 10 client.6840 fetch_fsmap learned FSMap version 69 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55fd8187b6d0 con 0x55fd81813cf0 2024-09-10T07:51:23.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d001a030 con 0x55fd81813cf0 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f77c8005990 con 0x55fd81813cf0 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 10 client.6840 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 10 client.6840 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 20 client.6840 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd81840a40 0x55fd81839a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 4 client.6840 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "status"}) -- 0x55fd8177ee90 con 0x55fd81840a40 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.467+0000 7f77df400640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd81840a40 0x55fd81839a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77df400640 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd81840a40 0x55fd81839a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fa2ae0d2439d6a server_cookie=1b0c685daafea292 in_seq=0 out_seq=0 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77ccc00640 10 client.6840 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:23.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77ccc00640 1 -- 172.21.15.3:0/3301201441 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fd8177ee90 con 0x55fd81840a40 2024-09-10T07:51:23.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77ccc00640 10 client.6840 handle_command_reply: tid=0 2024-09-10T07:51:23.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 client.6840 shutdown 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 msgr2=0x55fd818c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802c070 comp rx=0 tx=0).stop 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 shutdown_connections 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x55fd81840a40 0x55fd81839a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55fd81815480 0x55fd81812110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fd81813cf0 0x55fd818c0b10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 --2- 172.21.15.3:0/3301201441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fd818134e0 0x55fd81812cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77ccc00640 0 client.6840 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:23.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 >> 172.21.15.3:0/3301201441 conn(0x55fd818114b0 msgr2=0x55fd8185a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 shutdown_connections 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.468+0000 7f77e986e740 1 -- 172.21.15.3:0/3301201441 wait complete. 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e986e740 20 client.6840 trim_cache size 0 max 16384 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6702, 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:51:23.474 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.830604727000001, 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 69, 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 78, 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.932677998000001 2024-09-10T07:51:23.475 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 msgr2=0x7f77bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 0x7f77bc06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f77e0076300 tx=0x7f77d0008040 comp rx=0 tx=0).stop 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 msgr2=0x7f77e00b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001d30 tx=0x7f77d800a430 comp rx=0 tx=0).stop 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 shutdown_connections 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77e0161b30 0x7f77e00b1880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77e01601f0 0x7f77e0150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 --2- 172.21.15.3:0/2977913672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f77e015fdb0 0x7f77e01592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 >> 172.21.15.3:0/2977913672 conn(0x7f77e006d000 msgr2=0x7f77e015d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 shutdown_connections 2024-09-10T07:51:23.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.469+0000 7f77e6800640 1 -- 172.21.15.3:0/2977913672 wait complete. 2024-09-10T07:51:23.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:51:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.643+0000 7f8d74600640 1 Processor -- start 2024-09-10T07:51:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.643+0000 7f8d74600640 1 -- start start 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d74600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d74600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d6c0d07c0 0x7f8d6c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d74600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c0cfdd0 con 0x7f8d6c160af0 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d74600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6c15cc90 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d74600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c15cf70 con 0x7f8d6c160710 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d72600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d6c0d07c0 0x7f8d6c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d71200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d6c160af0 0x7f8d6c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35168/0 (socket says 172.21.15.3:35168) 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d71200640 1 -- 172.21.15.3:0/1258141720 learned_addr learned my addr 172.21.15.3:0/1258141720 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d71c00640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d72600640 1 -- 172.21.15.3:0/1258141720 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 msgr2=0x7f8d6c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d72600640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.645+0000 7f8d72600640 1 -- 172.21.15.3:0/1258141720 >> v1:172.21.15.3:6789/0 conn(0x7f8d6c160af0 legacy=0x7f8d6c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.646+0000 7f8d72600640 1 -- 172.21.15.3:0/1258141720 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c15d250 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.646+0000 7f8d6be00640 1 -- 172.21.15.3:0/1258141720 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945132986 0 0) 0x7f8d6c0cfdd0 con 0x7f8d6c160af0 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d72600640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c072070 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009d90 tx=0x7f8d6402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4141923b6b02ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d6be00640 1 -- 172.21.15.3:0/1258141720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64007740 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d6be00640 1 -- 172.21.15.3:0/1258141720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d640078e0 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d6be00640 1 -- 172.21.15.3:0/1258141720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64007cb0 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 -- 172.21.15.3:0/1258141720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c0d07c0 msgr2=0x7f8d6c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c072070 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009d90 tx=0x7f8d6402a660 comp rx=0 tx=0).stop 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 -- 172.21.15.3:0/1258141720 shutdown_connections 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c072070 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 --2- 172.21.15.3:0/1258141720 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.647+0000 7f8d74600640 1 -- 172.21.15.3:0/1258141720 >> 172.21.15.3:0/1258141720 conn(0x7f8d6c06d000 msgr2=0x7f8d6c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.648+0000 7f8d74600640 1 -- 172.21.15.3:0/1258141720 shutdown_connections 2024-09-10T07:51:23.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.648+0000 7f8d74600640 1 -- 172.21.15.3:0/1258141720 wait complete. 2024-09-10T07:51:23.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.649+0000 7f8d74600640 1 Processor -- start 2024-09-10T07:51:23.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.649+0000 7f8d74600640 1 -- start start 2024-09-10T07:51:23.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c1f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c15d770 con 0x7f8d6c160af0 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0cf800 con 0x7f8d6c0d07c0 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d74600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c154de0 con 0x7f8d6c160710 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d72600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d72600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55624/0 (socket says 172.21.15.3:55624) 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.650+0000 7f8d72600640 1 -- 172.21.15.3:0/1098267909 learned_addr learned my addr 172.21.15.3:0/1098267909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:23.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d71c00640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c1f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d71200640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 -- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 msgr2=0x7f8d6c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 -- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0d07c0 msgr2=0x7f8d6c1f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c1f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 -- 172.21.15.3:0/1098267909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c155080 con 0x7f8d6c160af0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d72600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009d90 tx=0x7f8d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64040070 con 0x7f8d6c160af0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64031c40 con 0x7f8d6c160af0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.651+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64009450 con 0x7f8d6c160af0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.652+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c155360 con 0x7f8d6c160af0 2024-09-10T07:51:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.652+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c155920 con 0x7f8d6c160af0 2024-09-10T07:51:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.653+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c0afd50 con 0x7f8d6c160af0 2024-09-10T07:51:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.654+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d64039040 con 0x7f8d6c160af0 2024-09-10T07:51:23.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.654+0000 7f8d69600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:23.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.655+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f8d640abcc0 con 0x7f8d6c160af0 2024-09-10T07:51:23.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.655+0000 7f8d71c00640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:23.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.655+0000 7f8d71c00640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 0x7f8d4c06e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d02d0 tx=0x7f8d60009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.659+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d640b2080 con 0x7f8d6c160af0 2024-09-10T07:51:23.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.878+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8d6c0d0020 con 0x7f8d6c160af0 2024-09-10T07:51:23.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.879+0000 7f8d69600640 1 -- 172.21.15.3:0/1098267909 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) ==== 53+0+73 (secure 0 0 0) 0x7f8d64079f80 con 0x7f8d6c160af0 2024-09-10T07:51:23.879 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 msgr2=0x7f8d4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 0x7f8d4c06e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d02d0 tx=0x7f8d60009190 comp rx=0 tx=0).stop 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 msgr2=0x7f8d6c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009d90 tx=0x7f8d64004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 shutdown_connections 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d6c160af0 0x7f8d6c156380 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d6c160710 0x7f8d6c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 --2- 172.21.15.3:0/1098267909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0d07c0 0x7f8d6c1f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 >> 172.21.15.3:0/1098267909 conn(0x7f8d6c06d000 msgr2=0x7f8d6c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 shutdown_connections 2024-09-10T07:51:23.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:23.881+0000 7f8d74600640 1 -- 172.21.15.3:0/1098267909 wait complete. 2024-09-10T07:51:23.897 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:51:23.897 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:51:23.898 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:51:23.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:23.898 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:51:23.935 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:51:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:51:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:51:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:51:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:51:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.939 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:51:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:51:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:51:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.941 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:51:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:51:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:51:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:51:23.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:23.941 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:51:24.029 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:51:24.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:24.032 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:51:24.068 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:51:24.068 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:51:24.068 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:51:24.069 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:51:24.069 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:51:24.069 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:51:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:51:24.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:51:24.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:24.266 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:51:24.301 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:51:24.301 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:51:24.301 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:51:24.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:24.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:51:24.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:51:24.398 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:51:24.433 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:51:24.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:51:24.433 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:51:24.508 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:51:24.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:24.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:51:24.638 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:51:24.638 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:51:24.638 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:51:24.654 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:51:24.654 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:51:24.654 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:51:24.654 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:51:24.654 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:51:24.675 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:51:24.676 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:51:24.677 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:51:24.677 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:51:24.677 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:51:24.677 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:51:24.677 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:51:24.680 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:51:24.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:24.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:51:24.737 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:51:24.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:51:24.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:51:24.831 INFO:teuthology.orchestra.run.smithi003.stdout:client6508 2024-09-10T07:51:24.831 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:24.831 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:24.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:51:25.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.066+0000 7f6628a00640 1 Processor -- start 2024-09-10T07:51:25.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.066+0000 7f6628a00640 1 -- start start 2024-09-10T07:51:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66241521d0 0x7f66240cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66240b48a0 0x7f66240cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66240d0130 0x7f66240d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f66240b3b20 con 0x7f66241521d0 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f66240b3dc0 con 0x7f66240d0130 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6628a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66240b40a0 con 0x7f66240b48a0 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6623400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66241521d0 0x7f66240cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6623400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66241521d0 0x7f66240cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60490/0 (socket says 172.21.15.3:60490) 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.067+0000 7f6623400640 1 -- 172.21.15.3:0/3495527039 learned_addr learned my addr 172.21.15.3:0/3495527039 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623e00640 1 --2- 172.21.15.3:0/3495527039 >> v2:172.21.15.70:3300/0 conn(0x7f66240d0130 0x7f66240d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f661be00640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66240b48a0 0x7f66240cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623400640 1 -- 172.21.15.3:0/3495527039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66240b48a0 msgr2=0x7f66240cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623400640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66240b48a0 0x7f66240cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623400640 1 -- 172.21.15.3:0/3495527039 >> v2:172.21.15.70:3300/0 conn(0x7f66240d0130 msgr2=0x7f66240d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623400640 1 --2- 172.21.15.3:0/3495527039 >> v2:172.21.15.70:3300/0 conn(0x7f66240d0130 0x7f66240d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.068+0000 7f6623400640 1 -- 172.21.15.3:0/3495527039 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66240b4380 con 0x7f66241521d0 2024-09-10T07:51:25.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.069+0000 7f6623400640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66241521d0 0x7f66240cf150 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f662415b820 tx=0x7f661402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2130733662f694f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.069+0000 7f6622a00640 1 -- 172.21.15.3:0/3495527039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614037070 con 0x7f66241521d0 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.069+0000 7f6622a00640 1 -- 172.21.15.3:0/3495527039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6614032440 con 0x7f66241521d0 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.069+0000 7f6622a00640 1 -- 172.21.15.3:0/3495527039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614032810 con 0x7f66241521d0 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 -- 172.21.15.3:0/3495527039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66241521d0 msgr2=0x7f66240cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66241521d0 0x7f66240cf150 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f662415b820 tx=0x7f661402a140 comp rx=0 tx=0).stop 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 -- 172.21.15.3:0/3495527039 shutdown_connections 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 --2- 172.21.15.3:0/3495527039 >> v2:172.21.15.70:3300/0 conn(0x7f66240d0130 0x7f66240d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66240b48a0 0x7f66240cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 --2- 172.21.15.3:0/3495527039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66241521d0 0x7f66240cf150 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 -- 172.21.15.3:0/3495527039 >> 172.21.15.3:0/3495527039 conn(0x7f66240b1e90 msgr2=0x7f66240b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 -- 172.21.15.3:0/3495527039 shutdown_connections 2024-09-10T07:51:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.070+0000 7f6628a00640 1 -- 172.21.15.3:0/3495527039 wait complete. 2024-09-10T07:51:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.072+0000 7f6628a00640 1 Processor -- start 2024-09-10T07:51:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.072+0000 7f6628a00640 1 -- start start 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.072+0000 7f6628a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6628a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 0x7f66241464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6628a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 0x7f6624146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6628a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6624152730 con 0x7f66240b48a0 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6628a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66241525b0 con 0x7f66240d0130 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6628a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66240d0ac0 con 0x7f66241521d0 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60500/0 (socket says 172.21.15.3:60500) 2024-09-10T07:51:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 -- 172.21.15.3:0/977878858 learned_addr learned my addr 172.21.15.3:0/977878858 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f661be00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 0x7f66241464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623e00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 0x7f6624146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 -- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 msgr2=0x7f6624146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 0x7f6624146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 -- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 msgr2=0x7f66241464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 0x7f66241464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623400640 1 -- 172.21.15.3:0/977878858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624147cf0 con 0x7f66240b48a0 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f6623e00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 0x7f6624146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.073+0000 7f661be00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 0x7f66241464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f6623400640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f661402a180 tx=0x7f66140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614037040 con 0x7f66240b48a0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6614042070 con 0x7f66240b48a0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614030880 con 0x7f66240b48a0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6624150860 con 0x7f66240b48a0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.074+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6624150580 con 0x7f66240b48a0 2024-09-10T07:51:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.075+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66240d0580 con 0x7f66240b48a0 2024-09-10T07:51:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.076+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6614030a20 con 0x7f66240b48a0 2024-09-10T07:51:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.077+0000 7f661b400640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 0x7f660006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.077+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f66140ab6c0 con 0x7f66240b48a0 2024-09-10T07:51:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.077+0000 7f661be00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 0x7f660006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.078+0000 7f661be00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 0x7f660006e320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f660c006fd0 tx=0x7f660c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.081+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f661407a070 con 0x7f66240b48a0 2024-09-10T07:51:25.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.300+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f66240b2520 con 0x7f66240b48a0 2024-09-10T07:51:25.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.621+0000 7f661b400640 1 -- 172.21.15.3:0/977878858 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v70) ==== 105+0+0 (secure 0 0 0) 0x7f6614079970 con 0x7f66240b48a0 2024-09-10T07:51:25.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 msgr2=0x7f660006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 0x7f660006e320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f660c006fd0 tx=0x7f660c008040 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 msgr2=0x7f6624073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f661402a180 tx=0x7f66140046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 shutdown_connections 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f660006be60 0x7f660006e320 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66241521d0 0x7f6624146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66240d0130 0x7f66241464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 --2- 172.21.15.3:0/977878858 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66240b48a0 0x7f6624073530 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 >> 172.21.15.3:0/977878858 conn(0x7f66240b1e90 msgr2=0x7f662406ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 shutdown_connections 2024-09-10T07:51:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.624+0000 7f6628a00640 1 -- 172.21.15.3:0/977878858 wait complete. 2024-09-10T07:51:25.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.801+0000 7f3e5f200640 1 Processor -- start 2024-09-10T07:51:25.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.801+0000 7f3e5f200640 1 -- start start 2024-09-10T07:51:25.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 0x7f3e580d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 0x7f3e580d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e580d3060 con 0x7f3e58161a00 2024-09-10T07:51:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3e580d1ae0 con 0x7f3e580d10f0 2024-09-10T07:51:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e580d3340 con 0x7f3e58161de0 2024-09-10T07:51:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5de00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e58161a00 0x7f3e580cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44676/0 (socket says 172.21.15.3:44676) 2024-09-10T07:51:25.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5de00640 1 -- 172.21.15.3:0/2137943184 learned_addr learned my addr 172.21.15.3:0/2137943184 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5d400640 1 --2- 172.21.15.3:0/2137943184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 0x7f3e580d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.802+0000 7f3e5e800640 1 --2- 172.21.15.3:0/2137943184 >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 0x7f3e580d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5ca00640 1 -- 172.21.15.3:0/2137943184 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409414592 0 0) 0x7f3e580d3060 con 0x7f3e58161a00 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5ca00640 1 -- 172.21.15.3:0/2137943184 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e40003520 con 0x7f3e58161a00 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5d400640 1 -- 172.21.15.3:0/2137943184 >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 msgr2=0x7f3e580d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5d400640 1 --2- 172.21.15.3:0/2137943184 >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 0x7f3e580d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5d400640 1 -- 172.21.15.3:0/2137943184 >> v1:172.21.15.3:6789/0 conn(0x7f3e58161a00 legacy=0x7f3e580cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5d400640 1 -- 172.21.15.3:0/2137943184 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580d3620 con 0x7f3e58161de0 2024-09-10T07:51:25.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.803+0000 7f3e5e800640 1 --2- 172.21.15.3:0/2137943184 >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 0x7f3e580d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.804+0000 7f3e5d400640 1 --2- 172.21.15.3:0/2137943184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 0x7f3e580d2a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e480097a0 tx=0x7f3e4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9730801586dc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.804+0000 7f3e5ca00640 1 -- 172.21.15.3:0/2137943184 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48037070 con 0x7f3e58161de0 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.804+0000 7f3e5ca00640 1 -- 172.21.15.3:0/2137943184 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e48032440 con 0x7f3e58161de0 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.804+0000 7f3e5ca00640 1 -- 172.21.15.3:0/2137943184 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48032810 con 0x7f3e58161de0 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 -- 172.21.15.3:0/2137943184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 msgr2=0x7f3e580d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 --2- 172.21.15.3:0/2137943184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 0x7f3e580d2a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e480097a0 tx=0x7f3e4802a180 comp rx=0 tx=0).stop 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 -- 172.21.15.3:0/2137943184 shutdown_connections 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 --2- 172.21.15.3:0/2137943184 >> v2:172.21.15.70:3300/0 conn(0x7f3e580d10f0 0x7f3e580d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 --2- 172.21.15.3:0/2137943184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161de0 0x7f3e580d2a80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 -- 172.21.15.3:0/2137943184 >> 172.21.15.3:0/2137943184 conn(0x7f3e58072770 msgr2=0x7f3e5806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 -- 172.21.15.3:0/2137943184 shutdown_connections 2024-09-10T07:51:25.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.805+0000 7f3e5f200640 1 -- 172.21.15.3:0/2137943184 wait complete. 2024-09-10T07:51:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.807+0000 7f3e5f200640 1 Processor -- start 2024-09-10T07:51:25.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.807+0000 7f3e5f200640 1 -- start start 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.807+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580d10f0 0x7f3e580b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161a00 0x7f3e580cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580b79f0 con 0x7f3e58161de0 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580d3be0 con 0x7f3e580d10f0 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e580b7b70 con 0x7f3e58161a00 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60508/0 (socket says 172.21.15.3:60508) 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 -- 172.21.15.3:0/197048710 learned_addr learned my addr 172.21.15.3:0/197048710 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5d400640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161a00 0x7f3e580cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5de00640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580d10f0 0x7f3e580b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 -- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161a00 msgr2=0x7f3e580cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161a00 0x7f3e580cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 -- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580d10f0 msgr2=0x7f3e580b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580d10f0 0x7f3e580b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:25.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.808+0000 7f3e5e800640 1 -- 172.21.15.3:0/197048710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580b69d0 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e5e800640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400efc0 tx=0x7f3e54009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54018030 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5400cde0 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54003bb0 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5815cbb0 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.809+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5815d130 con 0x7f3e58161de0 2024-09-10T07:51:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.810+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e580d15a0 con 0x7f3e58161de0 2024-09-10T07:51:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.811+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e54005d10 con 0x7f3e58161de0 2024-09-10T07:51:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.812+0000 7f3e52000640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 0x7f3e3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.812+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f3e5408b7b0 con 0x7f3e58161de0 2024-09-10T07:51:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.812+0000 7f3e5de00640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 0x7f3e3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.813+0000 7f3e5de00640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 0x7f3e3c06e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005880 tx=0x7f3e4c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:25.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:25.816+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e54022070 con 0x7f3e58161de0 2024-09-10T07:51:26.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.036+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e580b6f90 con 0x7f3e58161de0 2024-09-10T07:51:26.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.037+0000 7f3e52000640 1 -- 172.21.15.3:0/197048710 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5235 (secure 0 0 0) 0x7f3e54010040 con 0x7f3e58161de0 2024-09-10T07:51:26.037 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:26.037 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":71,"btime":"2024-09-10T07:51:25:624646+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6355,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:25.624624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6702,"mds_1":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702]},"id":5}]} 2024-09-10T07:51:26.038 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 71 2024-09-10T07:51:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 msgr2=0x7f3e3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 0x7f3e3c06e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005880 tx=0x7f3e4c009040 comp rx=0 tx=0).stop 2024-09-10T07:51:26.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 msgr2=0x7f3e580ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400efc0 tx=0x7f3e54009f90 comp rx=0 tx=0).stop 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 shutdown_connections 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3c06be30 0x7f3e3c06e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e58161de0 0x7f3e580ce0d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e58161a00 0x7f3e580cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 --2- 172.21.15.3:0/197048710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580d10f0 0x7f3e580b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 >> 172.21.15.3:0/197048710 conn(0x7f3e58072770 msgr2=0x7f3e580b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 shutdown_connections 2024-09-10T07:51:26.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:26.039+0000 7f3e5f200640 1 -- 172.21.15.3:0/197048710 wait complete. 2024-09-10T07:51:26.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:51:12.071594+0000', 'modified': '2024-09-10T07:51:25.624624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6702, 'mds_1': 6690}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6690': {'gid': 6690, 'name': 'a', 'rank': 1, 'incarnation': 71, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6835/4206303617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 4206303617}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 4206303617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6702': {'gid': 6702, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2373315703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6836', 'nonce': 2373315703}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2373315703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6702, 'qdb_cluster': [6702]} 2024-09-10T07:51:26.458 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6690:up:creating 2024-09-10T07:51:27.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:27.619 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:51:27.619+0000 7f2ad8a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:51:27.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.626+0000 7f7134e00640 1 Processor -- start 2024-09-10T07:51:27.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.626+0000 7f7134e00640 1 -- start start 2024-09-10T07:51:27.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7130160710 0x7f71300cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 0x7f71300d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 0x7f71300d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f71300cfdd0 con 0x7f7130160710 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f713015cc90 con 0x7f7130160af0 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7134e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f713015cf70 con 0x7f71300cf800 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7130160710 0x7f71300cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7130160710 0x7f71300cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60518/0 (socket says 172.21.15.3:60518) 2024-09-10T07:51:27.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 -- 172.21.15.3:0/114265714 learned_addr learned my addr 172.21.15.3:0/114265714 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:27.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 -- 172.21.15.3:0/114265714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 msgr2=0x7f71300d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:27.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712e800640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 0x7f71300d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7125400640 1 --2- 172.21.15.3:0/114265714 >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 0x7f71300d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 0x7f71300d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 -- 172.21.15.3:0/114265714 >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 msgr2=0x7f71300d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:27.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 --2- 172.21.15.3:0/114265714 >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 0x7f71300d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712de00640 1 -- 172.21.15.3:0/114265714 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713015d250 con 0x7f7130160710 2024-09-10T07:51:27.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f712e800640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 0x7f71300d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:27.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.628+0000 7f7125400640 1 --2- 172.21.15.3:0/114265714 >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 0x7f71300d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:27.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.629+0000 7f712de00640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160710 0x7f71300cf2c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f713006df70 tx=0x7f712002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1786001c5ef37c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:27.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.632+0000 7f712d400640 1 -- 172.21.15.3:0/114265714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120037070 con 0x7f7130160710 2024-09-10T07:51:27.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.632+0000 7f712d400640 1 -- 172.21.15.3:0/114265714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120032440 con 0x7f7130160710 2024-09-10T07:51:27.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.632+0000 7f712d400640 1 -- 172.21.15.3:0/114265714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120032810 con 0x7f7130160710 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 -- 172.21.15.3:0/114265714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160710 msgr2=0x7f71300cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160710 0x7f71300cf2c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f713006df70 tx=0x7f712002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 -- 172.21.15.3:0/114265714 shutdown_connections 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f71300cf800 0x7f71300d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 --2- 172.21.15.3:0/114265714 >> v2:172.21.15.70:3300/0 conn(0x7f7130160af0 0x7f71300d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 --2- 172.21.15.3:0/114265714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160710 0x7f71300cf2c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 -- 172.21.15.3:0/114265714 >> 172.21.15.3:0/114265714 conn(0x7f713006d000 msgr2=0x7f713006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 -- 172.21.15.3:0/114265714 shutdown_connections 2024-09-10T07:51:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.633+0000 7f7134e00640 1 -- 172.21.15.3:0/114265714 wait complete. 2024-09-10T07:51:27.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.635+0000 7f7134e00640 1 Processor -- start 2024-09-10T07:51:27.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.635+0000 7f7134e00640 1 -- start start 2024-09-10T07:51:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71300cf800 0x7f713015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7130160710 0x7f713014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f713015d770 con 0x7f7130160af0 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f713006bb20 con 0x7f71300cf800 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7134e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f713015e830 con 0x7f7130160710 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f712e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f712e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60522/0 (socket says 172.21.15.3:60522) 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f712e800640 1 -- 172.21.15.3:0/4004603697 learned_addr learned my addr 172.21.15.3:0/4004603697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f712de00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71300cf800 0x7f713015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.636+0000 7f7125400640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7130160710 0x7f713014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 -- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7130160710 msgr2=0x7f713014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7130160710 0x7f713014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 -- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71300cf800 msgr2=0x7f713015f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71300cf800 0x7f713015f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 -- 172.21.15.3:0/4004603697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713015eb10 con 0x7f7130160af0 2024-09-10T07:51:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f712e800640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f712800cef0 tx=0x7f712800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712801a070 con 0x7f7130160af0 2024-09-10T07:51:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712800d860 con 0x7f7130160af0 2024-09-10T07:51:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713015edf0 con 0x7f7130160af0 2024-09-10T07:51:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.637+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712800dc50 con 0x7f7130160af0 2024-09-10T07:51:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.638+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71300ce950 con 0x7f7130160af0 2024-09-10T07:51:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.639+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71300afd50 con 0x7f7130160af0 2024-09-10T07:51:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.640+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f712801d070 con 0x7f7130160af0 2024-09-10T07:51:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.641+0000 7f7126a00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 0x7f710c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.641+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f712808b5e0 con 0x7f7130160af0 2024-09-10T07:51:27.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.641+0000 7f712de00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 0x7f710c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.642+0000 7f712de00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 0x7f710c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7120034000 tx=0x7f712002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:27.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.647+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7128059a70 con 0x7f7130160af0 2024-09-10T07:51:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.872+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71301520b0 con 0x7f7130160af0 2024-09-10T07:51:27.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.873+0000 7f7126a00640 1 -- 172.21.15.3:0/4004603697 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5257 (secure 0 0 0) 0x7f712805dd70 con 0x7f7130160af0 2024-09-10T07:51:27.873 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:27.873 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-10T07:51:27:635322+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:27.635315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6355,6690,6702]},"id":5}]} 2024-09-10T07:51:27.873 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 msgr2=0x7f710c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 0x7f710c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7120034000 tx=0x7f712002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 msgr2=0x7f71300ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f712800cef0 tx=0x7f712800a710 comp rx=0 tx=0).stop 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 shutdown_connections 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f710c06be60 0x7f710c06e320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7130160af0 0x7f71300ce410 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7130160710 0x7f713014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 --2- 172.21.15.3:0/4004603697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71300cf800 0x7f713015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 >> 172.21.15.3:0/4004603697 conn(0x7f713006d000 msgr2=0x7f71300b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 shutdown_connections 2024-09-10T07:51:27.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:27.875+0000 7f7134e00640 1 -- 172.21.15.3:0/4004603697 wait complete. 2024-09-10T07:51:28.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:51:12.071594+0000', 'modified': '2024-09-10T07:51:27.635315+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6702, 'mds_1': 6690, 'mds_2': 6355}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6355': {'gid': 6355, 'name': 'b', 'rank': 2, 'incarnation': 73, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.70:6835/2352570209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 2352570209}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 2352570209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6690': {'gid': 6690, 'name': 'a', 'rank': 1, 'incarnation': 71, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6835/4206303617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 4206303617}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 4206303617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6702': {'gid': 6702, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2373315703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6836', 'nonce': 2373315703}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2373315703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6702, 'qdb_cluster': [6355, 6690, 6702]} 2024-09-10T07:51:28.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:51:28.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T07:51:28.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.456+0000 7f62d4600640 1 Processor -- start 2024-09-10T07:51:28.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.456+0000 7f62d4600640 1 -- start start 2024-09-10T07:51:28.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.457+0000 7f62d4600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62cc160710 0x7f62cc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d4600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc160af0 0x7f62cc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d4600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62cc0cfdd0 con 0x7f62cc0cf800 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d4600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f62cc15cc90 con 0x7f62cc160710 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d4600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc15cf70 con 0x7f62cc160af0 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62cc160710 0x7f62cc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d2600640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62cc0cf800 0x7f62cc06bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44678/0 (socket says 172.21.15.3:44678) 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d2600640 1 -- 172.21.15.3:0/1914930434 learned_addr learned my addr 172.21.15.3:0/1914930434 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1200640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc160af0 0x7f62cc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1c00640 1 -- 172.21.15.3:0/1914930434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc160af0 msgr2=0x7f62cc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1c00640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc160af0 0x7f62cc0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1c00640 1 -- 172.21.15.3:0/1914930434 >> v1:172.21.15.3:6789/0 conn(0x7f62cc0cf800 legacy=0x7f62cc06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62d1c00640 1 -- 172.21.15.3:0/1914930434 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc15d250 con 0x7f62cc160710 2024-09-10T07:51:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.458+0000 7f62cbe00640 1 -- 172.21.15.3:0/1914930434 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2582624969 0 0) 0x7f62cc0cfdd0 con 0x7f62cc0cf800 2024-09-10T07:51:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.459+0000 7f62d1c00640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160710 0x7f62cc0cf2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097a0 tx=0x7f62c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e38fc6885bd08f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62cbe00640 1 -- 172.21.15.3:0/1914930434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0037070 con 0x7f62cc160710 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62cbe00640 1 -- 172.21.15.3:0/1914930434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c0032440 con 0x7f62cc160710 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62cbe00640 1 -- 172.21.15.3:0/1914930434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0032810 con 0x7f62cc160710 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 -- 172.21.15.3:0/1914930434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160710 msgr2=0x7f62cc0cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160710 0x7f62cc0cf2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097a0 tx=0x7f62c002a180 comp rx=0 tx=0).stop 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 -- 172.21.15.3:0/1914930434 shutdown_connections 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc160af0 0x7f62cc0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 --2- 172.21.15.3:0/1914930434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160710 0x7f62cc0cf2c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 -- 172.21.15.3:0/1914930434 >> 172.21.15.3:0/1914930434 conn(0x7f62cc06d000 msgr2=0x7f62cc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 -- 172.21.15.3:0/1914930434 shutdown_connections 2024-09-10T07:51:28.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.460+0000 7f62d4600640 1 -- 172.21.15.3:0/1914930434 wait complete. 2024-09-10T07:51:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.462+0000 7f62d4600640 1 Processor -- start 2024-09-10T07:51:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.462+0000 7f62d4600640 1 -- start start 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc0cf800 0x7f62cc1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc160710 0x7f62cc154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc15d770 con 0x7f62cc160af0 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc073080 con 0x7f62cc160710 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d4600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc156840 con 0x7f62cc0cf800 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60538/0 (socket says 172.21.15.3:60538) 2024-09-10T07:51:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.463+0000 7f62d2600640 1 -- 172.21.15.3:0/16302814 learned_addr learned my addr 172.21.15.3:0/16302814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d1c00640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc0cf800 0x7f62cc1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d1200640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc160710 0x7f62cc154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 -- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc0cf800 msgr2=0x7f62cc1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc0cf800 0x7f62cc1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 -- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc160710 msgr2=0x7f62cc154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc160710 0x7f62cc154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 -- 172.21.15.3:0/16302814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1557a0 con 0x7f62cc160af0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d2600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7a0 tx=0x7f62c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62c9600640 1 -- 172.21.15.3:0/16302814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4018070 con 0x7f62cc160af0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62c9600640 1 -- 172.21.15.3:0/16302814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c400a640 con 0x7f62cc160af0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62c9600640 1 -- 172.21.15.3:0/16302814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c400aab0 con 0x7f62cc160af0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.464+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc155a80 con 0x7f62cc160af0 2024-09-10T07:51:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.465+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc156040 con 0x7f62cc160af0 2024-09-10T07:51:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.467+0000 7f62c9600640 1 -- 172.21.15.3:0/16302814 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62c400ac50 con 0x7f62cc160af0 2024-09-10T07:51:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.468+0000 7f62c9600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 0x7f62ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.468+0000 7f62c9600640 1 -- 172.21.15.3:0/16302814 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f62c408a440 con 0x7f62cc160af0 2024-09-10T07:51:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.468+0000 7f62d1c00640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 0x7f62ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.468+0000 7f62d7647740 1 Processor -- start 2024-09-10T07:51:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.468+0000 7f62d7647740 1 -- start start 2024-09-10T07:51:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d1c00640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 0x7f62ac06e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62c0034000 tx=0x7f62c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b325490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b2da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a06b23e3b0 con 0x55a06b2dd4b0 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a06b243410 con 0x55a06b2e1480 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a06b1eb620 con 0x55a06b2dfcf0 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60548/0 (socket says 172.21.15.3:60548) 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d1200640 1 -- 172.21.15.3:0/1814596854 learned_addr learned my addr 172.21.15.3:0/1814596854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d1c00640 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b325490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.469+0000 7f62d2600640 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b2da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 -- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 msgr2=0x55a06b325490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b325490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 -- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 msgr2=0x55a06b2da570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b2da570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 -- 172.21.15.3:0/1814596854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06af562d0 con 0x55a06b2dd4b0 2024-09-10T07:51:28.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62d1200640 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f62b800b900 tx=0x7f62b800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9563083cdbd9fd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62be000640 1 -- 172.21.15.3:0/1814596854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8018020 con 0x55a06b2dd4b0 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62be000640 1 -- 172.21.15.3:0/1814596854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b800f9c0 con 0x55a06b2dd4b0 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.470+0000 7f62be000640 1 -- 172.21.15.3:0/1814596854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b800fd90 con 0x55a06b2dd4b0 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 -- 172.21.15.3:0/1814596854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 msgr2=0x55a06b2e00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f62b800b900 tx=0x7f62b800bd00 comp rx=0 tx=0).stop 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 -- 172.21.15.3:0/1814596854 shutdown_connections 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b2da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dd4b0 0x55a06b2e00d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 --2- 172.21.15.3:0/1814596854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b325490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 -- 172.21.15.3:0/1814596854 >> 172.21.15.3:0/1814596854 conn(0x55a06b2dc740 msgr2=0x55a06b324960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 -- 172.21.15.3:0/1814596854 shutdown_connections 2024-09-10T07:51:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.471+0000 7f62d7647740 1 -- 172.21.15.3:0/1814596854 wait complete. 2024-09-10T07:51:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.472+0000 7f62d7647740 1 -- start start 2024-09-10T07:51:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.472+0000 7f62d7647740 1 Processor -- start 2024-09-10T07:51:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b2fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b329030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a06b1fe780 con 0x55a06b329570 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a06b23f3d0 con 0x55a06b2e1480 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a06b2dbbe0 con 0x55a06b2dfcf0 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d7647740 10 client.0 _ll_register_callbacks cb 0x55a06b23c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60562/0 (socket says 172.21.15.3:60562) 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d2600640 1 -- 172.21.15.3:0/4267332921 learned_addr learned my addr 172.21.15.3:0/4267332921 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d1c00640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b2fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.473+0000 7f62d1200640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b329030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 -- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 msgr2=0x55a06b2fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b2fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 -- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 msgr2=0x55a06b329030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b329030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 -- 172.21.15.3:0/4267332921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06b0206b0 con 0x55a06b329570 2024-09-10T07:51:28.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d2600640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f62c4054ab0 tx=0x7f62c40580e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d7cbfebfc57e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4060fa0 con 0x55a06b329570 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f62c405db90 con 0x55a06b329570 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b800f9c0 con 0x55a06b329570 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f62c4061230 con 0x55a06b329570 2024-09-10T07:51:28.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d7647740 10 client.6882 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.474+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a06b30a7f0 con 0x55a06b329570 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b800f9c0 con 0x55a06b329570 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f62c4057aa0 con 0x55a06b329570 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 10 client.6882 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 10 client.6882 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 20 client.6882 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b345390 0x55a06b347780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 4 client.6882 mds_command: new command op to 6355 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 --> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a06b250b40 con 0x55a06b345390 2024-09-10T07:51:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.475+0000 7f62d1c00640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b345390 0x55a06b347780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.476+0000 7f62d1c00640 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b345390 0x55a06b347780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cc01ea69f39b703 server_cookie=40b61484e12304c6 in_seq=0 out_seq=0 2024-09-10T07:51:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.476+0000 7f62b2a00640 10 client.6882 ms_handle_connect on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62b2a00640 1 -- 172.21.15.3:0/4267332921 <== mds.2 v2:172.21.15.70:6833/2352570209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a06b250b40 con 0x55a06b345390 2024-09-10T07:51:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62b2a00640 10 client.6882 handle_command_reply: tid=0 2024-09-10T07:51:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 client.6882 shutdown 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 msgr2=0x55a06b320620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f62c4054ab0 tx=0x7f62c40580e0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 shutdown_connections 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b345390 0x55a06b347780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b329570 0x55a06b320620 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2e1480 0x55a06b329030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 --2- 172.21.15.3:0/4267332921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2dfcf0 0x55a06b2fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62b2a00640 0 client.6882 ms_handle_reset on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 >> 172.21.15.3:0/4267332921 conn(0x55a06b2dd4b0 msgr2=0x55a06b323f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 shutdown_connections 2024-09-10T07:51:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.477+0000 7f62d7647740 1 -- 172.21.15.3:0/4267332921 wait complete. 2024-09-10T07:51:28.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.478+0000 7f62d7647740 20 client.6882 trim_cache size 0 max 16384 2024-09-10T07:51:28.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.496+0000 7f62d7647740 1 Processor -- start 2024-09-10T07:51:28.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.496+0000 7f62d7647740 1 -- start start 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 0x55a06b3229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b329570 0x55a06b342530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc0b47b0 con 0x55a06b2e1480 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc0b4e80 con 0x55a06b2dfcf0 2024-09-10T07:51:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.497+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc1544d0 con 0x55a06b329570 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60574/0 (socket says 172.21.15.3:60574) 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 -- 172.21.15.3:0/755049763 learned_addr learned my addr 172.21.15.3:0/755049763 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 -- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b329570 msgr2=0x55a06b342530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b329570 0x55a06b342530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1c00640 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 0x55a06b3229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 -- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 msgr2=0x55a06b3229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 0x55a06b3229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 -- 172.21.15.3:0/755049763 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06b2dc4e0 con 0x55a06b2e1480 2024-09-10T07:51:28.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1c00640 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 0x55a06b3229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62d1200640 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c000 tx=0x7f62b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad22db12623089a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.498+0000 7f62bd600640 1 -- 172.21.15.3:0/755049763 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8018070 con 0x55a06b2e1480 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62bd600640 1 -- 172.21.15.3:0/755049763 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b8004040 con 0x55a06b2e1480 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62bd600640 1 -- 172.21.15.3:0/755049763 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80043f0 con 0x55a06b2e1480 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 -- 172.21.15.3:0/755049763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 msgr2=0x55a06b322f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c000 tx=0x7f62b8014000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 -- 172.21.15.3:0/755049763 shutdown_connections 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b329570 0x55a06b342530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2e1480 0x55a06b322f20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 --2- 172.21.15.3:0/755049763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b2dfcf0 0x55a06b3229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 -- 172.21.15.3:0/755049763 >> 172.21.15.3:0/755049763 conn(0x55a06b2dd4b0 msgr2=0x55a06b323f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 -- 172.21.15.3:0/755049763 shutdown_connections 2024-09-10T07:51:28.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.499+0000 7f62d7647740 1 -- 172.21.15.3:0/755049763 wait complete. 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.500+0000 7f62d7647740 1 -- start start 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.500+0000 7f62d7647740 1 Processor -- start 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 0x55a06b2fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b329570 0x55a06b323e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a06b3212b0 con 0x55a06b2dfcf0 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55a06b321760 con 0x55a06b329570 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55a06b33a010 con 0x55a06b2e1480 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d7647740 10 client.0 _ll_register_callbacks cb 0x7f62c405f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60584/0 (socket says 172.21.15.3:60584) 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d1c00640 1 -- 172.21.15.3:0/2178731114 learned_addr learned my addr 172.21.15.3:0/2178731114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d2600640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b329570 0x55a06b323e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.501+0000 7f62d1200640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 0x55a06b2fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 -- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 msgr2=0x55a06b2fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 0x55a06b2fc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 -- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b329570 msgr2=0x55a06b323e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b329570 0x55a06b323e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 -- 172.21.15.3:0/2178731114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06b305dd0 con 0x55a06b2dfcf0 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1200640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 0x55a06b2fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d1c00640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098c0 tx=0x7f62c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9878dc41eba7d905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0037050 con 0x55a06b2dfcf0 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b8004040 con 0x55a06b2dfcf0 2024-09-10T07:51:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.502+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f62c0048040 con 0x55a06b2dfcf0 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f62c0062050 con 0x55a06b2dfcf0 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 10 client.6888 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a06b321a80 con 0x55a06b2dfcf0 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b8004040 con 0x55a06b2dfcf0 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f62c0065020 con 0x55a06b2dfcf0 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 10 client.6888 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 10 client.6888 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 20 client.6888 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b349f40 0x55a06b3375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 4 client.6888 mds_command: new command op to 6355 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:51:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 --> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] -- command(tid 0: {"prefix": "status"}) -- 0x55a06b31f7d0 con 0x55a06b349f40 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.503+0000 7f62d1200640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b349f40 0x55a06b3375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62d1200640 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b349f40 0x55a06b3375a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3da6ea043c00b557 server_cookie=3d2caf198c85ad91 in_seq=0 out_seq=0 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62b2a00640 10 client.6888 ms_handle_connect on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62b2a00640 1 -- 172.21.15.3:0/2178731114 <== mds.2 v2:172.21.15.70:6833/2352570209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a06b31f7d0 con 0x55a06b349f40 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62b2a00640 10 client.6888 handle_command_reply: tid=0 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62d7647740 1 client.6888 shutdown 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 msgr2=0x55a06b3a7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.504+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098c0 tx=0x7f62c0004730 comp rx=0 tx=0).stop 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 shutdown_connections 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55a06b349f40 0x55a06b3375a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a06b329570 0x55a06b323e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55a06b2e1480 0x55a06b2fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62b2a00640 0 client.6888 ms_handle_reset on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 --2- 172.21.15.3:0/2178731114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a06b2dfcf0 0x55a06b3a7850 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 >> 172.21.15.3:0/2178731114 conn(0x55a06b2dd4b0 msgr2=0x55a06b3229e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 shutdown_connections 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 1 -- 172.21.15.3:0/2178731114 wait complete. 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d7647740 20 client.6888 trim_cache size 0 max 16384 2024-09-10T07:51:28.513 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6355, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.8690395630000001, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 74, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 78, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 26.938381618000001 2024-09-10T07:51:28.514 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 msgr2=0x7f62ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 0x7f62ac06e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62c0034000 tx=0x7f62c002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 msgr2=0x7f62cc0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7a0 tx=0x7f62c400cc60 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.505+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 shutdown_connections 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f62ac06be60 0x7f62ac06e320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62cc160af0 0x7f62cc0b1fe0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc160710 0x7f62cc154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 --2- 172.21.15.3:0/16302814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f62cc0cf800 0x7f62cc1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 >> 172.21.15.3:0/16302814 conn(0x7f62cc06d000 msgr2=0x7f62cc0cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 shutdown_connections 2024-09-10T07:51:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.506+0000 7f62d4600640 1 -- 172.21.15.3:0/16302814 wait complete. 2024-09-10T07:51:28.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:51:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.683+0000 7f6399800640 1 Processor -- start 2024-09-10T07:51:28.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.683+0000 7f6399800640 1 -- start start 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.684+0000 7f6399800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 0x7f6394073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.684+0000 7f6399800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940d1b70 0x7f63940b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.684+0000 7f6399800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6394071780 con 0x7f63940b6f40 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.684+0000 7f6399800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f63940b7860 con 0x7f63940d1b70 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.684+0000 7f6399800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6394073e90 con 0x7f639415e520 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940d1b70 0x7f63940b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940d1b70 0x7f63940b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60596/0 (socket says 172.21.15.3:60596) 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 -- 172.21.15.3:0/1369807089 learned_addr learned my addr 172.21.15.3:0/1369807089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393e00640 1 --2- 172.21.15.3:0/1369807089 >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 0x7f6394073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 -- 172.21.15.3:0/1369807089 >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 msgr2=0x7f6394073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 --2- 172.21.15.3:0/1369807089 >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 0x7f6394073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 -- 172.21.15.3:0/1369807089 >> v1:172.21.15.70:6789/0 conn(0x7f63940b6f40 legacy=0x7f63941621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393400640 1 -- 172.21.15.3:0/1369807089 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63940709c0 con 0x7f63940d1b70 2024-09-10T07:51:28.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.685+0000 7f6393e00640 1 --2- 172.21.15.3:0/1369807089 >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 0x7f6394073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.686+0000 7f6393400640 1 --2- 172.21.15.3:0/1369807089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940d1b70 0x7f63940b6a00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6394050960 tx=0x7f638002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549ec11232b1a43f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.686+0000 7f6392a00640 1 -- 172.21.15.3:0/1369807089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380037070 con 0x7f63940d1b70 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.686+0000 7f6392a00640 1 -- 172.21.15.3:0/1369807089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6380032440 con 0x7f63940d1b70 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.686+0000 7f6392a00640 1 -- 172.21.15.3:0/1369807089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380032810 con 0x7f63940d1b70 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 -- 172.21.15.3:0/1369807089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940d1b70 msgr2=0x7f63940b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 --2- 172.21.15.3:0/1369807089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940d1b70 0x7f63940b6a00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6394050960 tx=0x7f638002a250 comp rx=0 tx=0).stop 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 -- 172.21.15.3:0/1369807089 shutdown_connections 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 --2- 172.21.15.3:0/1369807089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940d1b70 0x7f63940b6a00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 --2- 172.21.15.3:0/1369807089 >> v2:172.21.15.70:3300/0 conn(0x7f639415e520 0x7f6394073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 -- 172.21.15.3:0/1369807089 >> 172.21.15.3:0/1369807089 conn(0x7f63940b41e0 msgr2=0x7f63940b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 -- 172.21.15.3:0/1369807089 shutdown_connections 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.687+0000 7f6399800640 1 -- 172.21.15.3:0/1369807089 wait complete. 2024-09-10T07:51:28.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.689+0000 7f6399800640 1 Processor -- start 2024-09-10T07:51:28.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.689+0000 7f6399800640 1 -- start start 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63940d1b70 0x7f639406c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639415e520 0x7f63941507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6394071260 con 0x7f63940b6f40 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6394070f80 con 0x7f63940d1b70 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6399800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f639406cb30 con 0x7f639415e520 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60604/0 (socket says 172.21.15.3:60604) 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 -- 172.21.15.3:0/537753633 learned_addr learned my addr 172.21.15.3:0/537753633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 -- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639415e520 msgr2=0x7f63941507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639415e520 0x7f63941507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 -- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63940d1b70 msgr2=0x7f639406c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63940d1b70 0x7f639406c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.690+0000 7f6393e00640 1 -- 172.21.15.3:0/537753633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6394144370 con 0x7f63940b6f40 2024-09-10T07:51:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f6393e00640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f638400c820 tx=0x7f638400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f638be00640 1 -- 172.21.15.3:0/537753633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384015070 con 0x7f63940b6f40 2024-09-10T07:51:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f638be00640 1 -- 172.21.15.3:0/537753633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384007a30 con 0x7f63940b6f40 2024-09-10T07:51:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f638be00640 1 -- 172.21.15.3:0/537753633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384007de0 con 0x7f63940b6f40 2024-09-10T07:51:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6394153d40 con 0x7f63940b6f40 2024-09-10T07:51:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.691+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6394154300 con 0x7f63940b6f40 2024-09-10T07:51:28.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.693+0000 7f638be00640 1 -- 172.21.15.3:0/537753633 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6384018070 con 0x7f63940b6f40 2024-09-10T07:51:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.694+0000 7f638be00640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.694+0000 7f638be00640 1 -- 172.21.15.3:0/537753633 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f638408acd0 con 0x7f63940b6f40 2024-09-10T07:51:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.694+0000 7f6393400640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.695+0000 7f639dc35740 1 Processor -- start 2024-09-10T07:51:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.695+0000 7f639dc35740 1 -- start start 2024-09-10T07:51:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.695+0000 7f6393400640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6380034000 tx=0x7f638002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b1725aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b17255f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1724c33b0 con 0x55b172562660 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b1724c8410 con 0x55b172566630 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b172470620 con 0x55b172564ea0 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60612/0 (socket says 172.21.15.3:60612) 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 -- 172.21.15.3:0/167536758 learned_addr learned my addr 172.21.15.3:0/167536758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393e00640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b1725aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 -- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 msgr2=0x55b1725aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b1725aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 -- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 msgr2=0x55b17255f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6398e00640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b17255f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.696+0000 7f6393400640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b17255f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6393400640 1 -- 172.21.15.3:0/167536758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1721db2d0 con 0x55b172562660 2024-09-10T07:51:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6393e00640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b1725aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6398e00640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b17255f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6393400640 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f638002a840 tx=0x7f638002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23748f5365098ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6388c00640 1 -- 172.21.15.3:0/167536758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638004d070 con 0x55b172562660 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6388c00640 1 -- 172.21.15.3:0/167536758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638003d420 con 0x55b172562660 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.697+0000 7f6388c00640 1 -- 172.21.15.3:0/167536758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380037040 con 0x55b172562660 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 -- 172.21.15.3:0/167536758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 msgr2=0x55b172565280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f638002a840 tx=0x7f638002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 -- 172.21.15.3:0/167536758 shutdown_connections 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b17255f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172562660 0x55b172565280 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 --2- 172.21.15.3:0/167536758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b1725aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 -- 172.21.15.3:0/167536758 >> 172.21.15.3:0/167536758 conn(0x55b1725618f0 msgr2=0x55b1725a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 -- 172.21.15.3:0/167536758 shutdown_connections 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.698+0000 7f639dc35740 1 -- 172.21.15.3:0/167536758 wait complete. 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.699+0000 7f639dc35740 1 -- start start 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.699+0000 7f639dc35740 1 Processor -- start 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b1725a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b172483780 con 0x55b172563990 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b1724c43d0 con 0x55b172566630 2024-09-10T07:51:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b172560d90 con 0x55b172564ea0 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f639dc35740 10 client.0 _ll_register_callbacks cb 0x55b1724c15d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60616/0 (socket says 172.21.15.3:60616) 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f6393400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b1725a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f6398e00640 1 -- 172.21.15.3:0/2559551195 learned_addr learned my addr 172.21.15.3:0/2559551195 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.700+0000 7f6393e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 -- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 msgr2=0x55b17258c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 -- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 msgr2=0x55b1725a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b1725a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 -- 172.21.15.3:0/2559551195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1722a56b0 con 0x55b172563990 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f6398e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d480 tx=0x7f638c00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75addcaffc6cd8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f637d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c016070 con 0x55b172563990 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638003d420 con 0x55b172563990 2024-09-10T07:51:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f638c012800 con 0x55b172563990 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f638c021050 con 0x55b172563990 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f639dc35740 10 client.6900 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.701+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55b1725bd7b0 con 0x55b172563990 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638003d420 con 0x55b172563990 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f638c024020 con 0x55b172563990 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 10 client.6900 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 10 client.6900 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 20 client.6900 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b17255ef10 0x55b17255f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 4 client.6900 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1724d5b40 con 0x55b17255ef10 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.702+0000 7f6393e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b17255ef10 0x55b17255f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.703+0000 7f6393e00640 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b17255ef10 0x55b17255f2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=617b333638d31bd8 server_cookie=e14d25bb9a9f93f0 in_seq=0 out_seq=0 2024-09-10T07:51:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.703+0000 7f637d600640 10 client.6900 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.703+0000 7f637d600640 1 -- 172.21.15.3:0/2559551195 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1724d5b40 con 0x55b17255ef10 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.703+0000 7f637d600640 10 client.6900 handle_command_reply: tid=0 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 client.6900 shutdown 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 msgr2=0x55b172563e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d480 tx=0x7f638c00d940 comp rx=0 tx=0).stop 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 shutdown_connections 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b17255ef10 0x55b17255f2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172563990 0x55b172563e10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172566630 0x55b1725a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 --2- 172.21.15.3:0/2559551195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f637d600640 0 client.6900 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 >> 172.21.15.3:0/2559551195 conn(0x55b172562660 msgr2=0x55b1725a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 shutdown_connections 2024-09-10T07:51:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 1 -- 172.21.15.3:0/2559551195 wait complete. 2024-09-10T07:51:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.704+0000 7f639dc35740 20 client.6900 trim_cache size 0 max 16384 2024-09-10T07:51:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.723+0000 7f639dc35740 1 Processor -- start 2024-09-10T07:51:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.723+0000 7f639dc35740 1 -- start start 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 0x55b1725a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 0x55b1725ab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6394200ac0 con 0x55b172566630 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f63941ffde0 con 0x55b172564ea0 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f63942003f0 con 0x55b172563990 2024-09-10T07:51:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60626/0 (socket says 172.21.15.3:60626) 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 -- 172.21.15.3:0/2374776237 learned_addr learned my addr 172.21.15.3:0/2374776237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6393400640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 0x55b1725ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 -- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 msgr2=0x55b1725a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6393e00640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 0x55b1725a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 0x55b1725a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 -- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 msgr2=0x55b1725ab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 0x55b1725ab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6398e00640 1 -- 172.21.15.3:0/2374776237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1725655f0 con 0x55b172566630 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6393400640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 0x55b1725ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.724+0000 7f6393e00640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 0x55b1725a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f6398e00640 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e190 tx=0x7f638c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ecf76ee320c009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f637fe00640 1 -- 172.21.15.3:0/2374776237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c016050 con 0x55b172566630 2024-09-10T07:51:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f637fe00640 1 -- 172.21.15.3:0/2374776237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c012a50 con 0x55b172566630 2024-09-10T07:51:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f637fe00640 1 -- 172.21.15.3:0/2374776237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c012600 con 0x55b172566630 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f639dc35740 1 -- 172.21.15.3:0/2374776237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 msgr2=0x55b172562df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.725+0000 7f639dc35740 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e190 tx=0x7f638c012000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 -- 172.21.15.3:0/2374776237 shutdown_connections 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b172562df0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172564ea0 0x55b1725ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 --2- 172.21.15.3:0/2374776237 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172563990 0x55b1725a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 -- 172.21.15.3:0/2374776237 >> 172.21.15.3:0/2374776237 conn(0x55b172562660 msgr2=0x55b1725a9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 -- 172.21.15.3:0/2374776237 shutdown_connections 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 -- 172.21.15.3:0/2374776237 wait complete. 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 -- start start 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.726+0000 7f639dc35740 1 Processor -- start 2024-09-10T07:51:28.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172563990 0x55b17258ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1725be050 con 0x55b172566630 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6394155ff0 con 0x55b172563990 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b172583ff0 con 0x55b172564ea0 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.727+0000 7f639dc35740 10 client.0 _ll_register_callbacks cb 0x7f638c003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60636/0 (socket says 172.21.15.3:60636) 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 -- 172.21.15.3:0/784720065 learned_addr learned my addr 172.21.15.3:0/784720065 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6393e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172563990 0x55b17258ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6393400640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 -- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 msgr2=0x55b17258f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 -- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172563990 msgr2=0x55b17258ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172563990 0x55b17258ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 -- 172.21.15.3:0/784720065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b172611630 con 0x55b172566630 2024-09-10T07:51:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f6398e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f638c003e90 tx=0x7f638c003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d457f2c262a91e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f637d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.728+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c016040 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f638c012c00 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638c012a50 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f638c006b50 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 10 client.6906 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55b1725bc910 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c012a50 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f638c024040 con 0x55b172566630 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 10 client.6906 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 10 client.6906 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 20 client.6906 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b1726288d0 0x55b1725acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 4 client.6906 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "status"}) -- 0x55b17258d570 con 0x55b1726288d0 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.729+0000 7f6393e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b1726288d0 0x55b1725acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f6393e00640 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b1726288d0 0x55b1725acd40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee42911eef9df9b server_cookie=56f6d571f29468c1 in_seq=0 out_seq=0 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f637d600640 10 client.6906 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f637d600640 1 -- 172.21.15.3:0/784720065 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b17258d570 con 0x55b1726288d0 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f637d600640 10 client.6906 handle_command_reply: tid=0 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 client.6906 shutdown 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 msgr2=0x55b1726280e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f638c003e90 tx=0x7f638c003ae0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 shutdown_connections 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55b1726288d0 0x55b1725acd40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b172566630 0x55b1726280e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b172564ea0 0x55b17258f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 --2- 172.21.15.3:0/784720065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b172563990 0x55b17258ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f637d600640 0 client.6906 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.730+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 >> 172.21.15.3:0/784720065 conn(0x55b172562660 msgr2=0x55b172562df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 shutdown_connections 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f639dc35740 1 -- 172.21.15.3:0/784720065 wait complete. 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f639dc35740 20 client.6906 trim_cache size 0 max 16384 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:51:28.735 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6690, 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.1047743560000001, 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 74, 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 78, 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 27.166819148999998 2024-09-10T07:51:28.736 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 msgr2=0x7f637406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 0x7f637406e320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6380034000 tx=0x7f638002c000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 msgr2=0x7f6394143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f638400c820 tx=0x7f638400cce0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 shutdown_connections 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637406be60 0x7f637406e320 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639415e520 0x7f63941507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63940d1b70 0x7f639406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 --2- 172.21.15.3:0/537753633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b6f40 0x7f6394143e30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.731+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 >> 172.21.15.3:0/537753633 conn(0x7f63940b41e0 msgr2=0x7f639415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.732+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 shutdown_connections 2024-09-10T07:51:28.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.732+0000 7f6399800640 1 -- 172.21.15.3:0/537753633 wait complete. 2024-09-10T07:51:28.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:51:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.905+0000 7f7e91c00640 1 Processor -- start 2024-09-10T07:51:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.905+0000 7f7e91c00640 1 -- start start 2024-09-10T07:51:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e91c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e8c0d3c10 0x7f7e8c073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e91c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d4050 0x7f7e8c073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e91c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c0cfdc0 con 0x7f7e8c15e860 2024-09-10T07:51:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e91c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8c0d3780 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e91c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c0cee40 con 0x7f7e8c0d4050 2024-09-10T07:51:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.906+0000 7f7e8b400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e8c0d3c10 0x7f7e8c073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7e8c0d3c10 0x7f7e8c073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60650/0 (socket says 172.21.15.3:60650) 2024-09-10T07:51:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 -- 172.21.15.3:0/2796748826 learned_addr learned my addr 172.21.15.3:0/2796748826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e91200640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d4050 0x7f7e8c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 -- 172.21.15.3:0/2796748826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d4050 msgr2=0x7f7e8c073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d4050 0x7f7e8c073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 -- 172.21.15.3:0/2796748826 >> v1:172.21.15.70:6789/0 conn(0x7f7e8c15e860 legacy=0x7f7e8c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.907+0000 7f7e8b400640 1 -- 172.21.15.3:0/2796748826 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c0cf120 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.908+0000 7f7e8b400640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c0d3c10 0x7f7e8c073150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1721ade8ec1667 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.908+0000 7f7e8aa00640 1 -- 172.21.15.3:0/2796748826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78037070 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.908+0000 7f7e8aa00640 1 -- 172.21.15.3:0/2796748826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78032440 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.908+0000 7f7e8aa00640 1 -- 172.21.15.3:0/2796748826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78032810 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 -- 172.21.15.3:0/2796748826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c0d3c10 msgr2=0x7f7e8c073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c0d3c10 0x7f7e8c073150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802a270 comp rx=0 tx=0).stop 2024-09-10T07:51:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 -- 172.21.15.3:0/2796748826 shutdown_connections 2024-09-10T07:51:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d4050 0x7f7e8c073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 --2- 172.21.15.3:0/2796748826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c0d3c10 0x7f7e8c073150 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 -- 172.21.15.3:0/2796748826 >> 172.21.15.3:0/2796748826 conn(0x7f7e8c0d4650 msgr2=0x7f7e8c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 -- 172.21.15.3:0/2796748826 shutdown_connections 2024-09-10T07:51:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.909+0000 7f7e91c00640 1 -- 172.21.15.3:0/2796748826 wait complete. 2024-09-10T07:51:28.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.911+0000 7f7e91c00640 1 Processor -- start 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.911+0000 7f7e91c00640 1 -- start start 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.911+0000 7f7e91c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 0x7f7e8c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e8c0d4050 0x7f7e8c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0d2460 con 0x7f7e8c15e860 2024-09-10T07:51:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0cf400 con 0x7f7e8c0d4050 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c0b7880 con 0x7f7e8c0d3c10 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60658/0 (socket says 172.21.15.3:60658) 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 -- 172.21.15.3:0/1076952228 learned_addr learned my addr 172.21.15.3:0/1076952228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e8b400640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e8c0d4050 0x7f7e8c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e8be00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 0x7f7e8c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 -- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 msgr2=0x7f7e8c155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 0x7f7e8c155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 -- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e8c0d4050 msgr2=0x7f7e8c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e8c0d4050 0x7f7e8c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 -- 172.21.15.3:0/1076952228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1580f0 con 0x7f7e8c15e860 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e8be00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 0x7f7e8c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.912+0000 7f7e91200640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7e84007e40 tx=0x7f7e8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.913+0000 7f7e83e00640 1 -- 172.21.15.3:0/1076952228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84014620 con 0x7f7e8c15e860 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.913+0000 7f7e83e00640 1 -- 172.21.15.3:0/1076952228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84019070 con 0x7f7e8c15e860 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.913+0000 7f7e83e00640 1 -- 172.21.15.3:0/1076952228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84014a20 con 0x7f7e8c15e860 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.913+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c0ce110 con 0x7f7e8c15e860 2024-09-10T07:51:28.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.913+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c0cb7c0 con 0x7f7e8c15e860 2024-09-10T07:51:28.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.915+0000 7f7e83e00640 1 -- 172.21.15.3:0/1076952228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e84014bc0 con 0x7f7e8c15e860 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.916+0000 7f7e83e00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 0x7f7e6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.916+0000 7f7e83e00640 1 -- 172.21.15.3:0/1076952228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7e8408b690 con 0x7f7e8c15e860 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.916+0000 7f7e8be00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 0x7f7e6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.917+0000 7f7e8be00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 0x7f7e6c06e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c072360 tx=0x7f7e7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.917+0000 7f7e9627c740 1 Processor -- start 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.917+0000 7f7e9627c740 1 -- start start 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f9541640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fd630 0x5575f94f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5575f945a3b0 con 0x5575f94f9660 2024-09-10T07:51:28.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5575f945f410 con 0x5575f94fbea0 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5575f9407620 con 0x5575f94fd630 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60660/0 (socket says 172.21.15.3:60660) 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 -- 172.21.15.3:0/3558849356 learned_addr learned my addr 172.21.15.3:0/3558849356 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e91200640 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fd630 0x5575f94f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8be00640 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f9541640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 -- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fd630 msgr2=0x5575f94f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fd630 0x5575f94f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 -- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 msgr2=0x5575f9541640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f9541640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.918+0000 7f7e8b400640 1 -- 172.21.15.3:0/3558849356 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f91722d0 con 0x5575f94f9660 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5072a9e7ee182e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e80c00640 1 -- 172.21.15.3:0/3558849356 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78037040 con 0x5575f94f9660 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e80c00640 1 -- 172.21.15.3:0/3558849356 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78032780 con 0x5575f94f9660 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e80c00640 1 -- 172.21.15.3:0/3558849356 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78032b70 con 0x5575f94f9660 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e9627c740 1 -- 172.21.15.3:0/3558849356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 msgr2=0x5575f94fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.919+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802c000 comp rx=0 tx=0).stop 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 -- 172.21.15.3:0/3558849356 shutdown_connections 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fd630 0x5575f94f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94f9660 0x5575f94fc280 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3558849356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f9541640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 -- 172.21.15.3:0/3558849356 >> 172.21.15.3:0/3558849356 conn(0x5575f94f88f0 msgr2=0x5575f9540b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 -- 172.21.15.3:0/3558849356 shutdown_connections 2024-09-10T07:51:28.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 -- 172.21.15.3:0/3558849356 wait complete. 2024-09-10T07:51:28.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.920+0000 7f7e9627c740 1 -- start start 2024-09-10T07:51:28.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.921+0000 7f7e9627c740 1 Processor -- start 2024-09-10T07:51:28.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f95264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94f76e0 0x5575f94fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5575f941a780 con 0x5575f94fd630 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5575f945b3d0 con 0x5575f94fbea0 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5575f94f7d90 con 0x5575f94f76e0 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e9627c740 10 client.0 _ll_register_callbacks cb 0x5575f94585d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60664/0 (socket says 172.21.15.3:60664) 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f95264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 -- 172.21.15.3:0/3662516179 learned_addr learned my addr 172.21.15.3:0/3662516179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e91200640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94f76e0 0x5575f94fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 -- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94f76e0 msgr2=0x5575f94fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94f76e0 0x5575f94fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 -- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 msgr2=0x5575f95264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f95264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.922+0000 7f7e8b400640 1 -- 172.21.15.3:0/3662516179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f923c6b0 con 0x5575f94fd630 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e8b400640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7e7802a860 tx=0x7f7e78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35f66d50b3673ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78037040 con 0x5575f94fd630 2024-09-10T07:51:28.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7e78004440 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e78032780 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7e78051050 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 10 client.6918 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5575f951b8a0 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78032780 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f7e78054020 con 0x5575f94fd630 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 10 client.6918 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 10 client.6918 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 20 client.6918 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.923+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f59f0 0x5575f94f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.924+0000 7f7e9627c740 4 client.6918 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.924+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f946cb40 con 0x5575f94f59f0 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.924+0000 7f7e8be00640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f59f0 0x5575f94f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.924+0000 7f7e8be00640 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f59f0 0x5575f94f5dd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9df68194cdc93d8 server_cookie=d0af731c1aa200a4 in_seq=0 out_seq=0 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.924+0000 7f7e75600640 10 client.6918 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e75600640 1 -- 172.21.15.3:0/3662516179 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575f946cb40 con 0x5575f94f59f0 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e75600640 10 client.6918 handle_command_reply: tid=0 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 client.6918 shutdown 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 msgr2=0x5575f951a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7e7802a860 tx=0x7f7e78004730 comp rx=0 tx=0).stop 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 shutdown_connections 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f59f0 0x5575f94f5dd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94f76e0 0x5575f94fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e75600640 0 client.6918 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951a0d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 --2- 172.21.15.3:0/3662516179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94fbea0 0x5575f95264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.925+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 >> 172.21.15.3:0/3662516179 conn(0x5575f94f9660 msgr2=0x5575f9540070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.926+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 shutdown_connections 2024-09-10T07:51:28.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.926+0000 7f7e9627c740 1 -- 172.21.15.3:0/3662516179 wait complete. 2024-09-10T07:51:28.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.926+0000 7f7e9627c740 20 client.6918 trim_cache size 0 max 16384 2024-09-10T07:51:28.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.944+0000 7f7e9627c740 1 Processor -- start 2024-09-10T07:51:28.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.944+0000 7f7e9627c740 1 -- start start 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f9526520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f9541110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ffc80 con 0x5575f94fd630 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c200350 con 0x5575f94f76e0 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c1fed00 con 0x5575f94fbea0 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60678/0 (socket says 172.21.15.3:60678) 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.945+0000 7f7e91200640 1 -- 172.21.15.3:0/2005391529 learned_addr learned my addr 172.21.15.3:0/2005391529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 -- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 msgr2=0x5575f9541110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f9541110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 -- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 msgr2=0x5575f9526520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e8be00640 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f9526520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f9526520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 -- 172.21.15.3:0/2005391529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f9541650 con 0x5575f94fd630 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e8be00640 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f9526520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e91200640 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e84089bb0 tx=0x7f7e8408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5bf52fd50cd0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e77e00640 1 -- 172.21.15.3:0/2005391529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84088070 con 0x5575f94fd630 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e77e00640 1 -- 172.21.15.3:0/2005391529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8405dce0 con 0x5575f94fd630 2024-09-10T07:51:28.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.946+0000 7f7e77e00640 1 -- 172.21.15.3:0/2005391529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84062510 con 0x5575f94fd630 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- 172.21.15.3:0/2005391529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 msgr2=0x5575f9521c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e84089bb0 tx=0x7f7e8408b040 comp rx=0 tx=0).stop 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- 172.21.15.3:0/2005391529 shutdown_connections 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f9521c80 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f9541110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2005391529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f9526520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- 172.21.15.3:0/2005391529 >> 172.21.15.3:0/2005391529 conn(0x5575f94f9660 msgr2=0x5575f9540070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- 172.21.15.3:0/2005391529 shutdown_connections 2024-09-10T07:51:28.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- 172.21.15.3:0/2005391529 wait complete. 2024-09-10T07:51:28.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.947+0000 7f7e9627c740 1 -- start start 2024-09-10T07:51:28.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.948+0000 7f7e9627c740 1 Processor -- start 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f95beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f951f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5575f9524e50 con 0x5575f94fd630 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5575f951bb80 con 0x5575f94f76e0 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5575f951cd30 con 0x5575f94fbea0 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e9627c740 10 client.0 _ll_register_callbacks cb 0x7f7e7c001400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60686/0 (socket says 172.21.15.3:60686) 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 -- 172.21.15.3:0/2427557081 learned_addr learned my addr 172.21.15.3:0/2427557081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:28.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e8b400640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f951f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e8be00640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f95beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 -- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 msgr2=0x5575f951f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f951f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 -- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 msgr2=0x5575f95beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f95beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.949+0000 7f7e91200640 1 -- 172.21.15.3:0/2427557081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f9564aa0 con 0x5575f94fd630 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e91200640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9b0 tx=0x7f7e84089a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6b98b932416ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84075070 con 0x5575f94fd630 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7e840600b0 con 0x5575f94fd630 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8405dce0 con 0x5575f94fd630 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7e84080050 con 0x5575f94fd630 2024-09-10T07:51:28.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 10 client.6924 fetch_fsmap learned FSMap version 74 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5575f95bf690 con 0x5575f94fd630 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8405dce0 con 0x5575f94fd630 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3542+0+0 (secure 0 0 0) 0x7f7e84083020 con 0x5575f94fd630 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 10 client.6924 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 10 client.6924 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 20 client.6924 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f65b0 0x5575f94f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 4 client.6924 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.950+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "status"}) -- 0x5575f9466e90 con 0x5575f94f65b0 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.951+0000 7f7e8be00640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f65b0 0x5575f94f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.951+0000 7f7e8be00640 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f65b0 0x5575f94f6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9eda7565b6fd1a server_cookie=9058cdf2ab10ff34 in_seq=0 out_seq=0 2024-09-10T07:51:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.951+0000 7f7e75600640 10 client.6924 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.951+0000 7f7e75600640 1 -- 172.21.15.3:0/2427557081 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5575f9466e90 con 0x5575f94f65b0 2024-09-10T07:51:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e75600640 10 client.6924 handle_command_reply: tid=0 2024-09-10T07:51:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 client.6924 shutdown 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 msgr2=0x5575f951fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9b0 tx=0x7f7e84089a80 comp rx=0 tx=0).stop 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 shutdown_connections 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x5575f94f65b0 0x5575f94f6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5575f94fd630 0x5575f951fb20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5575f94fbea0 0x5575f951f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 --2- 172.21.15.3:0/2427557081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5575f94f76e0 0x5575f95beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 >> 172.21.15.3:0/2427557081 conn(0x5575f94f9660 msgr2=0x5575f9519e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e75600640 0 client.6924 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 shutdown_connections 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 1 -- 172.21.15.3:0/2427557081 wait complete. 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.952+0000 7f7e9627c740 20 client.6924 trim_cache size 0 max 16384 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6702, 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:51:28.956 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 16.314115770000001, 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 74, 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 78, 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 78, 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.416185097 2024-09-10T07:51:28.957 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 msgr2=0x7f7e6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 0x7f7e6c06e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c072360 tx=0x7f7e7c009040 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 msgr2=0x7f7e8c0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7e84007e40 tx=0x7f7e8400e4b0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 shutdown_connections 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7e6c06be30 0x7f7e6c06e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e8c15e860 0x7f7e8c0cbfc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e8c0d4050 0x7f7e8c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 --2- 172.21.15.3:0/1076952228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7e8c0d3c10 0x7f7e8c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 >> 172.21.15.3:0/1076952228 conn(0x7f7e8c0d4650 msgr2=0x7f7e8c0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 shutdown_connections 2024-09-10T07:51:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:28.953+0000 7f7e91c00640 1 -- 172.21.15.3:0/1076952228 wait complete. 2024-09-10T07:51:28.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:29.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.127+0000 7fa412a00640 1 Processor -- start 2024-09-10T07:51:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.127+0000 7fa412a00640 1 -- start start 2024-09-10T07:51:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.128+0000 7fa412a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa40c161a00 0x7fa40c0d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.128+0000 7fa412a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c0d10f0 0x7fa40c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.128+0000 7fa412a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40c0d3060 con 0x7fa40c161de0 2024-09-10T07:51:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.128+0000 7fa412a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c0d1ac0 con 0x7fa40c161a00 2024-09-10T07:51:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.128+0000 7fa412a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0d3340 con 0x7fa40c0d10f0 2024-09-10T07:51:29.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa412000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c0d10f0 0x7fa40c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa40c161a00 0x7fa40c0d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa40c161a00 0x7fa40c0d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:55780/0 (socket says 172.21.15.3:55780) 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 -- 172.21.15.3:0/137610706 learned_addr learned my addr 172.21.15.3:0/137610706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 -- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c0d10f0 msgr2=0x7fa40c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 --2- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c0d10f0 0x7fa40c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 -- 172.21.15.3:0/137610706 >> v1:172.21.15.70:6789/0 conn(0x7fa40c161de0 legacy=0x7fa40c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.129+0000 7fa411600640 1 -- 172.21.15.3:0/137610706 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c0d3620 con 0x7fa40c161a00 2024-09-10T07:51:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.130+0000 7fa411600640 1 --2- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c161a00 0x7fa40c0d2a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa400009870 tx=0x7fa40002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd7eecd5f9ff93b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.130+0000 7fa40be00640 1 -- 172.21.15.3:0/137610706 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400037070 con 0x7fa40c161a00 2024-09-10T07:51:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.130+0000 7fa40be00640 1 -- 172.21.15.3:0/137610706 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa400032440 con 0x7fa40c161a00 2024-09-10T07:51:29.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.130+0000 7fa40be00640 1 -- 172.21.15.3:0/137610706 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400032810 con 0x7fa40c161a00 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 -- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c161a00 msgr2=0x7fa40c0d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 --2- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c161a00 0x7fa40c0d2a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa400009870 tx=0x7fa40002a270 comp rx=0 tx=0).stop 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 -- 172.21.15.3:0/137610706 shutdown_connections 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 --2- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c0d10f0 0x7fa40c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 --2- 172.21.15.3:0/137610706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c161a00 0x7fa40c0d2a80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 -- 172.21.15.3:0/137610706 >> 172.21.15.3:0/137610706 conn(0x7fa40c072770 msgr2=0x7fa40c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 -- 172.21.15.3:0/137610706 shutdown_connections 2024-09-10T07:51:29.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.131+0000 7fa412a00640 1 -- 172.21.15.3:0/137610706 wait complete. 2024-09-10T07:51:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.133+0000 7fa412a00640 1 Processor -- start 2024-09-10T07:51:29.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.133+0000 7fa412a00640 1 -- start start 2024-09-10T07:51:29.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0d10f0 0x7fa40c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c161de0 0x7fa40c0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0b79f0 con 0x7fa40c161a00 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0d3be0 con 0x7fa40c0d10f0 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0b7b70 con 0x7fa40c161de0 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60692/0 (socket says 172.21.15.3:60692) 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 -- 172.21.15.3:0/3340150404 learned_addr learned my addr 172.21.15.3:0/3340150404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa411600640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0d10f0 0x7fa40c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa412000640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c161de0 0x7fa40c0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 -- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c161de0 msgr2=0x7fa40c0b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c161de0 0x7fa40c0b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 -- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0d10f0 msgr2=0x7fa40c0cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0d10f0 0x7fa40c0cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.134+0000 7fa410c00640 1 -- 172.21.15.3:0/3340150404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c147760 con 0x7fa40c161a00 2024-09-10T07:51:29.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa410c00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c470 tx=0x7fa3f800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80052d0 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f8005490 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8007260 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c0b5d40 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.135+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c0b65e0 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.136+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c0b6130 con 0x7fa40c161a00 2024-09-10T07:51:29.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.137+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3f8007400 con 0x7fa40c161a00 2024-09-10T07:51:29.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.138+0000 7fa409600640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 0x7fa3ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:29.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.138+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f808a2e0 con 0x7fa40c161a00 2024-09-10T07:51:29.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.138+0000 7fa411600640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 0x7fa3ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:29.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.139+0000 7fa411600640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 0x7fa3ec06e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa400034000 tx=0x7fa40002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:29.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.142+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa40c147760 con 0x7fa40c161a00 2024-09-10T07:51:29.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.362+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c0b96e0 con 0x7fa40c161a00 2024-09-10T07:51:29.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.362+0000 7fa409600640 1 -- 172.21.15.3:0/3340150404 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5257 (secure 0 0 0) 0x7fa3f80585b0 con 0x7fa40c161a00 2024-09-10T07:51:29.362 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:29.362 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-10T07:51:27:635322+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:27.635315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6355,6690,6702]},"id":5}]} 2024-09-10T07:51:29.363 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-10T07:51:29.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.364+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 msgr2=0x7fa3ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.364+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 0x7fa3ec06e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa400034000 tx=0x7fa40002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.364+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 msgr2=0x7fa40c0b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.364+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c470 tx=0x7fa3f800c930 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 shutdown_connections 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3ec06be30 0x7fa3ec06e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa40c161de0 0x7fa40c0b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa40c161a00 0x7fa40c0b5160 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 --2- 172.21.15.3:0/3340150404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0d10f0 0x7fa40c0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 >> 172.21.15.3:0/3340150404 conn(0x7fa40c072770 msgr2=0x7fa40c15f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 shutdown_connections 2024-09-10T07:51:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:29.365+0000 7fa412a00640 1 -- 172.21.15.3:0/3340150404 wait complete. 2024-09-10T07:51:29.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:29.381 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:51:29.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:29.428 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:51:29.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:29.511 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:51:29.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:29.588 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:51:29.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:29.665 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:51:29.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:51:29.957 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:51:29.957 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:29.958 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:51:31.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:51:32.143 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:51:32.144 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:32.144 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:51:34.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:51:34.331 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:51:34.332 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:34.332 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:51:36.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:51:36.520 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:51:36.521 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:36.521 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:51:38.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:51:38.710 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:51:38.710 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:51:38.711 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:51:38.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:51:38.885 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:51:38.886 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:38.886 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:51:40.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:51:41.071 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:51:41.072 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:41.072 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:51:43.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:51:43.259 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-10T07:51:43.259 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:51:43.260 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:51:43.260 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-10T07:51:43.440 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:51:43.441 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:51:43.441 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:51:43.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-10T07:51:43.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.601+0000 7f683ec00640 1 Processor -- start 2024-09-10T07:51:43.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.601+0000 7f683ec00640 1 -- start start 2024-09-10T07:51:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683ec00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6838161b70 0x7f68380d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683ec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d4cb0 0x7f68380cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683ec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68380d3de0 con 0x7f68380d3860 2024-09-10T07:51:43.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683ec00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6838161f50 con 0x7f6838161b70 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683ec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f68380d40c0 con 0x7f68380d4cb0 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683d800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6838161b70 0x7f68380d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683d800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6838161b70 0x7f68380d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:50910/0 (socket says 172.21.15.3:50910) 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683d800640 1 -- 172.21.15.3:0/1356641535 learned_addr learned my addr 172.21.15.3:0/1356641535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.602+0000 7f683e200640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d4cb0 0x7f68380cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.603+0000 7f683d800640 1 -- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d4cb0 msgr2=0x7f68380cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.603+0000 7f683d800640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d4cb0 0x7f68380cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.603+0000 7f683d800640 1 -- 172.21.15.3:0/1356641535 >> v1:172.21.15.70:6789/0 conn(0x7f68380d3860 legacy=0x7f68380d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.603+0000 7f683d800640 1 -- 172.21.15.3:0/1356641535 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68380cf690 con 0x7f6838161b70 2024-09-10T07:51:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f683d800640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838161b70 0x7f68380d3320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f682c009870 tx=0x7f682c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de410917867f0ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f6837e00640 1 -- 172.21.15.3:0/1356641535 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c037070 con 0x7f6838161b70 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f6837e00640 1 -- 172.21.15.3:0/1356641535 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682c032440 con 0x7f6838161b70 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f6837e00640 1 -- 172.21.15.3:0/1356641535 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c032810 con 0x7f6838161b70 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f683ec00640 1 -- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838161b70 msgr2=0x7f68380d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.604+0000 7f683ec00640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838161b70 0x7f68380d3320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f682c009870 tx=0x7f682c02a250 comp rx=0 tx=0).stop 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 -- 172.21.15.3:0/1356641535 shutdown_connections 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d4cb0 0x7f68380cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 --2- 172.21.15.3:0/1356641535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6838161b70 0x7f68380d3320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 -- 172.21.15.3:0/1356641535 >> 172.21.15.3:0/1356641535 conn(0x7f6838062630 msgr2=0x7f6838062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 -- 172.21.15.3:0/1356641535 shutdown_connections 2024-09-10T07:51:43.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.605+0000 7f683ec00640 1 -- 172.21.15.3:0/1356641535 wait complete. 2024-09-10T07:51:43.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.606+0000 7f683ec00640 1 Processor -- start 2024-09-10T07:51:43.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.606+0000 7f683ec00640 1 -- start start 2024-09-10T07:51:43.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 0x7f6838152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380d4cb0 0x7f6838160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 0x7f68381609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68380cf2d0 con 0x7f6838161b70 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f68380c9fc0 con 0x7f68380d4cb0 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.607+0000 7f683ec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f68380cf450 con 0x7f68380d3860 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 0x7f68381609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 0x7f68381609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58830/0 (socket says 172.21.15.3:58830) 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 0x7f6838152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683e200640 1 -- 172.21.15.3:0/1242029244 learned_addr learned my addr 172.21.15.3:0/1242029244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683ce00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380d4cb0 0x7f6838160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 -- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380d4cb0 msgr2=0x7f6838160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380d4cb0 0x7f6838160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 -- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 msgr2=0x7f68381609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 0x7f68381609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 -- 172.21.15.3:0/1242029244 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68380b4390 con 0x7f68380d3860 2024-09-10T07:51:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.608+0000 7f683d800640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 0x7f6838152ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f682c031000 tx=0x7f682c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.609+0000 7f6835600640 1 -- 172.21.15.3:0/1242029244 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c041020 con 0x7f68380d3860 2024-09-10T07:51:43.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.609+0000 7f6835600640 1 -- 172.21.15.3:0/1242029244 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682c030660 con 0x7f68380d3860 2024-09-10T07:51:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.609+0000 7f6835600640 1 -- 172.21.15.3:0/1242029244 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c037040 con 0x7f68380d3860 2024-09-10T07:51:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.609+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68380b4670 con 0x7f68380d3860 2024-09-10T07:51:43.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.609+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68380b4c30 con 0x7f68380d3860 2024-09-10T07:51:43.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.612+0000 7f6835600640 1 -- 172.21.15.3:0/1242029244 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f682c044020 con 0x7f68380d3860 2024-09-10T07:51:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.612+0000 7f6835600640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 0x7f681806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.613+0000 7f6835600640 1 -- 172.21.15.3:0/1242029244 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f682c0ab410 con 0x7f68380d3860 2024-09-10T07:51:43.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.613+0000 7f683ce00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 0x7f681806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.613+0000 7f684303a740 1 Processor -- start 2024-09-10T07:51:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.613+0000 7f684303a740 1 -- start start 2024-09-10T07:51:43.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.613+0000 7f683ce00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 0x7f681806e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f68380ca520 tx=0x7f6824009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8f3f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f259e0 0x563aa8f24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563aa8e8d660 con 0x563aa8f6fea0 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563aa8e35680 con 0x563aa8f3f4f0 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563aa8e89620 con 0x563aa8f259e0 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58844/0 (socket says 172.21.15.3:58844) 2024-09-10T07:51:43.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.614+0000 7f683e200640 1 -- 172.21.15.3:0/1639770202 learned_addr learned my addr 172.21.15.3:0/1639770202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683ce00640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f259e0 0x563aa8f24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 -- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f259e0 msgr2=0x563aa8f24240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f259e0 0x563aa8f24240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683d800640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8f3f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 -- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 msgr2=0x563aa8f3f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8f3f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 -- 172.21.15.3:0/1639770202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa8c6a7d0 con 0x563aa8f6fea0 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683d800640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8f3f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f683e200640 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6830011500 tx=0x7f6830009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44269011eedfe60f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f682a000640 1 -- 172.21.15.3:0/1639770202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830015840 con 0x563aa8f6fea0 2024-09-10T07:51:43.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f682a000640 1 -- 172.21.15.3:0/1639770202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68300159e0 con 0x563aa8f6fea0 2024-09-10T07:51:43.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.615+0000 7f682a000640 1 -- 172.21.15.3:0/1639770202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830015d90 con 0x563aa8f6fea0 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 -- 172.21.15.3:0/1639770202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 msgr2=0x563aa8f703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6830011500 tx=0x7f6830009750 comp rx=0 tx=0).stop 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 -- 172.21.15.3:0/1639770202 shutdown_connections 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f703f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f259e0 0x563aa8f24240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 --2- 172.21.15.3:0/1639770202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8f3f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 -- 172.21.15.3:0/1639770202 >> 172.21.15.3:0/1639770202 conn(0x563aa8f26410 msgr2=0x563aa8f6ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 -- 172.21.15.3:0/1639770202 shutdown_connections 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.616+0000 7f684303a740 1 -- 172.21.15.3:0/1639770202 wait complete. 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.617+0000 7f684303a740 1 -- start start 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.617+0000 7f684303a740 1 Processor -- start 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f6fea0 0x563aa8f68ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f7ee50 0x563aa8f7f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f24940 con 0x563aa8f3f4f0 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563aa8e164c0 con 0x563aa8f6fea0 2024-09-10T07:51:43.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563aa8f24c20 con 0x563aa8f7ee50 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f684303a740 10 client.0 _ll_register_callbacks cb 0x563aa8e85e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f683d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f683d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58852/0 (socket says 172.21.15.3:58852) 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f683d800640 1 -- 172.21.15.3:0/112569802 learned_addr learned my addr 172.21.15.3:0/112569802 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f683ce00640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f6fea0 0x563aa8f68ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.618+0000 7f683e200640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f7ee50 0x563aa8f7f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 -- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f7ee50 msgr2=0x563aa8f7f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f7ee50 0x563aa8f7f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 -- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f6fea0 msgr2=0x563aa8f68ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f6fea0 0x563aa8f68ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 -- 172.21.15.3:0/112569802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa8ba03f0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f683d800640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f682c0a98c0 tx=0x7f682c079170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f13b3ae10b0695 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f681ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0799b0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f682c079b50 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68300159e0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f682c081df0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f684303a740 10 client.6936 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.619+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x563aa8b600f0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68300159e0 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f682c03a030 con 0x563aa8f3f4f0 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 10 client.6936 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 10 client.6936 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 20 client.6936 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f89780 0x563aa8f8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 4 client.6936 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:43.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa8e9ad90 con 0x563aa8f89780 2024-09-10T07:51:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.620+0000 7f683ce00640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f89780 0x563aa8f8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.621+0000 7f683ce00640 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f89780 0x563aa8f8bbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c098a4cd4877e04d server_cookie=ad9399dcb4690e79 in_seq=0 out_seq=0 2024-09-10T07:51:43.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.621+0000 7f681ea00640 10 client.6936 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.621+0000 7f681ea00640 1 -- 172.21.15.3:0/112569802 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563aa8e9ad90 con 0x563aa8f89780 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.621+0000 7f681ea00640 10 client.6936 handle_command_reply: tid=0 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 client.6936 shutdown 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 msgr2=0x563aa8f68570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f682c0a98c0 tx=0x7f682c079170 comp rx=0 tx=0).stop 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 shutdown_connections 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f89780 0x563aa8f8bbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f7ee50 0x563aa8f7f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f6fea0 0x563aa8f68ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 --2- 172.21.15.3:0/112569802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f3f4f0 0x563aa8f68570 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f681ea00640 0 client.6936 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 >> 172.21.15.3:0/112569802 conn(0x563aa8f259e0 msgr2=0x563aa8f6dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 shutdown_connections 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 1 -- 172.21.15.3:0/112569802 wait complete. 2024-09-10T07:51:43.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.622+0000 7f684303a740 20 client.6936 trim_cache size 0 max 16384 2024-09-10T07:51:43.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 Processor -- start 2024-09-10T07:51:43.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 -- start start 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f3f4f0 0x563aa8f6ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f7ee50 0x563aa8f55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6838200b80 con 0x563aa8f6fea0 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f68381ffd50 con 0x563aa8f7ee50 2024-09-10T07:51:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.638+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6838200450 con 0x563aa8f3f4f0 2024-09-10T07:51:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58864/0 (socket says 172.21.15.3:58864) 2024-09-10T07:51:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 -- 172.21.15.3:0/4243447378 learned_addr learned my addr 172.21.15.3:0/4243447378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683d800640 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f3f4f0 0x563aa8f6ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683e200640 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f7ee50 0x563aa8f55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 -- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f3f4f0 msgr2=0x563aa8f6ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f3f4f0 0x563aa8f6ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 -- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f7ee50 msgr2=0x563aa8f55300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f7ee50 0x563aa8f55300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 -- 172.21.15.3:0/4243447378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa8f7f620 con 0x563aa8f6fea0 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.639+0000 7f683ce00640 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f68381f7850 tx=0x7f6824012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b34c7167f4f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f6829600640 1 -- 172.21.15.3:0/4243447378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682400fe10 con 0x563aa8f6fea0 2024-09-10T07:51:43.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f6829600640 1 -- 172.21.15.3:0/4243447378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824007030 con 0x563aa8f6fea0 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f6829600640 1 -- 172.21.15.3:0/4243447378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68240073e0 con 0x563aa8f6fea0 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 -- 172.21.15.3:0/4243447378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 msgr2=0x563aa8f72840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f68381f7850 tx=0x7f6824012000 comp rx=0 tx=0).stop 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 -- 172.21.15.3:0/4243447378 shutdown_connections 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f7ee50 0x563aa8f55300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f6fea0 0x563aa8f72840 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 --2- 172.21.15.3:0/4243447378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f3f4f0 0x563aa8f6ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.640+0000 7f684303a740 1 -- 172.21.15.3:0/4243447378 >> 172.21.15.3:0/4243447378 conn(0x563aa8f259e0 msgr2=0x563aa8f6bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.641+0000 7f684303a740 1 -- 172.21.15.3:0/4243447378 shutdown_connections 2024-09-10T07:51:43.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.641+0000 7f684303a740 1 -- 172.21.15.3:0/4243447378 wait complete. 2024-09-10T07:51:43.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.641+0000 7f684303a740 1 -- start start 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.641+0000 7f684303a740 1 Processor -- start 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8fe3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f6fea0 0x563aa8f3dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f6ef70 con 0x563aa8f7ee50 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6838157e30 con 0x563aa8f3f4f0 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563aa8f25ff0 con 0x563aa8f6fea0 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.642+0000 7f684303a740 10 client.0 _ll_register_callbacks cb 0x7f682c009240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58866/0 (socket says 172.21.15.3:58866) 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 -- 172.21.15.3:0/4160275245 learned_addr learned my addr 172.21.15.3:0/4160275245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683ce00640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f6fea0 0x563aa8f3dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683d800640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8fe3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 -- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f6fea0 msgr2=0x563aa8f3dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f6fea0 0x563aa8f3dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 -- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 msgr2=0x563aa8fe3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8fe3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 -- 172.21.15.3:0/4160275245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa8f2a680 con 0x563aa8f7ee50 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683d800640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8fe3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f683e200640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f68300114d0 tx=0x7f6830004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6a703bc2348cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.643+0000 7f681ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830028840 con 0x563aa8f7ee50 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f68300289e0 con 0x563aa8f7ee50 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6824007030 con 0x563aa8f7ee50 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6830035050 con 0x563aa8f7ee50 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 10 client.6942 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x563aa8f7e6c0 con 0x563aa8f7ee50 2024-09-10T07:51:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824007030 con 0x563aa8f7ee50 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f6830038020 con 0x563aa8f7ee50 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 10 client.6942 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 10 client.6942 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.644+0000 7f684303a740 20 client.6942 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f3e000 0x563aa8f6ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f684303a740 4 client.6942 mds_command: new command op to 6702 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 --> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563aa8e950e0 con 0x563aa8f3e000 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f683d800640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f3e000 0x563aa8f6ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f683d800640 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f3e000 0x563aa8f6ae30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b5f54aa11f5837 server_cookie=40238c1e3afb230f in_seq=0 out_seq=0 2024-09-10T07:51:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.645+0000 7f681ea00640 10 client.6942 ms_handle_connect on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f681ea00640 1 -- 172.21.15.3:0/4160275245 <== mds.0 v2:172.21.15.3:6836/2373315703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563aa8e950e0 con 0x563aa8f3e000 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f681ea00640 10 client.6942 handle_command_reply: tid=0 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 client.6942 shutdown 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 msgr2=0x563aa8f6a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f68300114d0 tx=0x7f6830004ab0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 shutdown_connections 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:6836/2373315703,v1:172.21.15.3:6837/2373315703] conn(0x563aa8f3e000 0x563aa8f6ae30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aa8f7ee50 0x563aa8f6a8f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563aa8f6fea0 0x563aa8f3dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 --2- 172.21.15.3:0/4160275245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563aa8f3f4f0 0x563aa8fe3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f681ea00640 0 client.6942 ms_handle_reset on v2:172.21.15.3:6836/2373315703 2024-09-10T07:51:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 >> 172.21.15.3:0/4160275245 conn(0x563aa8f259e0 msgr2=0x563aa8f69c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 shutdown_connections 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 1 -- 172.21.15.3:0/4160275245 wait complete. 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.659+0000 7f684303a740 20 client.6942 trim_cache size 0 max 16384 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:51:43.660 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 msgr2=0x7f681806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 0x7f681806e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f68380ca520 tx=0x7f6824009040 comp rx=0 tx=0).stop 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 msgr2=0x7f6838152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 0x7f6838152ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f682c031000 tx=0x7f682c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 shutdown_connections 2024-09-10T07:51:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f681806be30 0x7f681806e2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6838161b70 0x7f68381609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68380d4cb0 0x7f6838160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 --2- 172.21.15.3:0/1242029244 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f68380d3860 0x7f6838152ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 >> 172.21.15.3:0/1242029244 conn(0x7f6838062630 msgr2=0x7f683815eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 shutdown_connections 2024-09-10T07:51:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.660+0000 7f683ec00640 1 -- 172.21.15.3:0/1242029244 wait complete. 2024-09-10T07:51:43.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2024-09-10T07:51:43.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.835+0000 7f7cc6c00640 1 Processor -- start 2024-09-10T07:51:43.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.835+0000 7f7cc6c00640 1 -- start start 2024-09-10T07:51:43.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cc0161b70 0x7f7cc00d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc00d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc00d3de0 con 0x7f7cc00d4ae0 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc0161f50 con 0x7f7cc0161b70 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc00d40c0 con 0x7f7cc00d3860 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc5800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cc0161b70 0x7f7cc00d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc5800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cc0161b70 0x7f7cc00d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47700/0 (socket says 172.21.15.3:47700) 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc5800640 1 -- 172.21.15.3:0/2213175096 learned_addr learned my addr 172.21.15.3:0/2213175096 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.836+0000 7f7cc6200640 1 --1- 172.21.15.3:0/2213175096 >> v1:172.21.15.3:6789/0 conn(0x7f7cc00d4ae0 0x7f7cc00d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34606/0 (socket says 172.21.15.3:34606) 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc00d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cc5800640 1 -- 172.21.15.3:0/2213175096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 msgr2=0x7f7cc00d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc00d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cc5800640 1 -- 172.21.15.3:0/2213175096 >> v1:172.21.15.3:6789/0 conn(0x7f7cc00d4ae0 legacy=0x7f7cc00d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cc5800640 1 -- 172.21.15.3:0/2213175096 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc00cf690 con 0x7f7cc0161b70 2024-09-10T07:51:43.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.837+0000 7f7cbfe00640 1 -- 172.21.15.3:0/2213175096 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278504806 0 0) 0x7f7cc00d3de0 con 0x7f7cc00d4ae0 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.838+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00d3320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f5e545fb69daf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.838+0000 7f7cbfe00640 1 -- 172.21.15.3:0/2213175096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037070 con 0x7f7cc0161b70 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.838+0000 7f7cbfe00640 1 -- 172.21.15.3:0/2213175096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb4032440 con 0x7f7cc0161b70 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.838+0000 7f7cbfe00640 1 -- 172.21.15.3:0/2213175096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4032810 con 0x7f7cc0161b70 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 -- 172.21.15.3:0/2213175096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 msgr2=0x7f7cc00d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00d3320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb402a270 comp rx=0 tx=0).stop 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 -- 172.21.15.3:0/2213175096 shutdown_connections 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc00d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/2213175096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00d3320 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 -- 172.21.15.3:0/2213175096 >> 172.21.15.3:0/2213175096 conn(0x7f7cc0062630 msgr2=0x7f7cc0062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 -- 172.21.15.3:0/2213175096 shutdown_connections 2024-09-10T07:51:43.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.839+0000 7f7cc6c00640 1 -- 172.21.15.3:0/2213175096 wait complete. 2024-09-10T07:51:43.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.840+0000 7f7cc6c00640 1 Processor -- start 2024-09-10T07:51:43.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.840+0000 7f7cc6c00640 1 -- start start 2024-09-10T07:51:43.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc0154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cc00d4ae0 0x7f7cc00b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00cf2d0 con 0x7f7cc0161b70 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00c9fc0 con 0x7f7cc00d4ae0 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.841+0000 7f7cc6c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc00cf450 con 0x7f7cc00d3860 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47710/0 (socket says 172.21.15.3:47710) 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 -- 172.21.15.3:0/1951266107 learned_addr learned my addr 172.21.15.3:0/1951266107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cc00d4ae0 0x7f7cc00b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc5800640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc0154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 -- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 msgr2=0x7f7cc0154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc0154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 -- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cc00d4ae0 msgr2=0x7f7cc00b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cc00d4ae0 0x7f7cc00b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 -- 172.21.15.3:0/1951266107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc00b7060 con 0x7f7cc0161b70 2024-09-10T07:51:43.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.842+0000 7f7cc6200640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c7a0 tx=0x7f7cb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.843+0000 7f7cbd600640 1 -- 172.21.15.3:0/1951266107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8018070 con 0x7f7cc0161b70 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.843+0000 7f7cbd600640 1 -- 172.21.15.3:0/1951266107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb800a5e0 con 0x7f7cc0161b70 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.843+0000 7f7cbd600640 1 -- 172.21.15.3:0/1951266107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800aa50 con 0x7f7cc0161b70 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.843+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc015b3d0 con 0x7f7cc0161b70 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.843+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc015b990 con 0x7f7cc0161b70 2024-09-10T07:51:43.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.845+0000 7f7cbd600640 1 -- 172.21.15.3:0/1951266107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cb800abf0 con 0x7f7cc0161b70 2024-09-10T07:51:43.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.846+0000 7f7cbd600640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 0x7f7ca006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.846+0000 7f7cbd600640 1 -- 172.21.15.3:0/1951266107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7cb808a3b0 con 0x7f7cc0161b70 2024-09-10T07:51:43.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.846+0000 7f7cc5800640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 0x7f7ca006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.846+0000 7f7ccb248740 1 Processor -- start 2024-09-10T07:51:43.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7cc5800640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 0x7f7ca006e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4034000 tx=0x7f7cb402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 -- start start 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3092169e0 0x55d309215240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d3092613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d30917e660 con 0x55d3092169e0 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d309126680 con 0x55d309260ea0 2024-09-10T07:51:43.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.847+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d30917a620 con 0x55d3092304f0 2024-09-10T07:51:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3092169e0 0x55d309215240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:36424/0 (socket says 172.21.15.3:36424) 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 -- 172.21.15.3:0/339300345 learned_addr learned my addr 172.21.15.3:0/339300345 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc6200640 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d3092613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 -- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 msgr2=0x55d3092613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d3092613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 -- 172.21.15.3:0/339300345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3092169e0 msgr2=0x55d309215240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3092169e0 0x55d309215240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 -- 172.21.15.3:0/339300345 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d308f5b7d0 con 0x55d3092304f0 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.848+0000 7f7cc5800640 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009960 tx=0x7f7cb402c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee7028d4518ec98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7cb2000640 1 -- 172.21.15.3:0/339300345 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037040 con 0x55d3092304f0 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7cb2000640 1 -- 172.21.15.3:0/339300345 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb404e070 con 0x55d3092304f0 2024-09-10T07:51:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7cb2000640 1 -- 172.21.15.3:0/339300345 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4003b70 con 0x55d3092304f0 2024-09-10T07:51:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 -- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 msgr2=0x55d3092308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009960 tx=0x7f7cb402c000 comp rx=0 tx=0).stop 2024-09-10T07:51:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 -- 172.21.15.3:0/339300345 shutdown_connections 2024-09-10T07:51:43.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d3092613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d3092169e0 0x55d309215240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 --2- 172.21.15.3:0/339300345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092308d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.849+0000 7f7ccb248740 1 -- 172.21.15.3:0/339300345 >> 172.21.15.3:0/339300345 conn(0x55d309217410 msgr2=0x55d30925fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.850+0000 7f7ccb248740 1 -- 172.21.15.3:0/339300345 shutdown_connections 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.850+0000 7f7ccb248740 1 -- 172.21.15.3:0/339300345 wait complete. 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.850+0000 7f7ccb248740 1 -- start start 2024-09-10T07:51:43.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.850+0000 7f7ccb248740 1 Processor -- start 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.851+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.851+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d30925ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7ccb248740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d309215940 con 0x55d30922fee0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d3091074c0 con 0x55d309260ea0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d309215c20 con 0x55d3092304f0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7ccb248740 10 client.0 _ll_register_callbacks cb 0x55d309176e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc4e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d30925ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:36428/0 (socket says 172.21.15.3:36428) 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 -- 172.21.15.3:0/2037012184 learned_addr learned my addr 172.21.15.3:0/2037012184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc4e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d30925ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:43786/0 (socket says 172.21.15.3:43786) 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47726/0 (socket says 172.21.15.3:47726) 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 -- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 msgr2=0x55d30925ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d30925ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 -- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 msgr2=0x55d30925c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc5800640 1 -- 172.21.15.3:0/2037012184 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55d308e913f0 con 0x55d3092304f0 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.852+0000 7f7cc6200640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:43.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009630 tx=0x7f7cb4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f96a85639bd9db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037040 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7cb403f6e0 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb404e070 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7cb4003f10 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ccb248740 10 client.6128 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.853+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55d308e510f0 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb404e070 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f7cb403f080 con 0x55d3092304f0 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 10 client.6128 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 10 client.6128 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 20 client.6128 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d309257bc0 0x55d309257fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 4 client.6128 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30918bd90 con 0x55d309257bc0 2024-09-10T07:51:43.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.854+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d309257bc0 0x55d309257fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.855+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d309257bc0 0x55d309257fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfc1f95b50af4c8c server_cookie=2646c9bec9b37720 in_seq=0 out_seq=0 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.855+0000 7f7ca6a00640 10 client.6128 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.855+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2037012184 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d30918bd90 con 0x55d309257bc0 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.855+0000 7f7ca6a00640 10 client.6128 handle_command_reply: tid=0 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.855+0000 7f7ccb248740 1 client.6128 shutdown 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 msgr2=0x55d30922f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009630 tx=0x7f7cb4004730 comp rx=0 tx=0).stop 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 shutdown_connections 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d309257bc0 0x55d309257fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d30925c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d309260ea0 0x55d30925ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2037012184 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d30922f9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ca6a00640 0 client.6128 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 >> 172.21.15.3:0/2037012184 conn(0x55d3092169e0 msgr2=0x55d309262f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 shutdown_connections 2024-09-10T07:51:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 1 -- 172.21.15.3:0/2037012184 wait complete. 2024-09-10T07:51:43.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.856+0000 7f7ccb248740 20 client.6128 trim_cache size 0 max 16384 2024-09-10T07:51:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.871+0000 7f7ccb248740 1 Processor -- start 2024-09-10T07:51:43.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.871+0000 7f7ccb248740 1 -- start start 2024-09-10T07:51:43.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3092304f0 0x55d309263050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d309260ea0 0x55d309214140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc02008c0 con 0x55d30922fee0 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01ffae0 con 0x55d3092304f0 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc0200190 con 0x55d309260ea0 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47732/0 (socket says 172.21.15.3:47732) 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7cc5800640 1 -- 172.21.15.3:0/2732562294 learned_addr learned my addr 172.21.15.3:0/2732562294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3092304f0 0x55d309263050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.872+0000 7f7cc6200640 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d309260ea0 0x55d309214140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 -- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d309260ea0 msgr2=0x55d309214140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d309260ea0 0x55d309214140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 -- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3092304f0 msgr2=0x55d309263050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3092304f0 0x55d309263050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 -- 172.21.15.3:0/2732562294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3092181a0 con 0x55d30922fee0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009510 tx=0x7f7cb400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae23c43aa3ca54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cb1600640 1 -- 172.21.15.3:0/2732562294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb403f080 con 0x55d30922fee0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cb1600640 1 -- 172.21.15.3:0/2732562294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb404e070 con 0x55d30922fee0 2024-09-10T07:51:43.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.873+0000 7f7cb1600640 1 -- 172.21.15.3:0/2732562294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb403f930 con 0x55d30922fee0 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- 172.21.15.3:0/2732562294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 msgr2=0x55d309244720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009510 tx=0x7f7cb400b050 comp rx=0 tx=0).stop 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- 172.21.15.3:0/2732562294 shutdown_connections 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d309260ea0 0x55d309214140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d3092304f0 0x55d309263050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2732562294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30922fee0 0x55d309244720 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- 172.21.15.3:0/2732562294 >> 172.21.15.3:0/2732562294 conn(0x55d3092169e0 msgr2=0x55d30925be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- 172.21.15.3:0/2732562294 shutdown_connections 2024-09-10T07:51:43.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- 172.21.15.3:0/2732562294 wait complete. 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.874+0000 7f7ccb248740 1 -- start start 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.875+0000 7f7ccb248740 1 Processor -- start 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.875+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 0x55d309243bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d30925d210 con 0x55d309260ea0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00b5220 con 0x55d30922fee0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d30921b250 con 0x55d3092304f0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ccb248740 10 client.0 _ll_register_callbacks cb 0x7f7cac009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc5800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 0x55d309243bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47736/0 (socket says 172.21.15.3:47736) 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 -- 172.21.15.3:0/2932400248 learned_addr learned my addr 172.21.15.3:0/2932400248 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:43.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc4e00640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 -- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 msgr2=0x55d3092dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 -- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 msgr2=0x55d309243bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 0x55d309243bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 -- 172.21.15.3:0/2932400248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30926e8e0 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 0x55d309243bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7cc6200640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8058690 tx=0x7f7cb8058170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da9b29e8976760c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.876+0000 7f7ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8061230 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f7cb8057b50 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb404e070 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7cb8057d50 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 10 client.6963 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55d3092441a0 con 0x55d309260ea0 2024-09-10T07:51:43.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb404e070 con 0x55d309260ea0 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f7cb806cae0 con 0x55d309260ea0 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 10 client.6963 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 10 client.6963 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 20 client.6963 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d30922f900 0x55d309245610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 4 client.6963 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3091860e0 con 0x55d30922f900 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.877+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d30922f900 0x55d309245610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:43.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.878+0000 7f7cc5800640 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d30922f900 0x55d309245610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=591033978916d23f server_cookie=da3650df5f192f6d in_seq=0 out_seq=0 2024-09-10T07:51:43.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.878+0000 7f7ca6a00640 10 client.6963 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ca6a00640 1 -- 172.21.15.3:0/2932400248 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3091860e0 con 0x55d30922f900 2024-09-10T07:51:43.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ca6a00640 10 client.6963 handle_command_reply: tid=0 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 client.6963 shutdown 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 msgr2=0x55d30926dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8058690 tx=0x7f7cb8058170 comp rx=0 tx=0).stop 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 shutdown_connections 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55d30922f900 0x55d309245610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d309260ea0 0x55d30926dda0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d3092304f0 0x55d3092dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ca6a00640 0 client.6963 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 --2- 172.21.15.3:0/2932400248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d30922fee0 0x55d309243bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.887+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 >> 172.21.15.3:0/2932400248 conn(0x55d3092169e0 msgr2=0x55d309246310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 shutdown_connections 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7ccb248740 1 -- 172.21.15.3:0/2932400248 wait complete. 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7ccb248740 20 client.6963 trim_cache size 0 max 16384 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:51:43.889 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 msgr2=0x7f7ca006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 0x7f7ca006e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4034000 tx=0x7f7cb402e000 comp rx=0 tx=0).stop 2024-09-10T07:51:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 msgr2=0x7f7cc00b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c7a0 tx=0x7f7cb800cc60 comp rx=0 tx=0).stop 2024-09-10T07:51:43.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 shutdown_connections 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7ca006be30 0x7f7ca006e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cc0161b70 0x7f7cc00b93d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cc00d4ae0 0x7f7cc00b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.888+0000 7f7cc6c00640 1 --2- 172.21.15.3:0/1951266107 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7cc00d3860 0x7f7cc0154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.889+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 >> 172.21.15.3:0/1951266107 conn(0x7f7cc0062630 msgr2=0x7f7cc00b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.889+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 shutdown_connections 2024-09-10T07:51:43.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:43.889+0000 7f7cc6c00640 1 -- 172.21.15.3:0/1951266107 wait complete. 2024-09-10T07:51:43.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2024-09-10T07:51:44.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.062+0000 7f8243e00640 1 Processor -- start 2024-09-10T07:51:44.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.063+0000 7f8243e00640 1 -- start start 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8243e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c1451d0 0x7f823c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8243e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f823c0cf270 0x7f823c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8243e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c0cd900 con 0x7f823c145730 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8243e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f823c06d7c0 con 0x7f823c0cf270 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8243e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f823c0cdbe0 con 0x7f823c1451d0 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8241e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f823c0cf270 0x7f823c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8241e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f823c0cf270 0x7f823c0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47746/0 (socket says 172.21.15.3:47746) 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8241e00640 1 -- 172.21.15.3:0/4204732632 learned_addr learned my addr 172.21.15.3:0/4204732632 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.064+0000 7f8241400640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c1451d0 0x7f823c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.065+0000 7f8241e00640 1 -- 172.21.15.3:0/4204732632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c1451d0 msgr2=0x7f823c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.065+0000 7f8241e00640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c1451d0 0x7f823c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.065+0000 7f8241e00640 1 -- 172.21.15.3:0/4204732632 >> v1:172.21.15.70:6789/0 conn(0x7f823c145730 legacy=0x7f823c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.065+0000 7f8241e00640 1 -- 172.21.15.3:0/4204732632 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c06da10 con 0x7f823c0cf270 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.065+0000 7f823be00640 1 -- 172.21.15.3:0/4204732632 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787875854 0 0) 0x7f823c0cd900 con 0x7f823c145730 2024-09-10T07:51:44.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8241e00640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c0cf270 0x7f823c0cf720 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8234009e50 tx=0x7f823402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b0f482c84776cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f823be00640 1 -- 172.21.15.3:0/4204732632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234036070 con 0x7f823c0cf270 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f823be00640 1 -- 172.21.15.3:0/4204732632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8234031600 con 0x7f823c0cf270 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f823be00640 1 -- 172.21.15.3:0/4204732632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82340319d0 con 0x7f823c0cf270 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 -- 172.21.15.3:0/4204732632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c0cf270 msgr2=0x7f823c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c0cf270 0x7f823c0cf720 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8234009e50 tx=0x7f823402a680 comp rx=0 tx=0).stop 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 -- 172.21.15.3:0/4204732632 shutdown_connections 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c0cf270 0x7f823c0cf720 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 --2- 172.21.15.3:0/4204732632 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c1451d0 0x7f823c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.066+0000 7f8243e00640 1 -- 172.21.15.3:0/4204732632 >> 172.21.15.3:0/4204732632 conn(0x7f823c0b5cb0 msgr2=0x7f823c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.067+0000 7f8243e00640 1 -- 172.21.15.3:0/4204732632 shutdown_connections 2024-09-10T07:51:44.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.067+0000 7f8243e00640 1 -- 172.21.15.3:0/4204732632 wait complete. 2024-09-10T07:51:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.068+0000 7f8243e00640 1 Processor -- start 2024-09-10T07:51:44.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.068+0000 7f8243e00640 1 -- start start 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f823c0cf270 0x7f823c1404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c145730 0x7f823c15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f823c06df30 con 0x7f823c1451d0 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f823c145f10 con 0x7f823c0cf270 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8243e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f823c06e0b0 con 0x7f823c145730 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8240a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8240a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47750/0 (socket says 172.21.15.3:47750) 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.069+0000 7f8240a00640 1 -- 172.21.15.3:0/217942760 learned_addr learned my addr 172.21.15.3:0/217942760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8241400640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f823c0cf270 0x7f823c1404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8241e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c145730 0x7f823c15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 -- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c145730 msgr2=0x7f823c15f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c145730 0x7f823c15f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 -- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f823c0cf270 msgr2=0x7f823c1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f823c0cf270 0x7f823c1404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 -- 172.21.15.3:0/217942760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1f5130 con 0x7f823c1451d0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8240a00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f822800c410 tx=0x7f822800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8239600640 1 -- 172.21.15.3:0/217942760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228017070 con 0x7f823c1451d0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8239600640 1 -- 172.21.15.3:0/217942760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82280035f0 con 0x7f823c1451d0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8239600640 1 -- 172.21.15.3:0/217942760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228003a00 con 0x7f823c1451d0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.070+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c15fc30 con 0x7f823c1451d0 2024-09-10T07:51:44.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.071+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1f5690 con 0x7f823c1451d0 2024-09-10T07:51:44.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.073+0000 7f8239600640 1 -- 172.21.15.3:0/217942760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f822801a070 con 0x7f823c1451d0 2024-09-10T07:51:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.074+0000 7f8239600640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 0x7f821c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.074+0000 7f8239600640 1 -- 172.21.15.3:0/217942760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f822808a210 con 0x7f823c1451d0 2024-09-10T07:51:44.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.074+0000 7f8241400640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 0x7f821c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.074+0000 7f8246c6f740 1 Processor -- start 2024-09-10T07:51:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 -- start start 2024-09-10T07:51:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8241400640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 0x7f821c06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8230009ad0 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 0x55ab1c1fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 0x55ab1c1fe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab1c11b660 con 0x55ab1c1b3480 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ab1c0c3680 con 0x55ab1c1cd800 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.075+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab1c117620 con 0x55ab1c1cd420 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8241e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 0x55ab1c1fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8241400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 0x55ab1c1fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8241e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 0x55ab1c1fe740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47758/0 (socket says 172.21.15.3:47758) 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:43832/0 (socket says 172.21.15.3:43832) 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8241e00640 1 -- 172.21.15.3:0/882190374 learned_addr learned my addr 172.21.15.3:0/882190374 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 -- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 msgr2=0x55ab1c1fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 0x55ab1c1fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 -- 172.21.15.3:0/882190374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 msgr2=0x55ab1c1fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 0x55ab1c1fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 -- 172.21.15.3:0/882190374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1bef87d0 con 0x55ab1c1cd800 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8241400640 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 0x55ab1c1fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.076+0000 7f8240a00640 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8228088700 tx=0x7f8228058bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd8d04a4745169a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f822dc00640 1 -- 172.21.15.3:0/882190374 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822805c860 con 0x55ab1c1cd800 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f822dc00640 1 -- 172.21.15.3:0/882190374 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228061090 con 0x55ab1c1cd800 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f822dc00640 1 -- 172.21.15.3:0/882190374 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228054b20 con 0x55ab1c1cd800 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 -- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 msgr2=0x55ab1c1fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8228088700 tx=0x7f8228058bb0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 -- 172.21.15.3:0/882190374 shutdown_connections 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1b3480 0x55ab1c1fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd800 0x55ab1c1fe200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 --2- 172.21.15.3:0/882190374 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd420 0x55ab1c1fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.077+0000 7f8246c6f740 1 -- 172.21.15.3:0/882190374 >> 172.21.15.3:0/882190374 conn(0x55ab1c1b4220 msgr2=0x55ab1c1fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.078+0000 7f8246c6f740 1 -- 172.21.15.3:0/882190374 shutdown_connections 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.078+0000 7f8246c6f740 1 -- 172.21.15.3:0/882190374 wait complete. 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.078+0000 7f8246c6f740 1 -- start start 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.078+0000 7f8246c6f740 1 Processor -- start 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c1f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c1f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab1c1b23e0 con 0x55ab1c1cd420 2024-09-10T07:51:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ab1c0a44c0 con 0x55ab1c1f5090 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab1c1b26c0 con 0x55ab1c1cd800 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.079+0000 7f8246c6f740 10 client.0 _ll_register_callbacks cb 0x55ab1c113e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47762/0 (socket says 172.21.15.3:47762) 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 -- 172.21.15.3:0/3209177250 learned_addr learned my addr 172.21.15.3:0/3209177250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241e00640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c1f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8240a00640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c1f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 -- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 msgr2=0x55ab1c1f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c1f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 -- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 msgr2=0x55ab1c1f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c1f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 -- 172.21.15.3:0/3209177250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1be2e3f0 con 0x55ab1c1cd420 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8241400640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f82300157f0 tx=0x7f82300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec788f040c16ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.080+0000 7f8222000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230026070 con 0x55ab1c1cd420 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f8230014960 con 0x55ab1c1cd420 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8228061090 con 0x55ab1c1cd420 2024-09-10T07:51:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8230031050 con 0x55ab1c1cd420 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 10 client.6975 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ab1bdee0f0 con 0x55ab1c1cd420 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228061090 con 0x55ab1c1cd420 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f8230034020 con 0x55ab1c1cd420 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 10 client.6975 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 10 client.6975 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.b 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.081+0000 7f8246c6f740 20 client.6975 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c210890 0x55ab1c210cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8246c6f740 4 client.6975 mds_command: new command op to 6355 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 --> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab1c128d90 con 0x55ab1c210890 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8240a00640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c210890 0x55ab1c210cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8240a00640 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c210890 0x55ab1c210cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a11d6e9a221d78f8 server_cookie=740cd80a9e4591d1 in_seq=0 out_seq=0 2024-09-10T07:51:44.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.082+0000 7f8222000640 10 client.6975 ms_handle_connect on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8222000640 1 -- 172.21.15.3:0/3209177250 <== mds.2 v2:172.21.15.70:6833/2352570209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab1c128d90 con 0x55ab1c210890 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8222000640 10 client.6975 handle_command_reply: tid=0 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 client.6975 shutdown 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 msgr2=0x55ab1c20c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f82300157f0 tx=0x7f82300046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 shutdown_connections 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c210890 0x55ab1c210cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c1f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c1f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8222000640 0 client.6975 ms_handle_reset on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 --2- 172.21.15.3:0/3209177250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c20c890 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.083+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 >> 172.21.15.3:0/3209177250 conn(0x55ab1c1b3480 msgr2=0x55ab1c1b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.084+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 shutdown_connections 2024-09-10T07:51:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.084+0000 7f8246c6f740 1 -- 172.21.15.3:0/3209177250 wait complete. 2024-09-10T07:51:44.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.084+0000 7f8246c6f740 20 client.6975 trim_cache size 0 max 16384 2024-09-10T07:51:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.099+0000 7f8246c6f740 1 Processor -- start 2024-09-10T07:51:44.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.099+0000 7f8246c6f740 1 -- start start 2024-09-10T07:51:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd420 0x55ab1c221080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1f5090 0x55ab1c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f823c200bb0 con 0x55ab1c1cd800 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1f8a90 con 0x55ab1c1cd420 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f823c200550 con 0x55ab1c1f5090 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47776/0 (socket says 172.21.15.3:47776) 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 -- 172.21.15.3:0/2028941287 learned_addr learned my addr 172.21.15.3:0/2028941287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 -- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1f5090 msgr2=0x55ab1c1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1f5090 0x55ab1c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 -- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd420 msgr2=0x55ab1c221080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd420 0x55ab1c221080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.100+0000 7f8240a00640 1 -- 172.21.15.3:0/2028941287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1c1ae050 con 0x55ab1c1cd800 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8240a00640 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8228088700 tx=0x7f8228013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722c086289069154 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f822d200640 1 -- 172.21.15.3:0/2028941287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228085040 con 0x55ab1c1cd800 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f822d200640 1 -- 172.21.15.3:0/2028941287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228087040 con 0x55ab1c1cd800 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f822d200640 1 -- 172.21.15.3:0/2028941287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228088260 con 0x55ab1c1cd800 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 -- 172.21.15.3:0/2028941287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 msgr2=0x55ab1c1e23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8228088700 tx=0x7f8228013000 comp rx=0 tx=0).stop 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 -- 172.21.15.3:0/2028941287 shutdown_connections 2024-09-10T07:51:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1f5090 0x55ab1c1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd800 0x55ab1c1e23d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 --2- 172.21.15.3:0/2028941287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1cd420 0x55ab1c221080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 -- 172.21.15.3:0/2028941287 >> 172.21.15.3:0/2028941287 conn(0x55ab1c1b3480 msgr2=0x55ab1c1f5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 -- 172.21.15.3:0/2028941287 shutdown_connections 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.101+0000 7f8246c6f740 1 -- 172.21.15.3:0/2028941287 wait complete. 2024-09-10T07:51:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.102+0000 7f8246c6f740 1 -- start start 2024-09-10T07:51:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.102+0000 7f8246c6f740 1 Processor -- start 2024-09-10T07:51:44.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c263170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c2709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab1c1f3b60 con 0x55ab1c1cd420 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0b2000 con 0x55ab1c1f5090 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ab1c1f2b70 con 0x55ab1c1cd800 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8246c6f740 10 client.0 _ll_register_callbacks cb 0x7f82300119c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8241400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8241400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47786/0 (socket says 172.21.15.3:47786) 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.103+0000 7f8241400640 1 -- 172.21.15.3:0/1217154732 learned_addr learned my addr 172.21.15.3:0/1217154732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8240a00640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c263170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241e00640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c2709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 -- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 msgr2=0x55ab1c263170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c263170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 -- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 msgr2=0x55ab1c2709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c2709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 -- 172.21.15.3:0/1217154732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1c2636b0 con 0x55ab1c1cd420 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8241400640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8230000c00 tx=0x7f8230009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e8a6486e5790f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8222000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:44.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230039020 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.104+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f823002b650 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8228087040 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8230026070 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 10 client.6981 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ab1c21fcb0 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228087040 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f8230046020 con 0x55ab1c1cd420 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 10 client.6981 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 10 client.6981 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.b 2024-09-10T07:51:44.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 20 client.6981 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c21f870 0x55ab1c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 4 client.6981 mds_command: new command op to 6355 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 --> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab1c1230e0 con 0x55ab1c21f870 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.105+0000 7f8240a00640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c21f870 0x55ab1c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.106+0000 7f8240a00640 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c21f870 0x55ab1c1f4500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88ea4fef7aa501b7 server_cookie=a907081cf531e9f4 in_seq=0 out_seq=0 2024-09-10T07:51:44.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.106+0000 7f8222000640 10 client.6981 ms_handle_connect on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:44.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8222000640 1 -- 172.21.15.3:0/1217154732 <== mds.2 v2:172.21.15.70:6833/2352570209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab1c1230e0 con 0x55ab1c21f870 2024-09-10T07:51:44.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8222000640 10 client.6981 handle_command_reply: tid=0 2024-09-10T07:51:44.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8246c6f740 1 client.6981 shutdown 2024-09-10T07:51:44.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 msgr2=0x55ab1c1b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8230000c00 tx=0x7f8230009190 comp rx=0 tx=0).stop 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 shutdown_connections 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.114+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:6833/2352570209,v1:172.21.15.70:6835/2352570209] conn(0x55ab1c21f870 0x55ab1c1f4500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab1c1f5090 0x55ab1c2709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ab1c1cd800 0x55ab1c263170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 --2- 172.21.15.3:0/1217154732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab1c1cd420 0x55ab1c1b1040 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8222000640 0 client.6981 ms_handle_reset on v2:172.21.15.70:6833/2352570209 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 >> 172.21.15.3:0/1217154732 conn(0x55ab1c1b3480 msgr2=0x55ab1c1f5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 shutdown_connections 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 1 -- 172.21.15.3:0/1217154732 wait complete. 2024-09-10T07:51:44.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8246c6f740 20 client.6981 trim_cache size 0 max 16384 2024-09-10T07:51:44.119 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:44.119 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:51:44.119 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:51:44.119 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 msgr2=0x7f821c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 0x7f821c06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8230009ad0 tx=0x7f8230008040 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 msgr2=0x7f823c1f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.115+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f822800c410 tx=0x7f822800c8d0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 shutdown_connections 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f821c06be30 0x7f821c06e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f823c145730 0x7f823c15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f823c1451d0 0x7f823c1f4bf0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 --2- 172.21.15.3:0/217942760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f823c0cf270 0x7f823c1404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 >> 172.21.15.3:0/217942760 conn(0x7f823c0b5cb0 msgr2=0x7f823c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 shutdown_connections 2024-09-10T07:51:44.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.116+0000 7f8243e00640 1 -- 172.21.15.3:0/217942760 wait complete. 2024-09-10T07:51:44.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.286+0000 7fc442e00640 1 Processor -- start 2024-09-10T07:51:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.286+0000 7fc442e00640 1 -- start start 2024-09-10T07:51:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.287+0000 7fc442e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc43c15b110 0x7fc43c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.287+0000 7fc442e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.287+0000 7fc442e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c0b4250 con 0x7fc43c073d10 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.287+0000 7fc442e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc43c0b44f0 con 0x7fc43c15b110 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.287+0000 7fc442e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c0ccd80 con 0x7fc43c15b4f0 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc440e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc43c073d10 0x7fc43c15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34616/0 (socket says 172.21.15.3:34616) 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc440e00640 1 -- 172.21.15.3:0/2281450814 learned_addr learned my addr 172.21.15.3:0/2281450814 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43be00640 1 --2- 172.21.15.3:0/2281450814 >> v2:172.21.15.70:3300/0 conn(0x7fc43c15b110 0x7fc43c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc433e00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43b400640 1 -- 172.21.15.3:0/2281450814 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580024647 0 0) 0x7fc43c0b4250 con 0x7fc43c073d10 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43b400640 1 -- 172.21.15.3:0/2281450814 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc420003520 con 0x7fc43c073d10 2024-09-10T07:51:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43be00640 1 -- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 msgr2=0x7fc43c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43be00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43be00640 1 -- 172.21.15.3:0/2281450814 >> v1:172.21.15.3:6789/0 conn(0x7fc43c073d10 legacy=0x7fc43c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc43be00640 1 -- 172.21.15.3:0/2281450814 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c0cd060 con 0x7fc43c15b110 2024-09-10T07:51:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.288+0000 7fc433e00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc43be00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c15b110 0x7fc43c0b3d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0097a0 tx=0x7fc42c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef80be601cbdc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc43b400640 1 -- 172.21.15.3:0/2281450814 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c037070 con 0x7fc43c15b110 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc43b400640 1 -- 172.21.15.3:0/2281450814 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c032440 con 0x7fc43c15b110 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc43b400640 1 -- 172.21.15.3:0/2281450814 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c032810 con 0x7fc43c15b110 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc442e00640 1 -- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c15b110 msgr2=0x7fc43c0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc442e00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c15b110 0x7fc43c0b3d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0097a0 tx=0x7fc42c02a180 comp rx=0 tx=0).stop 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.290+0000 7fc442e00640 1 -- 172.21.15.3:0/2281450814 shutdown_connections 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.291+0000 7fc442e00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.291+0000 7fc442e00640 1 --2- 172.21.15.3:0/2281450814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c15b110 0x7fc43c0b3d10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.291+0000 7fc442e00640 1 -- 172.21.15.3:0/2281450814 >> 172.21.15.3:0/2281450814 conn(0x7fc43c06d000 msgr2=0x7fc43c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.291+0000 7fc442e00640 1 -- 172.21.15.3:0/2281450814 shutdown_connections 2024-09-10T07:51:44.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.291+0000 7fc442e00640 1 -- 172.21.15.3:0/2281450814 wait complete. 2024-09-10T07:51:44.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.292+0000 7fc442e00640 1 Processor -- start 2024-09-10T07:51:44.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.292+0000 7fc442e00640 1 -- start start 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 0x7fc43c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c1e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c0ceb50 con 0x7fc43c15b110 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c0cd620 con 0x7fc43c073d10 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc442e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c0cecd0 con 0x7fc43c15b4f0 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.293+0000 7fc433e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47800/0 (socket says 172.21.15.3:47800) 2024-09-10T07:51:44.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 -- 172.21.15.3:0/156620890 learned_addr learned my addr 172.21.15.3:0/156620890 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc43be00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 0x7fc43c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc440e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c1e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 -- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 msgr2=0x7fc43c1e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c1e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 -- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 msgr2=0x7fc43c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 0x7fc43c1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 -- 172.21.15.3:0/156620890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c1ea140 con 0x7fc43c15b110 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc43be00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 0x7fc43c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.294+0000 7fc433e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc42400e990 tx=0x7fc42400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.295+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424017070 con 0x7fc43c15b110 2024-09-10T07:51:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.295+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42400c9e0 con 0x7fc43c15b110 2024-09-10T07:51:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.295+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42400cdb0 con 0x7fc43c15b110 2024-09-10T07:51:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.295+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c1e7f10 con 0x7fc43c15b110 2024-09-10T07:51:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.295+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c1e89f0 con 0x7fc43c15b110 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.296+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c1e7d00 con 0x7fc43c15b110 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.297+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc42401a070 con 0x7fc43c15b110 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.298+0000 7fc438c00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 0x7fc41806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.298+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fc42408aff0 con 0x7fc43c15b110 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.298+0000 7fc43be00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 0x7fc41806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.298+0000 7fc43be00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 0x7fc41806e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc42c034000 tx=0x7fc42c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.302+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc424059310 con 0x7fc43c15b110 2024-09-10T07:51:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.526+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc43c0b3d10 con 0x7fc43c15b110 2024-09-10T07:51:44.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.526+0000 7fc438c00640 1 -- 172.21.15.3:0/156620890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5261 (secure 0 0 0) 0x7fc42405d610 con 0x7fc43c15b110 2024-09-10T07:51:44.526 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:44.526 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":76,"btime":"2024-09-10T07:51:41:614763+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:41.568595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6355,6690,6702]},"id":5}]} 2024-09-10T07:51:44.526 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 76 2024-09-10T07:51:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 msgr2=0x7fc41806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 0x7fc41806e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc42c034000 tx=0x7fc42c02e000 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 msgr2=0x7fc43c1e95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc42400e990 tx=0x7fc42400ee50 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 shutdown_connections 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc41806be60 0x7fc41806e320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc43c15b4f0 0x7fc43c1e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc43c15b110 0x7fc43c1e95a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 --2- 172.21.15.3:0/156620890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc43c073d10 0x7fc43c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 >> 172.21.15.3:0/156620890 conn(0x7fc43c06d000 msgr2=0x7fc43c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 shutdown_connections 2024-09-10T07:51:44.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.529+0000 7fc442e00640 1 -- 172.21.15.3:0/156620890 wait complete. 2024-09-10T07:51:44.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 2024-09-10T07:51:44.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.707+0000 7fac96200640 1 Processor -- start 2024-09-10T07:51:44.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.708+0000 7fac96200640 1 -- start start 2024-09-10T07:51:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac96200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac90145730 0x7fac9006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac96200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac900cf270 0x7fac900cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac96200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac900cd900 con 0x7fac901451d0 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac96200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fac9006d7c0 con 0x7fac90145730 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac96200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fac900cdbe0 con 0x7fac900cf270 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac8f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac901451d0 0x7fac900b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34620/0 (socket says 172.21.15.3:34620) 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac8f400640 1 -- 172.21.15.3:0/2911606016 learned_addr learned my addr 172.21.15.3:0/2911606016 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac87e00640 1 --2- 172.21.15.3:0/2911606016 >> v2:172.21.15.3:3300/0 conn(0x7fac90145730 0x7fac9006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac8fe00640 1 --2- 172.21.15.3:0/2911606016 >> v2:172.21.15.70:3300/0 conn(0x7fac900cf270 0x7fac900cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.709+0000 7fac8ea00640 1 -- 172.21.15.3:0/2911606016 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3729049161 0 0) 0x7fac900cd900 con 0x7fac901451d0 2024-09-10T07:51:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.710+0000 7fac8ea00640 1 -- 172.21.15.3:0/2911606016 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac74003520 con 0x7fac901451d0 2024-09-10T07:51:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.710+0000 7fac87e00640 1 -- 172.21.15.3:0/2911606016 >> v2:172.21.15.70:3300/0 conn(0x7fac900cf270 msgr2=0x7fac900cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.710+0000 7fac87e00640 1 --2- 172.21.15.3:0/2911606016 >> v2:172.21.15.70:3300/0 conn(0x7fac900cf270 0x7fac900cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.710+0000 7fac87e00640 1 -- 172.21.15.3:0/2911606016 >> v1:172.21.15.3:6789/0 conn(0x7fac901451d0 legacy=0x7fac900b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.710+0000 7fac87e00640 1 -- 172.21.15.3:0/2911606016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9006da10 con 0x7fac90145730 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac87e00640 1 --2- 172.21.15.3:0/2911606016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac9006cfc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fac900b5370 tx=0x7fac7802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb690d0b6e455ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac8ea00640 1 -- 172.21.15.3:0/2911606016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037070 con 0x7fac90145730 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac8ea00640 1 -- 172.21.15.3:0/2911606016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78032440 con 0x7fac90145730 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac8ea00640 1 -- 172.21.15.3:0/2911606016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7802ae60 con 0x7fac90145730 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac96200640 1 -- 172.21.15.3:0/2911606016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 msgr2=0x7fac9006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.711+0000 7fac96200640 1 --2- 172.21.15.3:0/2911606016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac9006cfc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fac900b5370 tx=0x7fac7802a110 comp rx=0 tx=0).stop 2024-09-10T07:51:44.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 -- 172.21.15.3:0/2911606016 shutdown_connections 2024-09-10T07:51:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 --2- 172.21.15.3:0/2911606016 >> v2:172.21.15.70:3300/0 conn(0x7fac900cf270 0x7fac900cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 --2- 172.21.15.3:0/2911606016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac9006cfc0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 -- 172.21.15.3:0/2911606016 >> 172.21.15.3:0/2911606016 conn(0x7fac900b5cb0 msgr2=0x7fac9006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 -- 172.21.15.3:0/2911606016 shutdown_connections 2024-09-10T07:51:44.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.712+0000 7fac96200640 1 -- 172.21.15.3:0/2911606016 wait complete. 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.713+0000 7fac96200640 1 Processor -- start 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.713+0000 7fac96200640 1 -- start start 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac900cf270 0x7fac900b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac901451d0 0x7fac900b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac901f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac9006df30 con 0x7fac90145730 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fac900cfc10 con 0x7fac900cf270 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.714+0000 7fac96200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac9006e0b0 con 0x7fac901451d0 2024-09-10T07:51:44.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac901f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac901f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47820/0 (socket says 172.21.15.3:47820) 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 -- 172.21.15.3:0/1101698895 learned_addr learned my addr 172.21.15.3:0/1101698895 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8f400640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac900cf270 0x7fac900b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac87e00640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac901451d0 0x7fac900b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 -- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac901451d0 msgr2=0x7fac900b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fac901451d0 0x7fac900b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 -- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac900cf270 msgr2=0x7fac900b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac900cf270 0x7fac900b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 -- 172.21.15.3:0/1101698895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac901f5030 con 0x7fac90145730 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.715+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac90145730 0x7fac901f4af0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fac8800efc0 tx=0x7fac8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.716+0000 7fac87400640 1 -- 172.21.15.3:0/1101698895 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88018020 con 0x7fac90145730 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.716+0000 7fac87400640 1 -- 172.21.15.3:0/1101698895 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac88014480 con 0x7fac90145730 2024-09-10T07:51:44.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.716+0000 7fac87400640 1 -- 172.21.15.3:0/1101698895 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88014870 con 0x7fac90145730 2024-09-10T07:51:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.716+0000 7fac96200640 1 -- 172.21.15.3:0/1101698895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac900b15e0 con 0x7fac90145730 2024-09-10T07:51:44.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.716+0000 7fac96200640 1 -- 172.21.15.3:0/1101698895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac900b1ba0 con 0x7fac90145730 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.719+0000 7fac87400640 1 -- 172.21.15.3:0/1101698895 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac88005d10 con 0x7fac90145730 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.720+0000 7fac87400640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fac6c06be30 0x7fac6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.720+0000 7fac87400640 1 -- 172.21.15.3:0/1101698895 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fac8808b810 con 0x7fac90145730 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.720+0000 7fac8f400640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fac6c06be30 0x7fac6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.720+0000 7fac99000740 1 Processor -- start 2024-09-10T07:51:44.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.720+0000 7fac99000740 1 -- start start 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac8f400640 1 --2- 172.21.15.3:0/1101698895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fac6c06be30 0x7fac6c06e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac80005880 tx=0x7fac80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 0x555c433dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 0x555c43391080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c432fa8c0 con 0x555c43394250 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555c432a2820 con 0x555c43396a90 2024-09-10T07:51:44.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.721+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555c432f6880 con 0x555c43398220 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47836/0 (socket says 172.21.15.3:47836) 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 -- 172.21.15.3:0/105648256 learned_addr learned my addr 172.21.15.3:0/105648256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac8f400640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 0x555c433dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac8fe00640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 0x555c43391080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 -- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 msgr2=0x555c43391080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 0x555c43391080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 -- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 msgr2=0x555c433dc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 0x555c433dc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 -- 172.21.15.3:0/105648256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c42ff1280 con 0x555c43394250 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac8fe00640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 0x555c43391080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac8f400640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 0x555c433dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:44.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.722+0000 7fac87e00640 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fac78009860 tx=0x7fac7802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4357486a214a7850 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac7f400640 1 -- 172.21.15.3:0/105648256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78043070 con 0x555c43394250 2024-09-10T07:51:44.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac7f400640 1 -- 172.21.15.3:0/105648256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7803c700 con 0x555c43394250 2024-09-10T07:51:44.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac7f400640 1 -- 172.21.15.3:0/105648256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037040 con 0x555c43394250 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 -- 172.21.15.3:0/105648256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 msgr2=0x555c43396e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fac78009860 tx=0x7fac7802c000 comp rx=0 tx=0).stop 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 -- 172.21.15.3:0/105648256 shutdown_connections 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43398220 0x555c43391080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43394250 0x555c43396e70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 --2- 172.21.15.3:0/105648256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43396a90 0x555c433dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 -- 172.21.15.3:0/105648256 >> 172.21.15.3:0/105648256 conn(0x555c43393250 msgr2=0x555c433db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 -- 172.21.15.3:0/105648256 shutdown_connections 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.723+0000 7fac99000740 1 -- 172.21.15.3:0/105648256 wait complete. 2024-09-10T07:51:44.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.724+0000 7fac99000740 1 -- start start 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.724+0000 7fac99000740 1 Processor -- start 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c433b2860 0x555c433df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c433926f0 con 0x555c43396a90 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555c432b5a40 con 0x555c43398220 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555c433dca60 con 0x555c433b2860 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac99000740 10 client.0 _ll_register_callbacks cb 0x555c432a7520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.725+0000 7fac8f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47848/0 (socket says 172.21.15.3:47848) 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 -- 172.21.15.3:0/1532821118 learned_addr learned my addr 172.21.15.3:0/1532821118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac87e00640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c433b2860 0x555c433df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 -- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c433b2860 msgr2=0x555c433df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c433b2860 0x555c433df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 -- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 msgr2=0x555c433dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 -- 172.21.15.3:0/1532821118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c430d9690 con 0x555c43396a90 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac8f400640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fac80015790 tx=0x7fac800010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b764ae190833e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.726+0000 7fac73e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80013700 con 0x555c43396a90 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7803c700 con 0x555c43396a90 2024-09-10T07:51:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fac80010040 con 0x555c43396a90 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fac8002e050 con 0x555c43396a90 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 10 client.7002 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x555c43139e20 con 0x555c43396a90 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7803c700 con 0x555c43396a90 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7fac80031020 con 0x555c43396a90 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 10 client.7002 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 10 client.7002 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 20 client.7002 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c433b63d0 0x555c433c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 4 client.7002 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.727+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c43307ff0 con 0x555c433b63d0 2024-09-10T07:51:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.728+0000 7fac87e00640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c433b63d0 0x555c433c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.728+0000 7fac87e00640 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c433b63d0 0x555c433c0510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e078cb3332e98e21 server_cookie=786d8536dad56aac in_seq=0 out_seq=0 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.728+0000 7fac73e00640 10 client.7002 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac73e00640 1 -- 172.21.15.3:0/1532821118 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c43307ff0 con 0x555c433b63d0 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac73e00640 10 client.7002 handle_command_reply: tid=0 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 client.7002 shutdown 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 msgr2=0x555c433b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fac80015790 tx=0x7fac800010a0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 shutdown_connections 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c433b63d0 0x555c433c0510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c433b2860 0x555c433df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 --2- 172.21.15.3:0/1532821118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43396a90 0x555c433b2150 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 >> 172.21.15.3:0/1532821118 conn(0x555c43394250 msgr2=0x555c433d96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac73e00640 0 client.7002 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 shutdown_connections 2024-09-10T07:51:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 1 -- 172.21.15.3:0/1532821118 wait complete. 2024-09-10T07:51:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.729+0000 7fac99000740 20 client.7002 trim_cache size 0 max 16384 2024-09-10T07:51:44.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 Processor -- start 2024-09-10T07:51:44.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 -- start start 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c433b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 0x555c433b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac901f7810 con 0x555c43398220 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fac901f7f10 con 0x555c433b2860 2024-09-10T07:51:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.748+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fac900ab470 con 0x555c43396a90 2024-09-10T07:51:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47858/0 (socket says 172.21.15.3:47858) 2024-09-10T07:51:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 -- 172.21.15.3:0/429731061 learned_addr learned my addr 172.21.15.3:0/429731061 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 -- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 msgr2=0x555c433b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c433b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac8fe00640 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 0x555c433b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 -- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 msgr2=0x555c433b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 0x555c433b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 -- 172.21.15.3:0/429731061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c42fccf80 con 0x555c43398220 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac8fe00640 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 0x555c433b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac87e00640 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fac7802a8c0 tx=0x7fac78008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed03f13687966282 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.749+0000 7fac7ea00640 1 -- 172.21.15.3:0/429731061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7804c070 con 0x555c43398220 2024-09-10T07:51:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac7ea00640 1 -- 172.21.15.3:0/429731061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78004360 con 0x555c43398220 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac7ea00640 1 -- 172.21.15.3:0/429731061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037040 con 0x555c43398220 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 -- 172.21.15.3:0/429731061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 msgr2=0x555c433b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fac7802a8c0 tx=0x7fac78008030 comp rx=0 tx=0).stop 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 -- 172.21.15.3:0/429731061 shutdown_connections 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c433b2860 0x555c433b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c43398220 0x555c433b7260 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 --2- 172.21.15.3:0/429731061 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c433b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 -- 172.21.15.3:0/429731061 >> 172.21.15.3:0/429731061 conn(0x555c43394250 msgr2=0x555c433dba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 -- 172.21.15.3:0/429731061 shutdown_connections 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.750+0000 7fac99000740 1 -- 172.21.15.3:0/429731061 wait complete. 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.751+0000 7fac99000740 1 -- start start 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.751+0000 7fac99000740 1 Processor -- start 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c43453f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c433b6250 con 0x555c433b2860 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555c43391160 con 0x555c43398220 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555c433b67e0 con 0x555c43396a90 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac99000740 10 client.0 _ll_register_callbacks cb 0x7fac8000fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47864/0 (socket says 172.21.15.3:47864) 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 -- 172.21.15.3:0/1009400313 learned_addr learned my addr 172.21.15.3:0/1009400313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 -- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 msgr2=0x555c43453f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c43453f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 -- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 msgr2=0x555c433b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.752+0000 7fac8fe00640 1 -- 172.21.15.3:0/1009400313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c433c10b0 con 0x555c433b2860 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac8fe00640 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fac88089cb0 tx=0x7fac8805dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfab047d6abe5692 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88062420 con 0x555c433b2860 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fac880600b0 con 0x555c433b2860 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac78004360 con 0x555c433b2860 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fac88078050 con 0x555c433b2860 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 10 client.7008 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x555c433debe0 con 0x555c433b2860 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78004360 con 0x555c433b2860 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7fac88058fc0 con 0x555c433b2860 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 10 client.7008 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 10 client.7008 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 20 client.7008 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c4345e2e0 0x555c433bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 4 client.7008 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.753+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) -- 0x555c43302340 con 0x555c4345e2e0 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac8f400640 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c4345e2e0 0x555c433bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac8f400640 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c4345e2e0 0x555c433bba90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faa0bebb3414e7eb server_cookie=7804370a59640be3 in_seq=0 out_seq=0 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac73e00640 10 client.7008 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac73e00640 1 -- 172.21.15.3:0/1009400313 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x555c43302340 con 0x555c4345e2e0 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac73e00640 10 client.7008 handle_command_reply: tid=0 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.754+0000 7fac99000740 1 client.7008 shutdown 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 msgr2=0x555c433b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fac88089cb0 tx=0x7fac8805dfb0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 shutdown_connections 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x555c4345e2e0 0x555c433bba90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c433b2860 0x555c433b0cc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555c43398220 0x555c433b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac73e00640 0 client.7008 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 --2- 172.21.15.3:0/1009400313 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555c43396a90 0x555c43453f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 >> 172.21.15.3:0/1009400313 conn(0x555c43394250 msgr2=0x555c433da5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 shutdown_connections 2024-09-10T07:51:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 1 -- 172.21.15.3:0/1009400313 wait complete. 2024-09-10T07:51:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.755+0000 7fac99000740 20 client.7008 trim_cache size 0 max 16384 2024-09-10T07:51:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:Error EXDEV: Not rank 0 2024-09-10T07:51:44.761 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-10T07:51:44.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-09-10T07:51:44.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.920+0000 7f8d82000640 1 Processor -- start 2024-09-10T07:51:44.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.920+0000 7f8d82000640 1 -- start start 2024-09-10T07:51:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d82000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d82000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7c15eff0 con 0x7f8d7c072c50 2024-09-10T07:51:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d82000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7c15f2b0 con 0x7f8d7c15b230 2024-09-10T07:51:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d82000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c15fbe0 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d7be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d7c072c50 0x7f8d7c071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34630/0 (socket says 172.21.15.3:34630) 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d7b400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8d7c15b230 0x7f8d7c0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:48414/0 (socket says 172.21.15.3:48414) 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d7be00640 1 -- 172.21.15.3:0/2490794160 learned_addr learned my addr 172.21.15.3:0/2490794160 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d73e00640 1 --2- 172.21.15.3:0/2490794160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d73e00640 1 -- 172.21.15.3:0/2490794160 >> v1:172.21.15.70:6789/0 conn(0x7f8d7c15b230 legacy=0x7f8d7c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d7aa00640 1 -- 172.21.15.3:0/2490794160 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695388407 0 0) 0x7f8d7c15f2b0 con 0x7f8d7c15b230 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d73e00640 1 -- 172.21.15.3:0/2490794160 >> v1:172.21.15.3:6789/0 conn(0x7f8d7c072c50 legacy=0x7f8d7c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.922+0000 7f8d73e00640 1 -- 172.21.15.3:0/2490794160 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c15f590 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.923+0000 7f8d73e00640 1 --2- 172.21.15.3:0/2490794160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c15c390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d640097d0 tx=0x7f8d6402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbbf07c186331034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.924+0000 7f8d7aa00640 1 -- 172.21.15.3:0/2490794160 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037070 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.924+0000 7f8d7aa00640 1 -- 172.21.15.3:0/2490794160 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6402ad90 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.924+0000 7f8d7aa00640 1 -- 172.21.15.3:0/2490794160 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64032660 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.924+0000 7f8d82000640 1 -- 172.21.15.3:0/2490794160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 msgr2=0x7f8d7c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.924+0000 7f8d82000640 1 --2- 172.21.15.3:0/2490794160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c15c390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d640097d0 tx=0x7f8d6402a100 comp rx=0 tx=0).stop 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.925+0000 7f8d82000640 1 -- 172.21.15.3:0/2490794160 shutdown_connections 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.925+0000 7f8d82000640 1 --2- 172.21.15.3:0/2490794160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c15c390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.925+0000 7f8d82000640 1 -- 172.21.15.3:0/2490794160 >> 172.21.15.3:0/2490794160 conn(0x7f8d7c0b5cb0 msgr2=0x7f8d7c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.925+0000 7f8d82000640 1 -- 172.21.15.3:0/2490794160 shutdown_connections 2024-09-10T07:51:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.925+0000 7f8d82000640 1 -- 172.21.15.3:0/2490794160 wait complete. 2024-09-10T07:51:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.926+0000 7f8d82000640 1 Processor -- start 2024-09-10T07:51:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.926+0000 7f8d82000640 1 -- start start 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d7c072c50 0x7f8d7c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7c15b230 0x7f8d7c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c15b610 con 0x7f8d7c15b230 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c15c8d0 con 0x7f8d7c072c50 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d82000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c15b790 con 0x7f8d7c0b4c90 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7c15b230 0x7f8d7c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.927+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7c15b230 0x7f8d7c0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47872/0 (socket says 172.21.15.3:47872) 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 -- 172.21.15.3:0/975445727 learned_addr learned my addr 172.21.15.3:0/975445727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7b400640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d7c072c50 0x7f8d7c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d73e00640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 -- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 msgr2=0x7f8d7c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 -- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d7c072c50 msgr2=0x7f8d7c14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d7c072c50 0x7f8d7c14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 -- 172.21.15.3:0/975445727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0b10c0 con 0x7f8d7c15b230 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d73e00640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8d7c0b4c90 0x7f8d7c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.928+0000 7f8d7be00640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d7c15b230 0x7f8d7c0aede0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400cb20 tx=0x7f8d74002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.929+0000 7f8d73400640 1 -- 172.21.15.3:0/975445727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74018070 con 0x7f8d7c15b230 2024-09-10T07:51:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.929+0000 7f8d73400640 1 -- 172.21.15.3:0/975445727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7400a700 con 0x7f8d7c15b230 2024-09-10T07:51:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.929+0000 7f8d82000640 1 -- 172.21.15.3:0/975445727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0b13a0 con 0x7f8d7c15b230 2024-09-10T07:51:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.929+0000 7f8d73400640 1 -- 172.21.15.3:0/975445727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400aad0 con 0x7f8d7c15b230 2024-09-10T07:51:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.929+0000 7f8d82000640 1 -- 172.21.15.3:0/975445727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1421b0 con 0x7f8d7c15b230 2024-09-10T07:51:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.932+0000 7f8d73400640 1 -- 172.21.15.3:0/975445727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d7401b070 con 0x7f8d7c15b230 2024-09-10T07:51:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.933+0000 7f8d73400640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d5806be30 0x7f8d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.933+0000 7f8d73400640 1 -- 172.21.15.3:0/975445727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f8d7408a5b0 con 0x7f8d7c15b230 2024-09-10T07:51:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.933+0000 7f8d84e8d740 1 Processor -- start 2024-09-10T07:51:44.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.933+0000 7f8d7b400640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d5806be30 0x7f8d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.933+0000 7f8d84e8d740 1 -- start start 2024-09-10T07:51:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d7b400640 1 --2- 172.21.15.3:0/975445727 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8d5806be30 0x7f8d5806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c000f60 tx=0x7f8d6c007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143645950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x560143690b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560143554280 con 0x56014368f1f0 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5601435ad7b0 con 0x560143645570 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5601435557d0 con 0x560143645e90 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47880/0 (socket says 172.21.15.3:47880) 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d7be00640 1 -- 172.21.15.3:0/2320706813 learned_addr learned my addr 172.21.15.3:0/2320706813 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d7b400640 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143645950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.934+0000 7f8d73e00640 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x560143690b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 -- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 msgr2=0x560143690b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x560143690b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 -- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 msgr2=0x560143645950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143645950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 -- 172.21.15.3:0/2320706813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601432a4120 con 0x56014368f1f0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d7be00640 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8d74054d60 tx=0x7f8d7400adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f92eecb10abedd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d6b400640 1 -- 172.21.15.3:0/2320706813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74061430 con 0x56014368f1f0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d6b400640 1 -- 172.21.15.3:0/2320706813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d740569a0 con 0x56014368f1f0 2024-09-10T07:51:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.935+0000 7f8d6b400640 1 -- 172.21.15.3:0/2320706813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74054f70 con 0x56014368f1f0 2024-09-10T07:51:44.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2320706813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 msgr2=0x560143691090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8d74054d60 tx=0x7f8d7400adb0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2320706813 shutdown_connections 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x560143691090 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x560143690b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2320706813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143645950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2320706813 >> 172.21.15.3:0/2320706813 conn(0x560143611ec0 msgr2=0x56014368fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2320706813 shutdown_connections 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.936+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2320706813 wait complete. 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.937+0000 7f8d84e8d740 1 -- start start 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.937+0000 7f8d84e8d740 1 Processor -- start 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645570 0x5601436a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645e90 0x560143672990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601435363f0 con 0x56014368f1f0 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5601435a9770 con 0x560143645e90 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56014364b800 con 0x560143645570 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d84e8d740 10 client.0 _ll_register_callbacks cb 0x56014355a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47896/0 (socket says 172.21.15.3:47896) 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 -- 172.21.15.3:0/1863664642 learned_addr learned my addr 172.21.15.3:0/1863664642 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7b400640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645570 0x5601436a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d73e00640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645e90 0x560143672990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 -- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645570 msgr2=0x5601436a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645570 0x5601436a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 -- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645e90 msgr2=0x560143672990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645e90 0x560143672990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.938+0000 7f8d7be00640 1 -- 172.21.15.3:0/1863664642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014338a500 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d7be00640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8d74058600 tx=0x7f8d7405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94519a8ea1db115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74059d80 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f8d7408a0c0 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d740569a0 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8d7408a2c0 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d84e8d740 10 client.7020 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5601433eac90 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d740569a0 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f8d7406cce0 con 0x56014368f1f0 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d84e8d740 10 client.7020 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.939+0000 7f8d84e8d740 10 client.7020 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d84e8d740 20 client.7020 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x56014366f6b0 0x5601436a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d84e8d740 4 client.7020 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601435baee0 con 0x56014366f6b0 2024-09-10T07:51:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d7b400640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x56014366f6b0 0x5601436a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d7b400640 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x56014366f6b0 0x5601436a3070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c661a5e1503709b server_cookie=6589c7645add7f4d in_seq=0 out_seq=0 2024-09-10T07:51:44.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.940+0000 7f8d5fe00640 10 client.7020 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d5fe00640 1 -- 172.21.15.3:0/1863664642 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601435baee0 con 0x56014366f6b0 2024-09-10T07:51:44.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d5fe00640 10 client.7020 handle_command_reply: tid=0 2024-09-10T07:51:44.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 client.7020 shutdown 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 msgr2=0x56014368c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8d74058600 tx=0x7f8d7405b040 comp rx=0 tx=0).stop 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 shutdown_connections 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x56014366f6b0 0x5601436a3070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x56014368c4b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645e90 0x560143672990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/1863664642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645570 0x5601436a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d5fe00640 0 client.7020 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.941+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 >> 172.21.15.3:0/1863664642 conn(0x560143611ec0 msgr2=0x56014368fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.942+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 shutdown_connections 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.942+0000 7f8d84e8d740 1 -- 172.21.15.3:0/1863664642 wait complete. 2024-09-10T07:51:44.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.942+0000 7f8d84e8d740 20 client.7020 trim_cache size 0 max 16384 2024-09-10T07:51:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.959+0000 7f8d84e8d740 1 Processor -- start 2024-09-10T07:51:44.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.959+0000 7f8d84e8d740 1 -- start start 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x56014364bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c200ca0 con 0x56014368f1f0 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ffe70 con 0x560143645570 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c200570 con 0x560143645e90 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47910/0 (socket says 172.21.15.3:47910) 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.960+0000 7f8d7be00640 1 -- 172.21.15.3:0/3843074022 learned_addr learned my addr 172.21.15.3:0/3843074022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 -- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 msgr2=0x56014364c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 -- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 msgr2=0x56014364bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x56014364bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 -- 172.21.15.3:0/3843074022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014327fe20 con 0x56014368f1f0 2024-09-10T07:51:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d7be00640 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8d74054d60 tx=0x7f8d7405e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92f11a4dda5f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d6aa00640 1 -- 172.21.15.3:0/3843074022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74059b80 con 0x56014368f1f0 2024-09-10T07:51:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d6aa00640 1 -- 172.21.15.3:0/3843074022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74070030 con 0x56014368f1f0 2024-09-10T07:51:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.961+0000 7f8d6aa00640 1 -- 172.21.15.3:0/3843074022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74059d80 con 0x56014368f1f0 2024-09-10T07:51:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 -- 172.21.15.3:0/3843074022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 msgr2=0x5601436b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8d74054d60 tx=0x7f8d7405e000 comp rx=0 tx=0).stop 2024-09-10T07:51:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 -- 172.21.15.3:0/3843074022 shutdown_connections 2024-09-10T07:51:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436b3550 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/3843074022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x56014364bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 -- 172.21.15.3:0/3843074022 >> 172.21.15.3:0/3843074022 conn(0x560143611ec0 msgr2=0x560143671ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 -- 172.21.15.3:0/3843074022 shutdown_connections 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.962+0000 7f8d84e8d740 1 -- 172.21.15.3:0/3843074022 wait complete. 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.963+0000 7f8d84e8d740 1 -- start start 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.963+0000 7f8d84e8d740 1 Processor -- start 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143670350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5601435a9770 con 0x56014368f1f0 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1f8c70 con 0x560143645570 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5601435363f0 con 0x560143645e90 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d84e8d740 10 client.0 _ll_register_callbacks cb 0x7f8d740582f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143670350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:36616/0 (socket says 172.21.15.3:36616) 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d73e00640 1 -- 172.21.15.3:0/2890204290 learned_addr learned my addr 172.21.15.3:0/2890204290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 -- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 msgr2=0x56014364be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 -- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 msgr2=0x5601436f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.964+0000 7f8d7b400640 1 -- 172.21.15.3:0/2890204290 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560143670890 con 0x560143645570 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d7b400640 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143670350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56014368a7b0 tx=0x7f8d6c007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e639ec32b09fdc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c024bb0 con 0x560143645570 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d74070030 con 0x560143645570 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mon.1 v2:172.21.15.70:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f8d6c024d50 con 0x560143645570 2024-09-10T07:51:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c036050 con 0x560143645570 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d84e8d740 10 client.6646 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.965+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x560143691770 con 0x560143645570 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mon.1 v2:172.21.15.70:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74070030 con 0x560143645570 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mon.1 v2:172.21.15.70:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f8d6c039020 con 0x560143645570 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 10 client.6646 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 10 client.6646 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 20 client.6646 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x560143710980 0x560143674450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 4 client.6646 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x5601435b5230 con 0x560143710980 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.966+0000 7f8d73e00640 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x560143710980 0x560143674450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d73e00640 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x560143710980 0x560143674450 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c362255d3380bcc9 server_cookie=eac16d80d6f66d3a in_seq=0 out_seq=0 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d5fe00640 10 client.6646 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d5fe00640 1 -- 172.21.15.3:0/2890204290 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x5601435b5230 con 0x560143710980 2024-09-10T07:51:44.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d5fe00640 10 client.6646 handle_command_reply: tid=0 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 client.6646 shutdown 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 msgr2=0x560143670350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143670350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56014368a7b0 tx=0x7f8d6c007f90 comp rx=0 tx=0).stop 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 shutdown_connections 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x560143710980 0x560143674450 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56014368f1f0 0x5601436f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560143645e90 0x56014364be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 --2- 172.21.15.3:0/2890204290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560143645570 0x560143670350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d5fe00640 0 client.6646 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 >> 172.21.15.3:0/2890204290 conn(0x560143611ec0 msgr2=0x5601436a0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 shutdown_connections 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 1 -- 172.21.15.3:0/2890204290 wait complete. 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:44.967+0000 7f8d84e8d740 20 client.6646 trim_cache size 0 max 16384 2024-09-10T07:51:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:Error EXDEV: Not rank 0 2024-09-10T07:51:44.974 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-10T07:51:44.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-09-10T07:51:45.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.132+0000 7f9ecb200640 1 Processor -- start 2024-09-10T07:51:45.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.132+0000 7f9ecb200640 1 -- start start 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ecb200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 0x7f9ecc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ecb200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0d1490 0x7f9ecc0b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ecb200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ecc0d3a70 con 0x7f9ecc158950 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ecb200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc0b83a0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ecb200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc0d1cc0 con 0x7f9ecc0b6900 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9eca800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ecc0d1490 0x7f9ecc0b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ec9e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ecc158950 0x7f9ecc0b61f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34634/0 (socket says 172.21.15.3:34634) 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ec9e00640 1 -- 172.21.15.3:0/247838793 learned_addr learned my addr 172.21.15.3:0/247838793 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/247838793 >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 0x7f9ecc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9eca800640 1 -- 172.21.15.3:0/247838793 >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 msgr2=0x7f9ecc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9eca800640 1 --2- 172.21.15.3:0/247838793 >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 0x7f9ecc0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.134+0000 7f9eca800640 1 -- 172.21.15.3:0/247838793 >> v1:172.21.15.3:6789/0 conn(0x7f9ecc158950 legacy=0x7f9ecc0b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.135+0000 7f9eca800640 1 -- 172.21.15.3:0/247838793 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc0d1fa0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.135+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/247838793 >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 0x7f9ecc0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:45.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.135+0000 7f9ec9400640 1 -- 172.21.15.3:0/247838793 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092048501 0 0) 0x7f9ebc001110 con 0x7f9ecc158950 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9eca800640 1 --2- 172.21.15.3:0/247838793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc0b8880 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc072ac0 tx=0x7f9ec402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cd64364fa799d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ec9400640 1 -- 172.21.15.3:0/247838793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007740 con 0x7f9ecc0d1490 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ec9400640 1 -- 172.21.15.3:0/247838793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec40078e0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ec9400640 1 -- 172.21.15.3:0/247838793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007cb0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 -- 172.21.15.3:0/247838793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 msgr2=0x7f9ecc0b8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 --2- 172.21.15.3:0/247838793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc0b8880 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc072ac0 tx=0x7f9ec402a5f0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 -- 172.21.15.3:0/247838793 shutdown_connections 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 --2- 172.21.15.3:0/247838793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc0b8880 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 --2- 172.21.15.3:0/247838793 >> v2:172.21.15.70:3300/0 conn(0x7f9ecc0b6900 0x7f9ecc0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 -- 172.21.15.3:0/247838793 >> 172.21.15.3:0/247838793 conn(0x7f9ecc062810 msgr2=0x7f9ecc072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 -- 172.21.15.3:0/247838793 shutdown_connections 2024-09-10T07:51:45.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.136+0000 7f9ecb200640 1 -- 172.21.15.3:0/247838793 wait complete. 2024-09-10T07:51:45.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.138+0000 7f9ecb200640 1 Processor -- start 2024-09-10T07:51:45.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.138+0000 7f9ecb200640 1 -- start start 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ecc0b6900 0x7f9ecc1f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ecc158950 0x7f9ecc15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d1900 con 0x7f9ecc0d1490 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d2560 con 0x7f9ecc158950 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ecb200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d1a80 con 0x7f9ecc0b6900 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47934/0 (socket says 172.21.15.3:47934) 2024-09-10T07:51:45.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.139+0000 7f9ec3e00640 1 -- 172.21.15.3:0/2607740847 learned_addr learned my addr 172.21.15.3:0/2607740847 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 -- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ecc0b6900 msgr2=0x7f9ecc1f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9eca800640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ecc158950 0x7f9ecc15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ecc0b6900 0x7f9ecc1f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ecc0b6900 0x7f9ecc1f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 -- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ecc158950 msgr2=0x7f9ecc15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ecc158950 0x7f9ecc15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 -- 172.21.15.3:0/2607740847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc159270 con 0x7f9ecc0d1490 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9eca800640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ecc158950 0x7f9ecc15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:45.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ecc0b6900 0x7f9ecc1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ecc0d1490 0x7f9ecc15b730 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400d7d0 tx=0x7f9eb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec2000640 1 -- 172.21.15.3:0/2607740847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004030 con 0x7f9ecc0d1490 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec2000640 1 -- 172.21.15.3:0/2607740847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb40041d0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ec2000640 1 -- 172.21.15.3:0/2607740847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400b6c0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.140+0000 7f9ecb200640 1 -- 172.21.15.3:0/2607740847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc159550 con 0x7f9ecc0d1490 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.141+0000 7f9ecb200640 1 -- 172.21.15.3:0/2607740847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc159b10 con 0x7f9ecc0d1490 2024-09-10T07:51:45.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.143+0000 7f9ec2000640 1 -- 172.21.15.3:0/2607740847 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9eb400b860 con 0x7f9ecc0d1490 2024-09-10T07:51:45.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.143+0000 7f9ec2000640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ea806be30 0x7f9ea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.144+0000 7f9ec2000640 1 -- 172.21.15.3:0/2607740847 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f9eb408b0f0 con 0x7f9ecc0d1490 2024-09-10T07:51:45.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.144+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ea806be30 0x7f9ea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.144+0000 7f9ed39d1740 1 Processor -- start 2024-09-10T07:51:45.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.144+0000 7f9ed39d1740 1 -- start start 2024-09-10T07:51:45.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.145+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/2607740847 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ea806be30 0x7f9ea806e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0013b0 tx=0x7f9ebc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.145+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd5228eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd5219d280 con 0x55bd522d83d0 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55bd521f67b0 con 0x55bd5228f070 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55bd5219e7d0 con 0x55bd5228e750 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47946/0 (socket says 172.21.15.3:47946) 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 -- 172.21.15.3:0/1022704970 learned_addr learned my addr 172.21.15.3:0/1022704970 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd5228eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 -- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 msgr2=0x55bd5228eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd5228eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 -- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 msgr2=0x55bd522d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9eca800640 1 -- 172.21.15.3:0/1022704970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd51eed120 con 0x55bd522d83d0 2024-09-10T07:51:45.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.146+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.147+0000 7f9eca800640 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009e80 tx=0x7f9ec402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d06f676147734 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.147+0000 7f9eba600640 1 -- 172.21.15.3:0/1022704970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40078c0 con 0x55bd522d83d0 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.147+0000 7f9eba600640 1 -- 172.21.15.3:0/1022704970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec4007a60 con 0x55bd522d83d0 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.147+0000 7f9eba600640 1 -- 172.21.15.3:0/1022704970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007e30 con 0x55bd522d83d0 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1022704970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 msgr2=0x55bd522da270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009e80 tx=0x7f9ec402e000 comp rx=0 tx=0).stop 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1022704970 shutdown_connections 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522da270 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1022704970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd5228eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1022704970 >> 172.21.15.3:0/1022704970 conn(0x55bd5225aec0 msgr2=0x55bd522d8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1022704970 shutdown_connections 2024-09-10T07:51:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1022704970 wait complete. 2024-09-10T07:51:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.148+0000 7f9ed39d1740 1 -- start start 2024-09-10T07:51:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.149+0000 7f9ed39d1740 1 Processor -- start 2024-09-10T07:51:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd522b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd5217f3f0 con 0x55bd522d83d0 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55bd521f2770 con 0x55bd5228f070 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55bd522949e0 con 0x55bd5228e750 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ed39d1740 10 client.0 _ll_register_callbacks cb 0x55bd521a35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd522b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47960/0 (socket says 172.21.15.3:47960) 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 -- 172.21.15.3:0/1546635588 learned_addr learned my addr 172.21.15.3:0/1546635588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 -- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 msgr2=0x55bd522b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd522b88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 -- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 msgr2=0x55bd522d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.150+0000 7f9eca800640 1 -- 172.21.15.3:0/1546635588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd51fd3500 con 0x55bd522d83d0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eca800640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40070c0 tx=0x7f9ec4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6429a8ebc57d058f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eaea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007c10 con 0x55bd522d83d0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec4007db0 con 0x55bd522d83d0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec4007a60 con 0x55bd522d83d0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9ec4058050 con 0x55bd522d83d0 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9ed39d1740 10 client.7041 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55bd52033c90 con 0x55bd522d83d0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.151+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec4007a60 con 0x55bd522d83d0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f9ec405b020 con 0x55bd522d83d0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 10 client.7041 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 10 client.7041 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 20 client.7041 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd522ec4f0 0x55bd522ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 4 client.7041 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd52203ee0 con 0x55bd522ec4f0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd522ec4f0 0x55bd522ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd522ec4f0 0x55bd522ec8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9c68ebcee735918 server_cookie=e013a4ae52f65c47 in_seq=0 out_seq=0 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.152+0000 7f9eaea00640 10 client.7041 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9eaea00640 1 -- 172.21.15.3:0/1546635588 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd52203ee0 con 0x55bd522ec4f0 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9eaea00640 10 client.7041 handle_command_reply: tid=0 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9ed39d1740 1 client.7041 shutdown 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 msgr2=0x55bd522d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40070c0 tx=0x7f9ec4004ab0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 shutdown_connections 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.153+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd522ec4f0 0x55bd522ec8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d3470 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228f070 0x55bd522d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/1546635588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228e750 0x55bd522b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9eaea00640 0 client.7041 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 >> 172.21.15.3:0/1546635588 conn(0x55bd5225aec0 msgr2=0x55bd522bb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 shutdown_connections 2024-09-10T07:51:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 1 -- 172.21.15.3:0/1546635588 wait complete. 2024-09-10T07:51:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.154+0000 7f9ed39d1740 20 client.7041 trim_cache size 0 max 16384 2024-09-10T07:51:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.172+0000 7f9ed39d1740 1 Processor -- start 2024-09-10T07:51:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.172+0000 7f9ed39d1740 1 -- start start 2024-09-10T07:51:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 0x55bd522eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0caff0 con 0x55bd5228e750 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0cb6f0 con 0x55bd522d83d0 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc0c9930 con 0x55bd5228f070 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47972/0 (socket says 172.21.15.3:47972) 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 -- 172.21.15.3:0/500345317 learned_addr learned my addr 172.21.15.3:0/500345317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9eca800640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 0x55bd522eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 -- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 msgr2=0x55bd522d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 -- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 msgr2=0x55bd522eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 0x55bd522eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 -- 172.21.15.3:0/500345317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd51ec8e20 con 0x55bd5228e750 2024-09-10T07:51:45.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9eca800640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 0x55bd522eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:45.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.173+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000f60 tx=0x7f9ebc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56af8f32e545ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9eb9c00640 1 -- 172.21.15.3:0/500345317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0058e0 con 0x55bd5228e750 2024-09-10T07:51:45.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9eb9c00640 1 -- 172.21.15.3:0/500345317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc005a80 con 0x55bd5228e750 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9eb9c00640 1 -- 172.21.15.3:0/500345317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00d7a0 con 0x55bd5228e750 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 -- 172.21.15.3:0/500345317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 msgr2=0x55bd522db460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc000f60 tx=0x7f9ebc010000 comp rx=0 tx=0).stop 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 -- 172.21.15.3:0/500345317 shutdown_connections 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd522d83d0 0x55bd522eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/500345317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd5228e750 0x55bd522db460 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 -- 172.21.15.3:0/500345317 >> 172.21.15.3:0/500345317 conn(0x55bd5225aec0 msgr2=0x55bd522d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 -- 172.21.15.3:0/500345317 shutdown_connections 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.174+0000 7f9ed39d1740 1 -- 172.21.15.3:0/500345317 wait complete. 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.175+0000 7f9ed39d1740 1 -- start start 2024-09-10T07:51:45.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.175+0000 7f9ed39d1740 1 Processor -- start 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 0x55bd522d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd522beb40 con 0x55bd522d83d0 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55bd522b8e10 con 0x55bd5228e750 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55bd522d2f80 con 0x55bd5228f070 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9ed39d1740 10 client.0 _ll_register_callbacks cb 0x7f9ec402aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.176+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47980/0 (socket says 172.21.15.3:47980) 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 -- 172.21.15.3:0/3661364522 learned_addr learned my addr 172.21.15.3:0/3661364522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 -- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 msgr2=0x55bd522bce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 0x55bd522d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522bce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 -- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 msgr2=0x55bd522d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 0x55bd522d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 -- 172.21.15.3:0/3661364522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd522fc020 con 0x55bd522d83d0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 0x55bd522d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9ec3e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522bce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eca800640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4000f80 tx=0x7f9ec403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde92004d0c93426 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eaea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.177+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007d30 con 0x55bd522d83d0 2024-09-10T07:51:45.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc005a80 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec4003ea0 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9ec4069050 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 10 client.7047 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55bd522fc2f0 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc005a80 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7f9ec406c020 con 0x55bd522d83d0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 10 client.7047 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 10 client.7047 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 20 client.7047 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd52355b90 0x55bd52359910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 4 client.7047 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55bd521fe230 con 0x55bd52355b90 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.178+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd52355b90 0x55bd52359910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ec9e00640 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd52355b90 0x55bd52359910 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a63bc216a1b4bf8 server_cookie=d8effc5483819cf0 in_seq=0 out_seq=0 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9eaea00640 10 client.7047 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9eaea00640 1 -- 172.21.15.3:0/3661364522 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x55bd521fe230 con 0x55bd52355b90 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9eaea00640 10 client.7047 handle_command_reply: tid=0 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 client.7047 shutdown 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 msgr2=0x55bd522d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4000f80 tx=0x7f9ec403e070 comp rx=0 tx=0).stop 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 shutdown_connections 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55bd52355b90 0x55bd52359910 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd522d83d0 0x55bd522d1020 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55bd5228f070 0x55bd522bce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 --2- 172.21.15.3:0/3661364522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55bd5228e750 0x55bd522d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 >> 172.21.15.3:0/3661364522 conn(0x55bd5225aec0 msgr2=0x55bd522f29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.179+0000 7f9eaea00640 0 client.7047 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.180+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 shutdown_connections 2024-09-10T07:51:45.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.180+0000 7f9ed39d1740 1 -- 172.21.15.3:0/3661364522 wait complete. 2024-09-10T07:51:45.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.180+0000 7f9ed39d1740 20 client.7047 trim_cache size 0 max 16384 2024-09-10T07:51:45.184 INFO:teuthology.orchestra.run.smithi003.stderr:Error EXDEV: Not rank 0 2024-09-10T07:51:45.187 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-10T07:51:45.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-09-10T07:51:45.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.347+0000 7fb2a3e00640 1 Processor -- start 2024-09-10T07:51:45.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.347+0000 7fb2a3e00640 1 -- start start 2024-09-10T07:51:45.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2a4161b70 0x7fb2a40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a3e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2a40d4cb0 0x7fb2a40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a3e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a40d3de0 con 0x7fb2a40d3860 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a3e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a4161f50 con 0x7fb2a4161b70 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a3e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a40d40c0 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2a4161b70 0x7fb2a40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2a4161b70 0x7fb2a40d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47992/0 (socket says 172.21.15.3:47992) 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a2a00640 1 -- 172.21.15.3:0/2575239560 learned_addr learned my addr 172.21.15.3:0/2575239560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.348+0000 7fb2a2000640 1 --1- 172.21.15.3:0/2575239560 >> v1:172.21.15.3:6789/0 conn(0x7fb2a40d3860 0x7fb2a40d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34638/0 (socket says 172.21.15.3:34638) 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.349+0000 7fb2a3400640 1 --2- 172.21.15.3:0/2575239560 >> v2:172.21.15.70:3300/0 conn(0x7fb2a40d4cb0 0x7fb2a40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.349+0000 7fb2a2a00640 1 -- 172.21.15.3:0/2575239560 >> v2:172.21.15.70:3300/0 conn(0x7fb2a40d4cb0 msgr2=0x7fb2a40cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.349+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/2575239560 >> v2:172.21.15.70:3300/0 conn(0x7fb2a40d4cb0 0x7fb2a40cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.349+0000 7fb2a2a00640 1 -- 172.21.15.3:0/2575239560 >> v1:172.21.15.3:6789/0 conn(0x7fb2a40d3860 legacy=0x7fb2a40d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.349+0000 7fb2a2a00640 1 -- 172.21.15.3:0/2575239560 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40cf690 con 0x7fb2a4161b70 2024-09-10T07:51:45.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.350+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/2575239560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a4161b70 0x7fb2a40d3320 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40b4160 tx=0x7fb29402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b549836905a3db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.350+0000 7fb2a1600640 1 -- 172.21.15.3:0/2575239560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294037070 con 0x7fb2a4161b70 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.350+0000 7fb2a1600640 1 -- 172.21.15.3:0/2575239560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294032440 con 0x7fb2a4161b70 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.350+0000 7fb2a1600640 1 -- 172.21.15.3:0/2575239560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294032810 con 0x7fb2a4161b70 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 -- 172.21.15.3:0/2575239560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a4161b70 msgr2=0x7fb2a40d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 --2- 172.21.15.3:0/2575239560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a4161b70 0x7fb2a40d3320 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40b4160 tx=0x7fb29402a270 comp rx=0 tx=0).stop 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 -- 172.21.15.3:0/2575239560 shutdown_connections 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 --2- 172.21.15.3:0/2575239560 >> v2:172.21.15.70:3300/0 conn(0x7fb2a40d4cb0 0x7fb2a40cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 --2- 172.21.15.3:0/2575239560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a4161b70 0x7fb2a40d3320 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 -- 172.21.15.3:0/2575239560 >> 172.21.15.3:0/2575239560 conn(0x7fb2a4062630 msgr2=0x7fb2a4062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 -- 172.21.15.3:0/2575239560 shutdown_connections 2024-09-10T07:51:45.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.351+0000 7fb2a3e00640 1 -- 172.21.15.3:0/2575239560 wait complete. 2024-09-10T07:51:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.352+0000 7fb2a3e00640 1 Processor -- start 2024-09-10T07:51:45.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.353+0000 7fb2a3e00640 1 -- start start 2024-09-10T07:51:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.353+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a40d3860 0x7fb2a40b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.353+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a40d4cb0 0x7fb2a40b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2a4161b70 0x7fb2a41f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40cf2d0 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a3e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40c9fc0 con 0x7fb2a40d3860 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a3e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a40cf450 con 0x7fb2a4161b70 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a40d4cb0 0x7fb2a40b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a40d4cb0 0x7fb2a40b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47994/0 (socket says 172.21.15.3:47994) 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 -- 172.21.15.3:0/4191855788 learned_addr learned my addr 172.21.15.3:0/4191855788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a40d3860 0x7fb2a40b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a3400640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2a4161b70 0x7fb2a41f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 -- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2a4161b70 msgr2=0x7fb2a41f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2a4161b70 0x7fb2a41f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 -- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a40d3860 msgr2=0x7fb2a40b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a40d3860 0x7fb2a40b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2000640 1 -- 172.21.15.3:0/4191855788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40ce400 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.354+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a40d3860 0x7fb2a40b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb2a2000640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2a40d4cb0 0x7fb2a40b4c90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c400 tx=0x7fb29000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb29aa00640 1 -- 172.21.15.3:0/4191855788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900074e0 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb29aa00640 1 -- 172.21.15.3:0/4191855788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290007680 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb29aa00640 1 -- 172.21.15.3:0/4191855788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290004a00 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb2a3e00640 1 -- 172.21.15.3:0/4191855788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41f4c80 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.355+0000 7fb2a3e00640 1 -- 172.21.15.3:0/4191855788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a40b41b0 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.357+0000 7fb29aa00640 1 -- 172.21.15.3:0/4191855788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb29001a020 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.358+0000 7fb29aa00640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb28406be30 0x7fb28406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.358+0000 7fb29aa00640 1 -- 172.21.15.3:0/4191855788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fb29008a540 con 0x7fb2a40d4cb0 2024-09-10T07:51:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.358+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb28406be30 0x7fb28406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.359+0000 7fb2acc5e740 1 Processor -- start 2024-09-10T07:51:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.359+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/4191855788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb28406be30 0x7fb28406e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb294034000 tx=0x7fb29402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.359+0000 7fb2acc5e740 1 -- start start 2024-09-10T07:51:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 0x55ec7ffed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffee3e0 0x55ec7ffe7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec7ffb3cf0 con 0x55ec7ffecc50 2024-09-10T07:51:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ec7ff4f7f0 con 0x55ec7ffea410 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ec7fef7800 con 0x55ec7ffee3e0 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47996/0 (socket says 172.21.15.3:47996) 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1986738240 learned_addr learned my addr 172.21.15.3:0/1986738240 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a3400640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffee3e0 0x55ec7ffe7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 0x55ec7ffed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffee3e0 msgr2=0x55ec7ffe7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffee3e0 0x55ec7ffe7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 msgr2=0x55ec7ffed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 0x55ec7ffed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1986738240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7fd2c7d0 con 0x55ec7ffecc50 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.360+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 0x55ec7ffed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb29402a860 tx=0x7fb29402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d0b16c611f83a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb28f400640 1 -- 172.21.15.3:0/1986738240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294037040 con 0x55ec7ffecc50 2024-09-10T07:51:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb28f400640 1 -- 172.21.15.3:0/1986738240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29404d070 con 0x55ec7ffecc50 2024-09-10T07:51:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb28f400640 1 -- 172.21.15.3:0/1986738240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294032e60 con 0x55ec7ffecc50 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1986738240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 msgr2=0x55ec800323f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.361+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb29402a860 tx=0x7fb29402c000 comp rx=0 tx=0).stop 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1986738240 shutdown_connections 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffee3e0 0x55ec7ffe7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffea410 0x55ec7ffed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1986738240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec800323f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1986738240 >> 172.21.15.3:0/1986738240 conn(0x55ec7ffe9410 msgr2=0x55ec800318c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1986738240 shutdown_connections 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1986738240 wait complete. 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.362+0000 7fb2acc5e740 1 -- start start 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.363+0000 7fb2acc5e740 1 Processor -- start 2024-09-10T07:51:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.363+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.363+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec80010c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.363+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec8000d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fed8750 con 0x55ec7ffecc50 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ec7ff4b7b0 con 0x55ec7ffee3e0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ec7ffe88b0 con 0x55ec800111b0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2acc5e740 10 client.0 _ll_register_callbacks cb 0x55ec7fc61be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48006/0 (socket says 172.21.15.3:48006) 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1839415471 learned_addr learned my addr 172.21.15.3:0/1839415471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a3400640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec8000d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec80010c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 msgr2=0x55ec8000d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec8000d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 msgr2=0x55ec80010c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec80010c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1839415471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7fc220f0 con 0x55ec7ffecc50 2024-09-10T07:51:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb294009630 tx=0x7fb294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6d81f40b2a1b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.364+0000 7fb283e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294037040 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fb29403cd40 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29404d070 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb294053460 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 10 client.7062 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ec7fd8cf60 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29404d070 con 0x55ec7ffecc50 2024-09-10T07:51:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7fb294053600 con 0x55ec7ffecc50 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 10 client.7062 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 10 client.7062 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 20 client.7062 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec8004d490 0x55ec8004f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 4 client.7062 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.365+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec7ff5cf20 con 0x55ec8004d490 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.366+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec8004d490 0x55ec8004f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.366+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec8004d490 0x55ec8004f880 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=146b8f8bf18c7911 server_cookie=38ded32d767a1149 in_seq=0 out_seq=0 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.366+0000 7fb283e00640 10 client.7062 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb283e00640 1 -- 172.21.15.3:0/1839415471 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec7ff5cf20 con 0x55ec8004d490 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb283e00640 10 client.7062 handle_command_reply: tid=0 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 client.7062 shutdown 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 msgr2=0x55ec8000b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb294009630 tx=0x7fb294004730 comp rx=0 tx=0).stop 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 shutdown_connections 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec8004d490 0x55ec8004f880 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec8000d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec80010c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1839415471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8000b690 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 >> 172.21.15.3:0/1839415471 conn(0x55ec7ffea410 msgr2=0x55ec80030fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb283e00640 0 client.7062 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 shutdown_connections 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1839415471 wait complete. 2024-09-10T07:51:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.367+0000 7fb2acc5e740 20 client.7062 trim_cache size 0 max 16384 2024-09-10T07:51:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.385+0000 7fb2acc5e740 1 Processor -- start 2024-09-10T07:51:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.385+0000 7fb2acc5e740 1 -- start start 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffecc50 0x55ec8000b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 0x55ec800a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41ffc20 con 0x55ec7ffee3e0 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a4200320 con 0x55ec800111b0 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a415b7e0 con 0x55ec7ffecc50 2024-09-10T07:51:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48020/0 (socket says 172.21.15.3:48020) 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.386+0000 7fb2a2000640 1 -- 172.21.15.3:0/79476848 learned_addr learned my addr 172.21.15.3:0/79476848 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffecc50 0x55ec8000b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a3400640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 0x55ec800a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 -- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffecc50 msgr2=0x55ec8000b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffecc50 0x55ec8000b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 -- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 msgr2=0x55ec800a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 0x55ec800a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 -- 172.21.15.3:0/79476848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7fbfb1b0 con 0x55ec7ffee3e0 2024-09-10T07:51:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a3400640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 0x55ec800a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb2a2000640 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb290000fb0 tx=0x7fb290088930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c096a1a4c054e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb28ea00640 1 -- 172.21.15.3:0/79476848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29005f0d0 con 0x55ec7ffee3e0 2024-09-10T07:51:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb28ea00640 1 -- 172.21.15.3:0/79476848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290054f00 con 0x55ec7ffee3e0 2024-09-10T07:51:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.387+0000 7fb28ea00640 1 -- 172.21.15.3:0/79476848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290058d10 con 0x55ec7ffee3e0 2024-09-10T07:51:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 -- 172.21.15.3:0/79476848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 msgr2=0x55ec8002fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb290000fb0 tx=0x7fb290088930 comp rx=0 tx=0).stop 2024-09-10T07:51:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 -- 172.21.15.3:0/79476848 shutdown_connections 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec800111b0 0x55ec800a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffee3e0 0x55ec8002fb30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/79476848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec7ffecc50 0x55ec8000b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 -- 172.21.15.3:0/79476848 >> 172.21.15.3:0/79476848 conn(0x55ec7ffea410 msgr2=0x55ec8000cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 -- 172.21.15.3:0/79476848 shutdown_connections 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.388+0000 7fb2acc5e740 1 -- 172.21.15.3:0/79476848 wait complete. 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.389+0000 7fb2acc5e740 1 -- start start 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.389+0000 7fb2acc5e740 1 Processor -- start 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec800132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec800a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec8000bbd0 con 0x55ec7ffecc50 2024-09-10T07:51:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fed8750 con 0x55ec7ffee3e0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ec8000c1e0 con 0x55ec800111b0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2acc5e740 10 client.0 _ll_register_callbacks cb 0x7fb2940096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48036/0 (socket says 172.21.15.3:48036) 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1683696897 learned_addr learned my addr 172.21.15.3:0/1683696897 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec800132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.390+0000 7fb2a3400640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec800a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 msgr2=0x55ec800a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec800a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 msgr2=0x55ec800132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec800132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 -- 172.21.15.3:0/1683696897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec80041ab0 con 0x55ec7ffecc50 2024-09-10T07:51:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2a2a00640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb294009630 tx=0x7fb294008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4efab9cf21bb92b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb283e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294037040 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fb294049ac0 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb290054f00 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb29403c750 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2acc5e740 10 client.7068 fetch_fsmap learned FSMap version 76 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.391+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ec800b0390 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290054f00 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 76) ==== 3554+0+0 (secure 0 0 0) 0x7fb294049cc0 con 0x55ec7ffecc50 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 10 client.7068 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 10 client.7068 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 20 client.7068 populate_metadata read hostname 'smithi003' 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec800afb70 0x55ec80030f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 4 client.7068 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-10T07:51:45.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 --> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55ec7ff57270 con 0x55ec800afb70 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.392+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec800afb70 0x55ec80030f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2a2000640 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec800afb70 0x55ec80030f40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cbca5d63766cdc3 server_cookie=ebb644d267eb605 in_seq=0 out_seq=0 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb283e00640 10 client.7068 ms_handle_connect on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb283e00640 1 -- 172.21.15.3:0/1683696897 <== mds.1 v2:172.21.15.3:6834/4206303617 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x55ec7ff57270 con 0x55ec800afb70 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb283e00640 10 client.7068 handle_command_reply: tid=0 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 client.7068 shutdown 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 msgr2=0x55ec8002e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb294009630 tx=0x7fb294008030 comp rx=0 tx=0).stop 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 shutdown_connections 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:6834/4206303617,v1:172.21.15.3:6835/4206303617] conn(0x55ec800afb70 0x55ec80030f40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ec800111b0 0x55ec800a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ec7ffee3e0 0x55ec800132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb283e00640 0 client.7068 ms_handle_reset on v2:172.21.15.3:6834/4206303617 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 --2- 172.21.15.3:0/1683696897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec7ffecc50 0x55ec8002e010 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 >> 172.21.15.3:0/1683696897 conn(0x55ec7ffea410 msgr2=0x55ec80010650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 shutdown_connections 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 1 -- 172.21.15.3:0/1683696897 wait complete. 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.393+0000 7fb2acc5e740 20 client.7068 trim_cache size 0 max 16384 2024-09-10T07:51:45.398 INFO:teuthology.orchestra.run.smithi003.stderr:Error EXDEV: Not rank 0 2024-09-10T07:51:45.400 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-10T07:51:45.401 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:51:45.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:45.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:45.579 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:51:45.579 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:51:45.579 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:45.579 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:45.614 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:45.614 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:45.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:51:45.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.888+0000 7f701e200640 1 Processor -- start 2024-09-10T07:51:45.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.888+0000 7f701e200640 1 -- start start 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701e200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7018141540 0x7f70180b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70180b2000 0x7f70180b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701e200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701815fc00 con 0x7f7018143de0 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701e200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f70180b29a0 con 0x7f7018141540 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70180aea40 con 0x7f70180b2000 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701ce00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7018141540 0x7f70180b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701d800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7018143de0 0x7f7018144270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34640/0 (socket says 172.21.15.3:34640) 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.889+0000 7f701d800640 1 -- 172.21.15.3:0/4163244441 learned_addr learned my addr 172.21.15.3:0/4163244441 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f7017e00640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70180b2000 0x7f70180b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f701ce00640 1 -- 172.21.15.3:0/4163244441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70180b2000 msgr2=0x7f70180b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f701ce00640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70180b2000 0x7f70180b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f701ce00640 1 -- 172.21.15.3:0/4163244441 >> v1:172.21.15.3:6789/0 conn(0x7f7018143de0 legacy=0x7f7018144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f701ce00640 1 -- 172.21.15.3:0/4163244441 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70180aed20 con 0x7f7018141540 2024-09-10T07:51:45.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.890+0000 7f7017400640 1 -- 172.21.15.3:0/4163244441 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313057506 0 0) 0x7f7010001490 con 0x7f7018143de0 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.891+0000 7f701ce00640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018141540 0x7f70180b50a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097a0 tx=0x7f700c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cccf70e95e92cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.891+0000 7f7017400640 1 -- 172.21.15.3:0/4163244441 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c037070 con 0x7f7018141540 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.891+0000 7f7017400640 1 -- 172.21.15.3:0/4163244441 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c032440 con 0x7f7018141540 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.891+0000 7f7017400640 1 -- 172.21.15.3:0/4163244441 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c032810 con 0x7f7018141540 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 -- 172.21.15.3:0/4163244441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018141540 msgr2=0x7f70180b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018141540 0x7f70180b50a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097a0 tx=0x7f700c02a180 comp rx=0 tx=0).stop 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 -- 172.21.15.3:0/4163244441 shutdown_connections 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70180b2000 0x7f70180b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 --2- 172.21.15.3:0/4163244441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018141540 0x7f70180b50a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 -- 172.21.15.3:0/4163244441 >> 172.21.15.3:0/4163244441 conn(0x7f701806bb20 msgr2=0x7f701806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:45.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 -- 172.21.15.3:0/4163244441 shutdown_connections 2024-09-10T07:51:45.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.892+0000 7f701e200640 1 -- 172.21.15.3:0/4163244441 wait complete. 2024-09-10T07:51:45.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.893+0000 7f701e200640 1 Processor -- start 2024-09-10T07:51:45.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.894+0000 7f701e200640 1 -- start start 2024-09-10T07:51:45.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.894+0000 7f701e200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70180b2000 0x7f70181f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.894+0000 7f701e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7018141540 0x7f7018070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.894+0000 7f701e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70180af5c0 con 0x7f7018143de0 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701e200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f70180af2e0 con 0x7f70180b2000 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70180b55e0 con 0x7f7018141540 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48052/0 (socket says 172.21.15.3:48052) 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 -- 172.21.15.3:0/1380190150 learned_addr learned my addr 172.21.15.3:0/1380190150 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:45.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f7017e00640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7018141540 0x7f7018070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701ce00640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70180b2000 0x7f70181f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 -- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7018141540 msgr2=0x7f7018070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7018141540 0x7f7018070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 -- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70180b2000 msgr2=0x7f70181f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:45.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70180b2000 0x7f70181f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.895+0000 7f701d800640 1 -- 172.21.15.3:0/1380190150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70180718f0 con 0x7f7018143de0 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f701d800640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f701000c7b0 tx=0x7f701000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010018070 con 0x7f7018143de0 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701000a640 con 0x7f7018143de0 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701000aa30 con 0x7f7018143de0 2024-09-10T07:51:45.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7018071bd0 con 0x7f7018143de0 2024-09-10T07:51:45.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.896+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7018072190 con 0x7f7018143de0 2024-09-10T07:51:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.897+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7018063610 con 0x7f7018143de0 2024-09-10T07:51:45.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.898+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f701001b070 con 0x7f7018143de0 2024-09-10T07:51:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.899+0000 7f7014c00640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.899+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f701008a3a0 con 0x7f7018143de0 2024-09-10T07:51:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.899+0000 7f701ce00640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.900+0000 7f701ce00640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 0x7f6ff806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f700c034000 tx=0x7f700c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:45.903+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f701815fc00 con 0x7f7018143de0 2024-09-10T07:51:46.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.101+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70181f53b0 con 0x7f7018143de0 2024-09-10T07:51:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.101+0000 7f7014c00640 1 -- 172.21.15.3:0/1380190150 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f70100586f0 con 0x7f7018143de0 2024-09-10T07:51:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:51:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.103+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 msgr2=0x7f6ff806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.103+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 0x7f6ff806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f700c034000 tx=0x7f700c02e000 comp rx=0 tx=0).stop 2024-09-10T07:51:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.103+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 msgr2=0x7f7018142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.103+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f701000c7b0 tx=0x7f701000cc70 comp rx=0 tx=0).stop 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 shutdown_connections 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ff806be30 0x7f6ff806e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7018143de0 0x7f7018142b80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7018141540 0x7f7018070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 --2- 172.21.15.3:0/1380190150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70180b2000 0x7f70181f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 >> 172.21.15.3:0/1380190150 conn(0x7f701806bb20 msgr2=0x7f7018144270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 shutdown_connections 2024-09-10T07:51:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.104+0000 7f701e200640 1 -- 172.21.15.3:0/1380190150 wait complete. 2024-09-10T07:51:46.119 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:51:46.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:46.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:51:46.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:51:46.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:51:46.183 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:51:46.219 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:51:46.219 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:46.219 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:46.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:46.259 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:51:46.259 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:46.259 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:46.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:51:46.294 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:51:46.295 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:46.295 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:46.337 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:46.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.493+0000 7fcd45e00640 1 Processor -- start 2024-09-10T07:51:46.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.493+0000 7fcd45e00640 1 -- start start 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd4006c230 0x7fcd40073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4015f0a0 con 0x7fcd4006e8e0 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd400737b0 con 0x7fcd4015e5b0 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcd40073580 con 0x7fcd4006c230 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd4006c230 0x7fcd40073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd4006c230 0x7fcd40073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48068/0 (socket says 172.21.15.3:48068) 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.494+0000 7fcd45400640 1 -- 172.21.15.3:0/1221843198 learned_addr learned my addr 172.21.15.3:0/1221843198 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.495+0000 7fcd45400640 1 -- 172.21.15.3:0/1221843198 >> v1:172.21.15.70:6789/0 conn(0x7fcd4015e5b0 legacy=0x7fcd4015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.495+0000 7fcd45400640 1 -- 172.21.15.3:0/1221843198 >> v1:172.21.15.3:6789/0 conn(0x7fcd4006e8e0 legacy=0x7fcd4006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.495+0000 7fcd45400640 1 -- 172.21.15.3:0/1221843198 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd40073a90 con 0x7fcd4006c230 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd45400640 1 --2- 172.21.15.3:0/1221843198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd40073040 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009e50 tx=0x7fcd3802a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cdce2fb2018a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd3f400640 1 -- 172.21.15.3:0/1221843198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38036070 con 0x7fcd4006c230 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd3f400640 1 -- 172.21.15.3:0/1221843198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd38031600 con 0x7fcd4006c230 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd3f400640 1 -- 172.21.15.3:0/1221843198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380319d0 con 0x7fcd4006c230 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd45e00640 1 -- 172.21.15.3:0/1221843198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 msgr2=0x7fcd40073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.496+0000 7fcd45e00640 1 --2- 172.21.15.3:0/1221843198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd40073040 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009e50 tx=0x7fcd3802a7b0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.497+0000 7fcd45e00640 1 -- 172.21.15.3:0/1221843198 shutdown_connections 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.497+0000 7fcd45e00640 1 --2- 172.21.15.3:0/1221843198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd40073040 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.497+0000 7fcd45e00640 1 -- 172.21.15.3:0/1221843198 >> 172.21.15.3:0/1221843198 conn(0x7fcd4006f210 msgr2=0x7fcd4006f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.497+0000 7fcd45e00640 1 -- 172.21.15.3:0/1221843198 shutdown_connections 2024-09-10T07:51:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.497+0000 7fcd45e00640 1 -- 172.21.15.3:0/1221843198 wait complete. 2024-09-10T07:51:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.498+0000 7fcd45e00640 1 Processor -- start 2024-09-10T07:51:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.498+0000 7fcd45e00640 1 -- start start 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 0x7fcd4015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 0x7fcd400cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd400b5940 con 0x7fcd4006c230 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4006ed50 con 0x7fcd4006e8e0 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd45e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcd400b5e60 con 0x7fcd4015e5b0 2024-09-10T07:51:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.499+0000 7fcd44a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2024-09-10T07:51:46.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 -- 172.21.15.3:0/3737795273 learned_addr learned my addr 172.21.15.3:0/3737795273 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd45400640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 0x7fcd400cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd3fe00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 0x7fcd4015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 -- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 msgr2=0x7fcd400cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 0x7fcd400cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 -- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 msgr2=0x7fcd4015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 0x7fcd4015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 -- 172.21.15.3:0/3737795273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd401e9240 con 0x7fcd4006c230 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd45400640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 0x7fcd400cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd3fe00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 0x7fcd4015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd44a00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c3c0 tx=0x7fcd3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.500+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340052d0 con 0x7fcd4006c230 2024-09-10T07:51:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.501+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd34005490 con 0x7fcd4006c230 2024-09-10T07:51:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.501+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34007260 con 0x7fcd4006c230 2024-09-10T07:51:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.501+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd401e9520 con 0x7fcd4006c230 2024-09-10T07:51:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.501+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd400cb900 con 0x7fcd4006c230 2024-09-10T07:51:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.502+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4015b740 con 0x7fcd4006c230 2024-09-10T07:51:46.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.502+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd34007400 con 0x7fcd4006c230 2024-09-10T07:51:46.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.503+0000 7fcd3cc00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 0x7fcd2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.503+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fcd3408a340 con 0x7fcd4006c230 2024-09-10T07:51:46.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.504+0000 7fcd3fe00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 0x7fcd2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.504+0000 7fcd3fe00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 0x7fcd2006e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd401f78a0 tx=0x7fcd2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.508+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd34058cf0 con 0x7fcd4006c230 2024-09-10T07:51:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.730+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd400b6530 con 0x7fcd4006c230 2024-09-10T07:51:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.730+0000 7fcd3cc00640 1 -- 172.21.15.3:0/3737795273 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5263 (secure 0 0 0) 0x7fcd340585f0 con 0x7fcd4006c230 2024-09-10T07:51:46.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:46.734 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"btime":"2024-09-10T07:51:46:626491+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:46.568413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702,6690,6355]},"id":5}]} 2024-09-10T07:51:46.735 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 77 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.732+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 msgr2=0x7fcd2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.732+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 0x7fcd2006e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd401f78a0 tx=0x7fcd2c009040 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.732+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 msgr2=0x7fcd401e89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.732+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c3c0 tx=0x7fcd3400c880 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.732+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 shutdown_connections 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd2006be30 0x7fcd2006e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcd4015e5b0 0x7fcd400cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd4006e8e0 0x7fcd4015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 --2- 172.21.15.3:0/3737795273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd4006c230 0x7fcd401e89f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 >> 172.21.15.3:0/3737795273 conn(0x7fcd4006f210 msgr2=0x7fcd4015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 shutdown_connections 2024-09-10T07:51:46.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.733+0000 7fcd45e00640 1 -- 172.21.15.3:0/3737795273 wait complete. 2024-09-10T07:51:46.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.905+0000 7f5b62200640 1 Processor -- start 2024-09-10T07:51:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.905+0000 7f5b62200640 1 -- start start 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.906+0000 7f5b62200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.906+0000 7f5b62200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b5c0d1ba0 0x7f5b5c0d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.906+0000 7f5b62200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c0b7920 con 0x7f5b5c0cf1f0 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.906+0000 7f5b62200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c0cf940 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.906+0000 7f5b62200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c0cfc20 con 0x7f5b5c15e8e0 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b5c0d1ba0 0x7f5b5c0d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b53e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b5c0cf1f0 0x7f5b5c0d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34656/0 (socket says 172.21.15.3:34656) 2024-09-10T07:51:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b53e00640 1 -- 172.21.15.3:0/164602553 learned_addr learned my addr 172.21.15.3:0/164602553 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5b400640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5be00640 1 -- 172.21.15.3:0/164602553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 msgr2=0x7f5b5c162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5be00640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5be00640 1 -- 172.21.15.3:0/164602553 >> v1:172.21.15.3:6789/0 conn(0x7f5b5c0cf1f0 legacy=0x7f5b5c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.907+0000 7f5b5be00640 1 -- 172.21.15.3:0/164602553 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c0d2ae0 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.908+0000 7f5b5be00640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c0d32e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009d90 tx=0x7f5b5402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dc213c20a5a538 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.908+0000 7f5b5aa00640 1 -- 172.21.15.3:0/164602553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54031600 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.908+0000 7f5b5aa00640 1 -- 172.21.15.3:0/164602553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b540317a0 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.908+0000 7f5b5aa00640 1 -- 172.21.15.3:0/164602553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54031b70 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 -- 172.21.15.3:0/164602553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 msgr2=0x7f5b5c0d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c0d32e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009d90 tx=0x7f5b5402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 -- 172.21.15.3:0/164602553 shutdown_connections 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c0d32e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 --2- 172.21.15.3:0/164602553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 -- 172.21.15.3:0/164602553 >> 172.21.15.3:0/164602553 conn(0x7f5b5c06bd00 msgr2=0x7f5b5c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 -- 172.21.15.3:0/164602553 shutdown_connections 2024-09-10T07:51:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.909+0000 7f5b62200640 1 -- 172.21.15.3:0/164602553 wait complete. 2024-09-10T07:51:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.911+0000 7f5b62200640 1 Processor -- start 2024-09-10T07:51:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.911+0000 7f5b62200640 1 -- start start 2024-09-10T07:51:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 0x7f5b5c0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d3820 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0cf630 con 0x7f5b5c0cf1f0 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b62200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d39a0 con 0x7f5b5c15e8e0 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48098/0 (socket says 172.21.15.3:48098) 2024-09-10T07:51:46.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 -- 172.21.15.3:0/3253900563 learned_addr learned my addr 172.21.15.3:0/3253900563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 -- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 msgr2=0x7f5b5c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b5b400640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 0x7f5b5c0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 -- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 msgr2=0x7f5b5c0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 0x7f5b5c0c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b53e00640 1 -- 172.21.15.3:0/3253900563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c1f4b60 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.912+0000 7f5b5b400640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 0x7f5b5c0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b53e00640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c440 tx=0x7f5b44001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017070 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44005ce0 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440072f0 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c1f4e40 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.913+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c1f5400 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.914+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c0d32e0 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.915+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b4401a070 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.916+0000 7f5b53400640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 0x7f5b3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.916+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f5b4408a380 con 0x7f5b5c0d1ba0 2024-09-10T07:51:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.916+0000 7f5b5b400640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 0x7f5b3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.917+0000 7f5b5b400640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 0x7f5b3806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:46.920+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b5c0b7920 con 0x7f5b5c0d1ba0 2024-09-10T07:51:47.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.140+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b5c0b9900 con 0x7f5b5c0d1ba0 2024-09-10T07:51:47.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.140+0000 7f5b53400640 1 -- 172.21.15.3:0/3253900563 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5263 (secure 0 0 0) 0x7f5b44058c90 con 0x7f5b5c0d1ba0 2024-09-10T07:51:47.142 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:47.142 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"btime":"2024-09-10T07:51:46:626491+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:46.568413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702,6690,6355]},"id":5}]} 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 77 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 msgr2=0x7f5b3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 0x7f5b3806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).stop 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 msgr2=0x7f5b5c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c440 tx=0x7f5b44001e70 comp rx=0 tx=0).stop 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 shutdown_connections 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5b3806be30 0x7f5b3806e2f0 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).stop 2024-09-10T07:51:47.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5b5c15e8e0 0x7f5b5c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b5c0d1ba0 0x7f5b5c15abd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 --2- 172.21.15.3:0/3253900563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5c0cf1f0 0x7f5b5c0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.142+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 >> 172.21.15.3:0/3253900563 conn(0x7f5b5c06bd00 msgr2=0x7f5b5c0b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.143+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 shutdown_connections 2024-09-10T07:51:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.143+0000 7f5b62200640 1 -- 172.21.15.3:0/3253900563 wait complete. 2024-09-10T07:51:47.158 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:47.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.315+0000 7f9c8d600640 1 Processor -- start 2024-09-10T07:51:47.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.315+0000 7f9c8d600640 1 -- start start 2024-09-10T07:51:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c8815e4f0 0x7f9c8815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8806bb20 0x7f9c8815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8806d160 0x7f9c8806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8806ccd0 con 0x7f9c8806bb20 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8806db30 con 0x7f9c8815e4f0 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c8d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8806c0c0 con 0x7f9c8806d160 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c7fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8806bb20 0x7f9c8815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c7fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c8806bb20 0x7f9c8815fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48110/0 (socket says 172.21.15.3:48110) 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.316+0000 7f9c7fe00640 1 -- 172.21.15.3:0/3273551094 learned_addr learned my addr 172.21.15.3:0/3273551094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c87400640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8806d160 0x7f9c8806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c86a00640 1 --2- 172.21.15.3:0/3273551094 >> v2:172.21.15.70:3300/0 conn(0x7f9c8815e4f0 0x7f9c8815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c7fe00640 1 -- 172.21.15.3:0/3273551094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8806d160 msgr2=0x7f9c8806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8806d160 0x7f9c8806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c7fe00640 1 -- 172.21.15.3:0/3273551094 >> v2:172.21.15.70:3300/0 conn(0x7f9c8815e4f0 msgr2=0x7f9c8815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/3273551094 >> v2:172.21.15.70:3300/0 conn(0x7f9c8815e4f0 0x7f9c8815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.317+0000 7f9c7fe00640 1 -- 172.21.15.3:0/3273551094 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8806c3a0 con 0x7f9c8806bb20 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.318+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c8815fcb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9c880741e0 tx=0x7f9c7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e37bb51c12bccb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.318+0000 7f9c86000640 1 -- 172.21.15.3:0/3273551094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037070 con 0x7f9c8806bb20 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.318+0000 7f9c86000640 1 -- 172.21.15.3:0/3273551094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c70032440 con 0x7f9c8806bb20 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.318+0000 7f9c86000640 1 -- 172.21.15.3:0/3273551094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70032810 con 0x7f9c8806bb20 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 -- 172.21.15.3:0/3273551094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 msgr2=0x7f9c8815fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c8815fcb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9c880741e0 tx=0x7f9c7002a1e0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 -- 172.21.15.3:0/3273551094 shutdown_connections 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8806d160 0x7f9c8806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 --2- 172.21.15.3:0/3273551094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c8815fcb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 --2- 172.21.15.3:0/3273551094 >> v2:172.21.15.70:3300/0 conn(0x7f9c8815e4f0 0x7f9c8815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 -- 172.21.15.3:0/3273551094 >> 172.21.15.3:0/3273551094 conn(0x7f9c8806e5b0 msgr2=0x7f9c8806f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:47.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 -- 172.21.15.3:0/3273551094 shutdown_connections 2024-09-10T07:51:47.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.319+0000 7f9c8d600640 1 -- 172.21.15.3:0/3273551094 wait complete. 2024-09-10T07:51:47.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.320+0000 7f9c8d600640 1 Processor -- start 2024-09-10T07:51:47.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.321+0000 7f9c8d600640 1 -- start start 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.321+0000 7f9c8d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.321+0000 7f9c8d600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 0x7f9c881f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.321+0000 7f9c8d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8815e4f0 0x7f9c8814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c8d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8815e8d0 con 0x7f9c8806bb20 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c8d600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8806c960 con 0x7f9c8806d160 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c8d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c881601f0 con 0x7f9c8815e4f0 2024-09-10T07:51:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48118/0 (socket says 172.21.15.3:48118) 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 -- 172.21.15.3:0/1652904578 learned_addr learned my addr 172.21.15.3:0/1652904578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 0x7f9c881f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c87400640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8815e4f0 0x7f9c8814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 -- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8815e4f0 msgr2=0x7f9c8814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8815e4f0 0x7f9c8814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 -- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 msgr2=0x7f9c881f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 0x7f9c881f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c86a00640 1 -- 172.21.15.3:0/1652904578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c881e7320 con 0x7f9c8806bb20 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.322+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 0x7f9c881f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c86a00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e870 tx=0x7f9c7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78017070 con 0x7f9c8806bb20 2024-09-10T07:51:47.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7800c940 con 0x7f9c8806bb20 2024-09-10T07:51:47.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800cd30 con 0x7f9c8806bb20 2024-09-10T07:51:47.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881e7600 con 0x7f9c8806bb20 2024-09-10T07:51:47.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.323+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881e7bc0 con 0x7f9c8806bb20 2024-09-10T07:51:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.324+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8815fcb0 con 0x7f9c8806bb20 2024-09-10T07:51:47.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.325+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c7801a070 con 0x7f9c8806bb20 2024-09-10T07:51:47.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.326+0000 7f9c7ea00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.326+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7808afe0 con 0x7f9c8806bb20 2024-09-10T07:51:47.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.326+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.327+0000 7f9c7fe00640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 0x7f9c6406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.330+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c780592c0 con 0x7f9c8806bb20 2024-09-10T07:51:47.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.515+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c880b5e30 con 0x7f9c8806bb20 2024-09-10T07:51:47.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.516+0000 7f9c7ea00640 1 -- 172.21.15.3:0/1652904578 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16596 (secure 0 0 0) 0x7f9c7805d5c0 con 0x7f9c8806bb20 2024-09-10T07:51:47.516 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:47.517 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":78,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:13.646589+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:09.867983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T07:51:11.065097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 msgr2=0x7f9c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 0x7f9c6406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 msgr2=0x7f9c880d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e870 tx=0x7f9c7800ed30 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 shutdown_connections 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c8815e4f0 0x7f9c8814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8806d160 0x7f9c881f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 --2- 172.21.15.3:0/1652904578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c8806bb20 0x7f9c880d2370 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 >> 172.21.15.3:0/1652904578 conn(0x7f9c8806e5b0 msgr2=0x7f9c880cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 shutdown_connections 2024-09-10T07:51:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.519+0000 7f9c8d600640 1 -- 172.21.15.3:0/1652904578 wait complete. 2024-09-10T07:51:47.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:51:47.535 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:47.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.693+0000 7fa286800640 1 Processor -- start 2024-09-10T07:51:47.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.693+0000 7fa286800640 1 -- start start 2024-09-10T07:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa286800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa280073860 0x7fa2800b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa286800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2800cfdc0 con 0x7fa2800d3c10 2024-09-10T07:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa286800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2800cee40 con 0x7fa28015e860 2024-09-10T07:51:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa286800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa2800d3780 con 0x7fa280073860 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa284a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa2800d3c10 0x7fa280073150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34660/0 (socket says 172.21.15.3:34660) 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa284a00640 1 -- 172.21.15.3:0/749345229 learned_addr learned my addr 172.21.15.3:0/749345229 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.694+0000 7fa285e00640 1 --2- 172.21.15.3:0/749345229 >> v2:172.21.15.3:3300/0 conn(0x7fa280073860 0x7fa2800b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.695+0000 7fa27fe00640 1 -- 172.21.15.3:0/749345229 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399464242 0 0) 0x7fa2800cfdc0 con 0x7fa2800d3c10 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.695+0000 7fa27fe00640 1 -- 172.21.15.3:0/749345229 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa268003520 con 0x7fa2800d3c10 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.695+0000 7fa285e00640 1 -- 172.21.15.3:0/749345229 >> v1:172.21.15.70:6789/0 conn(0x7fa28015e860 legacy=0x7fa2800cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.695+0000 7fa285e00640 1 -- 172.21.15.3:0/749345229 >> v1:172.21.15.3:6789/0 conn(0x7fa2800d3c10 legacy=0x7fa280073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.695+0000 7fa285e00640 1 -- 172.21.15.3:0/749345229 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800cf120 con 0x7fa280073860 2024-09-10T07:51:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.696+0000 7fa285e00640 1 --2- 172.21.15.3:0/749345229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa280073860 0x7fa2800b7880 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa278009d90 tx=0x7fa27802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f75ba9316833420 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.696+0000 7fa27fe00640 1 -- 172.21.15.3:0/749345229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278007740 con 0x7fa280073860 2024-09-10T07:51:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.696+0000 7fa27fe00640 1 -- 172.21.15.3:0/749345229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2780078e0 con 0x7fa280073860 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.696+0000 7fa27fe00640 1 -- 172.21.15.3:0/749345229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278007c90 con 0x7fa280073860 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 -- 172.21.15.3:0/749345229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa280073860 msgr2=0x7fa2800b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 --2- 172.21.15.3:0/749345229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa280073860 0x7fa2800b7880 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa278009d90 tx=0x7fa27802a6f0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 -- 172.21.15.3:0/749345229 shutdown_connections 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 --2- 172.21.15.3:0/749345229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa280073860 0x7fa2800b7880 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 -- 172.21.15.3:0/749345229 >> 172.21.15.3:0/749345229 conn(0x7fa2800d4650 msgr2=0x7fa2800d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 -- 172.21.15.3:0/749345229 shutdown_connections 2024-09-10T07:51:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.697+0000 7fa286800640 1 -- 172.21.15.3:0/749345229 wait complete. 2024-09-10T07:51:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.698+0000 7fa286800640 1 Processor -- start 2024-09-10T07:51:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.698+0000 7fa286800640 1 -- start start 2024-09-10T07:51:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa280073860 0x7fa280156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2800d3c10 0x7fa2801608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2800d2460 con 0x7fa28015e860 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa2800cf400 con 0x7fa280073860 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa286800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa2800d4050 con 0x7fa2800d3c10 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.699+0000 7fa285e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48144/0 (socket says 172.21.15.3:48144) 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 -- 172.21.15.3:0/1894504521 learned_addr learned my addr 172.21.15.3:0/1894504521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285400640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa280073860 0x7fa280156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa284a00640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2800d3c10 0x7fa2801608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 -- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2800d3c10 msgr2=0x7fa2801608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2800d3c10 0x7fa2801608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 -- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa280073860 msgr2=0x7fa280156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa280073860 0x7fa280156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 -- 172.21.15.3:0/1894504521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800cb8c0 con 0x7fa28015e860 2024-09-10T07:51:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.700+0000 7fa285e00640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa278009d90 tx=0x7fa278004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.701+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278040070 con 0x7fa28015e860 2024-09-10T07:51:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.701+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa278031c40 con 0x7fa28015e860 2024-09-10T07:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.701+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278009450 con 0x7fa28015e860 2024-09-10T07:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.701+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2800cbba0 con 0x7fa28015e860 2024-09-10T07:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.701+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa280157390 con 0x7fa28015e860 2024-09-10T07:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.702+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2800cf6b0 con 0x7fa28015e860 2024-09-10T07:51:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.703+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa278039040 con 0x7fa28015e860 2024-09-10T07:51:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.704+0000 7fa27d600640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 0x7fa26006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.704+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fa2780ac0c0 con 0x7fa28015e860 2024-09-10T07:51:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.704+0000 7fa285400640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 0x7fa26006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.705+0000 7fa285400640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 0x7fa26006e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa274000f60 tx=0x7fa274009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.708+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa27807a9d0 con 0x7fa28015e860 2024-09-10T07:51:47.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.930+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2800cfb10 con 0x7fa28015e860 2024-09-10T07:51:47.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.931+0000 7fa27d600640 1 -- 172.21.15.3:0/1894504521 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5263 (secure 0 0 0) 0x7fa27807a2d0 con 0x7fa28015e860 2024-09-10T07:51:47.931 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:47.931 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"btime":"2024-09-10T07:51:46:626491+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:46.568413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702,6690,6355]},"id":5}]} 2024-09-10T07:51:47.931 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 77 2024-09-10T07:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 msgr2=0x7fa26006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 0x7fa26006e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa274000f60 tx=0x7fa274009040 comp rx=0 tx=0).stop 2024-09-10T07:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 msgr2=0x7fa2800cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa278009d90 tx=0x7fa278004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:47.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 shutdown_connections 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa26006be60 0x7fa26006e320 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa28015e860 0x7fa2800cdd40 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2800d3c10 0x7fa2801608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 --2- 172.21.15.3:0/1894504521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa280073860 0x7fa280156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 >> 172.21.15.3:0/1894504521 conn(0x7fa2800d4650 msgr2=0x7fa2800b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 shutdown_connections 2024-09-10T07:51:47.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:47.933+0000 7fa286800640 1 -- 172.21.15.3:0/1894504521 wait complete. 2024-09-10T07:51:47.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:48.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.105+0000 7f089d200640 1 Processor -- start 2024-09-10T07:51:48.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.105+0000 7f089d200640 1 -- start start 2024-09-10T07:51:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.106+0000 7f089d200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f089815b4f0 0x7f08980d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.106+0000 7f089d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0898073d10 0x7f08980d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.106+0000 7f089d200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08980ccd80 con 0x7f089815b110 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.106+0000 7f089d200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f08980b44f0 con 0x7f0898073d10 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.106+0000 7f089d200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f08980cd060 con 0x7f089815b4f0 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0898073d10 0x7f08980d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0898073d10 0x7f08980d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48146/0 (socket says 172.21.15.3:48146) 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 -- 172.21.15.3:0/2329362099 learned_addr learned my addr 172.21.15.3:0/2329362099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0895400640 1 --2- 172.21.15.3:0/2329362099 >> v2:172.21.15.70:3300/0 conn(0x7f089815b4f0 0x7f08980d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 -- 172.21.15.3:0/2329362099 >> v2:172.21.15.70:3300/0 conn(0x7f089815b4f0 msgr2=0x7f08980d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 --2- 172.21.15.3:0/2329362099 >> v2:172.21.15.70:3300/0 conn(0x7f089815b4f0 0x7f08980d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 -- 172.21.15.3:0/2329362099 >> v1:172.21.15.70:6789/0 conn(0x7f089815b110 legacy=0x7f08980b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.107+0000 7f0896800640 1 -- 172.21.15.3:0/2329362099 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08980cd340 con 0x7f0898073d10 2024-09-10T07:51:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.108+0000 7f0896800640 1 --2- 172.21.15.3:0/2329362099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0898073d10 0x7f08980d0720 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f08980d1310 tx=0x7f089002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b7cbb85462b1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.108+0000 7f0894a00640 1 -- 172.21.15.3:0/2329362099 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890036070 con 0x7f0898073d10 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f0894a00640 1 -- 172.21.15.3:0/2329362099 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890031600 con 0x7f0898073d10 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f0894a00640 1 -- 172.21.15.3:0/2329362099 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08900319d0 con 0x7f0898073d10 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 -- 172.21.15.3:0/2329362099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0898073d10 msgr2=0x7f08980d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 --2- 172.21.15.3:0/2329362099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0898073d10 0x7f08980d0720 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f08980d1310 tx=0x7f089002a730 comp rx=0 tx=0).stop 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 -- 172.21.15.3:0/2329362099 shutdown_connections 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 --2- 172.21.15.3:0/2329362099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0898073d10 0x7f08980d0720 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 --2- 172.21.15.3:0/2329362099 >> v2:172.21.15.70:3300/0 conn(0x7f089815b4f0 0x7f08980d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 -- 172.21.15.3:0/2329362099 >> 172.21.15.3:0/2329362099 conn(0x7f089806d000 msgr2=0x7f089806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 -- 172.21.15.3:0/2329362099 shutdown_connections 2024-09-10T07:51:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.109+0000 7f089d200640 1 -- 172.21.15.3:0/2329362099 wait complete. 2024-09-10T07:51:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.111+0000 7f089d200640 1 Processor -- start 2024-09-10T07:51:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.111+0000 7f089d200640 1 -- start start 2024-09-10T07:51:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0898073d10 0x7f0898145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089815b110 0x7f089814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f089806bca0 con 0x7f089815b4f0 2024-09-10T07:51:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f089806bb20 con 0x7f089815b110 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f089d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f089806be20 con 0x7f0898073d10 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f0896800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f0896800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48158/0 (socket says 172.21.15.3:48158) 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f0896800640 1 -- 172.21.15.3:0/2698377684 learned_addr learned my addr 172.21.15.3:0/2698377684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f0895e00640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0898073d10 0x7f0898145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.112+0000 7f0895400640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089815b110 0x7f089814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 -- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0898073d10 msgr2=0x7f0898145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0898073d10 0x7f0898145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 -- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089815b110 msgr2=0x7f089814afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089815b110 0x7f089814afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 -- 172.21.15.3:0/2698377684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0898146040 con 0x7f089815b4f0 2024-09-10T07:51:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f0896800640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0890009f40 tx=0x7f0890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890036050 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890042070 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0898071020 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890009510 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.113+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08980715e0 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.114+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08980d1380 con 0x7f089815b4f0 2024-09-10T07:51:48.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.115+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0890007bf0 con 0x7f089815b4f0 2024-09-10T07:51:48.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.116+0000 7f088e000640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 0x7f087806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.116+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f08900abac0 con 0x7f089815b4f0 2024-09-10T07:51:48.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.116+0000 7f0895e00640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 0x7f087806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.117+0000 7f0895e00640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 0x7f087806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0898154f70 tx=0x7f0888009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.121+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08980d1380 con 0x7f089815b4f0 2024-09-10T07:51:48.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.342+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0898156240 con 0x7f089815b4f0 2024-09-10T07:51:48.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.342+0000 7f088e000640 1 -- 172.21.15.3:0/2698377684 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5263 (secure 0 0 0) 0x7f0890079e10 con 0x7f089815b4f0 2024-09-10T07:51:48.342 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:48.342 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"btime":"2024-09-10T07:51:46:626491+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:51:12.071594+0000","modified":"2024-09-10T07:51:46.568413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6702,"mds_1":6690,"mds_2":6355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6355":{"gid":6355,"name":"b","rank":2,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.70:6835/2352570209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2352570209},{"type":"v1","addr":"172.21.15.70:6835","nonce":2352570209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6690":{"gid":6690,"name":"a","rank":1,"incarnation":71,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6835/4206303617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4206303617},{"type":"v1","addr":"172.21.15.3:6835","nonce":4206303617}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6702":{"gid":6702,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2373315703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2373315703},{"type":"v1","addr":"172.21.15.3:6837","nonce":2373315703}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6702,"qdb_cluster":[6702,6690,6355]},"id":5}]} 2024-09-10T07:51:48.343 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 77 2024-09-10T07:51:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.344+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 msgr2=0x7f087806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.344+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 0x7f087806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0898154f70 tx=0x7f0888009040 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.344+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 msgr2=0x7f0898154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.344+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0890009f40 tx=0x7f0890004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 shutdown_connections 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f087806be30 0x7f087806e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089815b4f0 0x7f0898154420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f089815b110 0x7f089814afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 --2- 172.21.15.3:0/2698377684 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0898073d10 0x7f0898145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 >> 172.21.15.3:0/2698377684 conn(0x7f089806d000 msgr2=0x7f089806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 shutdown_connections 2024-09-10T07:51:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.345+0000 7f089d200640 1 -- 172.21.15.3:0/2698377684 wait complete. 2024-09-10T07:51:48.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.515+0000 7f4c36a00640 1 Processor -- start 2024-09-10T07:51:48.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.515+0000 7f4c36a00640 1 -- start start 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c36a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3015b110 0x7f4c300b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c36a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 0x7f4c300d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c36a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c300b4250 con 0x7f4c30073d10 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c36a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4c300b44f0 con 0x7f4c3015b110 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c36a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4c300ccd80 con 0x7f4c3015b4f0 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3015b110 0x7f4c300b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c3015b110 0x7f4c300b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48168/0 (socket says 172.21.15.3:48168) 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 -- 172.21.15.3:0/4159096442 learned_addr learned my addr 172.21.15.3:0/4159096442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c27400640 1 --2- 172.21.15.3:0/4159096442 >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 0x7f4c300d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 -- 172.21.15.3:0/4159096442 >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 msgr2=0x7f4c300d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 --2- 172.21.15.3:0/4159096442 >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 0x7f4c300d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 -- 172.21.15.3:0/4159096442 >> v1:172.21.15.3:6789/0 conn(0x7f4c30073d10 legacy=0x7f4c3015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c2fe00640 1 -- 172.21.15.3:0/4159096442 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c300cd060 con 0x7f4c3015b110 2024-09-10T07:51:48.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.517+0000 7f4c27400640 1 --2- 172.21.15.3:0/4159096442 >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 0x7f4c300d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.518+0000 7f4c2fe00640 1 --2- 172.21.15.3:0/4159096442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3015b110 0x7f4c300b3d10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b4670 tx=0x7f4c2002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a3ea6c0070421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c2f400640 1 -- 172.21.15.3:0/4159096442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20036020 con 0x7f4c3015b110 2024-09-10T07:51:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c2f400640 1 -- 172.21.15.3:0/4159096442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20032830 con 0x7f4c3015b110 2024-09-10T07:51:48.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c2f400640 1 -- 172.21.15.3:0/4159096442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20032c00 con 0x7f4c3015b110 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 -- 172.21.15.3:0/4159096442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3015b110 msgr2=0x7f4c300b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 --2- 172.21.15.3:0/4159096442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3015b110 0x7f4c300b3d10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b4670 tx=0x7f4c2002c930 comp rx=0 tx=0).stop 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 -- 172.21.15.3:0/4159096442 shutdown_connections 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 --2- 172.21.15.3:0/4159096442 >> v2:172.21.15.70:3300/0 conn(0x7f4c3015b4f0 0x7f4c300d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 --2- 172.21.15.3:0/4159096442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c3015b110 0x7f4c300b3d10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 -- 172.21.15.3:0/4159096442 >> 172.21.15.3:0/4159096442 conn(0x7f4c3006d000 msgr2=0x7f4c3006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 -- 172.21.15.3:0/4159096442 shutdown_connections 2024-09-10T07:51:48.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.519+0000 7f4c36a00640 1 -- 172.21.15.3:0/4159096442 wait complete. 2024-09-10T07:51:48.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.521+0000 7f4c36a00640 1 Processor -- start 2024-09-10T07:51:48.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.521+0000 7f4c36a00640 1 -- start start 2024-09-10T07:51:48.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c3015b110 0x7f4c30072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 0x7f4c30155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300ceb50 con 0x7f4c30073d10 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300cd620 con 0x7f4c3015b4f0 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c36a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4c300cecd0 con 0x7f4c3015b110 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48178/0 (socket says 172.21.15.3:48178) 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c2fe00640 1 -- 172.21.15.3:0/1639703609 learned_addr learned my addr 172.21.15.3:0/1639703609 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c27400640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c3015b110 0x7f4c30072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.522+0000 7f4c34a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 0x7f4c30155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 -- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c3015b110 msgr2=0x7f4c30072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c3015b110 0x7f4c30072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 -- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 msgr2=0x7f4c30155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 0x7f4c30155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 -- 172.21.15.3:0/1639703609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3015fc30 con 0x7f4c30073d10 2024-09-10T07:51:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c34a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 0x7f4c30155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:48.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2fe00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4c2002c970 tx=0x7f4c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20036040 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2003c960 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2003cd50 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.523+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c30072230 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.524+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c300727f0 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.525+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c300b3d10 con 0x7f4c30073d10 2024-09-10T07:51:48.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.525+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c2004d060 con 0x7f4c30073d10 2024-09-10T07:51:48.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.527+0000 7f4c2cc00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 0x7f4c0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.527+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f4c200add50 con 0x7f4c30073d10 2024-09-10T07:51:48.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.527+0000 7f4c27400640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 0x7f4c0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.528+0000 7f4c27400640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 0x7f4c0c06e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c300af7a0 tx=0x7f4c18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.531+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c200b1030 con 0x7f4c30073d10 2024-09-10T07:51:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.717+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c30158c40 con 0x7f4c30073d10 2024-09-10T07:51:48.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.718+0000 7f4c2cc00640 1 -- 172.21.15.3:0/1639703609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16596 (secure 0 0 0) 0x7f4c20041070 con 0x7f4c30073d10 2024-09-10T07:51:48.718 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:48.718 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":78,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:13.646589+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:09.867983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T07:51:11.065097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 msgr2=0x7f4c0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 0x7f4c0c06e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c300af7a0 tx=0x7f4c18008040 comp rx=0 tx=0).stop 2024-09-10T07:51:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 msgr2=0x7f4c30140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4c2002c970 tx=0x7f4c200046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 shutdown_connections 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4c0c06be30 0x7f4c0c06e2f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3015b4f0 0x7f4c30155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c3015b110 0x7f4c30072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 --2- 172.21.15.3:0/1639703609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30073d10 0x7f4c30140970 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 >> 172.21.15.3:0/1639703609 conn(0x7f4c3006d000 msgr2=0x7f4c300d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 shutdown_connections 2024-09-10T07:51:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.720+0000 7f4c36a00640 1 -- 172.21.15.3:0/1639703609 wait complete. 2024-09-10T07:51:48.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:51:48.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.893+0000 7fb4e4a00640 1 Processor -- start 2024-09-10T07:51:48.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.893+0000 7fb4e4a00640 1 -- start start 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4e015e860 0x7fb4e00cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 0x7fb4e00d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e00d4150 0x7fb4e00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e00d3780 con 0x7fb4e00d4150 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e00cee40 con 0x7fb4e015e860 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4e4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e00cf120 con 0x7fb4e00cfbf0 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4dfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e00d4150 0x7fb4e00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4dfe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb4e00d4150 0x7fb4e00b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48186/0 (socket says 172.21.15.3:48186) 2024-09-10T07:51:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4dfe00640 1 -- 172.21.15.3:0/1374304328 learned_addr learned my addr 172.21.15.3:0/1374304328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4df400640 1 --2- 172.21.15.3:0/1374304328 >> v2:172.21.15.70:3300/0 conn(0x7fb4e015e860 0x7fb4e00cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.894+0000 7fb4dea00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 0x7fb4e00d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dfe00640 1 -- 172.21.15.3:0/1374304328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 msgr2=0x7fb4e00d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dfe00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 0x7fb4e00d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dfe00640 1 -- 172.21.15.3:0/1374304328 >> v2:172.21.15.70:3300/0 conn(0x7fb4e015e860 msgr2=0x7fb4e00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dfe00640 1 --2- 172.21.15.3:0/1374304328 >> v2:172.21.15.70:3300/0 conn(0x7fb4e015e860 0x7fb4e00cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dfe00640 1 -- 172.21.15.3:0/1374304328 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e00d3410 con 0x7fb4e00d4150 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.895+0000 7fb4dea00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 0x7fb4e00d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4dfe00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e00b7880 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00ba7c0 tx=0x7fb4d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e7386c66b7960c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4de000640 1 -- 172.21.15.3:0/1374304328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8036070 con 0x7fb4e00d4150 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4de000640 1 -- 172.21.15.3:0/1374304328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8031600 con 0x7fb4e00d4150 2024-09-10T07:51:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4de000640 1 -- 172.21.15.3:0/1374304328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80319d0 con 0x7fb4e00d4150 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4e4a00640 1 -- 172.21.15.3:0/1374304328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 msgr2=0x7fb4e00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.896+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e00b7880 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00ba7c0 tx=0x7fb4d802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 -- 172.21.15.3:0/1374304328 shutdown_connections 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e00b7880 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/1374304328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e00cfbf0 0x7fb4e00d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/1374304328 >> v2:172.21.15.70:3300/0 conn(0x7fb4e015e860 0x7fb4e00cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 -- 172.21.15.3:0/1374304328 >> 172.21.15.3:0/1374304328 conn(0x7fb4e00d4650 msgr2=0x7fb4e00d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 -- 172.21.15.3:0/1374304328 shutdown_connections 2024-09-10T07:51:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.897+0000 7fb4e4a00640 1 -- 172.21.15.3:0/1374304328 wait complete. 2024-09-10T07:51:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.898+0000 7fb4e4a00640 1 Processor -- start 2024-09-10T07:51:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.898+0000 7fb4e4a00640 1 -- start start 2024-09-10T07:51:48.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e00cfbf0 0x7fb4e01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 0x7fb4e01e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00b7dc0 con 0x7fb4e00d4150 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00d2460 con 0x7fb4e00cfbf0 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4e4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e00b7f40 con 0x7fb4e015e860 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.899+0000 7fb4dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48202/0 (socket says 172.21.15.3:48202) 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 -- 172.21.15.3:0/2236084491 learned_addr learned my addr 172.21.15.3:0/2236084491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4df400640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e00cfbf0 0x7fb4e01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dfe00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 0x7fb4e01e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 -- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 msgr2=0x7fb4e01e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 0x7fb4e01e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 -- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e00cfbf0 msgr2=0x7fb4e01606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e00cfbf0 0x7fb4e01606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 -- 172.21.15.3:0/2236084491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e0161170 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dfe00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 0x7fb4e01e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.900+0000 7fb4dea00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e590 tx=0x7fb4cc00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.901+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.901+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc013610 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.901+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00f040 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.901+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e0161450 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.901+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e00ac950 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.902+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e00d0660 con 0x7fb4e00d4150 2024-09-10T07:51:48.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.903+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4cc01a070 con 0x7fb4e00d4150 2024-09-10T07:51:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.903+0000 7fb4d7400640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 0x7fb4c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.904+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fb4cc08a450 con 0x7fb4e00d4150 2024-09-10T07:51:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.904+0000 7fb4df400640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 0x7fb4c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.904+0000 7fb4df400640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 0x7fb4c006e320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00b3030 tx=0x7fb4d0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:48.908+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4e00d0660 con 0x7fb4e00d4150 2024-09-10T07:51:49.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.129+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb4e00b7880 con 0x7fb4e00d4150 2024-09-10T07:51:49.129 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:51:49.129+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:51:49.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.131+0000 7fb4d7400640 1 -- 172.21.15.3:0/2236084491 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7fb4cc058840 con 0x7fb4e00d4150 2024-09-10T07:51:49.131 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:51:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.134+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 msgr2=0x7fb4c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.134+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 0x7fb4c006e320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00b3030 tx=0x7fb4d0009190 comp rx=0 tx=0).stop 2024-09-10T07:51:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.134+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 msgr2=0x7fb4e01e8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.134+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e590 tx=0x7fb4cc00ea50 comp rx=0 tx=0).stop 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 shutdown_connections 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb4c006be60 0x7fb4c006e320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb4e015e860 0x7fb4e01e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4e00d4150 0x7fb4e01e8500 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 --2- 172.21.15.3:0/2236084491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e00cfbf0 0x7fb4e01606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 >> 172.21.15.3:0/2236084491 conn(0x7fb4e00d4650 msgr2=0x7fb4e00d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 shutdown_connections 2024-09-10T07:51:49.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.135+0000 7fb4e4a00640 1 -- 172.21.15.3:0/2236084491 wait complete. 2024-09-10T07:51:49.145 INFO:tasks.ceph.mds.c.smithi003.stderr: -9262> 2024-09-10T07:51:17.619+0000 7f2ad8a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:51:49.147 INFO:tasks.ceph.mds.c.smithi003.stderr: -7421> 2024-09-10T07:51:27.619+0000 7f2ad8a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:51:49.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:51:49.186 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:51:49.187 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:51:49.203 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:51:49.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.318+0000 7f9238a00640 1 Processor -- start 2024-09-10T07:51:49.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.318+0000 7f9238a00640 1 -- start start 2024-09-10T07:51:49.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.319+0000 7f9238a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f923415b230 0x7f923415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.319+0000 7f9238a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923415eff0 con 0x7f92340b4c90 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.319+0000 7f9238a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923415f2b0 con 0x7f9234072c50 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.319+0000 7f9238a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f923415fbe0 con 0x7f923415b230 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f923415b230 0x7f923415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f923415b230 0x7f923415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48242/0 (socket says 172.21.15.3:48242) 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 -- 172.21.15.3:0/4235410351 learned_addr learned my addr 172.21.15.3:0/4235410351 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9232800640 1 --1- 172.21.15.3:0/4235410351 >> v1:172.21.15.3:6789/0 conn(0x7f92340b4c90 0x7f9234071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34670/0 (socket says 172.21.15.3:34670) 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 -- 172.21.15.3:0/4235410351 >> v1:172.21.15.70:6789/0 conn(0x7f9234072c50 legacy=0x7f92340b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 -- 172.21.15.3:0/4235410351 >> v1:172.21.15.3:6789/0 conn(0x7f92340b4c90 legacy=0x7f9234071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.320+0000 7f9231e00640 1 -- 172.21.15.3:0/4235410351 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923415f590 con 0x7f923415b230 2024-09-10T07:51:49.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.321+0000 7f9231e00640 1 --2- 172.21.15.3:0/4235410351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f923415c390 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f922400bb10 tx=0x7f922402c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6979fcfa69b90459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:49.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.321+0000 7f9230a00640 1 -- 172.21.15.3:0/4235410351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224035070 con 0x7f923415b230 2024-09-10T07:51:49.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.321+0000 7f9230a00640 1 -- 172.21.15.3:0/4235410351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9224031830 con 0x7f923415b230 2024-09-10T07:51:49.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.321+0000 7f9230a00640 1 -- 172.21.15.3:0/4235410351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224031c00 con 0x7f923415b230 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 -- 172.21.15.3:0/4235410351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 msgr2=0x7f923415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 --2- 172.21.15.3:0/4235410351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f923415c390 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f922400bb10 tx=0x7f922402c9d0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 -- 172.21.15.3:0/4235410351 shutdown_connections 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 --2- 172.21.15.3:0/4235410351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f923415c390 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 -- 172.21.15.3:0/4235410351 >> 172.21.15.3:0/4235410351 conn(0x7f92340b5cb0 msgr2=0x7f923406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 -- 172.21.15.3:0/4235410351 shutdown_connections 2024-09-10T07:51:49.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.322+0000 7f9238a00640 1 -- 172.21.15.3:0/4235410351 wait complete. 2024-09-10T07:51:49.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.324+0000 7f9238a00640 1 Processor -- start 2024-09-10T07:51:49.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.324+0000 7f9238a00640 1 -- start start 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 0x7f923414fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 0x7f9234153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f923415b610 con 0x7f923415b230 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f923415c8d0 con 0x7f9234072c50 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9238a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f923415b790 con 0x7f92340b4c90 2024-09-10T07:51:49.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9232800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9232800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48258/0 (socket says 172.21.15.3:48258) 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.325+0000 7f9232800640 1 -- 172.21.15.3:0/3497644861 learned_addr learned my addr 172.21.15.3:0/3497644861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9231e00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 0x7f923414fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9231400640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 0x7f9234153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 -- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 msgr2=0x7f9234153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 0x7f9234153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 -- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 msgr2=0x7f923414fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 0x7f923414fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 -- 172.21.15.3:0/3497644861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9234150960 con 0x7f923415b230 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9231e00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 0x7f923414fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9231400640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 0x7f9234153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9232800640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c780 tx=0x7f922c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c017070 con 0x7f923415b230 2024-09-10T07:51:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c00a640 con 0x7f923415b230 2024-09-10T07:51:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9234150c40 con 0x7f923415b230 2024-09-10T07:51:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c00aa30 con 0x7f923415b230 2024-09-10T07:51:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.326+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9234151200 con 0x7f923415b230 2024-09-10T07:51:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.327+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923406e1e0 con 0x7f923415b230 2024-09-10T07:51:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.328+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f922c01a070 con 0x7f923415b230 2024-09-10T07:51:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.330+0000 7f922a000640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 0x7f921406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.330+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5967+0+0 (secure 0 0 0) 0x7f922c08a570 con 0x7f923415b230 2024-09-10T07:51:49.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.330+0000 7f9231e00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 0x7f921406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:49.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.331+0000 7f9231e00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 0x7f921406e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9224037000 tx=0x7f9224008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:49.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.334+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f922c058730 con 0x7f923415b230 2024-09-10T07:51:49.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:49.555+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9234151690 con 0x7f923415b230 2024-09-10T07:51:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.136+0000 7f922a000640 1 -- 172.21.15.3:0/3497644861 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7f922c05ca30 con 0x7f923415b230 2024-09-10T07:51:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 msgr2=0x7f921406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 0x7f921406e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9224037000 tx=0x7f9224008040 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 msgr2=0x7f92341597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c780 tx=0x7f922c00cc40 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 shutdown_connections 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f921406be60 0x7f921406e320 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f923415b230 0x7f92341597b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92340b4c90 0x7f9234153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 --2- 172.21.15.3:0/3497644861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9234072c50 0x7f923414fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 >> 172.21.15.3:0/3497644861 conn(0x7f92340b5cb0 msgr2=0x7f9234071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 shutdown_connections 2024-09-10T07:51:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.139+0000 7f9238a00640 1 -- 172.21.15.3:0/3497644861 wait complete. 2024-09-10T07:51:50.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:51:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.358+0000 7f1045e00640 1 Processor -- start 2024-09-10T07:51:50.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.358+0000 7f1045e00640 1 -- start start 2024-09-10T07:51:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f1045e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10400b26a0 0x7f10401670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f1045e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10400cf1f0 con 0x7f104015c530 2024-09-10T07:51:50.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f1045e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10400ce150 con 0x7f10400cd910 2024-09-10T07:51:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f1045e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f10400d00c0 con 0x7f10400b26a0 2024-09-10T07:51:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f103ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f104015c530 0x7f10400cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34676/0 (socket says 172.21.15.3:34676) 2024-09-10T07:51:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f103ea00640 1 -- 172.21.15.3:0/939898448 learned_addr learned my addr 172.21.15.3:0/939898448 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:50.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f103f400640 1 --2- 172.21.15.3:0/939898448 >> v2:172.21.15.70:3300/0 conn(0x7f10400b26a0 0x7f10401670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225736810 0 0) 0x7f10400cf1f0 con 0x7f104015c530 2024-09-10T07:51:50.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.360+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1024003520 con 0x7f104015c530 2024-09-10T07:51:50.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3793168099 0 0) 0x7f1024003520 con 0x7f104015c530 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 >> v2:172.21.15.70:3300/0 conn(0x7f10400b26a0 msgr2=0x7f10401670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 --2- 172.21.15.3:0/939898448 >> v2:172.21.15.70:3300/0 conn(0x7f10400b26a0 0x7f10401670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 >> v1:172.21.15.70:6789/0 conn(0x7f10400cd910 legacy=0x7f10400b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400ce430 con 0x7f104015c530 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f1030001670 con 0x7f104015c530 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 >> v1:172.21.15.3:6789/0 conn(0x7f104015c530 legacy=0x7f10400cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1024003a00 0x7f1024003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1024004800 0x7f1024004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f102400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f102400c090 con 0x7f10240051c0 2024-09-10T07:51:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f102400c240 con 0x7f1024003a00 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f102400c440 con 0x7f1024004800 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103f400640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f102400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f103ea00640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1024003a00 0x7f1024003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.361+0000 7f1036000640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1024004800 0x7f1024004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 -- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1024004800 msgr2=0x7f1024004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1024004800 0x7f1024004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 -- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1024003a00 msgr2=0x7f1024003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1024003a00 0x7f1024003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 -- 172.21.15.3:0/939898448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1024003520 con 0x7f10240051c0 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103ea00640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1024003a00 0x7f1024003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:51:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.362+0000 7f103f400640 1 -- 172.21.15.3:0/939898448 reap_dead start 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f103f400640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f102400b9c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1038009f90 tx=0x7f1024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3c527e7daa67e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038035020 con 0x7f10240051c0 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038003b90 con 0x7f10240051c0 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f103e000640 1 -- 172.21.15.3:0/939898448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030007c20 con 0x7f10240051c0 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 -- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 msgr2=0x7f102400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f102400b9c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1038009f90 tx=0x7f1024005830 comp rx=0 tx=0).stop 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 -- 172.21.15.3:0/939898448 shutdown_connections 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 --2- 172.21.15.3:0/939898448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f102400b9c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 -- 172.21.15.3:0/939898448 >> 172.21.15.3:0/939898448 conn(0x7f1040062cc0 msgr2=0x7f104006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 -- 172.21.15.3:0/939898448 shutdown_connections 2024-09-10T07:51:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.363+0000 7f1045e00640 1 -- 172.21.15.3:0/939898448 wait complete. 2024-09-10T07:51:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.365+0000 7f1045e00640 1 Processor -- start 2024-09-10T07:51:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.365+0000 7f1045e00640 1 -- start start 2024-09-10T07:51:50.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 0x7f104015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 0x7f10401f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10400cecd0 con 0x7f10240051c0 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10400ce9f0 con 0x7f10401f42a0 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f1045e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f104015c910 con 0x7f104015f910 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f103ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f103ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48290/0 (socket says 172.21.15.3:48290) 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.366+0000 7f103ea00640 1 -- 172.21.15.3:0/3187903939 learned_addr learned my addr 172.21.15.3:0/3187903939 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103f400640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 0x7f10401f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 -- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 msgr2=0x7f104015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1036000640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 0x7f104015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 0x7f104015d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 -- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 msgr2=0x7f10401f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 0x7f10401f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 -- 172.21.15.3:0/3187903939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104015e1d0 con 0x7f10240051c0 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103f400640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 0x7f10401f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1036000640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 0x7f104015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f103ea00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1030004f40 tx=0x7f10300074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103000f960 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1030018070 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104015e4b0 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.367+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103000fd20 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.368+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104015ea70 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.368+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10400b1f90 con 0x7f10240051c0 2024-09-10T07:51:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.370+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f103001d460 con 0x7f10240051c0 2024-09-10T07:51:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.371+0000 7f1037e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 0x7f101c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.371+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5967+0+0 (secure 0 0 0) 0x7f103008afb0 con 0x7f10240051c0 2024-09-10T07:51:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.371+0000 7f1036000640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 0x7f101c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.372+0000 7f1036000640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 0x7f101c06e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1028000c00 tx=0x7f1028009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:50.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.375+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f10300590e0 con 0x7f10240051c0 2024-09-10T07:51:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:50.559+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f104015ee10 con 0x7f10240051c0 2024-09-10T07:51:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.138+0000 7f1037e00640 1 -- 172.21.15.3:0/3187903939 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f103005d3e0 con 0x7f10240051c0 2024-09-10T07:51:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:51:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.140+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 msgr2=0x7f101c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.140+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 0x7f101c06e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1028000c00 tx=0x7f1028009040 comp rx=0 tx=0).stop 2024-09-10T07:51:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.140+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 msgr2=0x7f104014bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.140+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1030004f40 tx=0x7f10300074f0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 shutdown_connections 2024-09-10T07:51:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f101c06be30 0x7f101c06e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10401f42a0 0x7f10401f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f104015f910 0x7f104015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 --2- 172.21.15.3:0/3187903939 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10240051c0 0x7f104014bd40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.141+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 >> 172.21.15.3:0/3187903939 conn(0x7f1040062cc0 msgr2=0x7f104006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.143+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 shutdown_connections 2024-09-10T07:51:51.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.143+0000 7f1045e00640 1 -- 172.21.15.3:0/3187903939 wait complete. 2024-09-10T07:51:51.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:51:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.325+0000 7fb60ca00640 1 Processor -- start 2024-09-10T07:51:51.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.325+0000 7fb60ca00640 1 -- start start 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.326+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 0x7fb6080d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.326+0000 7fb60ca00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6080cf090 con 0x7fb60815e7c0 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.326+0000 7fb60ca00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6080cf370 con 0x7fb60806bb20 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.326+0000 7fb60ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb6080cfca0 con 0x7fb6080cead0 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb60815e7c0 0x7fb608072d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34682/0 (socket says 172.21.15.3:34682) 2024-09-10T07:51:51.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605e00640 1 -- 172.21.15.3:0/3280371898 learned_addr learned my addr 172.21.15.3:0/3280371898 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:51.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605400640 1 --2- 172.21.15.3:0/3280371898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 0x7fb6080d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:51.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018379697 0 0) 0x7fb6080cf090 con 0x7fb60815e7c0 2024-09-10T07:51:51.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5ec003520 con 0x7fb60815e7c0 2024-09-10T07:51:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569032025 0 0) 0x7fb6080cf370 con 0x7fb60806bb20 2024-09-10T07:51:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6080cf090 con 0x7fb60806bb20 2024-09-10T07:51:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605400640 1 -- 172.21.15.3:0/3280371898 >> v1:172.21.15.70:6789/0 conn(0x7fb60806bb20 legacy=0x7fb6080b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605400640 1 -- 172.21.15.3:0/3280371898 >> v1:172.21.15.3:6789/0 conn(0x7fb60815e7c0 legacy=0x7fb608072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.327+0000 7fb605400640 1 -- 172.21.15.3:0/3280371898 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080cf650 con 0x7fb6080cead0 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.328+0000 7fb605400640 1 --2- 172.21.15.3:0/3280371898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 0x7fb6080d0130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097d0 tx=0x7fb5f402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416470f24cd5c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037070 con 0x7fb6080cead0 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4032440 con 0x7fb6080cead0 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb604a00640 1 -- 172.21.15.3:0/3280371898 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032810 con 0x7fb6080cead0 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 -- 172.21.15.3:0/3280371898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 msgr2=0x7fb6080d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3280371898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 0x7fb6080d0130 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097d0 tx=0x7fb5f402a180 comp rx=0 tx=0).stop 2024-09-10T07:51:51.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 -- 172.21.15.3:0/3280371898 shutdown_connections 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3280371898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6080cead0 0x7fb6080d0130 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 -- 172.21.15.3:0/3280371898 >> 172.21.15.3:0/3280371898 conn(0x7fb6080b28a0 msgr2=0x7fb6080b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 -- 172.21.15.3:0/3280371898 shutdown_connections 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.329+0000 7fb60ca00640 1 -- 172.21.15.3:0/3280371898 wait complete. 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.331+0000 7fb60ca00640 1 Processor -- start 2024-09-10T07:51:51.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.331+0000 7fb60ca00640 1 -- start start 2024-09-10T07:51:51.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60806bb20 0x7fb60809da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:51.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:51.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 0x7fb60815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:51.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb60806bfb0 con 0x7fb6080cead0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb60815eba0 con 0x7fb60815e7c0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb60ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb60806c130 con 0x7fb60806bb20 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb605400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb605400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48292/0 (socket says 172.21.15.3:48292) 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb605400640 1 -- 172.21.15.3:0/3475885732 learned_addr learned my addr 172.21.15.3:0/3475885732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb605e00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60806bb20 0x7fb60809da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.332+0000 7fb606800640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 0x7fb60815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 -- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60806bb20 msgr2=0x7fb60809da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60806bb20 0x7fb60809da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 -- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 msgr2=0x7fb60815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 0x7fb60815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 -- 172.21.15.3:0/3475885732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60809df80 con 0x7fb6080cead0 2024-09-10T07:51:51.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb606800640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 0x7fb60815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:51.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb605400640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:51.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037050 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4043070 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f403cb30 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60814ec00 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.333+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60809c1a0 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.334+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6080d0130 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.335+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5f403ccd0 con 0x7fb6080cead0 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.336+0000 7fb5fe000640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 0x7fb5e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:51.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.336+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5529+0+0 (secure 0 0 0) 0x7fb5f40ab9f0 con 0x7fb6080cead0 2024-09-10T07:51:51.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.336+0000 7fb605e00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 0x7fb5e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:51.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.337+0000 7fb605e00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 0x7fb5e806e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f60 tx=0x7fb5f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:51.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.341+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb60814ec00 con 0x7fb6080cead0 2024-09-10T07:51:51.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:51.525+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6080b3000 con 0x7fb6080cead0 2024-09-10T07:51:52.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.143+0000 7fb5fe000640 1 -- 172.21.15.3:0/3475885732 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7fb5f407a3b0 con 0x7fb6080cead0 2024-09-10T07:51:52.143 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:51:52.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 msgr2=0x7fb5e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 0x7fb5e806e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f60 tx=0x7fb5f8009040 comp rx=0 tx=0).stop 2024-09-10T07:51:52.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 msgr2=0x7fb60814e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098f0 tx=0x7fb5f40046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 shutdown_connections 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb5e806be60 0x7fb5e806e320 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e7c0 0x7fb60815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6080cead0 0x7fb60814e6c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 --2- 172.21.15.3:0/3475885732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb60806bb20 0x7fb60809da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 >> 172.21.15.3:0/3475885732 conn(0x7fb6080b28a0 msgr2=0x7fb608072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:52.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 shutdown_connections 2024-09-10T07:51:52.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.146+0000 7fb60ca00640 1 -- 172.21.15.3:0/3475885732 wait complete. 2024-09-10T07:51:52.167 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:51:52.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:51:52.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:51:52.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.326+0000 7f9c2ca00640 1 Processor -- start 2024-09-10T07:51:52.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.326+0000 7f9c2ca00640 1 -- start start 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c2ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c281695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c2ca00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c280d3de0 con 0x7f9c28161b70 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c2ca00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c280d40c0 con 0x7f9c280d45a0 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c2ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c28161f50 con 0x7f9c281671b0 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c27400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9c28161b70 0x7f9c280d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34694/0 (socket says 172.21.15.3:34694) 2024-09-10T07:51:52.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c27400640 1 -- 172.21.15.3:0/1635472126 learned_addr learned my addr 172.21.15.3:0/1635472126 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:52.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.327+0000 7f9c27e00640 1 --2- 172.21.15.3:0/1635472126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c281695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971609647 0 0) 0x7f9c280d3de0 con 0x7f9c28161b70 2024-09-10T07:51:52.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c0c003520 con 0x7f9c28161b70 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698942847 0 0) 0x7f9c280d40c0 con 0x7f9c280d45a0 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c280d3de0 con 0x7f9c280d45a0 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c27e00640 1 -- 172.21.15.3:0/1635472126 >> v1:172.21.15.70:6789/0 conn(0x7f9c280d45a0 legacy=0x7f9c280d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c27e00640 1 -- 172.21.15.3:0/1635472126 >> v1:172.21.15.3:6789/0 conn(0x7f9c28161b70 legacy=0x7f9c280d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.328+0000 7f9c27e00640 1 -- 172.21.15.3:0/1635472126 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c280cf690 con 0x7f9c281671b0 2024-09-10T07:51:52.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.329+0000 7f9c27e00640 1 --2- 172.21.15.3:0/1635472126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c281695a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009dc0 tx=0x7f9c2002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c318797deb65549 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.329+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20007740 con 0x7f9c281671b0 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.329+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c200078e0 con 0x7f9c281671b0 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.329+0000 7f9c26000640 1 -- 172.21.15.3:0/1635472126 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20007cb0 con 0x7f9c281671b0 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 -- 172.21.15.3:0/1635472126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 msgr2=0x7f9c281695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/1635472126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c281695a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009dc0 tx=0x7f9c2002a6d0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 -- 172.21.15.3:0/1635472126 shutdown_connections 2024-09-10T07:51:52.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/1635472126 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c281695a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 -- 172.21.15.3:0/1635472126 >> 172.21.15.3:0/1635472126 conn(0x7f9c28062630 msgr2=0x7f9c28062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:52.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 -- 172.21.15.3:0/1635472126 shutdown_connections 2024-09-10T07:51:52.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.330+0000 7f9c2ca00640 1 -- 172.21.15.3:0/1635472126 wait complete. 2024-09-10T07:51:52.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.332+0000 7f9c2ca00640 1 Processor -- start 2024-09-10T07:51:52.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.333+0000 7f9c2ca00640 1 -- start start 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c28161b70 0x7f9c280cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c2814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c280d3a30 con 0x7f9c280d45a0 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9c280c9fc0 con 0x7f9c28161b70 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c2ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c280cc160 con 0x7f9c281671b0 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c27400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c27400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48298/0 (socket says 172.21.15.3:48298) 2024-09-10T07:51:52.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c27400640 1 -- 172.21.15.3:0/3045107606 learned_addr learned my addr 172.21.15.3:0/3045107606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:52.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c26a00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c28161b70 0x7f9c280cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.334+0000 7f9c27e00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c2814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 -- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 msgr2=0x7f9c2814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c2814d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 -- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c28161b70 msgr2=0x7f9c280cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c28161b70 0x7f9c280cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 -- 172.21.15.3:0/3045107606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c280cc440 con 0x7f9c280d45a0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c27400640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c490 tx=0x7f9c1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18017070 con 0x7f9c280d45a0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.335+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c18005ce0 con 0x7f9c280d45a0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.336+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18007290 con 0x7f9c280d45a0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.336+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c280cc720 con 0x7f9c280d45a0 2024-09-10T07:51:52.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.336+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c280ce2e0 con 0x7f9c280d45a0 2024-09-10T07:51:52.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.337+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c280cf360 con 0x7f9c280d45a0 2024-09-10T07:51:52.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.338+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c1801a070 con 0x7f9c280d45a0 2024-09-10T07:51:52.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.339+0000 7f9c1f400640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 0x7f9c0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.339+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f9c18089f30 con 0x7f9c280d45a0 2024-09-10T07:51:52.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.340+0000 7f9c26a00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 0x7f9c0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.341+0000 7f9c26a00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 0x7f9c0806e320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c14007fd0 tx=0x7f9c14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.343+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c18058440 con 0x7f9c280d45a0 2024-09-10T07:51:52.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.529+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9c2815c9b0 con 0x7f9c280d45a0 2024-09-10T07:51:52.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.533+0000 7f9c1f400640 1 -- 172.21.15.3:0/3045107606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f9c1805c740 con 0x7f9c280d45a0 2024-09-10T07:51:52.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 msgr2=0x7f9c0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 0x7f9c0806e320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c14007fd0 tx=0x7f9c14009040 comp rx=0 tx=0).stop 2024-09-10T07:51:52.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 msgr2=0x7f9c280b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c490 tx=0x7f9c1800c950 comp rx=0 tx=0).stop 2024-09-10T07:51:52.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 shutdown_connections 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9c0806be60 0x7f9c0806e320 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c281671b0 0x7f9c2814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c28161b70 0x7f9c280cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 --2- 172.21.15.3:0/3045107606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c280d45a0 0x7f9c280b9c90 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 >> 172.21.15.3:0/3045107606 conn(0x7f9c28062630 msgr2=0x7f9c280d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 shutdown_connections 2024-09-10T07:51:52.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.535+0000 7f9c2ca00640 1 -- 172.21.15.3:0/3045107606 wait complete. 2024-09-10T07:51:52.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-10T07:51:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.708+0000 7ff64b000640 1 Processor -- start 2024-09-10T07:51:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.708+0000 7ff64b000640 1 -- start start 2024-09-10T07:51:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff64b000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff64415b110 0x7ff6440b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff64b000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6440b4250 con 0x7ff64415b4f0 2024-09-10T07:51:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff64b000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6440ccd80 con 0x7ff6440d08f0 2024-09-10T07:51:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff64b000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff6440b44f0 con 0x7ff64415b110 2024-09-10T07:51:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff63be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff64415b4f0 0x7ff6440d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34704/0 (socket says 172.21.15.3:34704) 2024-09-10T07:51:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff63be00640 1 -- 172.21.15.3:0/1851679961 learned_addr learned my addr 172.21.15.3:0/1851679961 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.709+0000 7ff643e00640 1 --2- 172.21.15.3:0/1851679961 >> v2:172.21.15.70:3300/0 conn(0x7ff64415b110 0x7ff6440b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183758019 0 0) 0x7ff6440b4250 con 0x7ff64415b4f0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff628003520 con 0x7ff64415b4f0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357118055 0 0) 0x7ff6440ccd80 con 0x7ff6440d08f0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6440b4250 con 0x7ff6440d08f0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3363645912 0 0) 0x7ff628003520 con 0x7ff64415b4f0 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643e00640 1 -- 172.21.15.3:0/1851679961 >> v1:172.21.15.70:6789/0 conn(0x7ff6440d08f0 legacy=0x7ff6440cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643e00640 1 -- 172.21.15.3:0/1851679961 >> v1:172.21.15.3:6789/0 conn(0x7ff64415b4f0 legacy=0x7ff6440d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.710+0000 7ff643e00640 1 -- 172.21.15.3:0/1851679961 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6440cd060 con 0x7ff64415b110 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.711+0000 7ff643e00640 1 --2- 172.21.15.3:0/1851679961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b110 0x7ff6440b3d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff6340097a0 tx=0x7ff63402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc930bfc61bbd46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.711+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634037070 con 0x7ff64415b110 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.711+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63402ac50 con 0x7ff64415b110 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.711+0000 7ff643400640 1 -- 172.21.15.3:0/1851679961 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c001690 con 0x7ff64415b110 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 -- 172.21.15.3:0/1851679961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b110 msgr2=0x7ff6440b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 --2- 172.21.15.3:0/1851679961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b110 0x7ff6440b3d10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff6340097a0 tx=0x7ff63402a1c0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 -- 172.21.15.3:0/1851679961 shutdown_connections 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 --2- 172.21.15.3:0/1851679961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b110 0x7ff6440b3d10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 -- 172.21.15.3:0/1851679961 >> 172.21.15.3:0/1851679961 conn(0x7ff64406d000 msgr2=0x7ff64406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 -- 172.21.15.3:0/1851679961 shutdown_connections 2024-09-10T07:51:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.712+0000 7ff64b000640 1 -- 172.21.15.3:0/1851679961 wait complete. 2024-09-10T07:51:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.714+0000 7ff64b000640 1 Processor -- start 2024-09-10T07:51:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.714+0000 7ff64b000640 1 -- start start 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6440d08f0 0x7ff6441f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b4f0 0x7ff6440ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff644073d10 con 0x7ff64415b110 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff6440cd620 con 0x7ff64415b4f0 2024-09-10T07:51:52.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff64b000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff644073e90 con 0x7ff6440d08f0 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff63be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff63be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48314/0 (socket says 172.21.15.3:48314) 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff63be00640 1 -- 172.21.15.3:0/2570193942 learned_addr learned my addr 172.21.15.3:0/2570193942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff643e00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6440d08f0 0x7ff6441f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.715+0000 7ff649000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b4f0 0x7ff6440ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 -- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6440d08f0 msgr2=0x7ff6441f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6440d08f0 0x7ff6441f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 -- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b4f0 msgr2=0x7ff6440ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b4f0 0x7ff6440ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 -- 172.21.15.3:0/2570193942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441f8860 con 0x7ff64415b110 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff63be00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c520 tx=0x7ff62c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c017070 con 0x7ff64415b110 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c005ce0 con 0x7ff64415b110 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c007290 con 0x7ff64415b110 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6441f44c0 con 0x7ff64415b110 2024-09-10T07:51:52.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.716+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441f52a0 con 0x7ff64415b110 2024-09-10T07:51:52.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.717+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6440b3d10 con 0x7ff64415b110 2024-09-10T07:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.718+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff62c01a070 con 0x7ff64415b110 2024-09-10T07:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.719+0000 7ff640c00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 0x7ff62006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.719+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7ff62c089f90 con 0x7ff64415b110 2024-09-10T07:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.719+0000 7ff643e00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 0x7ff62006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.720+0000 7ff643e00640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 0x7ff62006e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff634034000 tx=0x7ff63402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:52.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.723+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff6441f8860 con 0x7ff64415b110 2024-09-10T07:51:52.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:52.979+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7ff644072fd0 con 0x7ff64415b110 2024-09-10T07:51:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.145+0000 7ff640c00640 1 -- 172.21.15.3:0/2570193942 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v278) ==== 141+0+0 (secure 0 0 0) 0x7ff62c058530 con 0x7ff64415b110 2024-09-10T07:51:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 msgr2=0x7ff62006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 0x7ff62006e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff634034000 tx=0x7ff63402e000 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 msgr2=0x7ff64414f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00c520 tx=0x7ff62c00c9e0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 shutdown_connections 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff62006be30 0x7ff62006e2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff64415b4f0 0x7ff6440ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff64415b110 0x7ff64414f8e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 --2- 172.21.15.3:0/2570193942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6440d08f0 0x7ff6441f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 >> 172.21.15.3:0/2570193942 conn(0x7ff64406d000 msgr2=0x7ff6440cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 shutdown_connections 2024-09-10T07:51:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.148+0000 7ff64b000640 1 -- 172.21.15.3:0/2570193942 wait complete. 2024-09-10T07:51:53.164 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:51:53.165 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:51:53.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:51:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.321+0000 7f0877800640 1 Processor -- start 2024-09-10T07:51:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.321+0000 7f0877800640 1 -- start start 2024-09-10T07:51:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0877800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015fdb0 0x7f087006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0877800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0870160220 0x7f087006c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0877800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08700b36a0 con 0x7f0870161b30 2024-09-10T07:51:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0877800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f08700b3980 con 0x7f087015fdb0 2024-09-10T07:51:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0877800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08700b3c60 con 0x7f0870160220 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f086fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015fdb0 0x7f087006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f086fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f087015fdb0 0x7f087006bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48316/0 (socket says 172.21.15.3:48316) 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f086fe00640 1 -- 172.21.15.3:0/720124526 learned_addr learned my addr 172.21.15.3:0/720124526 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f0875800640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0870160220 0x7f087006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f086fe00640 1 -- 172.21.15.3:0/720124526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0870160220 msgr2=0x7f087006c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.323+0000 7f086fe00640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0870160220 0x7f087006c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.324+0000 7f086fe00640 1 -- 172.21.15.3:0/720124526 >> v1:172.21.15.70:6789/0 conn(0x7f0870161b30 legacy=0x7f087015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.324+0000 7f086fe00640 1 -- 172.21.15.3:0/720124526 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08700b3f40 con 0x7f087015fdb0 2024-09-10T07:51:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f086fe00640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015fdb0 0x7f087006bb20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f085c009870 tx=0x7f085c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c59fbd37df2fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f086f400640 1 -- 172.21.15.3:0/720124526 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c037070 con 0x7f087015fdb0 2024-09-10T07:51:53.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f086f400640 1 -- 172.21.15.3:0/720124526 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c032440 con 0x7f087015fdb0 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f086f400640 1 -- 172.21.15.3:0/720124526 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c032810 con 0x7f087015fdb0 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 -- 172.21.15.3:0/720124526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015fdb0 msgr2=0x7f087006bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015fdb0 0x7f087006bb20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f085c009870 tx=0x7f085c02a270 comp rx=0 tx=0).stop 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 -- 172.21.15.3:0/720124526 shutdown_connections 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0870160220 0x7f087006c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 --2- 172.21.15.3:0/720124526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f087015fdb0 0x7f087006bb20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.325+0000 7f0877800640 1 -- 172.21.15.3:0/720124526 >> 172.21.15.3:0/720124526 conn(0x7f087006d000 msgr2=0x7f087006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.326+0000 7f0877800640 1 -- 172.21.15.3:0/720124526 shutdown_connections 2024-09-10T07:51:53.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.326+0000 7f0877800640 1 -- 172.21.15.3:0/720124526 wait complete. 2024-09-10T07:51:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.327+0000 7f0877800640 1 Processor -- start 2024-09-10T07:51:53.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.327+0000 7f0877800640 1 -- start start 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087015fdb0 0x7f0870158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 0x7f087014ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08700aefa0 con 0x7f0870160220 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0870161f10 con 0x7f0870161b30 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f0877800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08700b3300 con 0x7f087015fdb0 2024-09-10T07:51:53.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f086fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f086fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48332/0 (socket says 172.21.15.3:48332) 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.328+0000 7f086fe00640 1 -- 172.21.15.3:0/3721954799 learned_addr learned my addr 172.21.15.3:0/3721954799 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 -- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087015fdb0 msgr2=0x7f0870158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087015fdb0 0x7f0870158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 -- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 msgr2=0x7f087014ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f0875800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 0x7f087014ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 0x7f087014ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 -- 172.21.15.3:0/3721954799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08701584a0 con 0x7f0870160220 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f0875800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 0x7f087014ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086fe00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f085c02a2b0 tx=0x7f085c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c037040 con 0x7f0870160220 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c042070 con 0x7f0870160220 2024-09-10T07:51:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c030880 con 0x7f0870160220 2024-09-10T07:51:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.329+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0870155d90 con 0x7f0870160220 2024-09-10T07:51:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.330+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0870156350 con 0x7f0870160220 2024-09-10T07:51:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.331+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087015f670 con 0x7f0870160220 2024-09-10T07:51:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.332+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f085c030a20 con 0x7f0870160220 2024-09-10T07:51:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.332+0000 7f086cc00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 0x7f085006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.333+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f085c0ab450 con 0x7f0870160220 2024-09-10T07:51:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.333+0000 7f0874e00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 0x7f085006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.333+0000 7f0874e00640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 0x7f085006e320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f087015c7e0 tx=0x7f0864007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.337+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f085c07a050 con 0x7f0870160220 2024-09-10T07:51:53.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.522+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0870069f70 con 0x7f0870160220 2024-09-10T07:51:53.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.523+0000 7f086cc00640 1 -- 172.21.15.3:0/3721954799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13217 (secure 0 0 0) 0x7f085c079950 con 0x7f0870160220 2024-09-10T07:51:53.523 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:53.523 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:51:52.139606+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6837/2373315703":"2024-09-11T07:51:49.130538+0000","172.21.15.3:6834/4206303617":"2024-09-11T07:51:49.130534+0000","172.21.15.3:6836/2373315703":"2024-09-11T07:51:49.130538+0000","172.21.15.3:6835/4206303617":"2024-09-11T07:51:49.130534+0000","172.21.15.70:6835/2352570209":"2024-09-11T07:51:49.130523+0000","172.21.15.70:6833/2352570209":"2024-09-11T07:51:49.130523+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 msgr2=0x7f085006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 0x7f085006e320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f087015c7e0 tx=0x7f0864007970 comp rx=0 tx=0).stop 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 msgr2=0x7f08700d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f085c02a2b0 tx=0x7f085c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 shutdown_connections 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f085006be60 0x7f085006e320 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0870161b30 0x7f087014ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0870160220 0x7f08700d0580 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 --2- 172.21.15.3:0/3721954799 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f087015fdb0 0x7f0870158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 >> 172.21.15.3:0/3721954799 conn(0x7f087006d000 msgr2=0x7f08700b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 shutdown_connections 2024-09-10T07:51:53.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.525+0000 7f0877800640 1 -- 172.21.15.3:0/3721954799 wait complete. 2024-09-10T07:51:53.542 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:51:53.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:51:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.702+0000 7f1326a00640 1 Processor -- start 2024-09-10T07:51:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.702+0000 7f1326a00640 1 -- start start 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.703+0000 7f1326a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13201451d0 0x7f132006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1326a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13200cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1326a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13200cd900 con 0x7f1320145730 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1326a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f132006d7c0 con 0x7f13201451d0 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1326a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13200cdbe0 con 0x7f13200cf270 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13201451d0 0x7f132006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13201451d0 0x7f132006cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48344/0 (socket says 172.21.15.3:48344) 2024-09-10T07:51:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 -- 172.21.15.3:0/1596433586 learned_addr learned my addr 172.21.15.3:0/1596433586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1324a00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13200cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 -- 172.21.15.3:0/1596433586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 msgr2=0x7f13200cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13200cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 -- 172.21.15.3:0/1596433586 >> v1:172.21.15.3:6789/0 conn(0x7f1320145730 legacy=0x7f13200b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f131fe00640 1 -- 172.21.15.3:0/1596433586 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132006da10 con 0x7f13201451d0 2024-09-10T07:51:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.704+0000 7f1324a00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13200cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.705+0000 7f131fe00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201451d0 0x7f132006cfc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ad7914091ca0db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f131f400640 1 -- 172.21.15.3:0/1596433586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310037070 con 0x7f13201451d0 2024-09-10T07:51:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f131f400640 1 -- 172.21.15.3:0/1596433586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1310032440 con 0x7f13201451d0 2024-09-10T07:51:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f131f400640 1 -- 172.21.15.3:0/1596433586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310032810 con 0x7f13201451d0 2024-09-10T07:51:53.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 -- 172.21.15.3:0/1596433586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201451d0 msgr2=0x7f132006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201451d0 0x7f132006cfc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002a1c0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 -- 172.21.15.3:0/1596433586 shutdown_connections 2024-09-10T07:51:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13200cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 --2- 172.21.15.3:0/1596433586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13201451d0 0x7f132006cfc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 -- 172.21.15.3:0/1596433586 >> 172.21.15.3:0/1596433586 conn(0x7f13200b5cb0 msgr2=0x7f132006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 -- 172.21.15.3:0/1596433586 shutdown_connections 2024-09-10T07:51:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.706+0000 7f1326a00640 1 -- 172.21.15.3:0/1596433586 wait complete. 2024-09-10T07:51:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.708+0000 7f1326a00640 1 Processor -- start 2024-09-10T07:51:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.708+0000 7f1326a00640 1 -- start start 2024-09-10T07:51:53.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13201e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13201451d0 0x7f13201f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f132006df30 con 0x7f1320145730 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13200cfc10 con 0x7f13201451d0 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1326a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f132006e0b0 con 0x7f13200cf270 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1324a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1324a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48350/0 (socket says 172.21.15.3:48350) 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1324a00640 1 -- 172.21.15.3:0/3294299569 learned_addr learned my addr 172.21.15.3:0/3294299569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f131fe00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13201e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.709+0000 7f1317400640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13201451d0 0x7f13201f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 -- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 msgr2=0x7f13201e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13201e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 -- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13201451d0 msgr2=0x7f13201f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13201451d0 0x7f13201f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 -- 172.21.15.3:0/3294299569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13201e81b0 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1324a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1318007ef0 tx=0x7f131800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131801a070 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318003b90 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131800de20 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.710+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13201e8490 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.711+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13201e8a50 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.711+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13200cf6d0 con 0x7f1320145730 2024-09-10T07:51:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.713+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f131801d070 con 0x7f1320145730 2024-09-10T07:51:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.713+0000 7f131cc00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 0x7f12fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.713+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f131808ae40 con 0x7f1320145730 2024-09-10T07:51:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.714+0000 7f131fe00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 0x7f12fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.714+0000 7f131fe00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 0x7f12fc06e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1310034000 tx=0x7f131002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.718+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1318010070 con 0x7f1320145730 2024-09-10T07:51:53.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.900+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1320072d10 con 0x7f1320145730 2024-09-10T07:51:53.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.901+0000 7f131cc00640 1 -- 172.21.15.3:0/3294299569 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1318059350 con 0x7f1320145730 2024-09-10T07:51:53.901 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 msgr2=0x7f12fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 0x7f12fc06e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1310034000 tx=0x7f131002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 msgr2=0x7f13201f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1318007ef0 tx=0x7f131800d4a0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 shutdown_connections 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f12fc06be30 0x7f12fc06e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1320145730 0x7f13201f7dc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13201451d0 0x7f13201f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 --2- 172.21.15.3:0/3294299569 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13200cf270 0x7f13201e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 >> 172.21.15.3:0/3294299569 conn(0x7f13200b5cb0 msgr2=0x7f1320140710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 shutdown_connections 2024-09-10T07:51:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:53.903+0000 7f1326a00640 1 -- 172.21.15.3:0/3294299569 wait complete. 2024-09-10T07:51:53.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-10T07:51:54.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.076+0000 7f5717400640 1 Processor -- start 2024-09-10T07:51:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.076+0000 7f5717400640 1 -- start start 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5717400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f571015e970 0x7f57100b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5717400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 0x7f57100ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5717400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57100b70b0 con 0x7f5710162cd0 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5717400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f57100b78b0 con 0x7f571015e970 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5717400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f57100b62f0 con 0x7f57100d1c10 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5714a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f571015e970 0x7f57100b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5714a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f571015e970 0x7f57100b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42604/0 (socket says 172.21.15.3:42604) 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.077+0000 7f5714a00640 1 -- 172.21.15.3:0/1027217250 learned_addr learned my addr 172.21.15.3:0/1027217250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5715400640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 0x7f57100ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5714a00640 1 -- 172.21.15.3:0/1027217250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 msgr2=0x7f57100ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5714a00640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 0x7f57100ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5714a00640 1 -- 172.21.15.3:0/1027217250 >> v1:172.21.15.70:6789/0 conn(0x7f5710162cd0 legacy=0x7f57100d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5714a00640 1 -- 172.21.15.3:0/1027217250 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100b65d0 con 0x7f571015e970 2024-09-10T07:51:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.078+0000 7f5715400640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 0x7f57100ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.079+0000 7f5714a00640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b37a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f57040098a0 tx=0x7f570402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11813b9db0dd27a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.079+0000 7f570f400640 1 -- 172.21.15.3:0/1027217250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704037070 con 0x7f571015e970 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.079+0000 7f570f400640 1 -- 172.21.15.3:0/1027217250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5704032440 con 0x7f571015e970 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.079+0000 7f570f400640 1 -- 172.21.15.3:0/1027217250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704032810 con 0x7f571015e970 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 -- 172.21.15.3:0/1027217250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 msgr2=0x7f57100b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b37a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f57040098a0 tx=0x7f570402a200 comp rx=0 tx=0).stop 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 -- 172.21.15.3:0/1027217250 shutdown_connections 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57100d1c10 0x7f57100ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 --2- 172.21.15.3:0/1027217250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b37a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 -- 172.21.15.3:0/1027217250 >> 172.21.15.3:0/1027217250 conn(0x7f571006bd00 msgr2=0x7f571006c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 -- 172.21.15.3:0/1027217250 shutdown_connections 2024-09-10T07:51:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.080+0000 7f5717400640 1 -- 172.21.15.3:0/1027217250 wait complete. 2024-09-10T07:51:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.081+0000 7f5717400640 1 Processor -- start 2024-09-10T07:51:54.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 -- start start 2024-09-10T07:51:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57100d1c10 0x7f57100cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5710162cd0 0x7f57100b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5710163160 con 0x7f571015e970 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f57100b6b90 con 0x7f57100d1c10 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.082+0000 7f5717400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f57100cf280 con 0x7f5710162cd0 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42614/0 (socket says 172.21.15.3:42614) 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 -- 172.21.15.3:0/1276663522 learned_addr learned my addr 172.21.15.3:0/1276663522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f5714a00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57100d1c10 0x7f57100cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f5715400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5710162cd0 0x7f57100b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 -- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5710162cd0 msgr2=0x7f57100b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5710162cd0 0x7f57100b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 -- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57100d1c10 msgr2=0x7f57100cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57100d1c10 0x7f57100cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 -- 172.21.15.3:0/1276663522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5710149ee0 con 0x7f571015e970 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.083+0000 7f570fe00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c3f0 tx=0x7f56fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.084+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0052d0 con 0x7f571015e970 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.084+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc005490 con 0x7f571015e970 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.084+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc007260 con 0x7f571015e970 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.084+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571014a180 con 0x7f571015e970 2024-09-10T07:51:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.084+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571014a740 con 0x7f571015e970 2024-09-10T07:51:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.085+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57100ced40 con 0x7f571015e970 2024-09-10T07:51:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.086+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56fc007400 con 0x7f571015e970 2024-09-10T07:51:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.087+0000 7f570cc00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 0x7f56f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.087+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f56fc08a060 con 0x7f571015e970 2024-09-10T07:51:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.087+0000 7f5714a00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 0x7f56f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:54.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.088+0000 7f5714a00640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 0x7f56f006e320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5704034000 tx=0x7f570402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:54.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.092+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57100b70b0 con 0x7f571015e970 2024-09-10T07:51:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:54.346+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7f57100d2470 con 0x7f571015e970 2024-09-10T07:51:55.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.153+0000 7f570cc00640 1 -- 172.21.15.3:0/1276663522 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v280) ==== 157+0+0 (secure 0 0 0) 0x7f56fc058600 con 0x7f571015e970 2024-09-10T07:51:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 msgr2=0x7f56f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 0x7f56f006e320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5704034000 tx=0x7f570402e000 comp rx=0 tx=0).stop 2024-09-10T07:51:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 msgr2=0x7f57100b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c3f0 tx=0x7f56fc00c8b0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 shutdown_connections 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f56f006be60 0x7f56f006e320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5710162cd0 0x7f57100b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f571015e970 0x7f57100b4f00 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 --2- 172.21.15.3:0/1276663522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57100d1c10 0x7f57100cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.156+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 >> 172.21.15.3:0/1276663522 conn(0x7f571006bd00 msgr2=0x7f571006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.157+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 shutdown_connections 2024-09-10T07:51:55.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.157+0000 7f5717400640 1 -- 172.21.15.3:0/1276663522 wait complete. 2024-09-10T07:51:55.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:51:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.337+0000 7f5ab5000640 1 Processor -- start 2024-09-10T07:51:55.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.337+0000 7f5ab5000640 1 -- start start 2024-09-10T07:51:55.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.338+0000 7f5ab5000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ab00aefb0 0x7f5ab0072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.338+0000 7f5ab5000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.338+0000 7f5ab5000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab015faf0 con 0x7f5ab0147ce0 2024-09-10T07:51:55.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.338+0000 7f5ab5000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab015eaf0 con 0x7f5ab00aefb0 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.338+0000 7f5ab5000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab00730c0 con 0x7f5ab00728d0 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ab00aefb0 0x7f5ab0072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aade00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5ab0147ce0 0x7f5ab00ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35308/0 (socket says 172.21.15.3:35308) 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ab00aefb0 0x7f5ab0072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42626/0 (socket says 172.21.15.3:42626) 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aade00640 1 -- 172.21.15.3:0/4056284353 learned_addr learned my addr 172.21.15.3:0/4056284353 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:55.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aae800640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 -- 172.21.15.3:0/4056284353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 msgr2=0x7f5ab006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab006e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 -- 172.21.15.3:0/4056284353 >> v1:172.21.15.3:6789/0 conn(0x7f5ab0147ce0 legacy=0x7f5ab00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aa5400640 1 -- 172.21.15.3:0/4056284353 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab015ed30 con 0x7f5ab00aefb0 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.339+0000 7f5aad400640 1 -- 172.21.15.3:0/4056284353 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756467302 0 0) 0x7f5ab015faf0 con 0x7f5ab0147ce0 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.340+0000 7f5aa5400640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab00aefb0 0x7f5ab0072390 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5a980097a0 tx=0x7f5a9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf76a0e10cd33c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:55.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.340+0000 7f5aad400640 1 -- 172.21.15.3:0/4056284353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98037070 con 0x7f5ab00aefb0 2024-09-10T07:51:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5aad400640 1 -- 172.21.15.3:0/4056284353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a98032440 con 0x7f5ab00aefb0 2024-09-10T07:51:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5aad400640 1 -- 172.21.15.3:0/4056284353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98032810 con 0x7f5ab00aefb0 2024-09-10T07:51:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 -- 172.21.15.3:0/4056284353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab00aefb0 msgr2=0x7f5ab0072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab00aefb0 0x7f5ab0072390 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5a980097a0 tx=0x7f5a9802a180 comp rx=0 tx=0).stop 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 -- 172.21.15.3:0/4056284353 shutdown_connections 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab006e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 --2- 172.21.15.3:0/4056284353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab00aefb0 0x7f5ab0072390 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 -- 172.21.15.3:0/4056284353 >> 172.21.15.3:0/4056284353 conn(0x7f5ab00b5cb0 msgr2=0x7f5ab006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 -- 172.21.15.3:0/4056284353 shutdown_connections 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.341+0000 7f5ab5000640 1 -- 172.21.15.3:0/4056284353 wait complete. 2024-09-10T07:51:55.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.343+0000 7f5ab5000640 1 Processor -- start 2024-09-10T07:51:55.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.343+0000 7f5ab5000640 1 -- start start 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab00cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 0x7f5ab00cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab015f2f0 con 0x7f5ab0147ce0 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab015f660 con 0x7f5ab00aefb0 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5ab5000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab01480c0 con 0x7f5ab00728d0 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 -- 172.21.15.3:0/2808299833 learned_addr learned my addr 172.21.15.3:0/2808299833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aa5400640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 0x7f5ab00cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aade00640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab00cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 -- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 msgr2=0x7f5ab00cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab00cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 -- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 msgr2=0x7f5ab00cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 0x7f5ab00cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aae800640 1 -- 172.21.15.3:0/2808299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab01f4880 con 0x7f5ab0147ce0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aade00640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab00cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.344+0000 7f5aa5400640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 0x7f5ab00cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5aae800640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009dd0 tx=0x7f5aa800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8018020 con 0x7f5ab0147ce0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa8014480 con 0x7f5ab0147ce0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8014870 con 0x7f5ab0147ce0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab00cdeb0 con 0x7f5ab0147ce0 2024-09-10T07:51:55.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.345+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab00ce470 con 0x7f5ab0147ce0 2024-09-10T07:51:55.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.346+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab0161120 con 0x7f5ab0147ce0 2024-09-10T07:51:55.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.347+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5aa8005d10 con 0x7f5ab0147ce0 2024-09-10T07:51:55.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.348+0000 7f5aa6a00640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 0x7f5a8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:55.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.348+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f5aa808b780 con 0x7f5ab0147ce0 2024-09-10T07:51:55.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.348+0000 7f5aade00640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 0x7f5a8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:55.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.349+0000 7f5aade00640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 0x7f5a8c06e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000f60 tx=0x7f5aa0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:55.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.352+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ab0161120 con 0x7f5ab0147ce0 2024-09-10T07:51:55.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.537+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5ab01f4a50 con 0x7f5ab0147ce0 2024-09-10T07:51:55.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.541+0000 7f5aa6a00640 1 -- 172.21.15.3:0/2808299833 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f5aa8022070 con 0x7f5ab0147ce0 2024-09-10T07:51:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 msgr2=0x7f5a8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 0x7f5a8c06e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000f60 tx=0x7f5aa0009040 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 msgr2=0x7f5ab0140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009dd0 tx=0x7f5aa800c490 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 shutdown_connections 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5a8c06be30 0x7f5a8c06e2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ab0147ce0 0x7f5ab0140e40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab00aefb0 0x7f5ab00cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 --2- 172.21.15.3:0/2808299833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5ab00728d0 0x7f5ab00cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.543+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 >> 172.21.15.3:0/2808299833 conn(0x7f5ab00b5cb0 msgr2=0x7f5ab01617c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.544+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 shutdown_connections 2024-09-10T07:51:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:55.544+0000 7f5ab5000640 1 -- 172.21.15.3:0/2808299833 wait complete. 2024-09-10T07:51:55.559 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:51:55.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:55.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:51:55.745 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:51:55.746 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:55.746 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:55.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:51:55.782 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:55.782 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:55.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:55.819 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:51:55.819 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:55.856 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:51:55.857 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:51:55.857 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:51:55.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:51:55.895 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:51:55.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:51:56.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.052+0000 7f0313400640 1 Processor -- start 2024-09-10T07:51:56.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.052+0000 7f0313400640 1 -- start start 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0313400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f030c15b4f0 0x7f030c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0313400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f030c073d10 0x7f030c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0313400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c0ccd80 con 0x7f030c15b110 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0313400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f030c0b44f0 con 0x7f030c073d10 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0313400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f030c0cd060 con 0x7f030c15b4f0 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0311400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f030c073d10 0x7f030c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.053+0000 7f0310a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f030c15b110 0x7f030c0b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35314/0 (socket says 172.21.15.3:35314) 2024-09-10T07:51:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f0310a00640 1 -- 172.21.15.3:0/485804170 learned_addr learned my addr 172.21.15.3:0/485804170 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f030be00640 1 --2- 172.21.15.3:0/485804170 >> v2:172.21.15.70:3300/0 conn(0x7f030c15b4f0 0x7f030c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f030b400640 1 -- 172.21.15.3:0/485804170 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52051565 0 0) 0x7f030c0ccd80 con 0x7f030c15b110 2024-09-10T07:51:56.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f0311400640 1 -- 172.21.15.3:0/485804170 >> v2:172.21.15.70:3300/0 conn(0x7f030c15b4f0 msgr2=0x7f030c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f0311400640 1 --2- 172.21.15.3:0/485804170 >> v2:172.21.15.70:3300/0 conn(0x7f030c15b4f0 0x7f030c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f0311400640 1 -- 172.21.15.3:0/485804170 >> v1:172.21.15.3:6789/0 conn(0x7f030c15b110 legacy=0x7f030c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.054+0000 7f0311400640 1 -- 172.21.15.3:0/485804170 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c0cd340 con 0x7f030c073d10 2024-09-10T07:51:56.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.055+0000 7f0311400640 1 --2- 172.21.15.3:0/485804170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c073d10 0x7f030c0d0720 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f030c0d1310 tx=0x7f030402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc5b39e862cff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.055+0000 7f030b400640 1 -- 172.21.15.3:0/485804170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304007740 con 0x7f030c073d10 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.055+0000 7f030b400640 1 -- 172.21.15.3:0/485804170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040078e0 con 0x7f030c073d10 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.055+0000 7f030b400640 1 -- 172.21.15.3:0/485804170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304007cb0 con 0x7f030c073d10 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 -- 172.21.15.3:0/485804170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c073d10 msgr2=0x7f030c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 --2- 172.21.15.3:0/485804170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c073d10 0x7f030c0d0720 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f030c0d1310 tx=0x7f030402a680 comp rx=0 tx=0).stop 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 -- 172.21.15.3:0/485804170 shutdown_connections 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 --2- 172.21.15.3:0/485804170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c073d10 0x7f030c0d0720 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 --2- 172.21.15.3:0/485804170 >> v2:172.21.15.70:3300/0 conn(0x7f030c15b4f0 0x7f030c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 -- 172.21.15.3:0/485804170 >> 172.21.15.3:0/485804170 conn(0x7f030c06d000 msgr2=0x7f030c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 -- 172.21.15.3:0/485804170 shutdown_connections 2024-09-10T07:51:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.056+0000 7f0313400640 1 -- 172.21.15.3:0/485804170 wait complete. 2024-09-10T07:51:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.058+0000 7f0313400640 1 Processor -- start 2024-09-10T07:51:56.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.058+0000 7f0313400640 1 -- start start 2024-09-10T07:51:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 0x7f030c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 0x7f030c1e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0cdcc0 con 0x7f030c15b4f0 2024-09-10T07:51:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f030c0cdb40 con 0x7f030c15b110 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0313400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f030c0cde40 con 0x7f030c073d10 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42646/0 (socket says 172.21.15.3:42646) 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 -- 172.21.15.3:0/230432384 learned_addr learned my addr 172.21.15.3:0/230432384 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 -- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 msgr2=0x7f030c1513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0310a00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 0x7f030c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f030be00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 0x7f030c1e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 0x7f030c1513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 -- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 msgr2=0x7f030c1e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 0x7f030c1e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0311400640 1 -- 172.21.15.3:0/230432384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1563f0 con 0x7f030c15b4f0 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f0310a00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 0x7f030c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:56.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.059+0000 7f030be00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 0x7f030c1e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0311400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f030402ac70 tx=0x7f0304004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304040070 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0304031c40 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304009450 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1566d0 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.060+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c156c90 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.061+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c0cfbf0 con 0x7f030c15b4f0 2024-09-10T07:51:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.062+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0304039040 con 0x7f030c15b4f0 2024-09-10T07:51:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.063+0000 7f0308c00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.063+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5115+0+0 (secure 0 0 0) 0x7f03040abf30 con 0x7f030c15b4f0 2024-09-10T07:51:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.063+0000 7f0310a00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.064+0000 7f0310a00640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 0x7f02ec06e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0300000f60 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.067+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f030c0cfbf0 con 0x7f030c15b4f0 2024-09-10T07:51:56.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.287+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f030c0b3d10 con 0x7f030c15b4f0 2024-09-10T07:51:56.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.288+0000 7f0308c00640 1 -- 172.21.15.3:0/230432384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3899 (secure 0 0 0) 0x7f030407aa90 con 0x7f030c15b4f0 2024-09-10T07:51:56.287 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:56.287 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"btime":"2024-09-10T07:51:50:133003+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/470413742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":470413742},{"type":"v1","addr":"172.21.15.70:6835","nonce":470413742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/377756845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":377756845},{"type":"v1","addr":"172.21.15.3:6835","nonce":377756845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6352,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2196832681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2196832681},{"type":"v1","addr":"172.21.15.70:6834","nonce":2196832681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2217520077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2217520077},{"type":"v1","addr":"172.21.15.3:6837","nonce":2217520077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-09-10T07:51:56.288 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 79 2024-09-10T07:51:56.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 msgr2=0x7f02ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 0x7f02ec06e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0300000f60 tx=0x7f0300009040 comp rx=0 tx=0).stop 2024-09-10T07:51:56.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 msgr2=0x7f030c1e9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f030402ac70 tx=0x7f0304004a40 comp rx=0 tx=0).stop 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 shutdown_connections 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02ec06be30 0x7f02ec06e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c15b4f0 0x7f030c1e9610 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c15b110 0x7f030c1e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 --2- 172.21.15.3:0/230432384 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f030c073d10 0x7f030c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 >> 172.21.15.3:0/230432384 conn(0x7f030c06d000 msgr2=0x7f030c15f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 shutdown_connections 2024-09-10T07:51:56.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.290+0000 7f0313400640 1 -- 172.21.15.3:0/230432384 wait complete. 2024-09-10T07:51:56.306 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:51:56.307 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:51:56.307 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:51:56.307 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:56.307 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:51:56.307 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:51:56.307 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:51:56.307 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:56.308 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:51:56.308 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:51:56.308 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:51:56.308 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:56.308 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:51:56.308 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:51:56.308 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:51:56.309 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:51:56.325 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:51:56.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:51:56.327 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:51:56.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:51:56.327 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:51:56.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:51:56.328 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:51:56.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:51:56.370 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:51:56.370 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:56.371 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:51:56.371 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:56.373 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:51:56.373 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:56.373 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:51:56.374 DEBUG:teuthology.parallel:result is None 2024-09-10T07:51:56.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:51:56.498 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:51:56.498 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:51:56.504 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:51:56.504 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:51:56.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.542+0000 7f140a200640 1 Processor -- start 2024-09-10T07:51:56.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.542+0000 7f140a200640 1 -- start start 2024-09-10T07:51:56.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.543+0000 7f140a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040d1ba0 0x7f1404162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.543+0000 7f140a200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14040cf1f0 0x7f14040d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.543+0000 7f140a200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040b7920 con 0x7f140415e8e0 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f140a200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f14040cf6a0 con 0x7f14040cf1f0 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f140a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f14040cf940 con 0x7f14040d1ba0 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f140415e8e0 0x7f14040d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:39346/0 (socket says 172.21.15.3:39346) 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403400640 1 -- 172.21.15.3:0/1747653824 learned_addr learned my addr 172.21.15.3:0/1747653824 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1402a00640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040d1ba0 0x7f1404162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403e00640 1 --2- 172.21.15.3:0/1747653824 >> v2:172.21.15.3:3300/0 conn(0x7f14040cf1f0 0x7f14040d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1402000640 1 -- 172.21.15.3:0/1747653824 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219458811 0 0) 0x7f13f4001110 con 0x7f140415e8e0 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1402000640 1 -- 172.21.15.3:0/1747653824 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e8003520 con 0x7f140415e8e0 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403e00640 1 -- 172.21.15.3:0/1747653824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040d1ba0 msgr2=0x7f1404162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403e00640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040d1ba0 0x7f1404162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403e00640 1 -- 172.21.15.3:0/1747653824 >> v1:172.21.15.70:6789/0 conn(0x7f140415e8e0 legacy=0x7f14040d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.544+0000 7f1403e00640 1 -- 172.21.15.3:0/1747653824 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040cfc20 con 0x7f14040cf1f0 2024-09-10T07:51:56.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.545+0000 7f1403e00640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14040cf1f0 0x7f14040d32e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009d90 tx=0x7f13fc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2dd964f191a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.548+0000 7f1402000640 1 -- 172.21.15.3:0/1747653824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc031600 con 0x7f14040cf1f0 2024-09-10T07:51:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.548+0000 7f1402000640 1 -- 172.21.15.3:0/1747653824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc0317a0 con 0x7f14040cf1f0 2024-09-10T07:51:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.548+0000 7f1402000640 1 -- 172.21.15.3:0/1747653824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc031b50 con 0x7f14040cf1f0 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 -- 172.21.15.3:0/1747653824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14040cf1f0 msgr2=0x7f14040d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14040cf1f0 0x7f14040d32e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009d90 tx=0x7f13fc02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 -- 172.21.15.3:0/1747653824 shutdown_connections 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14040cf1f0 0x7f14040d32e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 --2- 172.21.15.3:0/1747653824 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040d1ba0 0x7f1404162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.549+0000 7f140a200640 1 -- 172.21.15.3:0/1747653824 >> 172.21.15.3:0/1747653824 conn(0x7f140406bd00 msgr2=0x7f140406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.550+0000 7f140a200640 1 -- 172.21.15.3:0/1747653824 shutdown_connections 2024-09-10T07:51:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.550+0000 7f140a200640 1 -- 172.21.15.3:0/1747653824 wait complete. 2024-09-10T07:51:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.552+0000 7f140a200640 1 Processor -- start 2024-09-10T07:51:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.552+0000 7f140a200640 1 -- start start 2024-09-10T07:51:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14040d1ba0 0x7f14041f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140415e8e0 0x7f14040cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14040d3820 con 0x7f140415e8e0 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f14040d2dc0 con 0x7f14040d1ba0 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f140a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f14040d39a0 con 0x7f14040cf1f0 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f1403400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f1403e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140415e8e0 0x7f14040cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f1403400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:42202/0 (socket says 172.21.15.3:42202) 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f1403400640 1 -- 172.21.15.3:0/28554183 learned_addr learned my addr 172.21.15.3:0/28554183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.554+0000 7f1402a00640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14040d1ba0 0x7f14041f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 -- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14040d1ba0 msgr2=0x7f14041f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14040d1ba0 0x7f14041f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 -- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140415e8e0 msgr2=0x7f14040cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140415e8e0 0x7f14040cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 -- 172.21.15.3:0/28554183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040ce820 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.555+0000 7f1403400640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c400 tx=0x7f13f400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.556+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.556+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4005ce0 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.556+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40072d0 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.556+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140415fda0 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.556+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1404160360 con 0x7f14040cf1f0 2024-09-10T07:51:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.557+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14040d32e0 con 0x7f14040cf1f0 2024-09-10T07:51:56.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.558+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13f401a070 con 0x7f14040cf1f0 2024-09-10T07:51:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.559+0000 7f13fb400640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 0x7f13e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.559+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(85..85 src has 1..85) ==== 5115+0+0 (secure 0 0 0) 0x7f13f408a370 con 0x7f14040cf1f0 2024-09-10T07:51:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.559+0000 7f1402a00640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 0x7f13e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:56.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.560+0000 7f1402a00640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 0x7f13e406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1404147660 tx=0x7f13f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:56.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.563+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14040b7920 con 0x7f14040cf1f0 2024-09-10T07:51:56.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:56.758+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f14040d23d0 con 0x7f14040cf1f0 2024-09-10T07:51:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.558+0000 7f13fb400640 1 -- 172.21.15.3:0/28554183 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7f13f405de20 con 0x7f14040cf1f0 2024-09-10T07:51:57.558 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 msgr2=0x7f13e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 0x7f13e406e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1404147660 tx=0x7f13f0009190 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 msgr2=0x7f14041444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c400 tx=0x7f13f400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 shutdown_connections 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f13e406be30 0x7f13e406e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f140415e8e0 0x7f14040cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14040d1ba0 0x7f14041f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 --2- 172.21.15.3:0/28554183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14040cf1f0 0x7f14041444d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 >> 172.21.15.3:0/28554183 conn(0x7f140406bd00 msgr2=0x7f14040b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:57.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 shutdown_connections 2024-09-10T07:51:57.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.561+0000 7f140a200640 1 -- 172.21.15.3:0/28554183 wait complete. 2024-09-10T07:51:57.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:51:57.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.737+0000 7feafac00640 1 Processor -- start 2024-09-10T07:51:57.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.737+0000 7feafac00640 1 -- start start 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feafac00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feaf40d3c10 0x7feaf4073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feafac00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf40cfdc0 con 0x7feaf415e860 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feafac00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf40cee40 con 0x7feaf40d4050 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feafac00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7feaf40d3780 con 0x7feaf40d3c10 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feaf9800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feaf415e860 0x7feaf40cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35334/0 (socket says 172.21.15.3:35334) 2024-09-10T07:51:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feaf9800640 1 -- 172.21.15.3:0/3074715435 learned_addr learned my addr 172.21.15.3:0/3074715435 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.738+0000 7feaf8e00640 1 --2- 172.21.15.3:0/3074715435 >> v2:172.21.15.3:3300/0 conn(0x7feaf40d3c10 0x7feaf4073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.739+0000 7feaf8e00640 1 -- 172.21.15.3:0/3074715435 >> v1:172.21.15.70:6789/0 conn(0x7feaf40d4050 legacy=0x7feaf40b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.739+0000 7feaf8e00640 1 -- 172.21.15.3:0/3074715435 >> v1:172.21.15.3:6789/0 conn(0x7feaf415e860 legacy=0x7feaf40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.739+0000 7feaf8e00640 1 -- 172.21.15.3:0/3074715435 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf40cf120 con 0x7feaf40d3c10 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.739+0000 7feaf3e00640 1 -- 172.21.15.3:0/3074715435 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150350978 0 0) 0x7feaf40cfdc0 con 0x7feaf415e860 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.740+0000 7feaf8e00640 1 --2- 172.21.15.3:0/3074715435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf4073150 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7feae00097a0 tx=0x7feae002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fae5b47613d2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.740+0000 7feaf3e00640 1 -- 172.21.15.3:0/3074715435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0037070 con 0x7feaf40d3c10 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.740+0000 7feaf3e00640 1 -- 172.21.15.3:0/3074715435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae0032440 con 0x7feaf40d3c10 2024-09-10T07:51:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.740+0000 7feaf3e00640 1 -- 172.21.15.3:0/3074715435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0032810 con 0x7feaf40d3c10 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 -- 172.21.15.3:0/3074715435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 msgr2=0x7feaf4073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 --2- 172.21.15.3:0/3074715435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf4073150 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7feae00097a0 tx=0x7feae002a230 comp rx=0 tx=0).stop 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 -- 172.21.15.3:0/3074715435 shutdown_connections 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 --2- 172.21.15.3:0/3074715435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf4073150 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 -- 172.21.15.3:0/3074715435 >> 172.21.15.3:0/3074715435 conn(0x7feaf40d4650 msgr2=0x7feaf40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 -- 172.21.15.3:0/3074715435 shutdown_connections 2024-09-10T07:51:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.741+0000 7feafac00640 1 -- 172.21.15.3:0/3074715435 wait complete. 2024-09-10T07:51:57.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.742+0000 7feafac00640 1 Processor -- start 2024-09-10T07:51:57.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.743+0000 7feafac00640 1 -- start start 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40d4050 0x7feaf415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf415e860 0x7feaf415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40d2460 con 0x7feaf40d3c10 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40cf400 con 0x7feaf40d4050 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feaf40b7f90 con 0x7feaf415e860 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42688/0 (socket says 172.21.15.3:42688) 2024-09-10T07:51:57.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 -- 172.21.15.3:0/3504649275 learned_addr learned my addr 172.21.15.3:0/3504649275 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feafa200640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf415e860 0x7feaf415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf8e00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40d4050 0x7feaf415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 -- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf415e860 msgr2=0x7feaf415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf415e860 0x7feaf415d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 -- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40d4050 msgr2=0x7feaf415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40d4050 0x7feaf415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.744+0000 7feaf9800640 1 -- 172.21.15.3:0/3504649275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf41f52f0 con 0x7feaf40d3c10 2024-09-10T07:51:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feaf9800640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7feae800c3f0 tx=0x7feae800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80074e0 con 0x7feaf40d3c10 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae80047e0 con 0x7feaf40d3c10 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf41601c0 con 0x7feaf40d3c10 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8004c10 con 0x7feaf40d3c10 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.745+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf4160780 con 0x7feaf40d3c10 2024-09-10T07:51:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.746+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf4161630 con 0x7feaf40d3c10 2024-09-10T07:51:57.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.747+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feae8011070 con 0x7feaf40d3c10 2024-09-10T07:51:57.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.748+0000 7feaf1600640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 0x7fead406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:57.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.748+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4857+0+0 (secure 0 0 0) 0x7feae808a690 con 0x7feaf40d3c10 2024-09-10T07:51:57.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.748+0000 7feaf8e00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 0x7fead406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:57.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.749+0000 7feaf8e00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 0x7fead406e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feae0034000 tx=0x7feae002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:57.752+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feaf41f52f0 con 0x7feaf40d3c10 2024-09-10T07:51:58.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.001+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feaf40b7880 con 0x7feaf40d3c10 2024-09-10T07:51:58.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.001+0000 7feaf1600640 1 -- 172.21.15.3:0/3504649275 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7feae8058ca0 con 0x7feaf40d3c10 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:51:58.005 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:58.006 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:51:58.008 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:51:58.010 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.011 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:51:58.012 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.013 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:51:58.014 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:58.015 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.017 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.018 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:51:58.019 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:51:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 msgr2=0x7fead406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 0x7fead406e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feae0034000 tx=0x7feae002e000 comp rx=0 tx=0).stop 2024-09-10T07:51:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 msgr2=0x7feaf41f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7feae800c3f0 tx=0x7feae800c8b0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 shutdown_connections 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fead406be30 0x7fead406e2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf415e860 0x7feaf415d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf40d4050 0x7feaf415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 --2- 172.21.15.3:0/3504649275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf40d3c10 0x7feaf41f4db0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.003+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 >> 172.21.15.3:0/3504649275 conn(0x7feaf40d4650 msgr2=0x7feaf40b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.004+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 shutdown_connections 2024-09-10T07:51:58.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.004+0000 7feafac00640 1 -- 172.21.15.3:0/3504649275 wait complete. 2024-09-10T07:51:58.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:51:58.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.178+0000 7fb45d600640 1 Processor -- start 2024-09-10T07:51:58.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.179+0000 7fb45d600640 1 -- start start 2024-09-10T07:51:58.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb45d600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4580b4aa0 0x7fb45806ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb45d600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4580ccd40 con 0x7fb4580cfac0 2024-09-10T07:51:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb45d600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4580cd020 con 0x7fb45815b410 2024-09-10T07:51:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb45d600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb45806f250 con 0x7fb4580b4aa0 2024-09-10T07:51:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb457400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4580cfac0 0x7fb4580b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35346/0 (socket says 172.21.15.3:35346) 2024-09-10T07:51:58.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb457400640 1 -- 172.21.15.3:0/3023200957 learned_addr learned my addr 172.21.15.3:0/3023200957 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:58.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb44fe00640 1 --2- 172.21.15.3:0/3023200957 >> v2:172.21.15.70:3300/0 conn(0x7fb4580b4aa0 0x7fb45806ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.180+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193616607 0 0) 0x7fb4580ccd40 con 0x7fb4580cfac0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb43c005730 con 0x7fb4580cfac0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 236196366 0 0) 0x7fb43c005730 con 0x7fb4580cfac0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 >> v2:172.21.15.70:3300/0 conn(0x7fb4580b4aa0 msgr2=0x7fb45806ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 --2- 172.21.15.3:0/3023200957 >> v2:172.21.15.70:3300/0 conn(0x7fb4580b4aa0 0x7fb45806ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 >> v1:172.21.15.70:6789/0 conn(0x7fb45815b410 legacy=0x7fb4580cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580cd300 con 0x7fb4580cfac0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fb4500019f0 con 0x7fb4580cfac0 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 >> v1:172.21.15.3:6789/0 conn(0x7fb4580cfac0 legacy=0x7fb4580b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 0x7fb43c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb43c006570 0x7fb43c006a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c007250 0x7fb43c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb45806f250 con 0x7fb43c005bc0 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c00e190 con 0x7fb43c007250 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb43c00e390 con 0x7fb43c006570 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.181+0000 7fb456a00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 0x7fb43c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb44fe00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb43c006570 0x7fb43c006a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb457400640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c007250 0x7fb43c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb456a00640 1 -- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb43c006570 msgr2=0x7fb43c006a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb456a00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb43c006570 0x7fb43c006a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb456a00640 1 -- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c007250 msgr2=0x7fb43c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb456a00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c007250 0x7fb43c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb456a00640 1 -- 172.21.15.3:0/3023200957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580cd300 con 0x7fb43c005bc0 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb44fe00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb43c006570 0x7fb43c006a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:51:58.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.182+0000 7fb457400640 1 -- 172.21.15.3:0/3023200957 reap_dead start 2024-09-10T07:51:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.183+0000 7fb456a00640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 0x7fb43c005fa0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb448001860 tx=0x7fb43c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48b99b2200349d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.183+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448037070 con 0x7fb43c005bc0 2024-09-10T07:51:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.183+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44802abf0 con 0x7fb43c005bc0 2024-09-10T07:51:58.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.183+0000 7fb456000640 1 -- 172.21.15.3:0/3023200957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448001430 con 0x7fb43c005bc0 2024-09-10T07:51:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 -- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 msgr2=0x7fb43c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 0x7fb43c005fa0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb448001860 tx=0x7fb43c007a40 comp rx=0 tx=0).stop 2024-09-10T07:51:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 -- 172.21.15.3:0/3023200957 shutdown_connections 2024-09-10T07:51:58.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 --2- 172.21.15.3:0/3023200957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb43c005bc0 0x7fb43c005fa0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 -- 172.21.15.3:0/3023200957 >> 172.21.15.3:0/3023200957 conn(0x7fb45806cfc0 msgr2=0x7fb45806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 -- 172.21.15.3:0/3023200957 shutdown_connections 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.184+0000 7fb45d600640 1 -- 172.21.15.3:0/3023200957 wait complete. 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.185+0000 7fb45d600640 1 Processor -- start 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 -- start start 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c005bc0 0x7fb4580c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb458156320 0x7fb45814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580cdba0 con 0x7fb45814bc10 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580cd8c0 con 0x7fb43c005bc0 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.186+0000 7fb45d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb458072540 con 0x7fb458156320 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42706/0 (socket says 172.21.15.3:42706) 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 -- 172.21.15.3:0/1279563711 learned_addr learned my addr 172.21.15.3:0/1279563711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb44fe00640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb458156320 0x7fb45814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb456a00640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c005bc0 0x7fb4580c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 -- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb458156320 msgr2=0x7fb45814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb458156320 0x7fb45814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 -- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c005bc0 msgr2=0x7fb4580c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c005bc0 0x7fb4580c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 -- 172.21.15.3:0/1279563711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580c6940 con 0x7fb45814bc10 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.187+0000 7fb457400640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb4580717e0 tx=0x7fb45000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.188+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45000ed40 con 0x7fb45814bc10 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.188+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb450018070 con 0x7fb45814bc10 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.188+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450013640 con 0x7fb45814bc10 2024-09-10T07:51:58.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.188+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4580c6c20 con 0x7fb45814bc10 2024-09-10T07:51:58.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.188+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4580ae270 con 0x7fb45814bc10 2024-09-10T07:51:58.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.189+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45815b7f0 con 0x7fb45814bc10 2024-09-10T07:51:58.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.190+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4500137e0 con 0x7fb45814bc10 2024-09-10T07:51:58.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.191+0000 7fb44ea00640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 0x7fb43406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.191+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4857+0+0 (secure 0 0 0) 0x7fb45008aec0 con 0x7fb45814bc10 2024-09-10T07:51:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.191+0000 7fb456a00640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 0x7fb43406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.192+0000 7fb456a00640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 0x7fb43406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb448034000 tx=0x7fb448009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.195+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb450059560 con 0x7fb45814bc10 2024-09-10T07:51:58.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.378+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb45806ee90 con 0x7fb45814bc10 2024-09-10T07:51:58.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.379+0000 7fb44ea00640 1 -- 172.21.15.3:0/1279563711 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7fb45005d860 con 0x7fb45814bc10 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:51:58.379 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 msgr2=0x7fb43406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 0x7fb43406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb448034000 tx=0x7fb448009030 comp rx=0 tx=0).stop 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 msgr2=0x7fb4580b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb4580717e0 tx=0x7fb45000e530 comp rx=0 tx=0).stop 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 shutdown_connections 2024-09-10T07:51:58.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43406be30 0x7fb43406e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45814bc10 0x7fb4580b2240 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb458156320 0x7fb45814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 --2- 172.21.15.3:0/1279563711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb43c005bc0 0x7fb4580c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 >> 172.21.15.3:0/1279563711 conn(0x7fb45806cfc0 msgr2=0x7fb4581598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 shutdown_connections 2024-09-10T07:51:58.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.381+0000 7fb45d600640 1 -- 172.21.15.3:0/1279563711 wait complete. 2024-09-10T07:51:58.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:51:58.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:51:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.558+0000 7fa254600640 1 Processor -- start 2024-09-10T07:51:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.558+0000 7fa254600640 1 -- start start 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.559+0000 7fa254600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa24c0d3320 0x7fa24c0d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.559+0000 7fa254600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa24c0d4ea0 0x7fa24c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.559+0000 7fa254600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0cfe30 con 0x7fa24c15e860 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.559+0000 7fa254600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0b3f10 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.559+0000 7fa254600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0d38c0 con 0x7fa24c0d3320 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa251c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa24c15e860 0x7fa24c0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35356/0 (socket says 172.21.15.3:35356) 2024-09-10T07:51:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa251c00640 1 -- 172.21.15.3:0/1945804333 learned_addr learned my addr 172.21.15.3:0/1945804333 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa252600640 1 --2- 172.21.15.3:0/1945804333 >> v2:172.21.15.3:3300/0 conn(0x7fa24c0d4ea0 0x7fa24c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa251200640 1 --2- 172.21.15.3:0/1945804333 >> v2:172.21.15.70:3300/0 conn(0x7fa24c0d3320 0x7fa24c0d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa252600640 1 -- 172.21.15.3:0/1945804333 >> v2:172.21.15.70:3300/0 conn(0x7fa24c0d3320 msgr2=0x7fa24c0d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa252600640 1 --2- 172.21.15.3:0/1945804333 >> v2:172.21.15.70:3300/0 conn(0x7fa24c0d3320 0x7fa24c0d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa252600640 1 -- 172.21.15.3:0/1945804333 >> v1:172.21.15.3:6789/0 conn(0x7fa24c15e860 legacy=0x7fa24c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa252600640 1 -- 172.21.15.3:0/1945804333 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0d3ba0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.560+0000 7fa24be00640 1 -- 172.21.15.3:0/1945804333 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230630938 0 0) 0x7fa24c0cfe30 con 0x7fa24c15e860 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.561+0000 7fa252600640 1 --2- 172.21.15.3:0/1945804333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c0b61f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa24c162170 tx=0x7fa24402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd72bdf6fa6c17a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.561+0000 7fa24be00640 1 -- 172.21.15.3:0/1945804333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244007740 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.561+0000 7fa24be00640 1 -- 172.21.15.3:0/1945804333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2440078e0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.561+0000 7fa24be00640 1 -- 172.21.15.3:0/1945804333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244007cb0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 -- 172.21.15.3:0/1945804333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 msgr2=0x7fa24c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 --2- 172.21.15.3:0/1945804333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c0b61f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa24c162170 tx=0x7fa24402a680 comp rx=0 tx=0).stop 2024-09-10T07:51:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 -- 172.21.15.3:0/1945804333 shutdown_connections 2024-09-10T07:51:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 --2- 172.21.15.3:0/1945804333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c0b61f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 --2- 172.21.15.3:0/1945804333 >> v2:172.21.15.70:3300/0 conn(0x7fa24c0d3320 0x7fa24c0d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 -- 172.21.15.3:0/1945804333 >> 172.21.15.3:0/1945804333 conn(0x7fa24c072560 msgr2=0x7fa24c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 -- 172.21.15.3:0/1945804333 shutdown_connections 2024-09-10T07:51:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.562+0000 7fa254600640 1 -- 172.21.15.3:0/1945804333 wait complete. 2024-09-10T07:51:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.564+0000 7fa254600640 1 Processor -- start 2024-09-10T07:51:58.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.564+0000 7fa254600640 1 -- start start 2024-09-10T07:51:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 0x7fa24c0ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c15e860 0x7fa24c1f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d4440 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d4160 con 0x7fa24c15e860 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa254600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0b6730 con 0x7fa24c0d3320 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42724/0 (socket says 172.21.15.3:42724) 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 -- 172.21.15.3:0/974412288 learned_addr learned my addr 172.21.15.3:0/974412288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa252600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c15e860 0x7fa24c1f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251c00640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 0x7fa24c0ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 -- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 msgr2=0x7fa24c0ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 0x7fa24c0ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 -- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c15e860 msgr2=0x7fa24c1f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:58.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c15e860 0x7fa24c1f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251200640 1 -- 172.21.15.3:0/974412288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0ab6a0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.565+0000 7fa251c00640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 0x7fa24c0ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa251200640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa23800e9b0 tx=0x7fa23800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238017070 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23800c9a0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23800cd90 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c0cf770 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.566+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c0cdab0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.567+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c0b6ac0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.568+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa23801a070 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.569+0000 7fa249600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:58.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.569+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4857+0+0 (secure 0 0 0) 0x7fa23808aea0 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.569+0000 7fa251c00640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:58.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.570+0000 7fa251c00640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 0x7fa22c06e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa240000f60 tx=0x7fa240009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:58.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.573+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa238059b90 con 0x7fa24c0d4ea0 2024-09-10T07:51:58.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:58.758+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa24c072ca0 con 0x7fa24c0d4ea0 2024-09-10T07:51:59.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.559+0000 7fa249600640 1 -- 172.21.15.3:0/974412288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7fa238059490 con 0x7fa24c0d4ea0 2024-09-10T07:51:59.559 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:51:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.562+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 msgr2=0x7fa22c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.562+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 0x7fa22c06e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa240000f60 tx=0x7fa240009040 comp rx=0 tx=0).stop 2024-09-10T07:51:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.562+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 msgr2=0x7fa24c1513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.562+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa23800e9b0 tx=0x7fa23800ee70 comp rx=0 tx=0).stop 2024-09-10T07:51:59.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 shutdown_connections 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa22c06be30 0x7fa22c06e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa24c15e860 0x7fa24c1f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa24c0d4ea0 0x7fa24c1513f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 --2- 172.21.15.3:0/974412288 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa24c0d3320 0x7fa24c0ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 >> 172.21.15.3:0/974412288 conn(0x7fa24c072560 msgr2=0x7fa24c0b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.563+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 shutdown_connections 2024-09-10T07:51:59.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.564+0000 7fa254600640 1 -- 172.21.15.3:0/974412288 wait complete. 2024-09-10T07:51:59.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:51:59.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.755+0000 7fb98c200640 1 Processor -- start 2024-09-10T07:51:59.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.755+0000 7fb98c200640 1 -- start start 2024-09-10T07:51:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.756+0000 7fb98c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb984072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb98c200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98415faf0 con 0x7fb984147ce0 2024-09-10T07:51:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb98c200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98415ed30 con 0x7fb9840728d0 2024-09-10T07:51:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb98c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb98415eaf0 con 0x7fb9840aefb0 2024-09-10T07:51:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb989800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb984147ce0 0x7fb9840ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35372/0 (socket says 172.21.15.3:35372) 2024-09-10T07:51:59.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb989800640 1 -- 172.21.15.3:0/3932956636 learned_addr learned my addr 172.21.15.3:0/3932956636 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:51:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb988e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb984072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321987666 0 0) 0x7fb98415faf0 con 0x7fb984147ce0 2024-09-10T07:51:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.757+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb96c003520 con 0x7fb984147ce0 2024-09-10T07:51:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405711416 0 0) 0x7fb96c003520 con 0x7fb984147ce0 2024-09-10T07:51:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 msgr2=0x7fb984072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb984072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 >> v1:172.21.15.70:6789/0 conn(0x7fb9840728d0 legacy=0x7fb98406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98415f010 con 0x7fb984147ce0 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fb978001670 con 0x7fb984147ce0 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 >> v1:172.21.15.3:6789/0 conn(0x7fb984147ce0 legacy=0x7fb9840ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb96c0039e0 0x7fb96c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb96c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c004d40 0x7fb96c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb98415eaf0 con 0x7fb96c0039e0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00baa0 con 0x7fb96c004d40 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c00bca0 con 0x7fb9840aefb0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb96c0039e0 0x7fb96c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb988e00640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb96c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb98a200640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c004d40 0x7fb96c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 -- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 msgr2=0x7fb96c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9840aefb0 0x7fb96c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 -- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c004d40 msgr2=0x7fb96c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c004d40 0x7fb96c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb989800640 1 -- 172.21.15.3:0/3932956636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c003520 con 0x7fb96c0039e0 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.758+0000 7fb98a200640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c004d40 0x7fb96c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:51:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.759+0000 7fb989800640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb96c0039e0 0x7fb96c003e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c210 tx=0x7fb96c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fbc17f82caea14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978007d10 con 0x7fb96c0039e0 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9780058d0 con 0x7fb96c0039e0 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb983e00640 1 -- 172.21.15.3:0/3932956636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978005ce0 con 0x7fb96c0039e0 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 -- 172.21.15.3:0/3932956636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb96c0039e0 msgr2=0x7fb96c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 --2- 172.21.15.3:0/3932956636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb96c0039e0 0x7fb96c003e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c210 tx=0x7fb96c005830 comp rx=0 tx=0).stop 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98a200640 1 -- 172.21.15.3:0/3932956636 reap_dead start 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 -- 172.21.15.3:0/3932956636 shutdown_connections 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 -- 172.21.15.3:0/3932956636 >> 172.21.15.3:0/3932956636 conn(0x7fb9840b5cb0 msgr2=0x7fb98406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 -- 172.21.15.3:0/3932956636 shutdown_connections 2024-09-10T07:51:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.760+0000 7fb98c200640 1 -- 172.21.15.3:0/3932956636 wait complete. 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.762+0000 7fb98c200640 1 Processor -- start 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.762+0000 7fb98c200640 1 -- start start 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c0039e0 0x7fb9841f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb98413b300 0x7fb98413c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb98415f660 con 0x7fb98413cda0 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840730c0 con 0x7fb96c0039e0 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9841480c0 con 0x7fb98413b300 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42744/0 (socket says 172.21.15.3:42744) 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.763+0000 7fb98a200640 1 -- 172.21.15.3:0/73420027 learned_addr learned my addr 172.21.15.3:0/73420027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb988e00640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb98413b300 0x7fb98413c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb989800640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c0039e0 0x7fb9841f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 -- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb98413b300 msgr2=0x7fb98413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb98413b300 0x7fb98413c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 -- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c0039e0 msgr2=0x7fb9841f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:51:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c0039e0 0x7fb9841f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 -- 172.21.15.3:0/73420027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841f52d0 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98a200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb98406c340 tx=0x7fb97c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c006d50 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c003970 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c003d60 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98413b7c0 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.764+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98413bd80 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.765+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb984072d30 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.766+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb97c01b020 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.767+0000 7fb981600640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 0x7fb96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.767+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5227+0+0 (secure 0 0 0) 0x7fb97c08a9d0 con 0x7fb98413cda0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.768+0000 7fb989800640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 0x7fb96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:51:59.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.768+0000 7fb989800640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 0x7fb96406e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb978033000 tx=0x7fb97802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:51:59.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.772+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb9841f52d0 con 0x7fb98413cda0 2024-09-10T07:51:59.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:51:59.958+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb984140e00 con 0x7fb98413cda0 2024-09-10T07:52:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.566+0000 7fb981600640 1 -- 172.21.15.3:0/73420027 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7fb97c01ce40 con 0x7fb98413cda0 2024-09-10T07:52:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:52:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.569+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 msgr2=0x7fb96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.569+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 0x7fb96406e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb978033000 tx=0x7fb97802e000 comp rx=0 tx=0).stop 2024-09-10T07:52:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.569+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 msgr2=0x7fb9841f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.569+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb98406c340 tx=0x7fb97c00dc00 comp rx=0 tx=0).stop 2024-09-10T07:52:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 shutdown_connections 2024-09-10T07:52:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb96406be30 0x7fb96406e2f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb98413cda0 0x7fb9841f54d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb98413b300 0x7fb98413c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 --2- 172.21.15.3:0/73420027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb96c0039e0 0x7fb9841f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 >> 172.21.15.3:0/73420027 conn(0x7fb9840b5cb0 msgr2=0x7fb984161990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:00.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 shutdown_connections 2024-09-10T07:52:00.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.570+0000 7fb98c200640 1 -- 172.21.15.3:0/73420027 wait complete. 2024-09-10T07:52:00.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:52:00.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.761+0000 7f5dadc00640 1 Processor -- start 2024-09-10T07:52:00.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.761+0000 7f5dadc00640 1 -- start start 2024-09-10T07:52:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.762+0000 7f5dadc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5da815c4e0 0x7f5da806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.762+0000 7f5dadc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da80722b0 con 0x7f5da80b5400 2024-09-10T07:52:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.762+0000 7f5dadc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da8072590 con 0x7f5da806c060 2024-09-10T07:52:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.762+0000 7f5dadc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5da806c4d0 con 0x7f5da815c4e0 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5da815c4e0 0x7f5da806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5da815c4e0 0x7f5da806bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42758/0 (socket says 172.21.15.3:42758) 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 -- 172.21.15.3:0/1302865923 learned_addr learned my addr 172.21.15.3:0/1302865923 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 -- 172.21.15.3:0/1302865923 >> v1:172.21.15.70:6789/0 conn(0x7f5da806c060 legacy=0x7f5da80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 -- 172.21.15.3:0/1302865923 >> v1:172.21.15.3:6789/0 conn(0x7f5da80b5400 legacy=0x7f5da80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.763+0000 7f5da6a00640 1 -- 172.21.15.3:0/1302865923 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8072870 con 0x7f5da815c4e0 2024-09-10T07:52:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.764+0000 7f5da6a00640 1 --2- 172.21.15.3:0/1302865923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da815c4e0 0x7f5da806bb20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009870 tx=0x7f5d9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3067f9b732d788d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5da6000640 1 -- 172.21.15.3:0/1302865923 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037070 con 0x7f5da815c4e0 2024-09-10T07:52:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5da6000640 1 -- 172.21.15.3:0/1302865923 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98032440 con 0x7f5da815c4e0 2024-09-10T07:52:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5da6000640 1 -- 172.21.15.3:0/1302865923 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98032810 con 0x7f5da815c4e0 2024-09-10T07:52:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 -- 172.21.15.3:0/1302865923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da815c4e0 msgr2=0x7f5da806bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 --2- 172.21.15.3:0/1302865923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da815c4e0 0x7f5da806bb20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009870 tx=0x7f5d9802a270 comp rx=0 tx=0).stop 2024-09-10T07:52:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 -- 172.21.15.3:0/1302865923 shutdown_connections 2024-09-10T07:52:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 --2- 172.21.15.3:0/1302865923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da815c4e0 0x7f5da806bb20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 -- 172.21.15.3:0/1302865923 >> 172.21.15.3:0/1302865923 conn(0x7f5da80b2a50 msgr2=0x7f5da80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 -- 172.21.15.3:0/1302865923 shutdown_connections 2024-09-10T07:52:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.765+0000 7f5dadc00640 1 -- 172.21.15.3:0/1302865923 wait complete. 2024-09-10T07:52:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.767+0000 7f5dadc00640 1 Processor -- start 2024-09-10T07:52:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.767+0000 7f5dadc00640 1 -- start start 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5da80b5400 0x7f5da81f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da815c4e0 0x7f5da8159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5da815c8c0 con 0x7f5da806c060 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80cf0c0 con 0x7f5da815c4e0 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5dadc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80cfc60 con 0x7f5da80b5400 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42768/0 (socket says 172.21.15.3:42768) 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.768+0000 7f5da6a00640 1 -- 172.21.15.3:0/3704186927 learned_addr learned my addr 172.21.15.3:0/3704186927 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 -- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5da80b5400 msgr2=0x7f5da81f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5da80b5400 0x7f5da81f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 -- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da815c4e0 msgr2=0x7f5da8159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da815c4e0 0x7f5da8159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 -- 172.21.15.3:0/3704186927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da81f3590 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5da6a00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037040 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98042070 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98030bd0 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da8156e50 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.769+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da8157410 con 0x7f5da806c060 2024-09-10T07:52:00.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.771+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8075ab0 con 0x7f5da806c060 2024-09-10T07:52:00.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.772+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d98047460 con 0x7f5da806c060 2024-09-10T07:52:00.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.773+0000 7f5d9f400640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 0x7f5d8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:00.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.773+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5609+0+0 (secure 0 0 0) 0x7f5d980ac430 con 0x7f5da806c060 2024-09-10T07:52:00.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.773+0000 7f5d9e000640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 0x7f5d8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:00.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.774+0000 7f5d9e000640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 0x7f5d8406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5da81546d0 tx=0x7f5d90007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:00.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.777+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5da8075ab0 con 0x7f5da806c060 2024-09-10T07:52:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:00.995+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5da806bb20 con 0x7f5da806c060 2024-09-10T07:52:01.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.575+0000 7f5d9f400640 1 -- 172.21.15.3:0/3704186927 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v81) ==== 174+0+0 (secure 0 0 0) 0x7f5da81f3590 con 0x7f5da806c060 2024-09-10T07:52:01.580 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T07:52:01.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.587+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 msgr2=0x7f5d8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.587+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 0x7f5d8406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5da81546d0 tx=0x7f5d90007990 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.587+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 msgr2=0x7f5da8156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.587+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d980046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 shutdown_connections 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d8406be30 0x7f5d8406e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da815c4e0 0x7f5da8159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5da80b5400 0x7f5da81f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 --2- 172.21.15.3:0/3704186927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5da806c060 0x7f5da8156870 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 >> 172.21.15.3:0/3704186927 conn(0x7f5da80b2a50 msgr2=0x7f5da80d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 shutdown_connections 2024-09-10T07:52:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.588+0000 7f5dadc00640 1 -- 172.21.15.3:0/3704186927 wait complete. 2024-09-10T07:52:01.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:52:01.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.780+0000 7f0f6b200640 1 Processor -- start 2024-09-10T07:52:01.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.780+0000 7f0f6b200640 1 -- start start 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 0x7f0f6c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c15ebb0 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c0cf2d0 con 0x7f0f6c0ced30 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.781+0000 7f0f6b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c0cf5b0 con 0x7f0f6c0d19d0 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42784/0 (socket says 172.21.15.3:42784) 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 -- 172.21.15.3:0/1041358089 learned_addr learned my addr 172.21.15.3:0/1041358089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f6a800640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69400640 1 --2- 172.21.15.3:0/1041358089 >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 0x7f0f6c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 -- 172.21.15.3:0/1041358089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c0d19d0 msgr2=0x7f0f6c0b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 -- 172.21.15.3:0/1041358089 >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 msgr2=0x7f0f6c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1041358089 >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 0x7f0f6c0d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69e00640 1 -- 172.21.15.3:0/1041358089 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c0cf890 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.782+0000 7f0f69400640 1 --2- 172.21.15.3:0/1041358089 >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 0x7f0f6c0d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.783+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0b76e0 tx=0x7f0f5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff582a2e6d15d4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.783+0000 7f0f68a00640 1 -- 172.21.15.3:0/1041358089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c037070 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.783+0000 7f0f68a00640 1 -- 172.21.15.3:0/1041358089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c032440 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.783+0000 7f0f68a00640 1 -- 172.21.15.3:0/1041358089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c032810 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 -- 172.21.15.3:0/1041358089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c15e7d0 msgr2=0x7f0f6c162ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0b76e0 tx=0x7f0f5c02a140 comp rx=0 tx=0).stop 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 -- 172.21.15.3:0/1041358089 shutdown_connections 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1041358089 >> v2:172.21.15.70:3300/0 conn(0x7f0f6c0ced30 0x7f0f6c0d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1041358089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c15e7d0 0x7f0f6c162ae0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 -- 172.21.15.3:0/1041358089 >> 172.21.15.3:0/1041358089 conn(0x7f0f6c06bd00 msgr2=0x7f0f6c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 -- 172.21.15.3:0/1041358089 shutdown_connections 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.784+0000 7f0f6b200640 1 -- 172.21.15.3:0/1041358089 wait complete. 2024-09-10T07:52:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.786+0000 7f0f6b200640 1 Processor -- start 2024-09-10T07:52:01.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.786+0000 7f0f6b200640 1 -- start start 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c15e7d0 0x7f0f6c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c163020 con 0x7f0f6c0ced30 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0cfe50 con 0x7f0f6c0d19d0 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c15ce80 con 0x7f0f6c15e7d0 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42786/0 (socket says 172.21.15.3:42786) 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 -- 172.21.15.3:0/1504481118 learned_addr learned my addr 172.21.15.3:0/1504481118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:01.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69400640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f6a800640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c15e7d0 0x7f0f6c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 -- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c15e7d0 msgr2=0x7f0f6c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c15e7d0 0x7f0f6c15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 -- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c0d19d0 msgr2=0x7f0f6c0b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.787+0000 7f0f69e00640 1 -- 172.21.15.3:0/1504481118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c1f4d80 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f69e00640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02a180 tx=0x7f0f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c037040 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c042070 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c030880 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c1f5060 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.788+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1f5620 con 0x7f0f6c0ced30 2024-09-10T07:52:01.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.789+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c0d1490 con 0x7f0f6c0ced30 2024-09-10T07:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.790+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f5c030a20 con 0x7f0f6c0ced30 2024-09-10T07:52:01.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.791+0000 7f0f62000640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 0x7f0f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.791+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f0f5c0ab670 con 0x7f0f6c0ced30 2024-09-10T07:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.791+0000 7f0f69400640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 0x7f0f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.792+0000 7f0f69400640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 0x7f0f4c06e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0b3be0 tx=0x7f0f58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:01.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.795+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f5c079930 con 0x7f0f6c0ced30 2024-09-10T07:52:01.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.983+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f6c1e9e40 con 0x7f0f6c0ced30 2024-09-10T07:52:01.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.984+0000 7f0f62000640 1 -- 172.21.15.3:0/1504481118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16673 (secure 0 0 0) 0x7f0f5c07dc30 con 0x7f0f6c0ced30 2024-09-10T07:52:01.984 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:52:00.997334+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:58.800477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T07:52:00.000902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 msgr2=0x7f0f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 0x7f0f4c06e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0b3be0 tx=0x7f0f58008040 comp rx=0 tx=0).stop 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 msgr2=0x7f0f6c15c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c02a180 tx=0x7f0f5c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 shutdown_connections 2024-09-10T07:52:01.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f4c06be30 0x7f0f4c06e2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f6c15e7d0 0x7f0f6c15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c0d19d0 0x7f0f6c0b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 --2- 172.21.15.3:0/1504481118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f6c0ced30 0x7f0f6c15c940 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 >> 172.21.15.3:0/1504481118 conn(0x7f0f6c06bd00 msgr2=0x7f0f6c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 shutdown_connections 2024-09-10T07:52:01.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:01.987+0000 7f0f6b200640 1 -- 172.21.15.3:0/1504481118 wait complete. 2024-09-10T07:52:02.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:52:02.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.160+0000 7f34dfa00640 1 Processor -- start 2024-09-10T07:52:02.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.160+0000 7f34dfa00640 1 -- start start 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34d8160af0 0x7f34d80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34d80cf800 0x7f34d80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34d80cfdd0 con 0x7f34d80cf800 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f34d815cc90 con 0x7f34d8160af0 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.161+0000 7f34dfa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34d815cf70 con 0x7f34d8160710 2024-09-10T07:52:02.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34d80cf800 0x7f34d80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34d80cf800 0x7f34d80d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42788/0 (socket says 172.21.15.3:42788) 2024-09-10T07:52:02.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 -- 172.21.15.3:0/1396179665 learned_addr learned my addr 172.21.15.3:0/1396179665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:02.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34d7e00640 1 --2- 172.21.15.3:0/1396179665 >> v2:172.21.15.70:3300/0 conn(0x7f34d8160af0 0x7f34d80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dd000640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 -- 172.21.15.3:0/1396179665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 msgr2=0x7f34d80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 -- 172.21.15.3:0/1396179665 >> v2:172.21.15.70:3300/0 conn(0x7f34d8160af0 msgr2=0x7f34d80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 --2- 172.21.15.3:0/1396179665 >> v2:172.21.15.70:3300/0 conn(0x7f34d8160af0 0x7f34d80d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.162+0000 7f34dda00640 1 -- 172.21.15.3:0/1396179665 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d815d250 con 0x7f34d80cf800 2024-09-10T07:52:02.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.163+0000 7f34dda00640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d80cf800 0x7f34d80d05f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f34d806cd40 tx=0x7f34d002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab58e68c05c5968 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.163+0000 7f34d7400640 1 -- 172.21.15.3:0/1396179665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0036070 con 0x7f34d80cf800 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34d7400640 1 -- 172.21.15.3:0/1396179665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d0031600 con 0x7f34d80cf800 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34d7400640 1 -- 172.21.15.3:0/1396179665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00319d0 con 0x7f34d80cf800 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 -- 172.21.15.3:0/1396179665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d80cf800 msgr2=0x7f34d80d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d80cf800 0x7f34d80d05f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f34d806cd40 tx=0x7f34d002a6a0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 -- 172.21.15.3:0/1396179665 shutdown_connections 2024-09-10T07:52:02.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d80cf800 0x7f34d80d05f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 --2- 172.21.15.3:0/1396179665 >> v2:172.21.15.70:3300/0 conn(0x7f34d8160af0 0x7f34d80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 --2- 172.21.15.3:0/1396179665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 -- 172.21.15.3:0/1396179665 >> 172.21.15.3:0/1396179665 conn(0x7f34d806d000 msgr2=0x7f34d806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 -- 172.21.15.3:0/1396179665 shutdown_connections 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.164+0000 7f34dfa00640 1 -- 172.21.15.3:0/1396179665 wait complete. 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.166+0000 7f34dfa00640 1 Processor -- start 2024-09-10T07:52:02.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.166+0000 7f34dfa00640 1 -- start start 2024-09-10T07:52:02.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 0x7f34d814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34d815d770 con 0x7f34d8160af0 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f34d806bb20 con 0x7f34d80cf800 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dfa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34d80b3400 con 0x7f34d8160710 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42802/0 (socket says 172.21.15.3:42802) 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dda00640 1 -- 172.21.15.3:0/2321100457 learned_addr learned my addr 172.21.15.3:0/2321100457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dd000640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 0x7f34d814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34dda00640 1 -- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 msgr2=0x7f34d80b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.167+0000 7f34d7e00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dda00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dda00640 1 -- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 msgr2=0x7f34d814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dda00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 0x7f34d814c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dda00640 1 -- 172.21.15.3:0/2321100457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d80b3650 con 0x7f34d8160af0 2024-09-10T07:52:02.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dd000640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 0x7f34d814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:02.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dda00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009f40 tx=0x7f34d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0036050 con 0x7f34d8160af0 2024-09-10T07:52:02.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d0042070 con 0x7f34d8160af0 2024-09-10T07:52:02.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0009510 con 0x7f34d8160af0 2024-09-10T07:52:02.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d80b3930 con 0x7f34d8160af0 2024-09-10T07:52:02.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.168+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d8154150 con 0x7f34d8160af0 2024-09-10T07:52:02.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.169+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d80afd50 con 0x7f34d8160af0 2024-09-10T07:52:02.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.170+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34d0007bb0 con 0x7f34d8160af0 2024-09-10T07:52:02.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.171+0000 7f34d4c00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 0x7f34b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.171+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f34d00abce0 con 0x7f34d8160af0 2024-09-10T07:52:02.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.171+0000 7f34dd000640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 0x7f34b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.172+0000 7f34dd000640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 0x7f34b806e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.175+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34d0045360 con 0x7f34d8160af0 2024-09-10T07:52:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.362+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34d80b1340 con 0x7f34d8160af0 2024-09-10T07:52:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.364+0000 7f34d4c00640 1 -- 172.21.15.3:0/2321100457 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16673 (secure 0 0 0) 0x7f34d007a680 con 0x7f34d8160af0 2024-09-10T07:52:02.363 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:02.364 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:52:00.997334+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:58.800477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T07:52:00.000902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 msgr2=0x7f34b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 0x7f34b806e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc008040 comp rx=0 tx=0).stop 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 msgr2=0x7f34d80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009f40 tx=0x7f34d0004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 shutdown_connections 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f34b806be30 0x7f34b806e2f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34d8160af0 0x7f34d80b4f40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34d8160710 0x7f34d80b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 --2- 172.21.15.3:0/2321100457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34d80cf800 0x7f34d814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 >> 172.21.15.3:0/2321100457 conn(0x7f34d806d000 msgr2=0x7f34d80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 shutdown_connections 2024-09-10T07:52:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.366+0000 7f34dfa00640 1 -- 172.21.15.3:0/2321100457 wait complete. 2024-09-10T07:52:02.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:52:02.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.542+0000 7fad12a00640 1 Processor -- start 2024-09-10T07:52:02.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.543+0000 7fad12a00640 1 -- start start 2024-09-10T07:52:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad12a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c1451d0 0x7fad0c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad12a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad0c0cf270 0x7fad0c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad12a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad0c0cd900 con 0x7fad0c145730 2024-09-10T07:52:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad12a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c06d7c0 con 0x7fad0c0cf270 2024-09-10T07:52:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad12a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c0cdbe0 con 0x7fad0c1451d0 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad10a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad0c0cf270 0x7fad0c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad0be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c1451d0 0x7fad0c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad10a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad0c0cf270 0x7fad0c0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46188/0 (socket says 172.21.15.3:46188) 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.544+0000 7fad10a00640 1 -- 172.21.15.3:0/3950051015 learned_addr learned my addr 172.21.15.3:0/3950051015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.545+0000 7fad10a00640 1 -- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c1451d0 msgr2=0x7fad0c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.545+0000 7fad10a00640 1 --2- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c1451d0 0x7fad0c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.545+0000 7fad10a00640 1 -- 172.21.15.3:0/3950051015 >> v1:172.21.15.70:6789/0 conn(0x7fad0c145730 legacy=0x7fad0c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.545+0000 7fad10a00640 1 -- 172.21.15.3:0/3950051015 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c06da10 con 0x7fad0c0cf270 2024-09-10T07:52:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.546+0000 7fad10a00640 1 --2- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c0cf720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fad04009e50 tx=0x7fad0402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5450ad0fa817eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.546+0000 7fad0b400640 1 -- 172.21.15.3:0/3950051015 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04036070 con 0x7fad0c0cf270 2024-09-10T07:52:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.546+0000 7fad0b400640 1 -- 172.21.15.3:0/3950051015 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad04031600 con 0x7fad0c0cf270 2024-09-10T07:52:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.546+0000 7fad0b400640 1 -- 172.21.15.3:0/3950051015 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040319d0 con 0x7fad0c0cf270 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 -- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 msgr2=0x7fad0c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 --2- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c0cf720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fad04009e50 tx=0x7fad0402a680 comp rx=0 tx=0).stop 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 -- 172.21.15.3:0/3950051015 shutdown_connections 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 --2- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c0cf720 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 --2- 172.21.15.3:0/3950051015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c1451d0 0x7fad0c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 -- 172.21.15.3:0/3950051015 >> 172.21.15.3:0/3950051015 conn(0x7fad0c0b5cb0 msgr2=0x7fad0c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 -- 172.21.15.3:0/3950051015 shutdown_connections 2024-09-10T07:52:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.547+0000 7fad12a00640 1 -- 172.21.15.3:0/3950051015 wait complete. 2024-09-10T07:52:02.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.549+0000 7fad12a00640 1 Processor -- start 2024-09-10T07:52:02.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.549+0000 7fad12a00640 1 -- start start 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 0x7fad0c0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c06df30 con 0x7fad0c1451d0 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c145f10 con 0x7fad0c0cf270 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad12a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c06e0b0 con 0x7fad0c145730 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad03e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42806/0 (socket says 172.21.15.3:42806) 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad03e00640 1 -- 172.21.15.3:0/1832816616 learned_addr learned my addr 172.21.15.3:0/1832816616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad0be00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.550+0000 7fad10a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 0x7fad0c0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 -- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 msgr2=0x7fad0c0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 0x7fad0c0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 -- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 msgr2=0x7fad0c15d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c15d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 -- 172.21.15.3:0/1832816616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c131ec0 con 0x7fad0c1451d0 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad10a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 0x7fad0c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad03e00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7facf400c440 tx=0x7facf400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40052d0 con 0x7fad0c1451d0 2024-09-10T07:52:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf4005470 con 0x7fad0c1451d0 2024-09-10T07:52:02.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c132200 con 0x7fad0c1451d0 2024-09-10T07:52:02.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4007260 con 0x7fad0c1451d0 2024-09-10T07:52:02.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.551+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c132a40 con 0x7fad0c1451d0 2024-09-10T07:52:02.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.553+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c0cf720 con 0x7fad0c1451d0 2024-09-10T07:52:02.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.553+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facf40074c0 con 0x7fad0c1451d0 2024-09-10T07:52:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.554+0000 7fad08c00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 0x7face806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.554+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7facf408a2c0 con 0x7fad0c1451d0 2024-09-10T07:52:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.555+0000 7fad0be00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 0x7face806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:02.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.555+0000 7fad0be00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 0x7face806e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7facfc009b40 tx=0x7facfc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.559+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad0c132200 con 0x7fad0c1451d0 2024-09-10T07:52:02.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:02.782+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fad0c06e990 con 0x7fad0c1451d0 2024-09-10T07:52:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.592+0000 7fad08c00640 1 -- 172.21.15.3:0/1832816616 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7facf40585a0 con 0x7fad0c1451d0 2024-09-10T07:52:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 msgr2=0x7face806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 0x7face806e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7facfc009b40 tx=0x7facfc009190 comp rx=0 tx=0).stop 2024-09-10T07:52:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 msgr2=0x7fad0c134120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7facf400c440 tx=0x7facf400c900 comp rx=0 tx=0).stop 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 shutdown_connections 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7face806be30 0x7face806e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad0c145730 0x7fad0c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad0c1451d0 0x7fad0c134120 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 --2- 172.21.15.3:0/1832816616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad0c0cf270 0x7fad0c15d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 >> 172.21.15.3:0/1832816616 conn(0x7fad0c0b5cb0 msgr2=0x7fad0c140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 shutdown_connections 2024-09-10T07:52:03.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.596+0000 7fad12a00640 1 -- 172.21.15.3:0/1832816616 wait complete. 2024-09-10T07:52:03.616 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:52:03.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:52:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.781+0000 7f5905800640 1 Processor -- start 2024-09-10T07:52:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.781+0000 7f5905800640 1 -- start start 2024-09-10T07:52:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.782+0000 7f5905800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5900145730 0x7f590006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.782+0000 7f5905800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59000cd900 con 0x7f59001451d0 2024-09-10T07:52:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.782+0000 7f5905800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59000cdbe0 con 0x7f59000cf270 2024-09-10T07:52:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f5905800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f590006d7c0 con 0x7f5900145730 2024-09-10T07:52:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59001451d0 0x7f59000b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57330/0 (socket says 172.21.15.3:57330) 2024-09-10T07:52:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fea00640 1 -- 172.21.15.3:0/1109535178 learned_addr learned my addr 172.21.15.3:0/1109535178 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112945227 0 0) 0x7f59000cd900 con 0x7f59001451d0 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58f6000640 1 --2- 172.21.15.3:0/1109535178 >> v2:172.21.15.70:3300/0 conn(0x7f5900145730 0x7f590006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58e4003520 con 0x7f59001451d0 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 317675579 0 0) 0x7f58e4003520 con 0x7f59001451d0 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 >> v2:172.21.15.70:3300/0 conn(0x7f5900145730 msgr2=0x7f590006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.783+0000 7f58fe000640 1 --2- 172.21.15.3:0/1109535178 >> v2:172.21.15.70:3300/0 conn(0x7f5900145730 0x7f590006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 >> v1:172.21.15.70:6789/0 conn(0x7f59000cf270 legacy=0x7f59000cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590006da10 con 0x7f59001451d0 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f58f0001670 con 0x7f59001451d0 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 >> v1:172.21.15.3:6789/0 conn(0x7f59001451d0 legacy=0x7f59000b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 0x7f58e4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58e4004800 0x7f58e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4005230 0x7f58e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f590006d7c0 con 0x7f58e4003a00 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f58e400c190 con 0x7f58e4004800 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f58e400c390 con 0x7f58e4005230 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 0x7f58e4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58f6000640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58e4004800 0x7f58e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58ff400640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4005230 0x7f58e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 -- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4005230 msgr2=0x7f58e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4005230 0x7f58e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 -- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58e4004800 msgr2=0x7f58e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58e4004800 0x7f58e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58fea00640 1 -- 172.21.15.3:0/1109535178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e4003520 con 0x7f58e4003a00 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58ff400640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4005230 0x7f58e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.784+0000 7f58ff400640 1 -- 172.21.15.3:0/1109535178 reap_dead start 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.785+0000 7f58fea00640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 0x7f58e4003e40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c110 tx=0x7f58e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6847ea82f1cdce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007c40 con 0x7f58e4003a00 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f0005af0 con 0x7f58e4003a00 2024-09-10T07:52:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f58fe000640 1 -- 172.21.15.3:0/1109535178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007070 con 0x7f58e4003a00 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 -- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 msgr2=0x7f58e4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 0x7f58e4003e40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c110 tx=0x7f58e4005830 comp rx=0 tx=0).stop 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 -- 172.21.15.3:0/1109535178 shutdown_connections 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 --2- 172.21.15.3:0/1109535178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e4003a00 0x7f58e4003e40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 -- 172.21.15.3:0/1109535178 >> 172.21.15.3:0/1109535178 conn(0x7f59000b5cb0 msgr2=0x7f590006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 -- 172.21.15.3:0/1109535178 shutdown_connections 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.786+0000 7f5905800640 1 -- 172.21.15.3:0/1109535178 wait complete. 2024-09-10T07:52:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.788+0000 7f5905800640 1 Processor -- start 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.788+0000 7f5905800640 1 -- start start 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 0x7f590015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 0x7f5900141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f590006d140 con 0x7f5900143360 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f590006cfc0 con 0x7f59000b4500 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f5905800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f590006d2c0 con 0x7f58e4003a00 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f58f6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f58f6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59698/0 (socket says 172.21.15.3:59698) 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.789+0000 7f58f6000640 1 -- 172.21.15.3:0/221964604 learned_addr learned my addr 172.21.15.3:0/221964604 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58fea00640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 0x7f590015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58ff400640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 0x7f5900141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 -- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 msgr2=0x7f590015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 0x7f590015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 -- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 msgr2=0x7f5900141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 0x7f5900141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 -- 172.21.15.3:0/221964604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59001422c0 con 0x7f5900143360 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58fea00640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 0x7f590015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58ff400640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 0x7f5900141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f6000640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f58e8011a70 tx=0x7f58e8011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800fb50 con 0x7f5900143360 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e800fcf0 con 0x7f5900143360 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001425a0 con 0x7f5900143360 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800fea0 con 0x7f5900143360 2024-09-10T07:52:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.790+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5900142b60 con 0x7f5900143360 2024-09-10T07:52:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.791+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59000af8b0 con 0x7f5900143360 2024-09-10T07:52:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.792+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58e8012040 con 0x7f5900143360 2024-09-10T07:52:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.793+0000 7f58f7400640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.793+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f58e808b560 con 0x7f5900143360 2024-09-10T07:52:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.793+0000 7f58fea00640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.794+0000 7f58fea00640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 0x7f58dc06e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58f0032000 tx=0x7f58f002d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:03.798+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59000af8b0 con 0x7f5900143360 2024-09-10T07:52:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.019+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f59000b4910 con 0x7f5900143360 2024-09-10T07:52:04.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.594+0000 7f58f7400640 1 -- 172.21.15.3:0/221964604 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f59000b4910 con 0x7f5900143360 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 msgr2=0x7f58dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 0x7f58dc06e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58f0032000 tx=0x7f58f002d000 comp rx=0 tx=0).stop 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 msgr2=0x7f59000b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f58e8011a70 tx=0x7f58e8011f30 comp rx=0 tx=0).stop 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 shutdown_connections 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59000b4500 0x7f5900141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5900143360 0x7f59000b3fc0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 --2- 172.21.15.3:0/221964604 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f58e4003a00 0x7f590015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 >> 172.21.15.3:0/221964604 conn(0x7f59000b5cb0 msgr2=0x7f59000b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 shutdown_connections 2024-09-10T07:52:04.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.597+0000 7f5905800640 1 -- 172.21.15.3:0/221964604 wait complete. 2024-09-10T07:52:04.615 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:52:04.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:04.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.773+0000 7ff33ec00640 1 Processor -- start 2024-09-10T07:52:04.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.774+0000 7ff33ec00640 1 -- start start 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33ec00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff33815b4f0 0x7ff3380d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33ec00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff338073d10 0x7ff3380d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33ec00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3380ccd80 con 0x7ff33815b110 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33ec00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff3380b44f0 con 0x7ff338073d10 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33ec00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff3380cd060 con 0x7ff33815b4f0 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff338073d10 0x7ff3380d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33cc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff338073d10 0x7ff3380d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59712/0 (socket says 172.21.15.3:59712) 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff33cc00640 1 -- 172.21.15.3:0/762400519 learned_addr learned my addr 172.21.15.3:0/762400519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:04.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.775+0000 7ff32fe00640 1 --2- 172.21.15.3:0/762400519 >> v2:172.21.15.70:3300/0 conn(0x7ff33815b4f0 0x7ff3380d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.776+0000 7ff33cc00640 1 -- 172.21.15.3:0/762400519 >> v2:172.21.15.70:3300/0 conn(0x7ff33815b4f0 msgr2=0x7ff3380d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.776+0000 7ff33cc00640 1 --2- 172.21.15.3:0/762400519 >> v2:172.21.15.70:3300/0 conn(0x7ff33815b4f0 0x7ff3380d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.776+0000 7ff33cc00640 1 -- 172.21.15.3:0/762400519 >> v1:172.21.15.70:6789/0 conn(0x7ff33815b110 legacy=0x7ff3380b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.776+0000 7ff33cc00640 1 -- 172.21.15.3:0/762400519 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3380cd340 con 0x7ff338073d10 2024-09-10T07:52:04.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33cc00640 1 --2- 172.21.15.3:0/762400519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff338073d10 0x7ff3380d0720 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff3380d1310 tx=0x7ff33002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a118b8b837b1537 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff337400640 1 -- 172.21.15.3:0/762400519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000b2e0 con 0x7ff338073d10 2024-09-10T07:52:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff337400640 1 -- 172.21.15.3:0/762400519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff330037070 con 0x7ff338073d10 2024-09-10T07:52:04.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff337400640 1 -- 172.21.15.3:0/762400519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000b480 con 0x7ff338073d10 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 -- 172.21.15.3:0/762400519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff338073d10 msgr2=0x7ff3380d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 --2- 172.21.15.3:0/762400519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff338073d10 0x7ff3380d0720 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff3380d1310 tx=0x7ff33002c940 comp rx=0 tx=0).stop 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 -- 172.21.15.3:0/762400519 shutdown_connections 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 --2- 172.21.15.3:0/762400519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff338073d10 0x7ff3380d0720 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 --2- 172.21.15.3:0/762400519 >> v2:172.21.15.70:3300/0 conn(0x7ff33815b4f0 0x7ff3380d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.777+0000 7ff33ec00640 1 -- 172.21.15.3:0/762400519 >> 172.21.15.3:0/762400519 conn(0x7ff33806d000 msgr2=0x7ff33806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.778+0000 7ff33ec00640 1 -- 172.21.15.3:0/762400519 shutdown_connections 2024-09-10T07:52:04.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.778+0000 7ff33ec00640 1 -- 172.21.15.3:0/762400519 wait complete. 2024-09-10T07:52:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.779+0000 7ff33ec00640 1 Processor -- start 2024-09-10T07:52:04.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.779+0000 7ff33ec00640 1 -- start start 2024-09-10T07:52:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff338073d10 0x7ff338148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 0x7ff338159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff33806bca0 con 0x7ff33815b110 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff33806bb20 con 0x7ff33815b4f0 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.780+0000 7ff33ec00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff33806be20 con 0x7ff338073d10 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59722/0 (socket says 172.21.15.3:59722) 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 -- 172.21.15.3:0/96744644 learned_addr learned my addr 172.21.15.3:0/96744644 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff337e00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff338073d10 0x7ff338148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff33cc00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 0x7ff338159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 -- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff338073d10 msgr2=0x7ff338148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff338073d10 0x7ff338148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 -- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 msgr2=0x7ff338159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 0x7ff338159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 -- 172.21.15.3:0/96744644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338071120 con 0x7ff33815b110 2024-09-10T07:52:04.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff33cc00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 0x7ff338159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff32fe00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e870 tx=0x7ff32000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.781+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320017070 con 0x7ff33815b110 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.782+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32000c940 con 0x7ff33815b110 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.782+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000cd30 con 0x7ff33815b110 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.782+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff338071400 con 0x7ff33815b110 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.782+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3380719c0 con 0x7ff33815b110 2024-09-10T07:52:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.783+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3380b3d10 con 0x7ff33815b110 2024-09-10T07:52:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.784+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff32001a070 con 0x7ff33815b110 2024-09-10T07:52:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.784+0000 7ff334c00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 0x7ff31406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.785+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7ff32008b040 con 0x7ff33815b110 2024-09-10T07:52:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.785+0000 7ff337e00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 0x7ff31406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.785+0000 7ff337e00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 0x7ff31406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff328001640 tx=0x7ff328009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:04.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:04.789+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff320059a80 con 0x7ff33815b110 2024-09-10T07:52:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.008+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3380b12a0 con 0x7ff33815b110 2024-09-10T07:52:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.009+0000 7ff334c00640 1 -- 172.21.15.3:0/96744644 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5216 (secure 0 0 0) 0x7ff320059380 con 0x7ff33815b110 2024-09-10T07:52:05.008 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:05.008 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":85,"btime":"2024-09-10T07:52:04:593834+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:04.020861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[7239]},"id":6}]} 2024-09-10T07:52:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 85 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 msgr2=0x7ff31406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 0x7ff31406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff328001640 tx=0x7ff328009040 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 msgr2=0x7ff3381592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e870 tx=0x7ff32000ed30 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 shutdown_connections 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff31406be30 0x7ff31406e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff33815b4f0 0x7ff338159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff33815b110 0x7ff3381592d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 --2- 172.21.15.3:0/96744644 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff338073d10 0x7ff338148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 >> 172.21.15.3:0/96744644 conn(0x7ff33806d000 msgr2=0x7ff33806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 shutdown_connections 2024-09-10T07:52:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.011+0000 7ff33ec00640 1 -- 172.21.15.3:0/96744644 wait complete. 2024-09-10T07:52:05.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:52:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.183+0000 7f40d4c00640 1 Processor -- start 2024-09-10T07:52:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.183+0000 7f40d4c00640 1 -- start start 2024-09-10T07:52:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40d00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40d4c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d00cfdd0 con 0x7f40d0160af0 2024-09-10T07:52:05.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40d4c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d015cc90 con 0x7f40d00d07c0 2024-09-10T07:52:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40d4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f40d015cf70 con 0x7f40d0160710 2024-09-10T07:52:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40c7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40d0160af0 0x7f40d00d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57332/0 (socket says 172.21.15.3:57332) 2024-09-10T07:52:05.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40c7e00640 1 -- 172.21.15.3:0/2736412970 learned_addr learned my addr 172.21.15.3:0/2736412970 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40cde00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40d00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2733108772 0 0) 0x7f40d00cfdd0 con 0x7f40d0160af0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.185+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40b4003520 con 0x7f40d0160af0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486122330 0 0) 0x7f40b4003520 con 0x7f40d0160af0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 msgr2=0x7f40d00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40d00cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 >> v1:172.21.15.70:6789/0 conn(0x7f40d00d07c0 legacy=0x7f40d0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d015d250 con 0x7f40d0160af0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f40b8001670 con 0x7f40d0160af0 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 >> v1:172.21.15.3:6789/0 conn(0x7f40d0160af0 legacy=0x7f40d00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40b40039e0 0x7f40b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40b4004800 0x7f40b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40d015cf70 con 0x7f40b40039e0 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40b400baa0 con 0x7f40b4004800 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f40b400bca0 con 0x7f40d0160710 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40b40039e0 0x7f40b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40c7e00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40b4004800 0x7f40b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40ce800640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 -- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 msgr2=0x7f40b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40d0160710 0x7f40b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 -- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40b4004800 msgr2=0x7f40b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40b4004800 0x7f40b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.186+0000 7f40cde00640 1 -- 172.21.15.3:0/2736412970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b4003520 con 0x7f40b40039e0 2024-09-10T07:52:05.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.187+0000 7f40cde00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40b40039e0 0x7f40b4003e00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009a90 tx=0x7f40b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3884192bef1311f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0036070 con 0x7f40b40039e0 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c0031400 con 0x7f40b40039e0 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40cd400640 1 -- 172.21.15.3:0/2736412970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0009690 con 0x7f40b40039e0 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 -- 172.21.15.3:0/2736412970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40b40039e0 msgr2=0x7f40b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 --2- 172.21.15.3:0/2736412970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40b40039e0 0x7f40b4003e00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009a90 tx=0x7f40b4005830 comp rx=0 tx=0).stop 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40ce800640 1 -- 172.21.15.3:0/2736412970 reap_dead start 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 -- 172.21.15.3:0/2736412970 shutdown_connections 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 -- 172.21.15.3:0/2736412970 >> 172.21.15.3:0/2736412970 conn(0x7f40d006d000 msgr2=0x7f40d006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 -- 172.21.15.3:0/2736412970 shutdown_connections 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.188+0000 7f40d4c00640 1 -- 172.21.15.3:0/2736412970 wait complete. 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.190+0000 7f40d4c00640 1 Processor -- start 2024-09-10T07:52:05.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.190+0000 7f40d4c00640 1 -- start start 2024-09-10T07:52:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40b40039e0 0x7f40d00b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 0x7f40d01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40d015d770 con 0x7f40d0073170 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40d00cf2c0 con 0x7f40d0072d10 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40d4c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f40d00b0a20 con 0x7f40b40039e0 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40ce800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40ce800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59730/0 (socket says 172.21.15.3:59730) 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40ce800640 1 -- 172.21.15.3:0/1119779496 learned_addr learned my addr 172.21.15.3:0/1119779496 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40cde00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40b40039e0 0x7f40d00b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.191+0000 7f40c7e00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 0x7f40d01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 -- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40b40039e0 msgr2=0x7f40d00b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40b40039e0 0x7f40d00b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 -- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 msgr2=0x7f40d01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 0x7f40d01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 -- 172.21.15.3:0/1119779496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d00b0cc0 con 0x7f40d0073170 2024-09-10T07:52:05.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40c7e00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 0x7f40d01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40ce800640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000f80 tx=0x7f40c800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003970 con 0x7f40d0073170 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c8018070 con 0x7f40d0073170 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d00b0fa0 con 0x7f40d0073170 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003d70 con 0x7f40d0073170 2024-09-10T07:52:05.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.192+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d00b1560 con 0x7f40d0073170 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.193+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d00b1760 con 0x7f40d0073170 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.194+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40c80050d0 con 0x7f40d0073170 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.195+0000 7f40c6000640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.195+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f40c808ab50 con 0x7f40d0073170 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.195+0000 7f40cde00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.196+0000 7f40cde00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 0x7f40ac06e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40c0033000 tx=0x7f40c0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.199+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40d00b1760 con 0x7f40d0073170 2024-09-10T07:52:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.386+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40d015d8f0 con 0x7f40d0073170 2024-09-10T07:52:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.387+0000 7f40c6000640 1 -- 172.21.15.3:0/1119779496 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16660 (secure 0 0 0) 0x7f40c80593d0 con 0x7f40d0073170 2024-09-10T07:52:05.387 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:05.387 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:52:02.576089+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:58.800477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T07:52:00.000902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:52:05.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.389+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 msgr2=0x7f40ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.389+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 0x7f40ac06e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40c0033000 tx=0x7f40c0008030 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.389+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 msgr2=0x7f40d014fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.389+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000f80 tx=0x7f40c800ec10 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 shutdown_connections 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d0073170 0x7f40d014fd80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d0072d10 0x7f40d01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 --2- 172.21.15.3:0/1119779496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f40b40039e0 0x7f40d00b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 >> 172.21.15.3:0/1119779496 conn(0x7f40d006d000 msgr2=0x7f40d015bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 shutdown_connections 2024-09-10T07:52:05.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.390+0000 7f40d4c00640 1 -- 172.21.15.3:0/1119779496 wait complete. 2024-09-10T07:52:05.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:52:05.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:52:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.562+0000 7f4639600640 1 Processor -- start 2024-09-10T07:52:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.563+0000 7f4639600640 1 -- start start 2024-09-10T07:52:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4639600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 0x7f46340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4639600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4634071cd0 0x7f46340b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4639600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46340ce730 con 0x7f4634145010 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4639600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f46340ced10 con 0x7f4634071cd0 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4639600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4634073e60 con 0x7f463415ca80 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4633e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4634145010 0x7f4634073410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57336/0 (socket says 172.21.15.3:57336) 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4633e00640 1 -- 172.21.15.3:0/3230985460 learned_addr learned my addr 172.21.15.3:0/3230985460 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f462be00640 1 --2- 172.21.15.3:0/3230985460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 0x7f46340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.564+0000 7f4638c00640 1 --2- 172.21.15.3:0/3230985460 >> v2:172.21.15.70:3300/0 conn(0x7f4634071cd0 0x7f46340b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747406582 0 0) 0x7f46340ce730 con 0x7f4634145010 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4618003520 con 0x7f4634145010 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f462be00640 1 -- 172.21.15.3:0/3230985460 >> v2:172.21.15.70:3300/0 conn(0x7f4634071cd0 msgr2=0x7f46340b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f462be00640 1 --2- 172.21.15.3:0/3230985460 >> v2:172.21.15.70:3300/0 conn(0x7f4634071cd0 0x7f46340b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f462be00640 1 -- 172.21.15.3:0/3230985460 >> v1:172.21.15.3:6789/0 conn(0x7f4634145010 legacy=0x7f4634073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1602928407 0 0) 0x7f4618003520 con 0x7f4634145010 2024-09-10T07:52:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.565+0000 7f462be00640 1 -- 172.21.15.3:0/3230985460 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46340ce9a0 con 0x7f463415ca80 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.566+0000 7f462be00640 1 --2- 172.21.15.3:0/3230985460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 0x7f46340b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097a0 tx=0x7f461c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a19888afc41ebc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.566+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c037070 con 0x7f463415ca80 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.566+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c02aba0 con 0x7f463415ca80 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.566+0000 7f4633400640 1 -- 172.21.15.3:0/3230985460 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624001670 con 0x7f463415ca80 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 -- 172.21.15.3:0/3230985460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 msgr2=0x7f46340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 --2- 172.21.15.3:0/3230985460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 0x7f46340b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097a0 tx=0x7f461c02a110 comp rx=0 tx=0).stop 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 -- 172.21.15.3:0/3230985460 shutdown_connections 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 --2- 172.21.15.3:0/3230985460 >> v2:172.21.15.70:3300/0 conn(0x7f4634071cd0 0x7f46340b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 --2- 172.21.15.3:0/3230985460 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f463415ca80 0x7f46340b2000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 -- 172.21.15.3:0/3230985460 >> 172.21.15.3:0/3230985460 conn(0x7f463406bb20 msgr2=0x7f463406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 -- 172.21.15.3:0/3230985460 shutdown_connections 2024-09-10T07:52:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.567+0000 7f4639600640 1 -- 172.21.15.3:0/3230985460 wait complete. 2024-09-10T07:52:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.569+0000 7f4639600640 1 Processor -- start 2024-09-10T07:52:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.569+0000 7f4639600640 1 -- start start 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4634145010 0x7f46340b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f463415ca80 0x7f463413be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f46341453f0 con 0x7f4634071cd0 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4634074140 con 0x7f463415ca80 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4639600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4634145570 con 0x7f4634145010 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59746/0 (socket says 172.21.15.3:59746) 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 -- 172.21.15.3:0/2526917828 learned_addr learned my addr 172.21.15.3:0/2526917828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f462be00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4634145010 0x7f46340b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4638c00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f463415ca80 0x7f463413be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 -- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4634145010 msgr2=0x7f46340b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4634145010 0x7f46340b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 -- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f463415ca80 msgr2=0x7f463413be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f463415ca80 0x7f463413be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.570+0000 7f4633e00640 1 -- 172.21.15.3:0/2526917828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341435a0 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4633e00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f462400c780 tx=0x7f462400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624017070 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4624005ad0 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4634143880 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624007070 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.571+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4634143e40 con 0x7f4634071cd0 2024-09-10T07:52:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.572+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4634063610 con 0x7f4634071cd0 2024-09-10T07:52:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.573+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f462401a070 con 0x7f4634071cd0 2024-09-10T07:52:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.574+0000 7f4630c00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 0x7f461006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.574+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f462408a3a0 con 0x7f4634071cd0 2024-09-10T07:52:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.574+0000 7f462be00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 0x7f461006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.575+0000 7f462be00640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 0x7f461006e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f461c034000 tx=0x7f461c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.578+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4624058660 con 0x7f4634071cd0 2024-09-10T07:52:05.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.761+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46340cdbd0 con 0x7f461006be60 2024-09-10T07:52:05.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.767+0000 7f4630c00640 1 -- 172.21.15.3:0/2526917828 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250215 (secure 0 0 0) 0x7f46340cdbd0 con 0x7f461006be60 2024-09-10T07:52:05.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:05.771 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:52:05.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 msgr2=0x7f461006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 0x7f461006e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f461c034000 tx=0x7f461c02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 msgr2=0x7f463415a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f462400c780 tx=0x7f462400cc40 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 shutdown_connections 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f461006be60 0x7f461006e320 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f463415ca80 0x7f463413be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4634145010 0x7f46340b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 --2- 172.21.15.3:0/2526917828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4634071cd0 0x7f463415a790 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 >> 172.21.15.3:0/2526917828 conn(0x7f463406bb20 msgr2=0x7f4634160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 shutdown_connections 2024-09-10T07:52:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.771+0000 7f4639600640 1 -- 172.21.15.3:0/2526917828 wait complete. 2024-09-10T07:52:05.788 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T07:52:05.564870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263204,"kb_used_data":9524,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711516,"statfs":{"total":773094113280,"available":772824592384,"internally_reserved":0,"allocated":9752576,"data_stored":4855566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000389"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597406+0000","last_change":"2024-09-10T07:52:01.591976+0000","last_active":"2024-09-10T07:52:01.597406+0000","last_peered":"2024-09-10T07:52:01.597406+0000","last_clean":"2024-09-10T07:52:01.597406+0000","last_became_active":"2024-09-10T07:52:01.591474+0000","last_became_peered":"2024-09-10T07:52:01.591474+0000","last_unstale":"2024-09-10T07:52:01.597406+0000","last_undegraded":"2024-09-10T07:52:01.597406+0000","last_fullsized":"2024-09-10T07:52:01.597406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:23.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586917+0000","last_change":"2024-09-10T07:52:01.588364+0000","last_active":"2024-09-10T07:52:02.586917+0000","last_peered":"2024-09-10T07:52:02.586917+0000","last_clean":"2024-09-10T07:52:02.586917+0000","last_became_active":"2024-09-10T07:52:01.585650+0000","last_became_peered":"2024-09-10T07:52:01.585650+0000","last_unstale":"2024-09-10T07:52:02.586917+0000","last_undegraded":"2024-09-10T07:52:02.586917+0000","last_fullsized":"2024-09-10T07:52:02.586917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:00.061836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607780+0000","last_change":"2024-09-10T07:52:01.590826+0000","last_active":"2024-09-10T07:52:01.607780+0000","last_peered":"2024-09-10T07:52:01.607780+0000","last_clean":"2024-09-10T07:52:01.607780+0000","last_became_active":"2024-09-10T07:52:01.590520+0000","last_became_peered":"2024-09-10T07:52:01.590520+0000","last_unstale":"2024-09-10T07:52:01.607780+0000","last_undegraded":"2024-09-10T07:52:01.607780+0000","last_fullsized":"2024-09-10T07:52:01.607780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:09.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579141+0000","last_change":"2024-09-10T07:52:00.579141+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579141+0000","last_undegraded":"2024-09-10T07:52:00.579141+0000","last_fullsized":"2024-09-10T07:52:00.579141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597009+0000","last_change":"2024-09-10T07:52:01.588987+0000","last_active":"2024-09-10T07:52:01.597009+0000","last_peered":"2024-09-10T07:52:01.597009+0000","last_clean":"2024-09-10T07:52:01.597009+0000","last_became_active":"2024-09-10T07:52:01.588704+0000","last_became_peered":"2024-09-10T07:52:01.588704+0000","last_unstale":"2024-09-10T07:52:01.597009+0000","last_undegraded":"2024-09-10T07:52:01.597009+0000","last_fullsized":"2024-09-10T07:52:01.597009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.917124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607788+0000","last_change":"2024-09-10T07:52:01.590162+0000","last_active":"2024-09-10T07:52:01.607788+0000","last_peered":"2024-09-10T07:52:01.607788+0000","last_clean":"2024-09-10T07:52:01.607788+0000","last_became_active":"2024-09-10T07:52:01.585077+0000","last_became_peered":"2024-09-10T07:52:01.585077+0000","last_unstale":"2024-09-10T07:52:01.607788+0000","last_undegraded":"2024-09-10T07:52:01.607788+0000","last_fullsized":"2024-09-10T07:52:01.607788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586493+0000","last_change":"2024-09-10T07:52:01.592098+0000","last_active":"2024-09-10T07:52:02.586493+0000","last_peered":"2024-09-10T07:52:02.586493+0000","last_clean":"2024-09-10T07:52:02.586493+0000","last_became_active":"2024-09-10T07:52:01.591441+0000","last_became_peered":"2024-09-10T07:52:01.591441+0000","last_unstale":"2024-09-10T07:52:02.586493+0000","last_undegraded":"2024-09-10T07:52:02.586493+0000","last_fullsized":"2024-09-10T07:52:02.586493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:03.328092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604052+0000","last_change":"2024-09-10T07:52:01.597132+0000","last_active":"2024-09-10T07:52:01.604052+0000","last_peered":"2024-09-10T07:52:01.604052+0000","last_clean":"2024-09-10T07:52:01.604052+0000","last_became_active":"2024-09-10T07:52:01.596923+0000","last_became_peered":"2024-09-10T07:52:01.596923+0000","last_unstale":"2024-09-10T07:52:01.604052+0000","last_undegraded":"2024-09-10T07:52:01.604052+0000","last_fullsized":"2024-09-10T07:52:01.604052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597340+0000","last_change":"2024-09-10T07:52:01.591154+0000","last_active":"2024-09-10T07:52:01.597340+0000","last_peered":"2024-09-10T07:52:01.597340+0000","last_clean":"2024-09-10T07:52:01.597340+0000","last_became_active":"2024-09-10T07:52:01.589021+0000","last_became_peered":"2024-09-10T07:52:01.589021+0000","last_unstale":"2024-09-10T07:52:01.597340+0000","last_undegraded":"2024-09-10T07:52:01.597340+0000","last_fullsized":"2024-09-10T07:52:01.597340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:27.551937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608406+0000","last_change":"2024-09-10T07:52:01.588436+0000","last_active":"2024-09-10T07:52:01.608406+0000","last_peered":"2024-09-10T07:52:01.608406+0000","last_clean":"2024-09-10T07:52:01.608406+0000","last_became_active":"2024-09-10T07:52:01.583416+0000","last_became_peered":"2024-09-10T07:52:01.583416+0000","last_unstale":"2024-09-10T07:52:01.608406+0000","last_undegraded":"2024-09-10T07:52:01.608406+0000","last_fullsized":"2024-09-10T07:52:01.608406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572168+0000","last_change":"2024-09-10T07:52:00.572168+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572168+0000","last_undegraded":"2024-09-10T07:52:00.572168+0000","last_fullsized":"2024-09-10T07:52:00.572168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604254+0000","last_change":"2024-09-10T07:52:01.596178+0000","last_active":"2024-09-10T07:52:01.604254+0000","last_peered":"2024-09-10T07:52:01.604254+0000","last_clean":"2024-09-10T07:52:01.604254+0000","last_became_active":"2024-09-10T07:52:01.595875+0000","last_became_peered":"2024-09-10T07:52:01.595875+0000","last_unstale":"2024-09-10T07:52:01.604254+0000","last_undegraded":"2024-09-10T07:52:01.604254+0000","last_fullsized":"2024-09-10T07:52:01.604254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:29.458387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.613750+0000","last_change":"2024-09-10T07:52:01.610337+0000","last_active":"2024-09-10T07:52:01.613750+0000","last_peered":"2024-09-10T07:52:01.613750+0000","last_clean":"2024-09-10T07:52:01.613750+0000","last_became_active":"2024-09-10T07:52:01.609110+0000","last_became_peered":"2024-09-10T07:52:01.609110+0000","last_unstale":"2024-09-10T07:52:01.613750+0000","last_undegraded":"2024-09-10T07:52:01.613750+0000","last_fullsized":"2024-09-10T07:52:01.613750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:46.374455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578583+0000","last_change":"2024-09-10T07:52:00.578583+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578583+0000","last_undegraded":"2024-09-10T07:52:00.578583+0000","last_fullsized":"2024-09-10T07:52:00.578583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586567+0000","last_change":"2024-09-10T07:52:01.591883+0000","last_active":"2024-09-10T07:52:02.586567+0000","last_peered":"2024-09-10T07:52:02.586567+0000","last_clean":"2024-09-10T07:52:02.586567+0000","last_became_active":"2024-09-10T07:52:01.591017+0000","last_became_peered":"2024-09-10T07:52:01.591017+0000","last_unstale":"2024-09-10T07:52:02.586567+0000","last_undegraded":"2024-09-10T07:52:02.586567+0000","last_fullsized":"2024-09-10T07:52:02.586567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:38.130494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607854+0000","last_change":"2024-09-10T07:52:01.586075+0000","last_active":"2024-09-10T07:52:01.607854+0000","last_peered":"2024-09-10T07:52:01.607854+0000","last_clean":"2024-09-10T07:52:01.607854+0000","last_became_active":"2024-09-10T07:52:01.585724+0000","last_became_peered":"2024-09-10T07:52:01.585724+0000","last_unstale":"2024-09-10T07:52:01.607854+0000","last_undegraded":"2024-09-10T07:52:01.607854+0000","last_fullsized":"2024-09-10T07:52:01.607854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604084+0000","last_change":"2024-09-10T07:52:01.598605+0000","last_active":"2024-09-10T07:52:01.604084+0000","last_peered":"2024-09-10T07:52:01.604084+0000","last_clean":"2024-09-10T07:52:01.604084+0000","last_became_active":"2024-09-10T07:52:01.596593+0000","last_became_peered":"2024-09-10T07:52:01.596593+0000","last_unstale":"2024-09-10T07:52:01.604084+0000","last_undegraded":"2024-09-10T07:52:01.604084+0000","last_fullsized":"2024-09-10T07:52:01.604084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572822+0000","last_change":"2024-09-10T07:52:00.572822+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572822+0000","last_undegraded":"2024-09-10T07:52:00.572822+0000","last_fullsized":"2024-09-10T07:52:00.572822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577424+0000","last_change":"2024-09-10T07:52:00.577424+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577424+0000","last_undegraded":"2024-09-10T07:52:00.577424+0000","last_fullsized":"2024-09-10T07:52:00.577424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578130+0000","last_change":"2024-09-10T07:52:00.578130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578130+0000","last_undegraded":"2024-09-10T07:52:00.578130+0000","last_fullsized":"2024-09-10T07:52:00.578130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579577+0000","last_change":"2024-09-10T07:52:00.579577+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579577+0000","last_undegraded":"2024-09-10T07:52:00.579577+0000","last_fullsized":"2024-09-10T07:52:00.579577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578055+0000","last_change":"2024-09-10T07:52:00.578055+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578055+0000","last_undegraded":"2024-09-10T07:52:00.578055+0000","last_fullsized":"2024-09-10T07:52:00.578055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603961+0000","last_change":"2024-09-10T07:52:01.597923+0000","last_active":"2024-09-10T07:52:01.603961+0000","last_peered":"2024-09-10T07:52:01.603961+0000","last_clean":"2024-09-10T07:52:01.603961+0000","last_became_active":"2024-09-10T07:52:01.597283+0000","last_became_peered":"2024-09-10T07:52:01.597283+0000","last_unstale":"2024-09-10T07:52:01.603961+0000","last_undegraded":"2024-09-10T07:52:01.603961+0000","last_fullsized":"2024-09-10T07:52:01.603961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573077+0000","last_change":"2024-09-10T07:52:00.573077+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573077+0000","last_undegraded":"2024-09-10T07:52:00.573077+0000","last_fullsized":"2024-09-10T07:52:00.573077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571664+0000","last_change":"2024-09-10T07:52:00.571664+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571664+0000","last_undegraded":"2024-09-10T07:52:00.571664+0000","last_fullsized":"2024-09-10T07:52:00.571664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597303+0000","last_change":"2024-09-10T07:52:01.594422+0000","last_active":"2024-09-10T07:52:01.597303+0000","last_peered":"2024-09-10T07:52:01.597303+0000","last_clean":"2024-09-10T07:52:01.597303+0000","last_became_active":"2024-09-10T07:52:01.593648+0000","last_became_peered":"2024-09-10T07:52:01.593648+0000","last_unstale":"2024-09-10T07:52:01.597303+0000","last_undegraded":"2024-09-10T07:52:01.597303+0000","last_fullsized":"2024-09-10T07:52:01.597303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:56.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.592573+0000","last_change":"2024-09-10T07:52:01.588890+0000","last_active":"2024-09-10T07:52:01.592573+0000","last_peered":"2024-09-10T07:52:01.592573+0000","last_clean":"2024-09-10T07:52:01.592573+0000","last_became_active":"2024-09-10T07:52:01.588642+0000","last_became_peered":"2024-09-10T07:52:01.588642+0000","last_unstale":"2024-09-10T07:52:01.592573+0000","last_undegraded":"2024-09-10T07:52:01.592573+0000","last_fullsized":"2024-09-10T07:52:01.592573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:55.090432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604635+0000","last_change":"2024-09-10T07:52:01.598808+0000","last_active":"2024-09-10T07:52:01.604635+0000","last_peered":"2024-09-10T07:52:01.604635+0000","last_clean":"2024-09-10T07:52:01.604635+0000","last_became_active":"2024-09-10T07:52:01.598556+0000","last_became_peered":"2024-09-10T07:52:01.598556+0000","last_unstale":"2024-09-10T07:52:01.604635+0000","last_undegraded":"2024-09-10T07:52:01.604635+0000","last_fullsized":"2024-09-10T07:52:01.604635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:53.560109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579083+0000","last_change":"2024-09-10T07:52:00.579083+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579083+0000","last_undegraded":"2024-09-10T07:52:00.579083+0000","last_fullsized":"2024-09-10T07:52:00.579083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577512+0000","last_change":"2024-09-10T07:52:00.577512+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577512+0000","last_undegraded":"2024-09-10T07:52:00.577512+0000","last_fullsized":"2024-09-10T07:52:00.577512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607922+0000","last_change":"2024-09-10T07:52:01.590994+0000","last_active":"2024-09-10T07:52:01.607922+0000","last_peered":"2024-09-10T07:52:01.607922+0000","last_clean":"2024-09-10T07:52:01.607922+0000","last_became_active":"2024-09-10T07:52:01.588304+0000","last_became_peered":"2024-09-10T07:52:01.588304+0000","last_unstale":"2024-09-10T07:52:01.607922+0000","last_undegraded":"2024-09-10T07:52:01.607922+0000","last_fullsized":"2024-09-10T07:52:01.607922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:48.624562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609278+0000","last_change":"2024-09-10T07:52:01.598304+0000","last_active":"2024-09-10T07:52:01.609278+0000","last_peered":"2024-09-10T07:52:01.609278+0000","last_clean":"2024-09-10T07:52:01.609278+0000","last_became_active":"2024-09-10T07:52:01.597760+0000","last_became_peered":"2024-09-10T07:52:01.597760+0000","last_unstale":"2024-09-10T07:52:01.609278+0000","last_undegraded":"2024-09-10T07:52:01.609278+0000","last_fullsized":"2024-09-10T07:52:01.609278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:36.143408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606093+0000","last_change":"2024-09-10T07:52:01.602531+0000","last_active":"2024-09-10T07:52:01.606093+0000","last_peered":"2024-09-10T07:52:01.606093+0000","last_clean":"2024-09-10T07:52:01.606093+0000","last_became_active":"2024-09-10T07:52:01.599463+0000","last_became_peered":"2024-09-10T07:52:01.599463+0000","last_unstale":"2024-09-10T07:52:01.606093+0000","last_undegraded":"2024-09-10T07:52:01.606093+0000","last_fullsized":"2024-09-10T07:52:01.606093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617284+0000","last_change":"2024-09-10T07:52:00.595443+0000","last_active":"2024-09-10T07:52:01.617284+0000","last_peered":"2024-09-10T07:52:01.617284+0000","last_clean":"2024-09-10T07:52:01.617284+0000","last_became_active":"2024-09-10T07:52:00.595138+0000","last_became_peered":"2024-09-10T07:52:00.595138+0000","last_unstale":"2024-09-10T07:52:01.617284+0000","last_undegraded":"2024-09-10T07:52:01.617284+0000","last_fullsized":"2024-09-10T07:52:01.617284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:11.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597386+0000","last_change":"2024-09-10T07:52:01.591760+0000","last_active":"2024-09-10T07:52:01.597386+0000","last_peered":"2024-09-10T07:52:01.597386+0000","last_clean":"2024-09-10T07:52:01.597386+0000","last_became_active":"2024-09-10T07:52:01.591343+0000","last_became_peered":"2024-09-10T07:52:01.591343+0000","last_unstale":"2024-09-10T07:52:01.597386+0000","last_undegraded":"2024-09-10T07:52:01.597386+0000","last_fullsized":"2024-09-10T07:52:01.597386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:52.869545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.612630+0000","last_change":"2024-09-10T07:52:00.586319+0000","last_active":"2024-09-10T07:52:01.612630+0000","last_peered":"2024-09-10T07:52:01.612630+0000","last_clean":"2024-09-10T07:52:01.612630+0000","last_became_active":"2024-09-10T07:52:00.586031+0000","last_became_peered":"2024-09-10T07:52:00.586031+0000","last_unstale":"2024-09-10T07:52:01.612630+0000","last_undegraded":"2024-09-10T07:52:01.612630+0000","last_fullsized":"2024-09-10T07:52:01.612630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.455287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604296+0000","last_change":"2024-09-10T07:52:01.596781+0000","last_active":"2024-09-10T07:52:01.604296+0000","last_peered":"2024-09-10T07:52:01.604296+0000","last_clean":"2024-09-10T07:52:01.604296+0000","last_became_active":"2024-09-10T07:52:01.596506+0000","last_became_peered":"2024-09-10T07:52:01.596506+0000","last_unstale":"2024-09-10T07:52:01.604296+0000","last_undegraded":"2024-09-10T07:52:01.604296+0000","last_fullsized":"2024-09-10T07:52:01.604296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:01.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586616+0000","last_change":"2024-09-10T07:52:00.583729+0000","last_active":"2024-09-10T07:52:00.586616+0000","last_peered":"2024-09-10T07:52:00.586616+0000","last_clean":"2024-09-10T07:52:00.586616+0000","last_became_active":"2024-09-10T07:52:00.582640+0000","last_became_peered":"2024-09-10T07:52:00.582640+0000","last_unstale":"2024-09-10T07:52:00.586616+0000","last_undegraded":"2024-09-10T07:52:00.586616+0000","last_fullsized":"2024-09-10T07:52:00.586616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597622+0000","last_change":"2024-09-10T07:52:01.590956+0000","last_active":"2024-09-10T07:52:01.597622+0000","last_peered":"2024-09-10T07:52:01.597622+0000","last_clean":"2024-09-10T07:52:01.597622+0000","last_became_active":"2024-09-10T07:52:01.590681+0000","last_became_peered":"2024-09-10T07:52:01.590681+0000","last_unstale":"2024-09-10T07:52:01.597622+0000","last_undegraded":"2024-09-10T07:52:01.597622+0000","last_fullsized":"2024-09-10T07:52:01.597622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:08.691480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.598631+0000","last_change":"2024-09-10T07:52:00.597568+0000","last_active":"2024-09-10T07:52:00.598631+0000","last_peered":"2024-09-10T07:52:00.598631+0000","last_clean":"2024-09-10T07:52:00.598631+0000","last_became_active":"2024-09-10T07:52:00.597236+0000","last_became_peered":"2024-09-10T07:52:00.597236+0000","last_unstale":"2024-09-10T07:52:00.598631+0000","last_undegraded":"2024-09-10T07:52:00.598631+0000","last_fullsized":"2024-09-10T07:52:00.598631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:59.599432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.587100+0000","last_change":"2024-09-10T07:52:00.587100+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.587100+0000","last_undegraded":"2024-09-10T07:52:00.587100+0000","last_fullsized":"2024-09-10T07:52:00.587100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.611712+0000","last_change":"2024-09-10T07:52:00.590175+0000","last_active":"2024-09-10T07:52:01.611712+0000","last_peered":"2024-09-10T07:52:01.611712+0000","last_clean":"2024-09-10T07:52:01.611712+0000","last_became_active":"2024-09-10T07:52:00.589851+0000","last_became_peered":"2024-09-10T07:52:00.589851+0000","last_unstale":"2024-09-10T07:52:01.611712+0000","last_undegraded":"2024-09-10T07:52:01.611712+0000","last_fullsized":"2024-09-10T07:52:01.611712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:05.625972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606112+0000","last_change":"2024-09-10T07:52:01.587513+0000","last_active":"2024-09-10T07:52:01.606112+0000","last_peered":"2024-09-10T07:52:01.606112+0000","last_clean":"2024-09-10T07:52:01.606112+0000","last_became_active":"2024-09-10T07:52:01.585495+0000","last_became_peered":"2024-09-10T07:52:01.585495+0000","last_unstale":"2024-09-10T07:52:01.606112+0000","last_undegraded":"2024-09-10T07:52:01.606112+0000","last_fullsized":"2024-09-10T07:52:01.606112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609541+0000","last_change":"2024-09-10T07:52:00.593997+0000","last_active":"2024-09-10T07:52:01.609541+0000","last_peered":"2024-09-10T07:52:01.609541+0000","last_clean":"2024-09-10T07:52:01.609541+0000","last_became_active":"2024-09-10T07:52:00.593699+0000","last_became_peered":"2024-09-10T07:52:00.593699+0000","last_unstale":"2024-09-10T07:52:01.609541+0000","last_undegraded":"2024-09-10T07:52:01.609541+0000","last_fullsized":"2024-09-10T07:52:01.609541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.601520+0000","last_change":"2024-09-10T07:52:01.597393+0000","last_active":"2024-09-10T07:52:01.601520+0000","last_peered":"2024-09-10T07:52:01.601520+0000","last_clean":"2024-09-10T07:52:01.601520+0000","last_became_active":"2024-09-10T07:52:01.596883+0000","last_became_peered":"2024-09-10T07:52:01.596883+0000","last_unstale":"2024-09-10T07:52:01.601520+0000","last_undegraded":"2024-09-10T07:52:01.601520+0000","last_fullsized":"2024-09-10T07:52:01.601520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:25.530868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594622+0000","last_change":"2024-09-10T07:52:00.592555+0000","last_active":"2024-09-10T07:52:00.594622+0000","last_peered":"2024-09-10T07:52:00.594622+0000","last_clean":"2024-09-10T07:52:00.594622+0000","last_became_active":"2024-09-10T07:52:00.592274+0000","last_became_peered":"2024-09-10T07:52:00.592274+0000","last_unstale":"2024-09-10T07:52:00.594622+0000","last_undegraded":"2024-09-10T07:52:00.594622+0000","last_fullsized":"2024-09-10T07:52:00.594622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:20.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577375+0000","last_change":"2024-09-10T07:52:00.577375+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577375+0000","last_undegraded":"2024-09-10T07:52:00.577375+0000","last_fullsized":"2024-09-10T07:52:00.577375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594746+0000","last_change":"2024-09-10T07:52:00.592915+0000","last_active":"2024-09-10T07:52:00.594746+0000","last_peered":"2024-09-10T07:52:00.594746+0000","last_clean":"2024-09-10T07:52:00.594746+0000","last_became_active":"2024-09-10T07:52:00.592202+0000","last_became_peered":"2024-09-10T07:52:00.592202+0000","last_unstale":"2024-09-10T07:52:00.594746+0000","last_undegraded":"2024-09-10T07:52:00.594746+0000","last_fullsized":"2024-09-10T07:52:00.594746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:52.317515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579928+0000","last_change":"2024-09-10T07:52:00.579928+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579928+0000","last_undegraded":"2024-09-10T07:52:00.579928+0000","last_fullsized":"2024-09-10T07:52:00.579928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582318+0000","last_change":"2024-09-10T07:52:00.589330+0000","last_active":"2024-09-10T07:52:01.582318+0000","last_peered":"2024-09-10T07:52:01.582318+0000","last_clean":"2024-09-10T07:52:01.582318+0000","last_became_active":"2024-09-10T07:52:00.589030+0000","last_became_peered":"2024-09-10T07:52:00.589030+0000","last_unstale":"2024-09-10T07:52:01.582318+0000","last_undegraded":"2024-09-10T07:52:01.582318+0000","last_fullsized":"2024-09-10T07:52:01.582318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597425+0000","last_change":"2024-09-10T07:52:01.591984+0000","last_active":"2024-09-10T07:52:01.597425+0000","last_peered":"2024-09-10T07:52:01.597425+0000","last_clean":"2024-09-10T07:52:01.597425+0000","last_became_active":"2024-09-10T07:52:01.591705+0000","last_became_peered":"2024-09-10T07:52:01.591705+0000","last_unstale":"2024-09-10T07:52:01.597425+0000","last_undegraded":"2024-09-10T07:52:01.597425+0000","last_fullsized":"2024-09-10T07:52:01.597425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:34.732740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573495+0000","last_change":"2024-09-10T07:52:00.593909+0000","last_active":"2024-09-10T07:52:01.573495+0000","last_peered":"2024-09-10T07:52:01.573495+0000","last_clean":"2024-09-10T07:52:01.573495+0000","last_became_active":"2024-09-10T07:52:00.593481+0000","last_became_peered":"2024-09-10T07:52:00.593481+0000","last_unstale":"2024-09-10T07:52:01.573495+0000","last_undegraded":"2024-09-10T07:52:01.573495+0000","last_fullsized":"2024-09-10T07:52:01.573495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:00.917860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573300+0000","last_change":"2024-09-10T07:52:00.573300+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573300+0000","last_undegraded":"2024-09-10T07:52:00.573300+0000","last_fullsized":"2024-09-10T07:52:00.573300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.615265+0000","last_change":"2024-09-10T07:52:00.581436+0000","last_active":"2024-09-10T07:52:01.615265+0000","last_peered":"2024-09-10T07:52:01.615265+0000","last_clean":"2024-09-10T07:52:01.615265+0000","last_became_active":"2024-09-10T07:52:00.581184+0000","last_became_peered":"2024-09-10T07:52:00.581184+0000","last_unstale":"2024-09-10T07:52:01.615265+0000","last_undegraded":"2024-09-10T07:52:01.615265+0000","last_fullsized":"2024-09-10T07:52:01.615265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608252+0000","last_change":"2024-09-10T07:52:01.598803+0000","last_active":"2024-09-10T07:52:01.608252+0000","last_peered":"2024-09-10T07:52:01.608252+0000","last_clean":"2024-09-10T07:52:01.608252+0000","last_became_active":"2024-09-10T07:52:01.598546+0000","last_became_peered":"2024-09-10T07:52:01.598546+0000","last_unstale":"2024-09-10T07:52:01.608252+0000","last_undegraded":"2024-09-10T07:52:01.608252+0000","last_fullsized":"2024-09-10T07:52:01.608252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.868625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617578+0000","last_change":"2024-09-10T07:52:00.593008+0000","last_active":"2024-09-10T07:52:01.617578+0000","last_peered":"2024-09-10T07:52:01.617578+0000","last_clean":"2024-09-10T07:52:01.617578+0000","last_became_active":"2024-09-10T07:52:00.592734+0000","last_became_peered":"2024-09-10T07:52:00.592734+0000","last_unstale":"2024-09-10T07:52:01.617578+0000","last_undegraded":"2024-09-10T07:52:01.617578+0000","last_fullsized":"2024-09-10T07:52:01.617578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572402+0000","last_change":"2024-09-10T07:52:00.572402+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572402+0000","last_undegraded":"2024-09-10T07:52:00.572402+0000","last_fullsized":"2024-09-10T07:52:00.572402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594717+0000","last_change":"2024-09-10T07:52:00.591803+0000","last_active":"2024-09-10T07:52:00.594717+0000","last_peered":"2024-09-10T07:52:00.594717+0000","last_clean":"2024-09-10T07:52:00.594717+0000","last_became_active":"2024-09-10T07:52:00.591495+0000","last_became_peered":"2024-09-10T07:52:00.591495+0000","last_unstale":"2024-09-10T07:52:00.594717+0000","last_undegraded":"2024-09-10T07:52:00.594717+0000","last_fullsized":"2024-09-10T07:52:00.594717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:07.100330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602204+0000","last_change":"2024-09-10T07:52:01.597959+0000","last_active":"2024-09-10T07:52:01.602204+0000","last_peered":"2024-09-10T07:52:01.602204+0000","last_clean":"2024-09-10T07:52:01.602204+0000","last_became_active":"2024-09-10T07:52:01.597689+0000","last_became_peered":"2024-09-10T07:52:01.597689+0000","last_unstale":"2024-09-10T07:52:01.602204+0000","last_undegraded":"2024-09-10T07:52:01.602204+0000","last_fullsized":"2024-09-10T07:52:01.602204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:21.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586638+0000","last_change":"2024-09-10T07:52:00.596444+0000","last_active":"2024-09-10T07:52:02.586638+0000","last_peered":"2024-09-10T07:52:02.586638+0000","last_clean":"2024-09-10T07:52:02.586638+0000","last_became_active":"2024-09-10T07:52:00.596005+0000","last_became_peered":"2024-09-10T07:52:00.596005+0000","last_unstale":"2024-09-10T07:52:02.586638+0000","last_undegraded":"2024-09-10T07:52:02.586638+0000","last_fullsized":"2024-09-10T07:52:02.586638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:45.516907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579130+0000","last_change":"2024-09-10T07:52:00.579130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579130+0000","last_undegraded":"2024-09-10T07:52:00.579130+0000","last_fullsized":"2024-09-10T07:52:00.579130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572551+0000","last_change":"2024-09-10T07:52:00.596629+0000","last_active":"2024-09-10T07:52:01.572551+0000","last_peered":"2024-09-10T07:52:01.572551+0000","last_clean":"2024-09-10T07:52:01.572551+0000","last_became_active":"2024-09-10T07:52:00.595725+0000","last_became_peered":"2024-09-10T07:52:00.595725+0000","last_unstale":"2024-09-10T07:52:01.572551+0000","last_undegraded":"2024-09-10T07:52:01.572551+0000","last_fullsized":"2024-09-10T07:52:01.572551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571585+0000","last_change":"2024-09-10T07:52:00.571585+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571585+0000","last_undegraded":"2024-09-10T07:52:00.571585+0000","last_fullsized":"2024-09-10T07:52:00.571585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597120+0000","last_change":"2024-09-10T07:52:00.595747+0000","last_active":"2024-09-10T07:52:00.597120+0000","last_peered":"2024-09-10T07:52:00.597120+0000","last_clean":"2024-09-10T07:52:00.597120+0000","last_became_active":"2024-09-10T07:52:00.595362+0000","last_became_peered":"2024-09-10T07:52:00.595362+0000","last_unstale":"2024-09-10T07:52:00.597120+0000","last_undegraded":"2024-09-10T07:52:00.597120+0000","last_fullsized":"2024-09-10T07:52:00.597120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604031+0000","last_change":"2024-09-10T07:52:01.595164+0000","last_active":"2024-09-10T07:52:01.604031+0000","last_peered":"2024-09-10T07:52:01.604031+0000","last_clean":"2024-09-10T07:52:01.604031+0000","last_became_active":"2024-09-10T07:52:01.594830+0000","last_became_peered":"2024-09-10T07:52:01.594830+0000","last_unstale":"2024-09-10T07:52:01.604031+0000","last_undegraded":"2024-09-10T07:52:01.604031+0000","last_fullsized":"2024-09-10T07:52:01.604031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:42.789252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617396+0000","last_change":"2024-09-10T07:52:00.596061+0000","last_active":"2024-09-10T07:52:01.617396+0000","last_peered":"2024-09-10T07:52:01.617396+0000","last_clean":"2024-09-10T07:52:01.617396+0000","last_became_active":"2024-09-10T07:52:00.595743+0000","last_became_peered":"2024-09-10T07:52:00.595743+0000","last_unstale":"2024-09-10T07:52:01.617396+0000","last_undegraded":"2024-09-10T07:52:01.617396+0000","last_fullsized":"2024-09-10T07:52:01.617396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:29.669564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608352+0000","last_change":"2024-09-10T07:52:01.594958+0000","last_active":"2024-09-10T07:52:01.608352+0000","last_peered":"2024-09-10T07:52:01.608352+0000","last_clean":"2024-09-10T07:52:01.608352+0000","last_became_active":"2024-09-10T07:52:01.594693+0000","last_became_peered":"2024-09-10T07:52:01.594693+0000","last_unstale":"2024-09-10T07:52:01.608352+0000","last_undegraded":"2024-09-10T07:52:01.608352+0000","last_fullsized":"2024-09-10T07:52:01.608352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:22.172497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.599721+0000","last_change":"2024-09-10T07:52:00.598950+0000","last_active":"2024-09-10T07:52:00.599721+0000","last_peered":"2024-09-10T07:52:00.599721+0000","last_clean":"2024-09-10T07:52:00.599721+0000","last_became_active":"2024-09-10T07:52:00.598731+0000","last_became_peered":"2024-09-10T07:52:00.598731+0000","last_unstale":"2024-09-10T07:52:00.599721+0000","last_undegraded":"2024-09-10T07:52:00.599721+0000","last_fullsized":"2024-09-10T07:52:00.599721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600611+0000","last_change":"2024-09-10T07:52:01.592487+0000","last_active":"2024-09-10T07:52:01.600611+0000","last_peered":"2024-09-10T07:52:01.600611+0000","last_clean":"2024-09-10T07:52:01.600611+0000","last_became_active":"2024-09-10T07:52:01.590383+0000","last_became_peered":"2024-09-10T07:52:01.590383+0000","last_unstale":"2024-09-10T07:52:01.600611+0000","last_undegraded":"2024-09-10T07:52:01.600611+0000","last_fullsized":"2024-09-10T07:52:01.600611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597893+0000","last_change":"2024-09-10T07:52:00.595658+0000","last_active":"2024-09-10T07:52:00.597893+0000","last_peered":"2024-09-10T07:52:00.597893+0000","last_clean":"2024-09-10T07:52:00.597893+0000","last_became_active":"2024-09-10T07:52:00.595374+0000","last_became_peered":"2024-09-10T07:52:00.595374+0000","last_unstale":"2024-09-10T07:52:00.597893+0000","last_undegraded":"2024-09-10T07:52:00.597893+0000","last_fullsized":"2024-09-10T07:52:00.597893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586631+0000","last_change":"2024-09-10T07:52:01.588191+0000","last_active":"2024-09-10T07:52:02.586631+0000","last_peered":"2024-09-10T07:52:02.586631+0000","last_clean":"2024-09-10T07:52:02.586631+0000","last_became_active":"2024-09-10T07:52:01.587568+0000","last_became_peered":"2024-09-10T07:52:01.587568+0000","last_unstale":"2024-09-10T07:52:02.586631+0000","last_undegraded":"2024-09-10T07:52:02.586631+0000","last_fullsized":"2024-09-10T07:52:02.586631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:28.983805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.590896+0000","last_change":"2024-09-10T07:52:00.586945+0000","last_active":"2024-09-10T07:52:00.590896+0000","last_peered":"2024-09-10T07:52:00.590896+0000","last_clean":"2024-09-10T07:52:00.590896+0000","last_became_active":"2024-09-10T07:52:00.586368+0000","last_became_peered":"2024-09-10T07:52:00.586368+0000","last_unstale":"2024-09-10T07:52:00.590896+0000","last_undegraded":"2024-09-10T07:52:00.590896+0000","last_fullsized":"2024-09-10T07:52:00.590896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:31.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607474+0000","last_change":"2024-09-10T07:52:01.596817+0000","last_active":"2024-09-10T07:52:01.607474+0000","last_peered":"2024-09-10T07:52:01.607474+0000","last_clean":"2024-09-10T07:52:01.607474+0000","last_became_active":"2024-09-10T07:52:01.595480+0000","last_became_peered":"2024-09-10T07:52:01.595480+0000","last_unstale":"2024-09-10T07:52:01.607474+0000","last_undegraded":"2024-09-10T07:52:01.607474+0000","last_fullsized":"2024-09-10T07:52:01.607474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:38.832201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586749+0000","last_change":"2024-09-10T07:52:00.590479+0000","last_active":"2024-09-10T07:52:02.586749+0000","last_peered":"2024-09-10T07:52:02.586749+0000","last_clean":"2024-09-10T07:52:02.586749+0000","last_became_active":"2024-09-10T07:52:00.589855+0000","last_became_peered":"2024-09-10T07:52:00.589855+0000","last_unstale":"2024-09-10T07:52:02.586749+0000","last_undegraded":"2024-09-10T07:52:02.586749+0000","last_fullsized":"2024-09-10T07:52:02.586749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:34.433999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578558+0000","last_change":"2024-09-10T07:52:00.578558+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578558+0000","last_undegraded":"2024-09-10T07:52:00.578558+0000","last_fullsized":"2024-09-10T07:52:00.578558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586935+0000","last_change":"2024-09-10T07:52:00.584260+0000","last_active":"2024-09-10T07:52:00.586935+0000","last_peered":"2024-09-10T07:52:00.586935+0000","last_clean":"2024-09-10T07:52:00.586935+0000","last_became_active":"2024-09-10T07:52:00.583375+0000","last_became_peered":"2024-09-10T07:52:00.583375+0000","last_unstale":"2024-09-10T07:52:00.586935+0000","last_undegraded":"2024-09-10T07:52:00.586935+0000","last_fullsized":"2024-09-10T07:52:00.586935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603982+0000","last_change":"2024-09-10T07:52:01.598146+0000","last_active":"2024-09-10T07:52:01.603982+0000","last_peered":"2024-09-10T07:52:01.603982+0000","last_clean":"2024-09-10T07:52:01.603982+0000","last_became_active":"2024-09-10T07:52:01.597673+0000","last_became_peered":"2024-09-10T07:52:01.597673+0000","last_unstale":"2024-09-10T07:52:01.603982+0000","last_undegraded":"2024-09-10T07:52:01.603982+0000","last_fullsized":"2024-09-10T07:52:01.603982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.593922+0000","last_change":"2024-09-10T07:52:00.592001+0000","last_active":"2024-09-10T07:52:00.593922+0000","last_peered":"2024-09-10T07:52:00.593922+0000","last_clean":"2024-09-10T07:52:00.593922+0000","last_became_active":"2024-09-10T07:52:00.590320+0000","last_became_peered":"2024-09-10T07:52:00.590320+0000","last_unstale":"2024-09-10T07:52:00.593922+0000","last_undegraded":"2024-09-10T07:52:00.593922+0000","last_fullsized":"2024-09-10T07:52:00.593922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:37.783344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597324+0000","last_change":"2024-09-10T07:52:01.594864+0000","last_active":"2024-09-10T07:52:01.597324+0000","last_peered":"2024-09-10T07:52:01.597324+0000","last_clean":"2024-09-10T07:52:01.597324+0000","last_became_active":"2024-09-10T07:52:01.594608+0000","last_became_peered":"2024-09-10T07:52:01.594608+0000","last_unstale":"2024-09-10T07:52:01.597324+0000","last_undegraded":"2024-09-10T07:52:01.597324+0000","last_fullsized":"2024-09-10T07:52:01.597324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.077924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.595080+0000","last_change":"2024-09-10T07:52:00.594080+0000","last_active":"2024-09-10T07:52:00.595080+0000","last_peered":"2024-09-10T07:52:00.595080+0000","last_clean":"2024-09-10T07:52:00.595080+0000","last_became_active":"2024-09-10T07:52:00.593703+0000","last_became_peered":"2024-09-10T07:52:00.593703+0000","last_unstale":"2024-09-10T07:52:00.595080+0000","last_undegraded":"2024-09-10T07:52:00.595080+0000","last_fullsized":"2024-09-10T07:52:00.595080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.614133+0000","last_change":"2024-09-10T07:52:01.606931+0000","last_active":"2024-09-10T07:52:01.614133+0000","last_peered":"2024-09-10T07:52:01.614133+0000","last_clean":"2024-09-10T07:52:01.614133+0000","last_became_active":"2024-09-10T07:52:01.606658+0000","last_became_peered":"2024-09-10T07:52:01.606658+0000","last_unstale":"2024-09-10T07:52:01.614133+0000","last_undegraded":"2024-09-10T07:52:01.614133+0000","last_fullsized":"2024-09-10T07:52:01.614133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:48.106660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.582768+0000","last_change":"2024-09-10T07:52:00.580977+0000","last_active":"2024-09-10T07:52:00.582768+0000","last_peered":"2024-09-10T07:52:00.582768+0000","last_clean":"2024-09-10T07:52:00.582768+0000","last_became_active":"2024-09-10T07:52:00.580503+0000","last_became_peered":"2024-09-10T07:52:00.580503+0000","last_unstale":"2024-09-10T07:52:00.582768+0000","last_undegraded":"2024-09-10T07:52:00.582768+0000","last_fullsized":"2024-09-10T07:52:00.582768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602646+0000","last_change":"2024-09-10T07:52:01.591171+0000","last_active":"2024-09-10T07:52:01.602646+0000","last_peered":"2024-09-10T07:52:01.602646+0000","last_clean":"2024-09-10T07:52:01.602646+0000","last_became_active":"2024-09-10T07:52:01.590850+0000","last_became_peered":"2024-09-10T07:52:01.590850+0000","last_unstale":"2024-09-10T07:52:01.602646+0000","last_undegraded":"2024-09-10T07:52:01.602646+0000","last_fullsized":"2024-09-10T07:52:01.602646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:24.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582479+0000","last_change":"2024-09-10T07:52:00.588158+0000","last_active":"2024-09-10T07:52:01.582479+0000","last_peered":"2024-09-10T07:52:01.582479+0000","last_clean":"2024-09-10T07:52:01.582479+0000","last_became_active":"2024-09-10T07:52:00.587861+0000","last_became_peered":"2024-09-10T07:52:00.587861+0000","last_unstale":"2024-09-10T07:52:01.582479+0000","last_undegraded":"2024-09-10T07:52:01.582479+0000","last_fullsized":"2024-09-10T07:52:01.582479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.605798+0000","last_change":"2024-09-10T07:52:01.585615+0000","last_active":"2024-09-10T07:52:01.605798+0000","last_peered":"2024-09-10T07:52:01.605798+0000","last_clean":"2024-09-10T07:52:01.605798+0000","last_became_active":"2024-09-10T07:52:01.585372+0000","last_became_peered":"2024-09-10T07:52:01.585372+0000","last_unstale":"2024-09-10T07:52:01.605798+0000","last_undegraded":"2024-09-10T07:52:01.605798+0000","last_fullsized":"2024-09-10T07:52:01.605798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573044+0000","last_change":"2024-09-10T07:52:00.586049+0000","last_active":"2024-09-10T07:52:01.573044+0000","last_peered":"2024-09-10T07:52:01.573044+0000","last_clean":"2024-09-10T07:52:01.573044+0000","last_became_active":"2024-09-10T07:52:00.585765+0000","last_became_peered":"2024-09-10T07:52:00.585765+0000","last_unstale":"2024-09-10T07:52:01.573044+0000","last_undegraded":"2024-09-10T07:52:01.573044+0000","last_fullsized":"2024-09-10T07:52:01.573044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:41.786152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609802+0000","last_change":"2024-09-10T07:52:01.605802+0000","last_active":"2024-09-10T07:52:01.609802+0000","last_peered":"2024-09-10T07:52:01.609802+0000","last_clean":"2024-09-10T07:52:01.609802+0000","last_became_active":"2024-09-10T07:52:01.605500+0000","last_became_peered":"2024-09-10T07:52:01.605500+0000","last_unstale":"2024-09-10T07:52:01.609802+0000","last_undegraded":"2024-09-10T07:52:01.609802+0000","last_fullsized":"2024-09-10T07:52:01.609802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:51.503646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586882+0000","last_change":"2024-09-10T07:52:00.587749+0000","last_active":"2024-09-10T07:52:02.586882+0000","last_peered":"2024-09-10T07:52:02.586882+0000","last_clean":"2024-09-10T07:52:02.586882+0000","last_became_active":"2024-09-10T07:52:00.587486+0000","last_became_peered":"2024-09-10T07:52:00.587486+0000","last_unstale":"2024-09-10T07:52:02.586882+0000","last_undegraded":"2024-09-10T07:52:02.586882+0000","last_fullsized":"2024-09-10T07:52:02.586882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:47.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572823+0000","last_change":"2024-09-10T07:52:00.572823+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572823+0000","last_undegraded":"2024-09-10T07:52:00.572823+0000","last_fullsized":"2024-09-10T07:52:00.572823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572808+0000","last_change":"2024-09-10T07:52:00.593914+0000","last_active":"2024-09-10T07:52:01.572808+0000","last_peered":"2024-09-10T07:52:01.572808+0000","last_clean":"2024-09-10T07:52:01.572808+0000","last_became_active":"2024-09-10T07:52:00.591741+0000","last_became_peered":"2024-09-10T07:52:00.591741+0000","last_unstale":"2024-09-10T07:52:01.572808+0000","last_undegraded":"2024-09-10T07:52:01.572808+0000","last_fullsized":"2024-09-10T07:52:01.572808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:40.147081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.598161+0000","last_change":"2024-09-10T07:52:01.587331+0000","last_active":"2024-09-10T07:52:01.598161+0000","last_peered":"2024-09-10T07:52:01.598161+0000","last_clean":"2024-09-10T07:52:01.598161+0000","last_became_active":"2024-09-10T07:52:01.587019+0000","last_became_peered":"2024-09-10T07:52:01.587019+0000","last_unstale":"2024-09-10T07:52:01.598161+0000","last_undegraded":"2024-09-10T07:52:01.598161+0000","last_fullsized":"2024-09-10T07:52:01.598161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:33.250694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.580356+0000","last_change":"2024-09-10T07:52:00.580768+0000","last_active":"2024-09-10T07:52:01.580356+0000","last_peered":"2024-09-10T07:52:01.580356+0000","last_clean":"2024-09-10T07:52:01.580356+0000","last_became_active":"2024-09-10T07:52:00.580422+0000","last_became_peered":"2024-09-10T07:52:00.580422+0000","last_unstale":"2024-09-10T07:52:01.580356+0000","last_undegraded":"2024-09-10T07:52:01.580356+0000","last_fullsized":"2024-09-10T07:52:01.580356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600973+0000","last_change":"2024-09-10T07:52:01.593290+0000","last_active":"2024-09-10T07:52:01.600973+0000","last_peered":"2024-09-10T07:52:01.600973+0000","last_clean":"2024-09-10T07:52:01.600973+0000","last_became_active":"2024-09-10T07:52:01.593023+0000","last_became_peered":"2024-09-10T07:52:01.593023+0000","last_unstale":"2024-09-10T07:52:01.600973+0000","last_undegraded":"2024-09-10T07:52:01.600973+0000","last_fullsized":"2024-09-10T07:52:01.600973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:53.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.577180+0000","last_change":"2024-09-10T07:52:00.592995+0000","last_active":"2024-09-10T07:52:01.577180+0000","last_peered":"2024-09-10T07:52:01.577180+0000","last_clean":"2024-09-10T07:52:01.577180+0000","last_became_active":"2024-09-10T07:52:00.592712+0000","last_became_peered":"2024-09-10T07:52:00.592712+0000","last_unstale":"2024-09-10T07:52:01.577180+0000","last_undegraded":"2024-09-10T07:52:01.577180+0000","last_fullsized":"2024-09-10T07:52:01.577180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:40.800624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571582+0000","last_change":"2024-09-10T07:52:00.571582+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571582+0000","last_undegraded":"2024-09-10T07:52:00.571582+0000","last_fullsized":"2024-09-10T07:52:00.571582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.616068+0000","last_change":"2024-09-10T07:52:00.592703+0000","last_active":"2024-09-10T07:52:01.616068+0000","last_peered":"2024-09-10T07:52:01.616068+0000","last_clean":"2024-09-10T07:52:01.616068+0000","last_became_active":"2024-09-10T07:52:00.592405+0000","last_became_peered":"2024-09-10T07:52:00.592405+0000","last_unstale":"2024-09-10T07:52:01.616068+0000","last_undegraded":"2024-09-10T07:52:01.616068+0000","last_fullsized":"2024-09-10T07:52:01.616068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:50.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.579327+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:52:01.579327+0000","last_peered":"2024-09-10T07:52:01.579327+0000","last_clean":"2024-09-10T07:52:01.579327+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:52:01.579327+0000","last_undegraded":"2024-09-10T07:52:01.579327+0000","last_fullsized":"2024-09-10T07:52:01.579327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476808,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1069056,"data_stored":459078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":465853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1662976,"data_stored":1049446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476807,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":466525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1650688,"data_stored":1043171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1085440,"data_stored":465887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:52:05.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:52:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.982+0000 7f90f0400640 1 Processor -- start 2024-09-10T07:52:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.983+0000 7f90f0400640 1 -- start start 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e8161b30 0x7f90e815f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e815fdb0 0x7f90e806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 0x7f90e806c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f90e815fbb0 con 0x7f90e8161b30 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f90e80b36a0 con 0x7f90e81601f0 2024-09-10T07:52:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90f0400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f90e80b3980 con 0x7f90e815fdb0 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90eda00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e8161b30 0x7f90e815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90eda00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90e8161b30 0x7f90e815f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90eda00640 1 -- 172.21.15.3:0/389492183 learned_addr learned my addr 172.21.15.3:0/389492183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90ed000640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e815fdb0 0x7f90e806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.984+0000 7f90ee400640 1 --2- 172.21.15.3:0/389492183 >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 0x7f90e806c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90eda00640 1 -- 172.21.15.3:0/389492183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e815fdb0 msgr2=0x7f90e806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90eda00640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e815fdb0 0x7f90e806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90eda00640 1 -- 172.21.15.3:0/389492183 >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 msgr2=0x7f90e806c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90eda00640 1 --2- 172.21.15.3:0/389492183 >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 0x7f90e806c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90eda00640 1 -- 172.21.15.3:0/389492183 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e80b3c60 con 0x7f90e8161b30 2024-09-10T07:52:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.985+0000 7f90ee400640 1 --2- 172.21.15.3:0/389492183 >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 0x7f90e806c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90eda00640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e8161b30 0x7f90e815f670 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f90e806f280 tx=0x7f90dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56a6707837df9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90e7e00640 1 -- 172.21.15.3:0/389492183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc037070 con 0x7f90e8161b30 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90e7e00640 1 -- 172.21.15.3:0/389492183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90dc032440 con 0x7f90e8161b30 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90e7e00640 1 -- 172.21.15.3:0/389492183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc032810 con 0x7f90e8161b30 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 -- 172.21.15.3:0/389492183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e8161b30 msgr2=0x7f90e815f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e8161b30 0x7f90e815f670 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f90e806f280 tx=0x7f90dc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 -- 172.21.15.3:0/389492183 shutdown_connections 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 --2- 172.21.15.3:0/389492183 >> v2:172.21.15.70:3300/0 conn(0x7f90e81601f0 0x7f90e806c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e815fdb0 0x7f90e806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 --2- 172.21.15.3:0/389492183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e8161b30 0x7f90e815f670 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.986+0000 7f90f0400640 1 -- 172.21.15.3:0/389492183 >> 172.21.15.3:0/389492183 conn(0x7f90e806d000 msgr2=0x7f90e806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.987+0000 7f90f0400640 1 -- 172.21.15.3:0/389492183 shutdown_connections 2024-09-10T07:52:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.987+0000 7f90f0400640 1 -- 172.21.15.3:0/389492183 wait complete. 2024-09-10T07:52:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.988+0000 7f90f0400640 1 Processor -- start 2024-09-10T07:52:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.988+0000 7f90f0400640 1 -- start start 2024-09-10T07:52:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 0x7f90e814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e8161b30 0x7f90e80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8161f10 con 0x7f90e81601f0 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f90e80b4220 con 0x7f90e815fdb0 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90f0400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f90e80aefa0 con 0x7f90e8161b30 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.989+0000 7f90ed000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59776/0 (socket says 172.21.15.3:59776) 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 -- 172.21.15.3:0/1824689655 learned_addr learned my addr 172.21.15.3:0/1824689655 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90eda00640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 0x7f90e814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ee400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e8161b30 0x7f90e80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 -- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e8161b30 msgr2=0x7f90e80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e8161b30 0x7f90e80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 -- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 msgr2=0x7f90e814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 0x7f90e814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 -- 172.21.15.3:0/1824689655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e815aae0 con 0x7f90e81601f0 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90eda00640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 0x7f90e814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90ed000640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f90d400d800 tx=0x7f90d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.990+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4018030 con 0x7f90e81601f0 2024-09-10T07:52:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.991+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d40096a0 con 0x7f90e81601f0 2024-09-10T07:52:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.991+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4009870 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.991+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e815adc0 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.991+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e815b380 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.992+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e80b3300 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.993+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90d401b070 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.993+0000 7f90e5600640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 0x7f90c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.993+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f90d408b130 con 0x7f90e81601f0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.994+0000 7f90eda00640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 0x7f90c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:05.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.994+0000 7f90eda00640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 0x7f90c806e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90dc034000 tx=0x7f90dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:05.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:05.998+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90d4059ad0 con 0x7f90e81601f0 2024-09-10T07:52:06.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.181+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90e815a890 con 0x7f90c806be60 2024-09-10T07:52:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.187+0000 7f90e5600640 1 -- 172.21.15.3:0/1824689655 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250215 (secure 0 0 0) 0x7f90e815a890 con 0x7f90c806be60 2024-09-10T07:52:06.189 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.190+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 msgr2=0x7f90c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.190+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 0x7f90c806e320 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90dc034000 tx=0x7f90dc02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.190+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 msgr2=0x7f90e80cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.190+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f90d400d800 tx=0x7f90d400dcc0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 shutdown_connections 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f90c806be60 0x7f90c806e320 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90e8161b30 0x7f90e80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90e81601f0 0x7f90e80cf530 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 --2- 172.21.15.3:0/1824689655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90e815fdb0 0x7f90e814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 >> 172.21.15.3:0/1824689655 conn(0x7f90e806d000 msgr2=0x7f90e815ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 shutdown_connections 2024-09-10T07:52:06.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.191+0000 7f90f0400640 1 -- 172.21.15.3:0/1824689655 wait complete. 2024-09-10T07:52:06.206 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T07:52:05.564870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263204,"kb_used_data":9524,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711516,"statfs":{"total":773094113280,"available":772824592384,"internally_reserved":0,"allocated":9752576,"data_stored":4855566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000389"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597406+0000","last_change":"2024-09-10T07:52:01.591976+0000","last_active":"2024-09-10T07:52:01.597406+0000","last_peered":"2024-09-10T07:52:01.597406+0000","last_clean":"2024-09-10T07:52:01.597406+0000","last_became_active":"2024-09-10T07:52:01.591474+0000","last_became_peered":"2024-09-10T07:52:01.591474+0000","last_unstale":"2024-09-10T07:52:01.597406+0000","last_undegraded":"2024-09-10T07:52:01.597406+0000","last_fullsized":"2024-09-10T07:52:01.597406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:23.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586917+0000","last_change":"2024-09-10T07:52:01.588364+0000","last_active":"2024-09-10T07:52:02.586917+0000","last_peered":"2024-09-10T07:52:02.586917+0000","last_clean":"2024-09-10T07:52:02.586917+0000","last_became_active":"2024-09-10T07:52:01.585650+0000","last_became_peered":"2024-09-10T07:52:01.585650+0000","last_unstale":"2024-09-10T07:52:02.586917+0000","last_undegraded":"2024-09-10T07:52:02.586917+0000","last_fullsized":"2024-09-10T07:52:02.586917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:00.061836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607780+0000","last_change":"2024-09-10T07:52:01.590826+0000","last_active":"2024-09-10T07:52:01.607780+0000","last_peered":"2024-09-10T07:52:01.607780+0000","last_clean":"2024-09-10T07:52:01.607780+0000","last_became_active":"2024-09-10T07:52:01.590520+0000","last_became_peered":"2024-09-10T07:52:01.590520+0000","last_unstale":"2024-09-10T07:52:01.607780+0000","last_undegraded":"2024-09-10T07:52:01.607780+0000","last_fullsized":"2024-09-10T07:52:01.607780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:09.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579141+0000","last_change":"2024-09-10T07:52:00.579141+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579141+0000","last_undegraded":"2024-09-10T07:52:00.579141+0000","last_fullsized":"2024-09-10T07:52:00.579141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597009+0000","last_change":"2024-09-10T07:52:01.588987+0000","last_active":"2024-09-10T07:52:01.597009+0000","last_peered":"2024-09-10T07:52:01.597009+0000","last_clean":"2024-09-10T07:52:01.597009+0000","last_became_active":"2024-09-10T07:52:01.588704+0000","last_became_peered":"2024-09-10T07:52:01.588704+0000","last_unstale":"2024-09-10T07:52:01.597009+0000","last_undegraded":"2024-09-10T07:52:01.597009+0000","last_fullsized":"2024-09-10T07:52:01.597009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.917124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607788+0000","last_change":"2024-09-10T07:52:01.590162+0000","last_active":"2024-09-10T07:52:01.607788+0000","last_peered":"2024-09-10T07:52:01.607788+0000","last_clean":"2024-09-10T07:52:01.607788+0000","last_became_active":"2024-09-10T07:52:01.585077+0000","last_became_peered":"2024-09-10T07:52:01.585077+0000","last_unstale":"2024-09-10T07:52:01.607788+0000","last_undegraded":"2024-09-10T07:52:01.607788+0000","last_fullsized":"2024-09-10T07:52:01.607788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586493+0000","last_change":"2024-09-10T07:52:01.592098+0000","last_active":"2024-09-10T07:52:02.586493+0000","last_peered":"2024-09-10T07:52:02.586493+0000","last_clean":"2024-09-10T07:52:02.586493+0000","last_became_active":"2024-09-10T07:52:01.591441+0000","last_became_peered":"2024-09-10T07:52:01.591441+0000","last_unstale":"2024-09-10T07:52:02.586493+0000","last_undegraded":"2024-09-10T07:52:02.586493+0000","last_fullsized":"2024-09-10T07:52:02.586493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:03.328092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604052+0000","last_change":"2024-09-10T07:52:01.597132+0000","last_active":"2024-09-10T07:52:01.604052+0000","last_peered":"2024-09-10T07:52:01.604052+0000","last_clean":"2024-09-10T07:52:01.604052+0000","last_became_active":"2024-09-10T07:52:01.596923+0000","last_became_peered":"2024-09-10T07:52:01.596923+0000","last_unstale":"2024-09-10T07:52:01.604052+0000","last_undegraded":"2024-09-10T07:52:01.604052+0000","last_fullsized":"2024-09-10T07:52:01.604052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597340+0000","last_change":"2024-09-10T07:52:01.591154+0000","last_active":"2024-09-10T07:52:01.597340+0000","last_peered":"2024-09-10T07:52:01.597340+0000","last_clean":"2024-09-10T07:52:01.597340+0000","last_became_active":"2024-09-10T07:52:01.589021+0000","last_became_peered":"2024-09-10T07:52:01.589021+0000","last_unstale":"2024-09-10T07:52:01.597340+0000","last_undegraded":"2024-09-10T07:52:01.597340+0000","last_fullsized":"2024-09-10T07:52:01.597340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:27.551937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608406+0000","last_change":"2024-09-10T07:52:01.588436+0000","last_active":"2024-09-10T07:52:01.608406+0000","last_peered":"2024-09-10T07:52:01.608406+0000","last_clean":"2024-09-10T07:52:01.608406+0000","last_became_active":"2024-09-10T07:52:01.583416+0000","last_became_peered":"2024-09-10T07:52:01.583416+0000","last_unstale":"2024-09-10T07:52:01.608406+0000","last_undegraded":"2024-09-10T07:52:01.608406+0000","last_fullsized":"2024-09-10T07:52:01.608406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572168+0000","last_change":"2024-09-10T07:52:00.572168+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572168+0000","last_undegraded":"2024-09-10T07:52:00.572168+0000","last_fullsized":"2024-09-10T07:52:00.572168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604254+0000","last_change":"2024-09-10T07:52:01.596178+0000","last_active":"2024-09-10T07:52:01.604254+0000","last_peered":"2024-09-10T07:52:01.604254+0000","last_clean":"2024-09-10T07:52:01.604254+0000","last_became_active":"2024-09-10T07:52:01.595875+0000","last_became_peered":"2024-09-10T07:52:01.595875+0000","last_unstale":"2024-09-10T07:52:01.604254+0000","last_undegraded":"2024-09-10T07:52:01.604254+0000","last_fullsized":"2024-09-10T07:52:01.604254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:29.458387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.613750+0000","last_change":"2024-09-10T07:52:01.610337+0000","last_active":"2024-09-10T07:52:01.613750+0000","last_peered":"2024-09-10T07:52:01.613750+0000","last_clean":"2024-09-10T07:52:01.613750+0000","last_became_active":"2024-09-10T07:52:01.609110+0000","last_became_peered":"2024-09-10T07:52:01.609110+0000","last_unstale":"2024-09-10T07:52:01.613750+0000","last_undegraded":"2024-09-10T07:52:01.613750+0000","last_fullsized":"2024-09-10T07:52:01.613750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:46.374455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578583+0000","last_change":"2024-09-10T07:52:00.578583+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578583+0000","last_undegraded":"2024-09-10T07:52:00.578583+0000","last_fullsized":"2024-09-10T07:52:00.578583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586567+0000","last_change":"2024-09-10T07:52:01.591883+0000","last_active":"2024-09-10T07:52:02.586567+0000","last_peered":"2024-09-10T07:52:02.586567+0000","last_clean":"2024-09-10T07:52:02.586567+0000","last_became_active":"2024-09-10T07:52:01.591017+0000","last_became_peered":"2024-09-10T07:52:01.591017+0000","last_unstale":"2024-09-10T07:52:02.586567+0000","last_undegraded":"2024-09-10T07:52:02.586567+0000","last_fullsized":"2024-09-10T07:52:02.586567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:38.130494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607854+0000","last_change":"2024-09-10T07:52:01.586075+0000","last_active":"2024-09-10T07:52:01.607854+0000","last_peered":"2024-09-10T07:52:01.607854+0000","last_clean":"2024-09-10T07:52:01.607854+0000","last_became_active":"2024-09-10T07:52:01.585724+0000","last_became_peered":"2024-09-10T07:52:01.585724+0000","last_unstale":"2024-09-10T07:52:01.607854+0000","last_undegraded":"2024-09-10T07:52:01.607854+0000","last_fullsized":"2024-09-10T07:52:01.607854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604084+0000","last_change":"2024-09-10T07:52:01.598605+0000","last_active":"2024-09-10T07:52:01.604084+0000","last_peered":"2024-09-10T07:52:01.604084+0000","last_clean":"2024-09-10T07:52:01.604084+0000","last_became_active":"2024-09-10T07:52:01.596593+0000","last_became_peered":"2024-09-10T07:52:01.596593+0000","last_unstale":"2024-09-10T07:52:01.604084+0000","last_undegraded":"2024-09-10T07:52:01.604084+0000","last_fullsized":"2024-09-10T07:52:01.604084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572822+0000","last_change":"2024-09-10T07:52:00.572822+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572822+0000","last_undegraded":"2024-09-10T07:52:00.572822+0000","last_fullsized":"2024-09-10T07:52:00.572822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577424+0000","last_change":"2024-09-10T07:52:00.577424+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577424+0000","last_undegraded":"2024-09-10T07:52:00.577424+0000","last_fullsized":"2024-09-10T07:52:00.577424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578130+0000","last_change":"2024-09-10T07:52:00.578130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578130+0000","last_undegraded":"2024-09-10T07:52:00.578130+0000","last_fullsized":"2024-09-10T07:52:00.578130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579577+0000","last_change":"2024-09-10T07:52:00.579577+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579577+0000","last_undegraded":"2024-09-10T07:52:00.579577+0000","last_fullsized":"2024-09-10T07:52:00.579577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578055+0000","last_change":"2024-09-10T07:52:00.578055+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578055+0000","last_undegraded":"2024-09-10T07:52:00.578055+0000","last_fullsized":"2024-09-10T07:52:00.578055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603961+0000","last_change":"2024-09-10T07:52:01.597923+0000","last_active":"2024-09-10T07:52:01.603961+0000","last_peered":"2024-09-10T07:52:01.603961+0000","last_clean":"2024-09-10T07:52:01.603961+0000","last_became_active":"2024-09-10T07:52:01.597283+0000","last_became_peered":"2024-09-10T07:52:01.597283+0000","last_unstale":"2024-09-10T07:52:01.603961+0000","last_undegraded":"2024-09-10T07:52:01.603961+0000","last_fullsized":"2024-09-10T07:52:01.603961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573077+0000","last_change":"2024-09-10T07:52:00.573077+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573077+0000","last_undegraded":"2024-09-10T07:52:00.573077+0000","last_fullsized":"2024-09-10T07:52:00.573077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571664+0000","last_change":"2024-09-10T07:52:00.571664+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571664+0000","last_undegraded":"2024-09-10T07:52:00.571664+0000","last_fullsized":"2024-09-10T07:52:00.571664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597303+0000","last_change":"2024-09-10T07:52:01.594422+0000","last_active":"2024-09-10T07:52:01.597303+0000","last_peered":"2024-09-10T07:52:01.597303+0000","last_clean":"2024-09-10T07:52:01.597303+0000","last_became_active":"2024-09-10T07:52:01.593648+0000","last_became_peered":"2024-09-10T07:52:01.593648+0000","last_unstale":"2024-09-10T07:52:01.597303+0000","last_undegraded":"2024-09-10T07:52:01.597303+0000","last_fullsized":"2024-09-10T07:52:01.597303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:56.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.592573+0000","last_change":"2024-09-10T07:52:01.588890+0000","last_active":"2024-09-10T07:52:01.592573+0000","last_peered":"2024-09-10T07:52:01.592573+0000","last_clean":"2024-09-10T07:52:01.592573+0000","last_became_active":"2024-09-10T07:52:01.588642+0000","last_became_peered":"2024-09-10T07:52:01.588642+0000","last_unstale":"2024-09-10T07:52:01.592573+0000","last_undegraded":"2024-09-10T07:52:01.592573+0000","last_fullsized":"2024-09-10T07:52:01.592573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:55.090432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604635+0000","last_change":"2024-09-10T07:52:01.598808+0000","last_active":"2024-09-10T07:52:01.604635+0000","last_peered":"2024-09-10T07:52:01.604635+0000","last_clean":"2024-09-10T07:52:01.604635+0000","last_became_active":"2024-09-10T07:52:01.598556+0000","last_became_peered":"2024-09-10T07:52:01.598556+0000","last_unstale":"2024-09-10T07:52:01.604635+0000","last_undegraded":"2024-09-10T07:52:01.604635+0000","last_fullsized":"2024-09-10T07:52:01.604635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:53.560109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579083+0000","last_change":"2024-09-10T07:52:00.579083+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579083+0000","last_undegraded":"2024-09-10T07:52:00.579083+0000","last_fullsized":"2024-09-10T07:52:00.579083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577512+0000","last_change":"2024-09-10T07:52:00.577512+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577512+0000","last_undegraded":"2024-09-10T07:52:00.577512+0000","last_fullsized":"2024-09-10T07:52:00.577512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607922+0000","last_change":"2024-09-10T07:52:01.590994+0000","last_active":"2024-09-10T07:52:01.607922+0000","last_peered":"2024-09-10T07:52:01.607922+0000","last_clean":"2024-09-10T07:52:01.607922+0000","last_became_active":"2024-09-10T07:52:01.588304+0000","last_became_peered":"2024-09-10T07:52:01.588304+0000","last_unstale":"2024-09-10T07:52:01.607922+0000","last_undegraded":"2024-09-10T07:52:01.607922+0000","last_fullsized":"2024-09-10T07:52:01.607922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:48.624562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609278+0000","last_change":"2024-09-10T07:52:01.598304+0000","last_active":"2024-09-10T07:52:01.609278+0000","last_peered":"2024-09-10T07:52:01.609278+0000","last_clean":"2024-09-10T07:52:01.609278+0000","last_became_active":"2024-09-10T07:52:01.597760+0000","last_became_peered":"2024-09-10T07:52:01.597760+0000","last_unstale":"2024-09-10T07:52:01.609278+0000","last_undegraded":"2024-09-10T07:52:01.609278+0000","last_fullsized":"2024-09-10T07:52:01.609278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:36.143408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606093+0000","last_change":"2024-09-10T07:52:01.602531+0000","last_active":"2024-09-10T07:52:01.606093+0000","last_peered":"2024-09-10T07:52:01.606093+0000","last_clean":"2024-09-10T07:52:01.606093+0000","last_became_active":"2024-09-10T07:52:01.599463+0000","last_became_peered":"2024-09-10T07:52:01.599463+0000","last_unstale":"2024-09-10T07:52:01.606093+0000","last_undegraded":"2024-09-10T07:52:01.606093+0000","last_fullsized":"2024-09-10T07:52:01.606093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617284+0000","last_change":"2024-09-10T07:52:00.595443+0000","last_active":"2024-09-10T07:52:01.617284+0000","last_peered":"2024-09-10T07:52:01.617284+0000","last_clean":"2024-09-10T07:52:01.617284+0000","last_became_active":"2024-09-10T07:52:00.595138+0000","last_became_peered":"2024-09-10T07:52:00.595138+0000","last_unstale":"2024-09-10T07:52:01.617284+0000","last_undegraded":"2024-09-10T07:52:01.617284+0000","last_fullsized":"2024-09-10T07:52:01.617284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:11.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597386+0000","last_change":"2024-09-10T07:52:01.591760+0000","last_active":"2024-09-10T07:52:01.597386+0000","last_peered":"2024-09-10T07:52:01.597386+0000","last_clean":"2024-09-10T07:52:01.597386+0000","last_became_active":"2024-09-10T07:52:01.591343+0000","last_became_peered":"2024-09-10T07:52:01.591343+0000","last_unstale":"2024-09-10T07:52:01.597386+0000","last_undegraded":"2024-09-10T07:52:01.597386+0000","last_fullsized":"2024-09-10T07:52:01.597386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:52.869545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.612630+0000","last_change":"2024-09-10T07:52:00.586319+0000","last_active":"2024-09-10T07:52:01.612630+0000","last_peered":"2024-09-10T07:52:01.612630+0000","last_clean":"2024-09-10T07:52:01.612630+0000","last_became_active":"2024-09-10T07:52:00.586031+0000","last_became_peered":"2024-09-10T07:52:00.586031+0000","last_unstale":"2024-09-10T07:52:01.612630+0000","last_undegraded":"2024-09-10T07:52:01.612630+0000","last_fullsized":"2024-09-10T07:52:01.612630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.455287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604296+0000","last_change":"2024-09-10T07:52:01.596781+0000","last_active":"2024-09-10T07:52:01.604296+0000","last_peered":"2024-09-10T07:52:01.604296+0000","last_clean":"2024-09-10T07:52:01.604296+0000","last_became_active":"2024-09-10T07:52:01.596506+0000","last_became_peered":"2024-09-10T07:52:01.596506+0000","last_unstale":"2024-09-10T07:52:01.604296+0000","last_undegraded":"2024-09-10T07:52:01.604296+0000","last_fullsized":"2024-09-10T07:52:01.604296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:01.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586616+0000","last_change":"2024-09-10T07:52:00.583729+0000","last_active":"2024-09-10T07:52:00.586616+0000","last_peered":"2024-09-10T07:52:00.586616+0000","last_clean":"2024-09-10T07:52:00.586616+0000","last_became_active":"2024-09-10T07:52:00.582640+0000","last_became_peered":"2024-09-10T07:52:00.582640+0000","last_unstale":"2024-09-10T07:52:00.586616+0000","last_undegraded":"2024-09-10T07:52:00.586616+0000","last_fullsized":"2024-09-10T07:52:00.586616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597622+0000","last_change":"2024-09-10T07:52:01.590956+0000","last_active":"2024-09-10T07:52:01.597622+0000","last_peered":"2024-09-10T07:52:01.597622+0000","last_clean":"2024-09-10T07:52:01.597622+0000","last_became_active":"2024-09-10T07:52:01.590681+0000","last_became_peered":"2024-09-10T07:52:01.590681+0000","last_unstale":"2024-09-10T07:52:01.597622+0000","last_undegraded":"2024-09-10T07:52:01.597622+0000","last_fullsized":"2024-09-10T07:52:01.597622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:08.691480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.598631+0000","last_change":"2024-09-10T07:52:00.597568+0000","last_active":"2024-09-10T07:52:00.598631+0000","last_peered":"2024-09-10T07:52:00.598631+0000","last_clean":"2024-09-10T07:52:00.598631+0000","last_became_active":"2024-09-10T07:52:00.597236+0000","last_became_peered":"2024-09-10T07:52:00.597236+0000","last_unstale":"2024-09-10T07:52:00.598631+0000","last_undegraded":"2024-09-10T07:52:00.598631+0000","last_fullsized":"2024-09-10T07:52:00.598631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:59.599432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.587100+0000","last_change":"2024-09-10T07:52:00.587100+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.587100+0000","last_undegraded":"2024-09-10T07:52:00.587100+0000","last_fullsized":"2024-09-10T07:52:00.587100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.611712+0000","last_change":"2024-09-10T07:52:00.590175+0000","last_active":"2024-09-10T07:52:01.611712+0000","last_peered":"2024-09-10T07:52:01.611712+0000","last_clean":"2024-09-10T07:52:01.611712+0000","last_became_active":"2024-09-10T07:52:00.589851+0000","last_became_peered":"2024-09-10T07:52:00.589851+0000","last_unstale":"2024-09-10T07:52:01.611712+0000","last_undegraded":"2024-09-10T07:52:01.611712+0000","last_fullsized":"2024-09-10T07:52:01.611712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:05.625972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606112+0000","last_change":"2024-09-10T07:52:01.587513+0000","last_active":"2024-09-10T07:52:01.606112+0000","last_peered":"2024-09-10T07:52:01.606112+0000","last_clean":"2024-09-10T07:52:01.606112+0000","last_became_active":"2024-09-10T07:52:01.585495+0000","last_became_peered":"2024-09-10T07:52:01.585495+0000","last_unstale":"2024-09-10T07:52:01.606112+0000","last_undegraded":"2024-09-10T07:52:01.606112+0000","last_fullsized":"2024-09-10T07:52:01.606112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609541+0000","last_change":"2024-09-10T07:52:00.593997+0000","last_active":"2024-09-10T07:52:01.609541+0000","last_peered":"2024-09-10T07:52:01.609541+0000","last_clean":"2024-09-10T07:52:01.609541+0000","last_became_active":"2024-09-10T07:52:00.593699+0000","last_became_peered":"2024-09-10T07:52:00.593699+0000","last_unstale":"2024-09-10T07:52:01.609541+0000","last_undegraded":"2024-09-10T07:52:01.609541+0000","last_fullsized":"2024-09-10T07:52:01.609541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.601520+0000","last_change":"2024-09-10T07:52:01.597393+0000","last_active":"2024-09-10T07:52:01.601520+0000","last_peered":"2024-09-10T07:52:01.601520+0000","last_clean":"2024-09-10T07:52:01.601520+0000","last_became_active":"2024-09-10T07:52:01.596883+0000","last_became_peered":"2024-09-10T07:52:01.596883+0000","last_unstale":"2024-09-10T07:52:01.601520+0000","last_undegraded":"2024-09-10T07:52:01.601520+0000","last_fullsized":"2024-09-10T07:52:01.601520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:25.530868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594622+0000","last_change":"2024-09-10T07:52:00.592555+0000","last_active":"2024-09-10T07:52:00.594622+0000","last_peered":"2024-09-10T07:52:00.594622+0000","last_clean":"2024-09-10T07:52:00.594622+0000","last_became_active":"2024-09-10T07:52:00.592274+0000","last_became_peered":"2024-09-10T07:52:00.592274+0000","last_unstale":"2024-09-10T07:52:00.594622+0000","last_undegraded":"2024-09-10T07:52:00.594622+0000","last_fullsized":"2024-09-10T07:52:00.594622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:20.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577375+0000","last_change":"2024-09-10T07:52:00.577375+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577375+0000","last_undegraded":"2024-09-10T07:52:00.577375+0000","last_fullsized":"2024-09-10T07:52:00.577375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594746+0000","last_change":"2024-09-10T07:52:00.592915+0000","last_active":"2024-09-10T07:52:00.594746+0000","last_peered":"2024-09-10T07:52:00.594746+0000","last_clean":"2024-09-10T07:52:00.594746+0000","last_became_active":"2024-09-10T07:52:00.592202+0000","last_became_peered":"2024-09-10T07:52:00.592202+0000","last_unstale":"2024-09-10T07:52:00.594746+0000","last_undegraded":"2024-09-10T07:52:00.594746+0000","last_fullsized":"2024-09-10T07:52:00.594746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:52.317515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579928+0000","last_change":"2024-09-10T07:52:00.579928+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579928+0000","last_undegraded":"2024-09-10T07:52:00.579928+0000","last_fullsized":"2024-09-10T07:52:00.579928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582318+0000","last_change":"2024-09-10T07:52:00.589330+0000","last_active":"2024-09-10T07:52:01.582318+0000","last_peered":"2024-09-10T07:52:01.582318+0000","last_clean":"2024-09-10T07:52:01.582318+0000","last_became_active":"2024-09-10T07:52:00.589030+0000","last_became_peered":"2024-09-10T07:52:00.589030+0000","last_unstale":"2024-09-10T07:52:01.582318+0000","last_undegraded":"2024-09-10T07:52:01.582318+0000","last_fullsized":"2024-09-10T07:52:01.582318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597425+0000","last_change":"2024-09-10T07:52:01.591984+0000","last_active":"2024-09-10T07:52:01.597425+0000","last_peered":"2024-09-10T07:52:01.597425+0000","last_clean":"2024-09-10T07:52:01.597425+0000","last_became_active":"2024-09-10T07:52:01.591705+0000","last_became_peered":"2024-09-10T07:52:01.591705+0000","last_unstale":"2024-09-10T07:52:01.597425+0000","last_undegraded":"2024-09-10T07:52:01.597425+0000","last_fullsized":"2024-09-10T07:52:01.597425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:34.732740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573495+0000","last_change":"2024-09-10T07:52:00.593909+0000","last_active":"2024-09-10T07:52:01.573495+0000","last_peered":"2024-09-10T07:52:01.573495+0000","last_clean":"2024-09-10T07:52:01.573495+0000","last_became_active":"2024-09-10T07:52:00.593481+0000","last_became_peered":"2024-09-10T07:52:00.593481+0000","last_unstale":"2024-09-10T07:52:01.573495+0000","last_undegraded":"2024-09-10T07:52:01.573495+0000","last_fullsized":"2024-09-10T07:52:01.573495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:00.917860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573300+0000","last_change":"2024-09-10T07:52:00.573300+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573300+0000","last_undegraded":"2024-09-10T07:52:00.573300+0000","last_fullsized":"2024-09-10T07:52:00.573300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.615265+0000","last_change":"2024-09-10T07:52:00.581436+0000","last_active":"2024-09-10T07:52:01.615265+0000","last_peered":"2024-09-10T07:52:01.615265+0000","last_clean":"2024-09-10T07:52:01.615265+0000","last_became_active":"2024-09-10T07:52:00.581184+0000","last_became_peered":"2024-09-10T07:52:00.581184+0000","last_unstale":"2024-09-10T07:52:01.615265+0000","last_undegraded":"2024-09-10T07:52:01.615265+0000","last_fullsized":"2024-09-10T07:52:01.615265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608252+0000","last_change":"2024-09-10T07:52:01.598803+0000","last_active":"2024-09-10T07:52:01.608252+0000","last_peered":"2024-09-10T07:52:01.608252+0000","last_clean":"2024-09-10T07:52:01.608252+0000","last_became_active":"2024-09-10T07:52:01.598546+0000","last_became_peered":"2024-09-10T07:52:01.598546+0000","last_unstale":"2024-09-10T07:52:01.608252+0000","last_undegraded":"2024-09-10T07:52:01.608252+0000","last_fullsized":"2024-09-10T07:52:01.608252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.868625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617578+0000","last_change":"2024-09-10T07:52:00.593008+0000","last_active":"2024-09-10T07:52:01.617578+0000","last_peered":"2024-09-10T07:52:01.617578+0000","last_clean":"2024-09-10T07:52:01.617578+0000","last_became_active":"2024-09-10T07:52:00.592734+0000","last_became_peered":"2024-09-10T07:52:00.592734+0000","last_unstale":"2024-09-10T07:52:01.617578+0000","last_undegraded":"2024-09-10T07:52:01.617578+0000","last_fullsized":"2024-09-10T07:52:01.617578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572402+0000","last_change":"2024-09-10T07:52:00.572402+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572402+0000","last_undegraded":"2024-09-10T07:52:00.572402+0000","last_fullsized":"2024-09-10T07:52:00.572402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594717+0000","last_change":"2024-09-10T07:52:00.591803+0000","last_active":"2024-09-10T07:52:00.594717+0000","last_peered":"2024-09-10T07:52:00.594717+0000","last_clean":"2024-09-10T07:52:00.594717+0000","last_became_active":"2024-09-10T07:52:00.591495+0000","last_became_peered":"2024-09-10T07:52:00.591495+0000","last_unstale":"2024-09-10T07:52:00.594717+0000","last_undegraded":"2024-09-10T07:52:00.594717+0000","last_fullsized":"2024-09-10T07:52:00.594717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:07.100330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602204+0000","last_change":"2024-09-10T07:52:01.597959+0000","last_active":"2024-09-10T07:52:01.602204+0000","last_peered":"2024-09-10T07:52:01.602204+0000","last_clean":"2024-09-10T07:52:01.602204+0000","last_became_active":"2024-09-10T07:52:01.597689+0000","last_became_peered":"2024-09-10T07:52:01.597689+0000","last_unstale":"2024-09-10T07:52:01.602204+0000","last_undegraded":"2024-09-10T07:52:01.602204+0000","last_fullsized":"2024-09-10T07:52:01.602204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:21.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586638+0000","last_change":"2024-09-10T07:52:00.596444+0000","last_active":"2024-09-10T07:52:02.586638+0000","last_peered":"2024-09-10T07:52:02.586638+0000","last_clean":"2024-09-10T07:52:02.586638+0000","last_became_active":"2024-09-10T07:52:00.596005+0000","last_became_peered":"2024-09-10T07:52:00.596005+0000","last_unstale":"2024-09-10T07:52:02.586638+0000","last_undegraded":"2024-09-10T07:52:02.586638+0000","last_fullsized":"2024-09-10T07:52:02.586638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:45.516907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579130+0000","last_change":"2024-09-10T07:52:00.579130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579130+0000","last_undegraded":"2024-09-10T07:52:00.579130+0000","last_fullsized":"2024-09-10T07:52:00.579130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572551+0000","last_change":"2024-09-10T07:52:00.596629+0000","last_active":"2024-09-10T07:52:01.572551+0000","last_peered":"2024-09-10T07:52:01.572551+0000","last_clean":"2024-09-10T07:52:01.572551+0000","last_became_active":"2024-09-10T07:52:00.595725+0000","last_became_peered":"2024-09-10T07:52:00.595725+0000","last_unstale":"2024-09-10T07:52:01.572551+0000","last_undegraded":"2024-09-10T07:52:01.572551+0000","last_fullsized":"2024-09-10T07:52:01.572551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571585+0000","last_change":"2024-09-10T07:52:00.571585+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571585+0000","last_undegraded":"2024-09-10T07:52:00.571585+0000","last_fullsized":"2024-09-10T07:52:00.571585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597120+0000","last_change":"2024-09-10T07:52:00.595747+0000","last_active":"2024-09-10T07:52:00.597120+0000","last_peered":"2024-09-10T07:52:00.597120+0000","last_clean":"2024-09-10T07:52:00.597120+0000","last_became_active":"2024-09-10T07:52:00.595362+0000","last_became_peered":"2024-09-10T07:52:00.595362+0000","last_unstale":"2024-09-10T07:52:00.597120+0000","last_undegraded":"2024-09-10T07:52:00.597120+0000","last_fullsized":"2024-09-10T07:52:00.597120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604031+0000","last_change":"2024-09-10T07:52:01.595164+0000","last_active":"2024-09-10T07:52:01.604031+0000","last_peered":"2024-09-10T07:52:01.604031+0000","last_clean":"2024-09-10T07:52:01.604031+0000","last_became_active":"2024-09-10T07:52:01.594830+0000","last_became_peered":"2024-09-10T07:52:01.594830+0000","last_unstale":"2024-09-10T07:52:01.604031+0000","last_undegraded":"2024-09-10T07:52:01.604031+0000","last_fullsized":"2024-09-10T07:52:01.604031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:42.789252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617396+0000","last_change":"2024-09-10T07:52:00.596061+0000","last_active":"2024-09-10T07:52:01.617396+0000","last_peered":"2024-09-10T07:52:01.617396+0000","last_clean":"2024-09-10T07:52:01.617396+0000","last_became_active":"2024-09-10T07:52:00.595743+0000","last_became_peered":"2024-09-10T07:52:00.595743+0000","last_unstale":"2024-09-10T07:52:01.617396+0000","last_undegraded":"2024-09-10T07:52:01.617396+0000","last_fullsized":"2024-09-10T07:52:01.617396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:29.669564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608352+0000","last_change":"2024-09-10T07:52:01.594958+0000","last_active":"2024-09-10T07:52:01.608352+0000","last_peered":"2024-09-10T07:52:01.608352+0000","last_clean":"2024-09-10T07:52:01.608352+0000","last_became_active":"2024-09-10T07:52:01.594693+0000","last_became_peered":"2024-09-10T07:52:01.594693+0000","last_unstale":"2024-09-10T07:52:01.608352+0000","last_undegraded":"2024-09-10T07:52:01.608352+0000","last_fullsized":"2024-09-10T07:52:01.608352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:22.172497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.599721+0000","last_change":"2024-09-10T07:52:00.598950+0000","last_active":"2024-09-10T07:52:00.599721+0000","last_peered":"2024-09-10T07:52:00.599721+0000","last_clean":"2024-09-10T07:52:00.599721+0000","last_became_active":"2024-09-10T07:52:00.598731+0000","last_became_peered":"2024-09-10T07:52:00.598731+0000","last_unstale":"2024-09-10T07:52:00.599721+0000","last_undegraded":"2024-09-10T07:52:00.599721+0000","last_fullsized":"2024-09-10T07:52:00.599721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600611+0000","last_change":"2024-09-10T07:52:01.592487+0000","last_active":"2024-09-10T07:52:01.600611+0000","last_peered":"2024-09-10T07:52:01.600611+0000","last_clean":"2024-09-10T07:52:01.600611+0000","last_became_active":"2024-09-10T07:52:01.590383+0000","last_became_peered":"2024-09-10T07:52:01.590383+0000","last_unstale":"2024-09-10T07:52:01.600611+0000","last_undegraded":"2024-09-10T07:52:01.600611+0000","last_fullsized":"2024-09-10T07:52:01.600611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597893+0000","last_change":"2024-09-10T07:52:00.595658+0000","last_active":"2024-09-10T07:52:00.597893+0000","last_peered":"2024-09-10T07:52:00.597893+0000","last_clean":"2024-09-10T07:52:00.597893+0000","last_became_active":"2024-09-10T07:52:00.595374+0000","last_became_peered":"2024-09-10T07:52:00.595374+0000","last_unstale":"2024-09-10T07:52:00.597893+0000","last_undegraded":"2024-09-10T07:52:00.597893+0000","last_fullsized":"2024-09-10T07:52:00.597893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586631+0000","last_change":"2024-09-10T07:52:01.588191+0000","last_active":"2024-09-10T07:52:02.586631+0000","last_peered":"2024-09-10T07:52:02.586631+0000","last_clean":"2024-09-10T07:52:02.586631+0000","last_became_active":"2024-09-10T07:52:01.587568+0000","last_became_peered":"2024-09-10T07:52:01.587568+0000","last_unstale":"2024-09-10T07:52:02.586631+0000","last_undegraded":"2024-09-10T07:52:02.586631+0000","last_fullsized":"2024-09-10T07:52:02.586631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:28.983805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.590896+0000","last_change":"2024-09-10T07:52:00.586945+0000","last_active":"2024-09-10T07:52:00.590896+0000","last_peered":"2024-09-10T07:52:00.590896+0000","last_clean":"2024-09-10T07:52:00.590896+0000","last_became_active":"2024-09-10T07:52:00.586368+0000","last_became_peered":"2024-09-10T07:52:00.586368+0000","last_unstale":"2024-09-10T07:52:00.590896+0000","last_undegraded":"2024-09-10T07:52:00.590896+0000","last_fullsized":"2024-09-10T07:52:00.590896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:31.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607474+0000","last_change":"2024-09-10T07:52:01.596817+0000","last_active":"2024-09-10T07:52:01.607474+0000","last_peered":"2024-09-10T07:52:01.607474+0000","last_clean":"2024-09-10T07:52:01.607474+0000","last_became_active":"2024-09-10T07:52:01.595480+0000","last_became_peered":"2024-09-10T07:52:01.595480+0000","last_unstale":"2024-09-10T07:52:01.607474+0000","last_undegraded":"2024-09-10T07:52:01.607474+0000","last_fullsized":"2024-09-10T07:52:01.607474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:38.832201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586749+0000","last_change":"2024-09-10T07:52:00.590479+0000","last_active":"2024-09-10T07:52:02.586749+0000","last_peered":"2024-09-10T07:52:02.586749+0000","last_clean":"2024-09-10T07:52:02.586749+0000","last_became_active":"2024-09-10T07:52:00.589855+0000","last_became_peered":"2024-09-10T07:52:00.589855+0000","last_unstale":"2024-09-10T07:52:02.586749+0000","last_undegraded":"2024-09-10T07:52:02.586749+0000","last_fullsized":"2024-09-10T07:52:02.586749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:34.433999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578558+0000","last_change":"2024-09-10T07:52:00.578558+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578558+0000","last_undegraded":"2024-09-10T07:52:00.578558+0000","last_fullsized":"2024-09-10T07:52:00.578558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586935+0000","last_change":"2024-09-10T07:52:00.584260+0000","last_active":"2024-09-10T07:52:00.586935+0000","last_peered":"2024-09-10T07:52:00.586935+0000","last_clean":"2024-09-10T07:52:00.586935+0000","last_became_active":"2024-09-10T07:52:00.583375+0000","last_became_peered":"2024-09-10T07:52:00.583375+0000","last_unstale":"2024-09-10T07:52:00.586935+0000","last_undegraded":"2024-09-10T07:52:00.586935+0000","last_fullsized":"2024-09-10T07:52:00.586935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603982+0000","last_change":"2024-09-10T07:52:01.598146+0000","last_active":"2024-09-10T07:52:01.603982+0000","last_peered":"2024-09-10T07:52:01.603982+0000","last_clean":"2024-09-10T07:52:01.603982+0000","last_became_active":"2024-09-10T07:52:01.597673+0000","last_became_peered":"2024-09-10T07:52:01.597673+0000","last_unstale":"2024-09-10T07:52:01.603982+0000","last_undegraded":"2024-09-10T07:52:01.603982+0000","last_fullsized":"2024-09-10T07:52:01.603982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.593922+0000","last_change":"2024-09-10T07:52:00.592001+0000","last_active":"2024-09-10T07:52:00.593922+0000","last_peered":"2024-09-10T07:52:00.593922+0000","last_clean":"2024-09-10T07:52:00.593922+0000","last_became_active":"2024-09-10T07:52:00.590320+0000","last_became_peered":"2024-09-10T07:52:00.590320+0000","last_unstale":"2024-09-10T07:52:00.593922+0000","last_undegraded":"2024-09-10T07:52:00.593922+0000","last_fullsized":"2024-09-10T07:52:00.593922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:37.783344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597324+0000","last_change":"2024-09-10T07:52:01.594864+0000","last_active":"2024-09-10T07:52:01.597324+0000","last_peered":"2024-09-10T07:52:01.597324+0000","last_clean":"2024-09-10T07:52:01.597324+0000","last_became_active":"2024-09-10T07:52:01.594608+0000","last_became_peered":"2024-09-10T07:52:01.594608+0000","last_unstale":"2024-09-10T07:52:01.597324+0000","last_undegraded":"2024-09-10T07:52:01.597324+0000","last_fullsized":"2024-09-10T07:52:01.597324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.077924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.595080+0000","last_change":"2024-09-10T07:52:00.594080+0000","last_active":"2024-09-10T07:52:00.595080+0000","last_peered":"2024-09-10T07:52:00.595080+0000","last_clean":"2024-09-10T07:52:00.595080+0000","last_became_active":"2024-09-10T07:52:00.593703+0000","last_became_peered":"2024-09-10T07:52:00.593703+0000","last_unstale":"2024-09-10T07:52:00.595080+0000","last_undegraded":"2024-09-10T07:52:00.595080+0000","last_fullsized":"2024-09-10T07:52:00.595080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.614133+0000","last_change":"2024-09-10T07:52:01.606931+0000","last_active":"2024-09-10T07:52:01.614133+0000","last_peered":"2024-09-10T07:52:01.614133+0000","last_clean":"2024-09-10T07:52:01.614133+0000","last_became_active":"2024-09-10T07:52:01.606658+0000","last_became_peered":"2024-09-10T07:52:01.606658+0000","last_unstale":"2024-09-10T07:52:01.614133+0000","last_undegraded":"2024-09-10T07:52:01.614133+0000","last_fullsized":"2024-09-10T07:52:01.614133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:48.106660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.582768+0000","last_change":"2024-09-10T07:52:00.580977+0000","last_active":"2024-09-10T07:52:00.582768+0000","last_peered":"2024-09-10T07:52:00.582768+0000","last_clean":"2024-09-10T07:52:00.582768+0000","last_became_active":"2024-09-10T07:52:00.580503+0000","last_became_peered":"2024-09-10T07:52:00.580503+0000","last_unstale":"2024-09-10T07:52:00.582768+0000","last_undegraded":"2024-09-10T07:52:00.582768+0000","last_fullsized":"2024-09-10T07:52:00.582768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602646+0000","last_change":"2024-09-10T07:52:01.591171+0000","last_active":"2024-09-10T07:52:01.602646+0000","last_peered":"2024-09-10T07:52:01.602646+0000","last_clean":"2024-09-10T07:52:01.602646+0000","last_became_active":"2024-09-10T07:52:01.590850+0000","last_became_peered":"2024-09-10T07:52:01.590850+0000","last_unstale":"2024-09-10T07:52:01.602646+0000","last_undegraded":"2024-09-10T07:52:01.602646+0000","last_fullsized":"2024-09-10T07:52:01.602646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:24.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582479+0000","last_change":"2024-09-10T07:52:00.588158+0000","last_active":"2024-09-10T07:52:01.582479+0000","last_peered":"2024-09-10T07:52:01.582479+0000","last_clean":"2024-09-10T07:52:01.582479+0000","last_became_active":"2024-09-10T07:52:00.587861+0000","last_became_peered":"2024-09-10T07:52:00.587861+0000","last_unstale":"2024-09-10T07:52:01.582479+0000","last_undegraded":"2024-09-10T07:52:01.582479+0000","last_fullsized":"2024-09-10T07:52:01.582479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.605798+0000","last_change":"2024-09-10T07:52:01.585615+0000","last_active":"2024-09-10T07:52:01.605798+0000","last_peered":"2024-09-10T07:52:01.605798+0000","last_clean":"2024-09-10T07:52:01.605798+0000","last_became_active":"2024-09-10T07:52:01.585372+0000","last_became_peered":"2024-09-10T07:52:01.585372+0000","last_unstale":"2024-09-10T07:52:01.605798+0000","last_undegraded":"2024-09-10T07:52:01.605798+0000","last_fullsized":"2024-09-10T07:52:01.605798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573044+0000","last_change":"2024-09-10T07:52:00.586049+0000","last_active":"2024-09-10T07:52:01.573044+0000","last_peered":"2024-09-10T07:52:01.573044+0000","last_clean":"2024-09-10T07:52:01.573044+0000","last_became_active":"2024-09-10T07:52:00.585765+0000","last_became_peered":"2024-09-10T07:52:00.585765+0000","last_unstale":"2024-09-10T07:52:01.573044+0000","last_undegraded":"2024-09-10T07:52:01.573044+0000","last_fullsized":"2024-09-10T07:52:01.573044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:41.786152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609802+0000","last_change":"2024-09-10T07:52:01.605802+0000","last_active":"2024-09-10T07:52:01.609802+0000","last_peered":"2024-09-10T07:52:01.609802+0000","last_clean":"2024-09-10T07:52:01.609802+0000","last_became_active":"2024-09-10T07:52:01.605500+0000","last_became_peered":"2024-09-10T07:52:01.605500+0000","last_unstale":"2024-09-10T07:52:01.609802+0000","last_undegraded":"2024-09-10T07:52:01.609802+0000","last_fullsized":"2024-09-10T07:52:01.609802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:51.503646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586882+0000","last_change":"2024-09-10T07:52:00.587749+0000","last_active":"2024-09-10T07:52:02.586882+0000","last_peered":"2024-09-10T07:52:02.586882+0000","last_clean":"2024-09-10T07:52:02.586882+0000","last_became_active":"2024-09-10T07:52:00.587486+0000","last_became_peered":"2024-09-10T07:52:00.587486+0000","last_unstale":"2024-09-10T07:52:02.586882+0000","last_undegraded":"2024-09-10T07:52:02.586882+0000","last_fullsized":"2024-09-10T07:52:02.586882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:47.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572823+0000","last_change":"2024-09-10T07:52:00.572823+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572823+0000","last_undegraded":"2024-09-10T07:52:00.572823+0000","last_fullsized":"2024-09-10T07:52:00.572823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572808+0000","last_change":"2024-09-10T07:52:00.593914+0000","last_active":"2024-09-10T07:52:01.572808+0000","last_peered":"2024-09-10T07:52:01.572808+0000","last_clean":"2024-09-10T07:52:01.572808+0000","last_became_active":"2024-09-10T07:52:00.591741+0000","last_became_peered":"2024-09-10T07:52:00.591741+0000","last_unstale":"2024-09-10T07:52:01.572808+0000","last_undegraded":"2024-09-10T07:52:01.572808+0000","last_fullsized":"2024-09-10T07:52:01.572808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:40.147081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.598161+0000","last_change":"2024-09-10T07:52:01.587331+0000","last_active":"2024-09-10T07:52:01.598161+0000","last_peered":"2024-09-10T07:52:01.598161+0000","last_clean":"2024-09-10T07:52:01.598161+0000","last_became_active":"2024-09-10T07:52:01.587019+0000","last_became_peered":"2024-09-10T07:52:01.587019+0000","last_unstale":"2024-09-10T07:52:01.598161+0000","last_undegraded":"2024-09-10T07:52:01.598161+0000","last_fullsized":"2024-09-10T07:52:01.598161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:33.250694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.580356+0000","last_change":"2024-09-10T07:52:00.580768+0000","last_active":"2024-09-10T07:52:01.580356+0000","last_peered":"2024-09-10T07:52:01.580356+0000","last_clean":"2024-09-10T07:52:01.580356+0000","last_became_active":"2024-09-10T07:52:00.580422+0000","last_became_peered":"2024-09-10T07:52:00.580422+0000","last_unstale":"2024-09-10T07:52:01.580356+0000","last_undegraded":"2024-09-10T07:52:01.580356+0000","last_fullsized":"2024-09-10T07:52:01.580356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600973+0000","last_change":"2024-09-10T07:52:01.593290+0000","last_active":"2024-09-10T07:52:01.600973+0000","last_peered":"2024-09-10T07:52:01.600973+0000","last_clean":"2024-09-10T07:52:01.600973+0000","last_became_active":"2024-09-10T07:52:01.593023+0000","last_became_peered":"2024-09-10T07:52:01.593023+0000","last_unstale":"2024-09-10T07:52:01.600973+0000","last_undegraded":"2024-09-10T07:52:01.600973+0000","last_fullsized":"2024-09-10T07:52:01.600973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:53.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.577180+0000","last_change":"2024-09-10T07:52:00.592995+0000","last_active":"2024-09-10T07:52:01.577180+0000","last_peered":"2024-09-10T07:52:01.577180+0000","last_clean":"2024-09-10T07:52:01.577180+0000","last_became_active":"2024-09-10T07:52:00.592712+0000","last_became_peered":"2024-09-10T07:52:00.592712+0000","last_unstale":"2024-09-10T07:52:01.577180+0000","last_undegraded":"2024-09-10T07:52:01.577180+0000","last_fullsized":"2024-09-10T07:52:01.577180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:40.800624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571582+0000","last_change":"2024-09-10T07:52:00.571582+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571582+0000","last_undegraded":"2024-09-10T07:52:00.571582+0000","last_fullsized":"2024-09-10T07:52:00.571582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.616068+0000","last_change":"2024-09-10T07:52:00.592703+0000","last_active":"2024-09-10T07:52:01.616068+0000","last_peered":"2024-09-10T07:52:01.616068+0000","last_clean":"2024-09-10T07:52:01.616068+0000","last_became_active":"2024-09-10T07:52:00.592405+0000","last_became_peered":"2024-09-10T07:52:00.592405+0000","last_unstale":"2024-09-10T07:52:01.616068+0000","last_undegraded":"2024-09-10T07:52:01.616068+0000","last_fullsized":"2024-09-10T07:52:01.616068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:50.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.579327+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:52:01.579327+0000","last_peered":"2024-09-10T07:52:01.579327+0000","last_clean":"2024-09-10T07:52:01.579327+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:52:01.579327+0000","last_undegraded":"2024-09-10T07:52:01.579327+0000","last_fullsized":"2024-09-10T07:52:01.579327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476808,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1069056,"data_stored":459078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":465853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1662976,"data_stored":1049446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476807,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":466525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1650688,"data_stored":1043171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1085440,"data_stored":465887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:52:06.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.403+0000 7f35e2c00640 1 Processor -- start 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.403+0000 7f35e2c00640 1 -- start start 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35e2c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35dc161b30 0x7f35dc15f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35e2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35e2c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35dc15fbb0 con 0x7f35dc1601f0 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35e2c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f35dc0b36a0 con 0x7f35dc161b30 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35e2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35dc0b3980 con 0x7f35dc15fdb0 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35dbe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35dc161b30 0x7f35dc15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:57244/0 (socket says 172.21.15.3:57244) 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 -- 172.21.15.3:0/2810371961 learned_addr learned my addr 172.21.15.3:0/2810371961 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 -- 172.21.15.3:0/2810371961 >> v2:172.21.15.70:3300/0 conn(0x7f35dc161b30 msgr2=0x7f35dc15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 --2- 172.21.15.3:0/2810371961 >> v2:172.21.15.70:3300/0 conn(0x7f35dc161b30 0x7f35dc15f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 -- 172.21.15.3:0/2810371961 >> v1:172.21.15.70:6789/0 conn(0x7f35dc1601f0 legacy=0x7f35dc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.405+0000 7f35d3e00640 1 -- 172.21.15.3:0/2810371961 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc0b3c60 con 0x7f35dc15fdb0 2024-09-10T07:52:06.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35d3e00640 1 --2- 172.21.15.3:0/2810371961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009870 tx=0x7f35c402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=723bfb85b70dc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35db400640 1 -- 172.21.15.3:0/2810371961 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35dc15fdb0 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35db400640 1 -- 172.21.15.3:0/2810371961 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4032440 con 0x7f35dc15fdb0 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35db400640 1 -- 172.21.15.3:0/2810371961 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4032810 con 0x7f35dc15fdb0 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 -- 172.21.15.3:0/2810371961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 msgr2=0x7f35dc06bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 --2- 172.21.15.3:0/2810371961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009870 tx=0x7f35c402a270 comp rx=0 tx=0).stop 2024-09-10T07:52:06.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 -- 172.21.15.3:0/2810371961 shutdown_connections 2024-09-10T07:52:06.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 --2- 172.21.15.3:0/2810371961 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc06bb20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 --2- 172.21.15.3:0/2810371961 >> v2:172.21.15.70:3300/0 conn(0x7f35dc161b30 0x7f35dc15f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.407+0000 7f35e2c00640 1 -- 172.21.15.3:0/2810371961 >> 172.21.15.3:0/2810371961 conn(0x7f35dc06d000 msgr2=0x7f35dc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.408+0000 7f35e2c00640 1 -- 172.21.15.3:0/2810371961 shutdown_connections 2024-09-10T07:52:06.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.408+0000 7f35e2c00640 1 -- 172.21.15.3:0/2810371961 wait complete. 2024-09-10T07:52:06.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.409+0000 7f35e2c00640 1 Processor -- start 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.409+0000 7f35e2c00640 1 -- start start 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35dc161b30 0x7f35dc158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc161f10 con 0x7f35dc1601f0 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc0b4220 con 0x7f35dc161b30 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35dc0aefa0 con 0x7f35dc15fdb0 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59788/0 (socket says 172.21.15.3:59788) 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35d3e00640 1 -- 172.21.15.3:0/73980341 learned_addr learned my addr 172.21.15.3:0/73980341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35dbe00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.410+0000 7f35e0c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35dc161b30 0x7f35dc158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 -- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 msgr2=0x7f35dc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 -- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35dc161b30 msgr2=0x7f35dc158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35dc161b30 0x7f35dc158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 -- 172.21.15.3:0/73980341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc156e20 con 0x7f35dc1601f0 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35dbe00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d3e00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000c00 tx=0x7f35c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037040 con 0x7f35dc1601f0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.411+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4042070 con 0x7f35dc1601f0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.412+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc157100 con 0x7f35dc1601f0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.412+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4030880 con 0x7f35dc1601f0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.412+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc15abe0 con 0x7f35dc1601f0 2024-09-10T07:52:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.413+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35dc0d1380 con 0x7f35dc1601f0 2024-09-10T07:52:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.414+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35c4030a20 con 0x7f35dc1601f0 2024-09-10T07:52:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.414+0000 7f35d8c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 0x7f35b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.415+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f35c40ab590 con 0x7f35dc1601f0 2024-09-10T07:52:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.415+0000 7f35dbe00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 0x7f35b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.415+0000 7f35dbe00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 0x7f35b806e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0cc700 tx=0x7f35cc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.419+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35dc0d1380 con 0x7f35dc1601f0 2024-09-10T07:52:06.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.599+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35dc0b3300 con 0x7f35b806be30 2024-09-10T07:52:06.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.605+0000 7f35d8c00640 1 -- 172.21.15.3:0/73980341 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250215 (secure 0 0 0) 0x7f35dc0b3300 con 0x7f35b806be30 2024-09-10T07:52:06.608 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 msgr2=0x7f35b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 0x7f35b806e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0cc700 tx=0x7f35cc009190 comp rx=0 tx=0).stop 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 msgr2=0x7f35dc156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000c00 tx=0x7f35c40046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 shutdown_connections 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f35b806be30 0x7f35b806e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35dc161b30 0x7f35dc158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35dc1601f0 0x7f35dc156720 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 --2- 172.21.15.3:0/73980341 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35dc15fdb0 0x7f35dc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 >> 172.21.15.3:0/73980341 conn(0x7f35dc06d000 msgr2=0x7f35dc15df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 shutdown_connections 2024-09-10T07:52:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.610+0000 7f35e2c00640 1 -- 172.21.15.3:0/73980341 wait complete. 2024-09-10T07:52:06.625 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T07:52:05.564870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263204,"kb_used_data":9524,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711516,"statfs":{"total":773094113280,"available":772824592384,"internally_reserved":0,"allocated":9752576,"data_stored":4855566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000389"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597406+0000","last_change":"2024-09-10T07:52:01.591976+0000","last_active":"2024-09-10T07:52:01.597406+0000","last_peered":"2024-09-10T07:52:01.597406+0000","last_clean":"2024-09-10T07:52:01.597406+0000","last_became_active":"2024-09-10T07:52:01.591474+0000","last_became_peered":"2024-09-10T07:52:01.591474+0000","last_unstale":"2024-09-10T07:52:01.597406+0000","last_undegraded":"2024-09-10T07:52:01.597406+0000","last_fullsized":"2024-09-10T07:52:01.597406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:23.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586917+0000","last_change":"2024-09-10T07:52:01.588364+0000","last_active":"2024-09-10T07:52:02.586917+0000","last_peered":"2024-09-10T07:52:02.586917+0000","last_clean":"2024-09-10T07:52:02.586917+0000","last_became_active":"2024-09-10T07:52:01.585650+0000","last_became_peered":"2024-09-10T07:52:01.585650+0000","last_unstale":"2024-09-10T07:52:02.586917+0000","last_undegraded":"2024-09-10T07:52:02.586917+0000","last_fullsized":"2024-09-10T07:52:02.586917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:00.061836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607780+0000","last_change":"2024-09-10T07:52:01.590826+0000","last_active":"2024-09-10T07:52:01.607780+0000","last_peered":"2024-09-10T07:52:01.607780+0000","last_clean":"2024-09-10T07:52:01.607780+0000","last_became_active":"2024-09-10T07:52:01.590520+0000","last_became_peered":"2024-09-10T07:52:01.590520+0000","last_unstale":"2024-09-10T07:52:01.607780+0000","last_undegraded":"2024-09-10T07:52:01.607780+0000","last_fullsized":"2024-09-10T07:52:01.607780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:09.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579141+0000","last_change":"2024-09-10T07:52:00.579141+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579141+0000","last_undegraded":"2024-09-10T07:52:00.579141+0000","last_fullsized":"2024-09-10T07:52:00.579141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597009+0000","last_change":"2024-09-10T07:52:01.588987+0000","last_active":"2024-09-10T07:52:01.597009+0000","last_peered":"2024-09-10T07:52:01.597009+0000","last_clean":"2024-09-10T07:52:01.597009+0000","last_became_active":"2024-09-10T07:52:01.588704+0000","last_became_peered":"2024-09-10T07:52:01.588704+0000","last_unstale":"2024-09-10T07:52:01.597009+0000","last_undegraded":"2024-09-10T07:52:01.597009+0000","last_fullsized":"2024-09-10T07:52:01.597009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.917124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607788+0000","last_change":"2024-09-10T07:52:01.590162+0000","last_active":"2024-09-10T07:52:01.607788+0000","last_peered":"2024-09-10T07:52:01.607788+0000","last_clean":"2024-09-10T07:52:01.607788+0000","last_became_active":"2024-09-10T07:52:01.585077+0000","last_became_peered":"2024-09-10T07:52:01.585077+0000","last_unstale":"2024-09-10T07:52:01.607788+0000","last_undegraded":"2024-09-10T07:52:01.607788+0000","last_fullsized":"2024-09-10T07:52:01.607788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586493+0000","last_change":"2024-09-10T07:52:01.592098+0000","last_active":"2024-09-10T07:52:02.586493+0000","last_peered":"2024-09-10T07:52:02.586493+0000","last_clean":"2024-09-10T07:52:02.586493+0000","last_became_active":"2024-09-10T07:52:01.591441+0000","last_became_peered":"2024-09-10T07:52:01.591441+0000","last_unstale":"2024-09-10T07:52:02.586493+0000","last_undegraded":"2024-09-10T07:52:02.586493+0000","last_fullsized":"2024-09-10T07:52:02.586493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:03.328092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604052+0000","last_change":"2024-09-10T07:52:01.597132+0000","last_active":"2024-09-10T07:52:01.604052+0000","last_peered":"2024-09-10T07:52:01.604052+0000","last_clean":"2024-09-10T07:52:01.604052+0000","last_became_active":"2024-09-10T07:52:01.596923+0000","last_became_peered":"2024-09-10T07:52:01.596923+0000","last_unstale":"2024-09-10T07:52:01.604052+0000","last_undegraded":"2024-09-10T07:52:01.604052+0000","last_fullsized":"2024-09-10T07:52:01.604052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597340+0000","last_change":"2024-09-10T07:52:01.591154+0000","last_active":"2024-09-10T07:52:01.597340+0000","last_peered":"2024-09-10T07:52:01.597340+0000","last_clean":"2024-09-10T07:52:01.597340+0000","last_became_active":"2024-09-10T07:52:01.589021+0000","last_became_peered":"2024-09-10T07:52:01.589021+0000","last_unstale":"2024-09-10T07:52:01.597340+0000","last_undegraded":"2024-09-10T07:52:01.597340+0000","last_fullsized":"2024-09-10T07:52:01.597340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:27.551937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608406+0000","last_change":"2024-09-10T07:52:01.588436+0000","last_active":"2024-09-10T07:52:01.608406+0000","last_peered":"2024-09-10T07:52:01.608406+0000","last_clean":"2024-09-10T07:52:01.608406+0000","last_became_active":"2024-09-10T07:52:01.583416+0000","last_became_peered":"2024-09-10T07:52:01.583416+0000","last_unstale":"2024-09-10T07:52:01.608406+0000","last_undegraded":"2024-09-10T07:52:01.608406+0000","last_fullsized":"2024-09-10T07:52:01.608406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572168+0000","last_change":"2024-09-10T07:52:00.572168+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572168+0000","last_undegraded":"2024-09-10T07:52:00.572168+0000","last_fullsized":"2024-09-10T07:52:00.572168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604254+0000","last_change":"2024-09-10T07:52:01.596178+0000","last_active":"2024-09-10T07:52:01.604254+0000","last_peered":"2024-09-10T07:52:01.604254+0000","last_clean":"2024-09-10T07:52:01.604254+0000","last_became_active":"2024-09-10T07:52:01.595875+0000","last_became_peered":"2024-09-10T07:52:01.595875+0000","last_unstale":"2024-09-10T07:52:01.604254+0000","last_undegraded":"2024-09-10T07:52:01.604254+0000","last_fullsized":"2024-09-10T07:52:01.604254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:29.458387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.613750+0000","last_change":"2024-09-10T07:52:01.610337+0000","last_active":"2024-09-10T07:52:01.613750+0000","last_peered":"2024-09-10T07:52:01.613750+0000","last_clean":"2024-09-10T07:52:01.613750+0000","last_became_active":"2024-09-10T07:52:01.609110+0000","last_became_peered":"2024-09-10T07:52:01.609110+0000","last_unstale":"2024-09-10T07:52:01.613750+0000","last_undegraded":"2024-09-10T07:52:01.613750+0000","last_fullsized":"2024-09-10T07:52:01.613750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:46.374455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578583+0000","last_change":"2024-09-10T07:52:00.578583+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578583+0000","last_undegraded":"2024-09-10T07:52:00.578583+0000","last_fullsized":"2024-09-10T07:52:00.578583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586567+0000","last_change":"2024-09-10T07:52:01.591883+0000","last_active":"2024-09-10T07:52:02.586567+0000","last_peered":"2024-09-10T07:52:02.586567+0000","last_clean":"2024-09-10T07:52:02.586567+0000","last_became_active":"2024-09-10T07:52:01.591017+0000","last_became_peered":"2024-09-10T07:52:01.591017+0000","last_unstale":"2024-09-10T07:52:02.586567+0000","last_undegraded":"2024-09-10T07:52:02.586567+0000","last_fullsized":"2024-09-10T07:52:02.586567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:38.130494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607854+0000","last_change":"2024-09-10T07:52:01.586075+0000","last_active":"2024-09-10T07:52:01.607854+0000","last_peered":"2024-09-10T07:52:01.607854+0000","last_clean":"2024-09-10T07:52:01.607854+0000","last_became_active":"2024-09-10T07:52:01.585724+0000","last_became_peered":"2024-09-10T07:52:01.585724+0000","last_unstale":"2024-09-10T07:52:01.607854+0000","last_undegraded":"2024-09-10T07:52:01.607854+0000","last_fullsized":"2024-09-10T07:52:01.607854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604084+0000","last_change":"2024-09-10T07:52:01.598605+0000","last_active":"2024-09-10T07:52:01.604084+0000","last_peered":"2024-09-10T07:52:01.604084+0000","last_clean":"2024-09-10T07:52:01.604084+0000","last_became_active":"2024-09-10T07:52:01.596593+0000","last_became_peered":"2024-09-10T07:52:01.596593+0000","last_unstale":"2024-09-10T07:52:01.604084+0000","last_undegraded":"2024-09-10T07:52:01.604084+0000","last_fullsized":"2024-09-10T07:52:01.604084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572822+0000","last_change":"2024-09-10T07:52:00.572822+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572822+0000","last_undegraded":"2024-09-10T07:52:00.572822+0000","last_fullsized":"2024-09-10T07:52:00.572822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577424+0000","last_change":"2024-09-10T07:52:00.577424+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577424+0000","last_undegraded":"2024-09-10T07:52:00.577424+0000","last_fullsized":"2024-09-10T07:52:00.577424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578130+0000","last_change":"2024-09-10T07:52:00.578130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578130+0000","last_undegraded":"2024-09-10T07:52:00.578130+0000","last_fullsized":"2024-09-10T07:52:00.578130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579577+0000","last_change":"2024-09-10T07:52:00.579577+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579577+0000","last_undegraded":"2024-09-10T07:52:00.579577+0000","last_fullsized":"2024-09-10T07:52:00.579577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578055+0000","last_change":"2024-09-10T07:52:00.578055+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578055+0000","last_undegraded":"2024-09-10T07:52:00.578055+0000","last_fullsized":"2024-09-10T07:52:00.578055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603961+0000","last_change":"2024-09-10T07:52:01.597923+0000","last_active":"2024-09-10T07:52:01.603961+0000","last_peered":"2024-09-10T07:52:01.603961+0000","last_clean":"2024-09-10T07:52:01.603961+0000","last_became_active":"2024-09-10T07:52:01.597283+0000","last_became_peered":"2024-09-10T07:52:01.597283+0000","last_unstale":"2024-09-10T07:52:01.603961+0000","last_undegraded":"2024-09-10T07:52:01.603961+0000","last_fullsized":"2024-09-10T07:52:01.603961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573077+0000","last_change":"2024-09-10T07:52:00.573077+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573077+0000","last_undegraded":"2024-09-10T07:52:00.573077+0000","last_fullsized":"2024-09-10T07:52:00.573077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571664+0000","last_change":"2024-09-10T07:52:00.571664+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571664+0000","last_undegraded":"2024-09-10T07:52:00.571664+0000","last_fullsized":"2024-09-10T07:52:00.571664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597303+0000","last_change":"2024-09-10T07:52:01.594422+0000","last_active":"2024-09-10T07:52:01.597303+0000","last_peered":"2024-09-10T07:52:01.597303+0000","last_clean":"2024-09-10T07:52:01.597303+0000","last_became_active":"2024-09-10T07:52:01.593648+0000","last_became_peered":"2024-09-10T07:52:01.593648+0000","last_unstale":"2024-09-10T07:52:01.597303+0000","last_undegraded":"2024-09-10T07:52:01.597303+0000","last_fullsized":"2024-09-10T07:52:01.597303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:56.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.592573+0000","last_change":"2024-09-10T07:52:01.588890+0000","last_active":"2024-09-10T07:52:01.592573+0000","last_peered":"2024-09-10T07:52:01.592573+0000","last_clean":"2024-09-10T07:52:01.592573+0000","last_became_active":"2024-09-10T07:52:01.588642+0000","last_became_peered":"2024-09-10T07:52:01.588642+0000","last_unstale":"2024-09-10T07:52:01.592573+0000","last_undegraded":"2024-09-10T07:52:01.592573+0000","last_fullsized":"2024-09-10T07:52:01.592573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:55.090432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604635+0000","last_change":"2024-09-10T07:52:01.598808+0000","last_active":"2024-09-10T07:52:01.604635+0000","last_peered":"2024-09-10T07:52:01.604635+0000","last_clean":"2024-09-10T07:52:01.604635+0000","last_became_active":"2024-09-10T07:52:01.598556+0000","last_became_peered":"2024-09-10T07:52:01.598556+0000","last_unstale":"2024-09-10T07:52:01.604635+0000","last_undegraded":"2024-09-10T07:52:01.604635+0000","last_fullsized":"2024-09-10T07:52:01.604635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:53.560109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579083+0000","last_change":"2024-09-10T07:52:00.579083+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579083+0000","last_undegraded":"2024-09-10T07:52:00.579083+0000","last_fullsized":"2024-09-10T07:52:00.579083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577512+0000","last_change":"2024-09-10T07:52:00.577512+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577512+0000","last_undegraded":"2024-09-10T07:52:00.577512+0000","last_fullsized":"2024-09-10T07:52:00.577512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607922+0000","last_change":"2024-09-10T07:52:01.590994+0000","last_active":"2024-09-10T07:52:01.607922+0000","last_peered":"2024-09-10T07:52:01.607922+0000","last_clean":"2024-09-10T07:52:01.607922+0000","last_became_active":"2024-09-10T07:52:01.588304+0000","last_became_peered":"2024-09-10T07:52:01.588304+0000","last_unstale":"2024-09-10T07:52:01.607922+0000","last_undegraded":"2024-09-10T07:52:01.607922+0000","last_fullsized":"2024-09-10T07:52:01.607922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:48.624562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609278+0000","last_change":"2024-09-10T07:52:01.598304+0000","last_active":"2024-09-10T07:52:01.609278+0000","last_peered":"2024-09-10T07:52:01.609278+0000","last_clean":"2024-09-10T07:52:01.609278+0000","last_became_active":"2024-09-10T07:52:01.597760+0000","last_became_peered":"2024-09-10T07:52:01.597760+0000","last_unstale":"2024-09-10T07:52:01.609278+0000","last_undegraded":"2024-09-10T07:52:01.609278+0000","last_fullsized":"2024-09-10T07:52:01.609278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:36.143408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606093+0000","last_change":"2024-09-10T07:52:01.602531+0000","last_active":"2024-09-10T07:52:01.606093+0000","last_peered":"2024-09-10T07:52:01.606093+0000","last_clean":"2024-09-10T07:52:01.606093+0000","last_became_active":"2024-09-10T07:52:01.599463+0000","last_became_peered":"2024-09-10T07:52:01.599463+0000","last_unstale":"2024-09-10T07:52:01.606093+0000","last_undegraded":"2024-09-10T07:52:01.606093+0000","last_fullsized":"2024-09-10T07:52:01.606093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617284+0000","last_change":"2024-09-10T07:52:00.595443+0000","last_active":"2024-09-10T07:52:01.617284+0000","last_peered":"2024-09-10T07:52:01.617284+0000","last_clean":"2024-09-10T07:52:01.617284+0000","last_became_active":"2024-09-10T07:52:00.595138+0000","last_became_peered":"2024-09-10T07:52:00.595138+0000","last_unstale":"2024-09-10T07:52:01.617284+0000","last_undegraded":"2024-09-10T07:52:01.617284+0000","last_fullsized":"2024-09-10T07:52:01.617284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:11.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597386+0000","last_change":"2024-09-10T07:52:01.591760+0000","last_active":"2024-09-10T07:52:01.597386+0000","last_peered":"2024-09-10T07:52:01.597386+0000","last_clean":"2024-09-10T07:52:01.597386+0000","last_became_active":"2024-09-10T07:52:01.591343+0000","last_became_peered":"2024-09-10T07:52:01.591343+0000","last_unstale":"2024-09-10T07:52:01.597386+0000","last_undegraded":"2024-09-10T07:52:01.597386+0000","last_fullsized":"2024-09-10T07:52:01.597386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:52.869545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.612630+0000","last_change":"2024-09-10T07:52:00.586319+0000","last_active":"2024-09-10T07:52:01.612630+0000","last_peered":"2024-09-10T07:52:01.612630+0000","last_clean":"2024-09-10T07:52:01.612630+0000","last_became_active":"2024-09-10T07:52:00.586031+0000","last_became_peered":"2024-09-10T07:52:00.586031+0000","last_unstale":"2024-09-10T07:52:01.612630+0000","last_undegraded":"2024-09-10T07:52:01.612630+0000","last_fullsized":"2024-09-10T07:52:01.612630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.455287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604296+0000","last_change":"2024-09-10T07:52:01.596781+0000","last_active":"2024-09-10T07:52:01.604296+0000","last_peered":"2024-09-10T07:52:01.604296+0000","last_clean":"2024-09-10T07:52:01.604296+0000","last_became_active":"2024-09-10T07:52:01.596506+0000","last_became_peered":"2024-09-10T07:52:01.596506+0000","last_unstale":"2024-09-10T07:52:01.604296+0000","last_undegraded":"2024-09-10T07:52:01.604296+0000","last_fullsized":"2024-09-10T07:52:01.604296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:01.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586616+0000","last_change":"2024-09-10T07:52:00.583729+0000","last_active":"2024-09-10T07:52:00.586616+0000","last_peered":"2024-09-10T07:52:00.586616+0000","last_clean":"2024-09-10T07:52:00.586616+0000","last_became_active":"2024-09-10T07:52:00.582640+0000","last_became_peered":"2024-09-10T07:52:00.582640+0000","last_unstale":"2024-09-10T07:52:00.586616+0000","last_undegraded":"2024-09-10T07:52:00.586616+0000","last_fullsized":"2024-09-10T07:52:00.586616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597622+0000","last_change":"2024-09-10T07:52:01.590956+0000","last_active":"2024-09-10T07:52:01.597622+0000","last_peered":"2024-09-10T07:52:01.597622+0000","last_clean":"2024-09-10T07:52:01.597622+0000","last_became_active":"2024-09-10T07:52:01.590681+0000","last_became_peered":"2024-09-10T07:52:01.590681+0000","last_unstale":"2024-09-10T07:52:01.597622+0000","last_undegraded":"2024-09-10T07:52:01.597622+0000","last_fullsized":"2024-09-10T07:52:01.597622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:08.691480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.598631+0000","last_change":"2024-09-10T07:52:00.597568+0000","last_active":"2024-09-10T07:52:00.598631+0000","last_peered":"2024-09-10T07:52:00.598631+0000","last_clean":"2024-09-10T07:52:00.598631+0000","last_became_active":"2024-09-10T07:52:00.597236+0000","last_became_peered":"2024-09-10T07:52:00.597236+0000","last_unstale":"2024-09-10T07:52:00.598631+0000","last_undegraded":"2024-09-10T07:52:00.598631+0000","last_fullsized":"2024-09-10T07:52:00.598631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:59.599432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.587100+0000","last_change":"2024-09-10T07:52:00.587100+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.587100+0000","last_undegraded":"2024-09-10T07:52:00.587100+0000","last_fullsized":"2024-09-10T07:52:00.587100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.611712+0000","last_change":"2024-09-10T07:52:00.590175+0000","last_active":"2024-09-10T07:52:01.611712+0000","last_peered":"2024-09-10T07:52:01.611712+0000","last_clean":"2024-09-10T07:52:01.611712+0000","last_became_active":"2024-09-10T07:52:00.589851+0000","last_became_peered":"2024-09-10T07:52:00.589851+0000","last_unstale":"2024-09-10T07:52:01.611712+0000","last_undegraded":"2024-09-10T07:52:01.611712+0000","last_fullsized":"2024-09-10T07:52:01.611712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:05.625972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606112+0000","last_change":"2024-09-10T07:52:01.587513+0000","last_active":"2024-09-10T07:52:01.606112+0000","last_peered":"2024-09-10T07:52:01.606112+0000","last_clean":"2024-09-10T07:52:01.606112+0000","last_became_active":"2024-09-10T07:52:01.585495+0000","last_became_peered":"2024-09-10T07:52:01.585495+0000","last_unstale":"2024-09-10T07:52:01.606112+0000","last_undegraded":"2024-09-10T07:52:01.606112+0000","last_fullsized":"2024-09-10T07:52:01.606112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609541+0000","last_change":"2024-09-10T07:52:00.593997+0000","last_active":"2024-09-10T07:52:01.609541+0000","last_peered":"2024-09-10T07:52:01.609541+0000","last_clean":"2024-09-10T07:52:01.609541+0000","last_became_active":"2024-09-10T07:52:00.593699+0000","last_became_peered":"2024-09-10T07:52:00.593699+0000","last_unstale":"2024-09-10T07:52:01.609541+0000","last_undegraded":"2024-09-10T07:52:01.609541+0000","last_fullsized":"2024-09-10T07:52:01.609541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.601520+0000","last_change":"2024-09-10T07:52:01.597393+0000","last_active":"2024-09-10T07:52:01.601520+0000","last_peered":"2024-09-10T07:52:01.601520+0000","last_clean":"2024-09-10T07:52:01.601520+0000","last_became_active":"2024-09-10T07:52:01.596883+0000","last_became_peered":"2024-09-10T07:52:01.596883+0000","last_unstale":"2024-09-10T07:52:01.601520+0000","last_undegraded":"2024-09-10T07:52:01.601520+0000","last_fullsized":"2024-09-10T07:52:01.601520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:25.530868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594622+0000","last_change":"2024-09-10T07:52:00.592555+0000","last_active":"2024-09-10T07:52:00.594622+0000","last_peered":"2024-09-10T07:52:00.594622+0000","last_clean":"2024-09-10T07:52:00.594622+0000","last_became_active":"2024-09-10T07:52:00.592274+0000","last_became_peered":"2024-09-10T07:52:00.592274+0000","last_unstale":"2024-09-10T07:52:00.594622+0000","last_undegraded":"2024-09-10T07:52:00.594622+0000","last_fullsized":"2024-09-10T07:52:00.594622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:20.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577375+0000","last_change":"2024-09-10T07:52:00.577375+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577375+0000","last_undegraded":"2024-09-10T07:52:00.577375+0000","last_fullsized":"2024-09-10T07:52:00.577375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594746+0000","last_change":"2024-09-10T07:52:00.592915+0000","last_active":"2024-09-10T07:52:00.594746+0000","last_peered":"2024-09-10T07:52:00.594746+0000","last_clean":"2024-09-10T07:52:00.594746+0000","last_became_active":"2024-09-10T07:52:00.592202+0000","last_became_peered":"2024-09-10T07:52:00.592202+0000","last_unstale":"2024-09-10T07:52:00.594746+0000","last_undegraded":"2024-09-10T07:52:00.594746+0000","last_fullsized":"2024-09-10T07:52:00.594746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:52.317515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579928+0000","last_change":"2024-09-10T07:52:00.579928+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579928+0000","last_undegraded":"2024-09-10T07:52:00.579928+0000","last_fullsized":"2024-09-10T07:52:00.579928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582318+0000","last_change":"2024-09-10T07:52:00.589330+0000","last_active":"2024-09-10T07:52:01.582318+0000","last_peered":"2024-09-10T07:52:01.582318+0000","last_clean":"2024-09-10T07:52:01.582318+0000","last_became_active":"2024-09-10T07:52:00.589030+0000","last_became_peered":"2024-09-10T07:52:00.589030+0000","last_unstale":"2024-09-10T07:52:01.582318+0000","last_undegraded":"2024-09-10T07:52:01.582318+0000","last_fullsized":"2024-09-10T07:52:01.582318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597425+0000","last_change":"2024-09-10T07:52:01.591984+0000","last_active":"2024-09-10T07:52:01.597425+0000","last_peered":"2024-09-10T07:52:01.597425+0000","last_clean":"2024-09-10T07:52:01.597425+0000","last_became_active":"2024-09-10T07:52:01.591705+0000","last_became_peered":"2024-09-10T07:52:01.591705+0000","last_unstale":"2024-09-10T07:52:01.597425+0000","last_undegraded":"2024-09-10T07:52:01.597425+0000","last_fullsized":"2024-09-10T07:52:01.597425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:34.732740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573495+0000","last_change":"2024-09-10T07:52:00.593909+0000","last_active":"2024-09-10T07:52:01.573495+0000","last_peered":"2024-09-10T07:52:01.573495+0000","last_clean":"2024-09-10T07:52:01.573495+0000","last_became_active":"2024-09-10T07:52:00.593481+0000","last_became_peered":"2024-09-10T07:52:00.593481+0000","last_unstale":"2024-09-10T07:52:01.573495+0000","last_undegraded":"2024-09-10T07:52:01.573495+0000","last_fullsized":"2024-09-10T07:52:01.573495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:00.917860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573300+0000","last_change":"2024-09-10T07:52:00.573300+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573300+0000","last_undegraded":"2024-09-10T07:52:00.573300+0000","last_fullsized":"2024-09-10T07:52:00.573300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.615265+0000","last_change":"2024-09-10T07:52:00.581436+0000","last_active":"2024-09-10T07:52:01.615265+0000","last_peered":"2024-09-10T07:52:01.615265+0000","last_clean":"2024-09-10T07:52:01.615265+0000","last_became_active":"2024-09-10T07:52:00.581184+0000","last_became_peered":"2024-09-10T07:52:00.581184+0000","last_unstale":"2024-09-10T07:52:01.615265+0000","last_undegraded":"2024-09-10T07:52:01.615265+0000","last_fullsized":"2024-09-10T07:52:01.615265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608252+0000","last_change":"2024-09-10T07:52:01.598803+0000","last_active":"2024-09-10T07:52:01.608252+0000","last_peered":"2024-09-10T07:52:01.608252+0000","last_clean":"2024-09-10T07:52:01.608252+0000","last_became_active":"2024-09-10T07:52:01.598546+0000","last_became_peered":"2024-09-10T07:52:01.598546+0000","last_unstale":"2024-09-10T07:52:01.608252+0000","last_undegraded":"2024-09-10T07:52:01.608252+0000","last_fullsized":"2024-09-10T07:52:01.608252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.868625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617578+0000","last_change":"2024-09-10T07:52:00.593008+0000","last_active":"2024-09-10T07:52:01.617578+0000","last_peered":"2024-09-10T07:52:01.617578+0000","last_clean":"2024-09-10T07:52:01.617578+0000","last_became_active":"2024-09-10T07:52:00.592734+0000","last_became_peered":"2024-09-10T07:52:00.592734+0000","last_unstale":"2024-09-10T07:52:01.617578+0000","last_undegraded":"2024-09-10T07:52:01.617578+0000","last_fullsized":"2024-09-10T07:52:01.617578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572402+0000","last_change":"2024-09-10T07:52:00.572402+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572402+0000","last_undegraded":"2024-09-10T07:52:00.572402+0000","last_fullsized":"2024-09-10T07:52:00.572402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594717+0000","last_change":"2024-09-10T07:52:00.591803+0000","last_active":"2024-09-10T07:52:00.594717+0000","last_peered":"2024-09-10T07:52:00.594717+0000","last_clean":"2024-09-10T07:52:00.594717+0000","last_became_active":"2024-09-10T07:52:00.591495+0000","last_became_peered":"2024-09-10T07:52:00.591495+0000","last_unstale":"2024-09-10T07:52:00.594717+0000","last_undegraded":"2024-09-10T07:52:00.594717+0000","last_fullsized":"2024-09-10T07:52:00.594717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:07.100330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602204+0000","last_change":"2024-09-10T07:52:01.597959+0000","last_active":"2024-09-10T07:52:01.602204+0000","last_peered":"2024-09-10T07:52:01.602204+0000","last_clean":"2024-09-10T07:52:01.602204+0000","last_became_active":"2024-09-10T07:52:01.597689+0000","last_became_peered":"2024-09-10T07:52:01.597689+0000","last_unstale":"2024-09-10T07:52:01.602204+0000","last_undegraded":"2024-09-10T07:52:01.602204+0000","last_fullsized":"2024-09-10T07:52:01.602204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:21.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586638+0000","last_change":"2024-09-10T07:52:00.596444+0000","last_active":"2024-09-10T07:52:02.586638+0000","last_peered":"2024-09-10T07:52:02.586638+0000","last_clean":"2024-09-10T07:52:02.586638+0000","last_became_active":"2024-09-10T07:52:00.596005+0000","last_became_peered":"2024-09-10T07:52:00.596005+0000","last_unstale":"2024-09-10T07:52:02.586638+0000","last_undegraded":"2024-09-10T07:52:02.586638+0000","last_fullsized":"2024-09-10T07:52:02.586638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:45.516907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579130+0000","last_change":"2024-09-10T07:52:00.579130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579130+0000","last_undegraded":"2024-09-10T07:52:00.579130+0000","last_fullsized":"2024-09-10T07:52:00.579130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572551+0000","last_change":"2024-09-10T07:52:00.596629+0000","last_active":"2024-09-10T07:52:01.572551+0000","last_peered":"2024-09-10T07:52:01.572551+0000","last_clean":"2024-09-10T07:52:01.572551+0000","last_became_active":"2024-09-10T07:52:00.595725+0000","last_became_peered":"2024-09-10T07:52:00.595725+0000","last_unstale":"2024-09-10T07:52:01.572551+0000","last_undegraded":"2024-09-10T07:52:01.572551+0000","last_fullsized":"2024-09-10T07:52:01.572551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571585+0000","last_change":"2024-09-10T07:52:00.571585+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571585+0000","last_undegraded":"2024-09-10T07:52:00.571585+0000","last_fullsized":"2024-09-10T07:52:00.571585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597120+0000","last_change":"2024-09-10T07:52:00.595747+0000","last_active":"2024-09-10T07:52:00.597120+0000","last_peered":"2024-09-10T07:52:00.597120+0000","last_clean":"2024-09-10T07:52:00.597120+0000","last_became_active":"2024-09-10T07:52:00.595362+0000","last_became_peered":"2024-09-10T07:52:00.595362+0000","last_unstale":"2024-09-10T07:52:00.597120+0000","last_undegraded":"2024-09-10T07:52:00.597120+0000","last_fullsized":"2024-09-10T07:52:00.597120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604031+0000","last_change":"2024-09-10T07:52:01.595164+0000","last_active":"2024-09-10T07:52:01.604031+0000","last_peered":"2024-09-10T07:52:01.604031+0000","last_clean":"2024-09-10T07:52:01.604031+0000","last_became_active":"2024-09-10T07:52:01.594830+0000","last_became_peered":"2024-09-10T07:52:01.594830+0000","last_unstale":"2024-09-10T07:52:01.604031+0000","last_undegraded":"2024-09-10T07:52:01.604031+0000","last_fullsized":"2024-09-10T07:52:01.604031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:42.789252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617396+0000","last_change":"2024-09-10T07:52:00.596061+0000","last_active":"2024-09-10T07:52:01.617396+0000","last_peered":"2024-09-10T07:52:01.617396+0000","last_clean":"2024-09-10T07:52:01.617396+0000","last_became_active":"2024-09-10T07:52:00.595743+0000","last_became_peered":"2024-09-10T07:52:00.595743+0000","last_unstale":"2024-09-10T07:52:01.617396+0000","last_undegraded":"2024-09-10T07:52:01.617396+0000","last_fullsized":"2024-09-10T07:52:01.617396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:29.669564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608352+0000","last_change":"2024-09-10T07:52:01.594958+0000","last_active":"2024-09-10T07:52:01.608352+0000","last_peered":"2024-09-10T07:52:01.608352+0000","last_clean":"2024-09-10T07:52:01.608352+0000","last_became_active":"2024-09-10T07:52:01.594693+0000","last_became_peered":"2024-09-10T07:52:01.594693+0000","last_unstale":"2024-09-10T07:52:01.608352+0000","last_undegraded":"2024-09-10T07:52:01.608352+0000","last_fullsized":"2024-09-10T07:52:01.608352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:22.172497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.599721+0000","last_change":"2024-09-10T07:52:00.598950+0000","last_active":"2024-09-10T07:52:00.599721+0000","last_peered":"2024-09-10T07:52:00.599721+0000","last_clean":"2024-09-10T07:52:00.599721+0000","last_became_active":"2024-09-10T07:52:00.598731+0000","last_became_peered":"2024-09-10T07:52:00.598731+0000","last_unstale":"2024-09-10T07:52:00.599721+0000","last_undegraded":"2024-09-10T07:52:00.599721+0000","last_fullsized":"2024-09-10T07:52:00.599721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600611+0000","last_change":"2024-09-10T07:52:01.592487+0000","last_active":"2024-09-10T07:52:01.600611+0000","last_peered":"2024-09-10T07:52:01.600611+0000","last_clean":"2024-09-10T07:52:01.600611+0000","last_became_active":"2024-09-10T07:52:01.590383+0000","last_became_peered":"2024-09-10T07:52:01.590383+0000","last_unstale":"2024-09-10T07:52:01.600611+0000","last_undegraded":"2024-09-10T07:52:01.600611+0000","last_fullsized":"2024-09-10T07:52:01.600611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597893+0000","last_change":"2024-09-10T07:52:00.595658+0000","last_active":"2024-09-10T07:52:00.597893+0000","last_peered":"2024-09-10T07:52:00.597893+0000","last_clean":"2024-09-10T07:52:00.597893+0000","last_became_active":"2024-09-10T07:52:00.595374+0000","last_became_peered":"2024-09-10T07:52:00.595374+0000","last_unstale":"2024-09-10T07:52:00.597893+0000","last_undegraded":"2024-09-10T07:52:00.597893+0000","last_fullsized":"2024-09-10T07:52:00.597893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586631+0000","last_change":"2024-09-10T07:52:01.588191+0000","last_active":"2024-09-10T07:52:02.586631+0000","last_peered":"2024-09-10T07:52:02.586631+0000","last_clean":"2024-09-10T07:52:02.586631+0000","last_became_active":"2024-09-10T07:52:01.587568+0000","last_became_peered":"2024-09-10T07:52:01.587568+0000","last_unstale":"2024-09-10T07:52:02.586631+0000","last_undegraded":"2024-09-10T07:52:02.586631+0000","last_fullsized":"2024-09-10T07:52:02.586631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:28.983805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.590896+0000","last_change":"2024-09-10T07:52:00.586945+0000","last_active":"2024-09-10T07:52:00.590896+0000","last_peered":"2024-09-10T07:52:00.590896+0000","last_clean":"2024-09-10T07:52:00.590896+0000","last_became_active":"2024-09-10T07:52:00.586368+0000","last_became_peered":"2024-09-10T07:52:00.586368+0000","last_unstale":"2024-09-10T07:52:00.590896+0000","last_undegraded":"2024-09-10T07:52:00.590896+0000","last_fullsized":"2024-09-10T07:52:00.590896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:31.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607474+0000","last_change":"2024-09-10T07:52:01.596817+0000","last_active":"2024-09-10T07:52:01.607474+0000","last_peered":"2024-09-10T07:52:01.607474+0000","last_clean":"2024-09-10T07:52:01.607474+0000","last_became_active":"2024-09-10T07:52:01.595480+0000","last_became_peered":"2024-09-10T07:52:01.595480+0000","last_unstale":"2024-09-10T07:52:01.607474+0000","last_undegraded":"2024-09-10T07:52:01.607474+0000","last_fullsized":"2024-09-10T07:52:01.607474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:38.832201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586749+0000","last_change":"2024-09-10T07:52:00.590479+0000","last_active":"2024-09-10T07:52:02.586749+0000","last_peered":"2024-09-10T07:52:02.586749+0000","last_clean":"2024-09-10T07:52:02.586749+0000","last_became_active":"2024-09-10T07:52:00.589855+0000","last_became_peered":"2024-09-10T07:52:00.589855+0000","last_unstale":"2024-09-10T07:52:02.586749+0000","last_undegraded":"2024-09-10T07:52:02.586749+0000","last_fullsized":"2024-09-10T07:52:02.586749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:34.433999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578558+0000","last_change":"2024-09-10T07:52:00.578558+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578558+0000","last_undegraded":"2024-09-10T07:52:00.578558+0000","last_fullsized":"2024-09-10T07:52:00.578558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586935+0000","last_change":"2024-09-10T07:52:00.584260+0000","last_active":"2024-09-10T07:52:00.586935+0000","last_peered":"2024-09-10T07:52:00.586935+0000","last_clean":"2024-09-10T07:52:00.586935+0000","last_became_active":"2024-09-10T07:52:00.583375+0000","last_became_peered":"2024-09-10T07:52:00.583375+0000","last_unstale":"2024-09-10T07:52:00.586935+0000","last_undegraded":"2024-09-10T07:52:00.586935+0000","last_fullsized":"2024-09-10T07:52:00.586935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603982+0000","last_change":"2024-09-10T07:52:01.598146+0000","last_active":"2024-09-10T07:52:01.603982+0000","last_peered":"2024-09-10T07:52:01.603982+0000","last_clean":"2024-09-10T07:52:01.603982+0000","last_became_active":"2024-09-10T07:52:01.597673+0000","last_became_peered":"2024-09-10T07:52:01.597673+0000","last_unstale":"2024-09-10T07:52:01.603982+0000","last_undegraded":"2024-09-10T07:52:01.603982+0000","last_fullsized":"2024-09-10T07:52:01.603982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.593922+0000","last_change":"2024-09-10T07:52:00.592001+0000","last_active":"2024-09-10T07:52:00.593922+0000","last_peered":"2024-09-10T07:52:00.593922+0000","last_clean":"2024-09-10T07:52:00.593922+0000","last_became_active":"2024-09-10T07:52:00.590320+0000","last_became_peered":"2024-09-10T07:52:00.590320+0000","last_unstale":"2024-09-10T07:52:00.593922+0000","last_undegraded":"2024-09-10T07:52:00.593922+0000","last_fullsized":"2024-09-10T07:52:00.593922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:37.783344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597324+0000","last_change":"2024-09-10T07:52:01.594864+0000","last_active":"2024-09-10T07:52:01.597324+0000","last_peered":"2024-09-10T07:52:01.597324+0000","last_clean":"2024-09-10T07:52:01.597324+0000","last_became_active":"2024-09-10T07:52:01.594608+0000","last_became_peered":"2024-09-10T07:52:01.594608+0000","last_unstale":"2024-09-10T07:52:01.597324+0000","last_undegraded":"2024-09-10T07:52:01.597324+0000","last_fullsized":"2024-09-10T07:52:01.597324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.077924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.595080+0000","last_change":"2024-09-10T07:52:00.594080+0000","last_active":"2024-09-10T07:52:00.595080+0000","last_peered":"2024-09-10T07:52:00.595080+0000","last_clean":"2024-09-10T07:52:00.595080+0000","last_became_active":"2024-09-10T07:52:00.593703+0000","last_became_peered":"2024-09-10T07:52:00.593703+0000","last_unstale":"2024-09-10T07:52:00.595080+0000","last_undegraded":"2024-09-10T07:52:00.595080+0000","last_fullsized":"2024-09-10T07:52:00.595080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.614133+0000","last_change":"2024-09-10T07:52:01.606931+0000","last_active":"2024-09-10T07:52:01.614133+0000","last_peered":"2024-09-10T07:52:01.614133+0000","last_clean":"2024-09-10T07:52:01.614133+0000","last_became_active":"2024-09-10T07:52:01.606658+0000","last_became_peered":"2024-09-10T07:52:01.606658+0000","last_unstale":"2024-09-10T07:52:01.614133+0000","last_undegraded":"2024-09-10T07:52:01.614133+0000","last_fullsized":"2024-09-10T07:52:01.614133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:48.106660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.582768+0000","last_change":"2024-09-10T07:52:00.580977+0000","last_active":"2024-09-10T07:52:00.582768+0000","last_peered":"2024-09-10T07:52:00.582768+0000","last_clean":"2024-09-10T07:52:00.582768+0000","last_became_active":"2024-09-10T07:52:00.580503+0000","last_became_peered":"2024-09-10T07:52:00.580503+0000","last_unstale":"2024-09-10T07:52:00.582768+0000","last_undegraded":"2024-09-10T07:52:00.582768+0000","last_fullsized":"2024-09-10T07:52:00.582768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602646+0000","last_change":"2024-09-10T07:52:01.591171+0000","last_active":"2024-09-10T07:52:01.602646+0000","last_peered":"2024-09-10T07:52:01.602646+0000","last_clean":"2024-09-10T07:52:01.602646+0000","last_became_active":"2024-09-10T07:52:01.590850+0000","last_became_peered":"2024-09-10T07:52:01.590850+0000","last_unstale":"2024-09-10T07:52:01.602646+0000","last_undegraded":"2024-09-10T07:52:01.602646+0000","last_fullsized":"2024-09-10T07:52:01.602646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:24.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582479+0000","last_change":"2024-09-10T07:52:00.588158+0000","last_active":"2024-09-10T07:52:01.582479+0000","last_peered":"2024-09-10T07:52:01.582479+0000","last_clean":"2024-09-10T07:52:01.582479+0000","last_became_active":"2024-09-10T07:52:00.587861+0000","last_became_peered":"2024-09-10T07:52:00.587861+0000","last_unstale":"2024-09-10T07:52:01.582479+0000","last_undegraded":"2024-09-10T07:52:01.582479+0000","last_fullsized":"2024-09-10T07:52:01.582479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.605798+0000","last_change":"2024-09-10T07:52:01.585615+0000","last_active":"2024-09-10T07:52:01.605798+0000","last_peered":"2024-09-10T07:52:01.605798+0000","last_clean":"2024-09-10T07:52:01.605798+0000","last_became_active":"2024-09-10T07:52:01.585372+0000","last_became_peered":"2024-09-10T07:52:01.585372+0000","last_unstale":"2024-09-10T07:52:01.605798+0000","last_undegraded":"2024-09-10T07:52:01.605798+0000","last_fullsized":"2024-09-10T07:52:01.605798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573044+0000","last_change":"2024-09-10T07:52:00.586049+0000","last_active":"2024-09-10T07:52:01.573044+0000","last_peered":"2024-09-10T07:52:01.573044+0000","last_clean":"2024-09-10T07:52:01.573044+0000","last_became_active":"2024-09-10T07:52:00.585765+0000","last_became_peered":"2024-09-10T07:52:00.585765+0000","last_unstale":"2024-09-10T07:52:01.573044+0000","last_undegraded":"2024-09-10T07:52:01.573044+0000","last_fullsized":"2024-09-10T07:52:01.573044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:41.786152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609802+0000","last_change":"2024-09-10T07:52:01.605802+0000","last_active":"2024-09-10T07:52:01.609802+0000","last_peered":"2024-09-10T07:52:01.609802+0000","last_clean":"2024-09-10T07:52:01.609802+0000","last_became_active":"2024-09-10T07:52:01.605500+0000","last_became_peered":"2024-09-10T07:52:01.605500+0000","last_unstale":"2024-09-10T07:52:01.609802+0000","last_undegraded":"2024-09-10T07:52:01.609802+0000","last_fullsized":"2024-09-10T07:52:01.609802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:51.503646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586882+0000","last_change":"2024-09-10T07:52:00.587749+0000","last_active":"2024-09-10T07:52:02.586882+0000","last_peered":"2024-09-10T07:52:02.586882+0000","last_clean":"2024-09-10T07:52:02.586882+0000","last_became_active":"2024-09-10T07:52:00.587486+0000","last_became_peered":"2024-09-10T07:52:00.587486+0000","last_unstale":"2024-09-10T07:52:02.586882+0000","last_undegraded":"2024-09-10T07:52:02.586882+0000","last_fullsized":"2024-09-10T07:52:02.586882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:47.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572823+0000","last_change":"2024-09-10T07:52:00.572823+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572823+0000","last_undegraded":"2024-09-10T07:52:00.572823+0000","last_fullsized":"2024-09-10T07:52:00.572823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572808+0000","last_change":"2024-09-10T07:52:00.593914+0000","last_active":"2024-09-10T07:52:01.572808+0000","last_peered":"2024-09-10T07:52:01.572808+0000","last_clean":"2024-09-10T07:52:01.572808+0000","last_became_active":"2024-09-10T07:52:00.591741+0000","last_became_peered":"2024-09-10T07:52:00.591741+0000","last_unstale":"2024-09-10T07:52:01.572808+0000","last_undegraded":"2024-09-10T07:52:01.572808+0000","last_fullsized":"2024-09-10T07:52:01.572808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:40.147081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.598161+0000","last_change":"2024-09-10T07:52:01.587331+0000","last_active":"2024-09-10T07:52:01.598161+0000","last_peered":"2024-09-10T07:52:01.598161+0000","last_clean":"2024-09-10T07:52:01.598161+0000","last_became_active":"2024-09-10T07:52:01.587019+0000","last_became_peered":"2024-09-10T07:52:01.587019+0000","last_unstale":"2024-09-10T07:52:01.598161+0000","last_undegraded":"2024-09-10T07:52:01.598161+0000","last_fullsized":"2024-09-10T07:52:01.598161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:33.250694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.580356+0000","last_change":"2024-09-10T07:52:00.580768+0000","last_active":"2024-09-10T07:52:01.580356+0000","last_peered":"2024-09-10T07:52:01.580356+0000","last_clean":"2024-09-10T07:52:01.580356+0000","last_became_active":"2024-09-10T07:52:00.580422+0000","last_became_peered":"2024-09-10T07:52:00.580422+0000","last_unstale":"2024-09-10T07:52:01.580356+0000","last_undegraded":"2024-09-10T07:52:01.580356+0000","last_fullsized":"2024-09-10T07:52:01.580356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600973+0000","last_change":"2024-09-10T07:52:01.593290+0000","last_active":"2024-09-10T07:52:01.600973+0000","last_peered":"2024-09-10T07:52:01.600973+0000","last_clean":"2024-09-10T07:52:01.600973+0000","last_became_active":"2024-09-10T07:52:01.593023+0000","last_became_peered":"2024-09-10T07:52:01.593023+0000","last_unstale":"2024-09-10T07:52:01.600973+0000","last_undegraded":"2024-09-10T07:52:01.600973+0000","last_fullsized":"2024-09-10T07:52:01.600973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:53.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.577180+0000","last_change":"2024-09-10T07:52:00.592995+0000","last_active":"2024-09-10T07:52:01.577180+0000","last_peered":"2024-09-10T07:52:01.577180+0000","last_clean":"2024-09-10T07:52:01.577180+0000","last_became_active":"2024-09-10T07:52:00.592712+0000","last_became_peered":"2024-09-10T07:52:00.592712+0000","last_unstale":"2024-09-10T07:52:01.577180+0000","last_undegraded":"2024-09-10T07:52:01.577180+0000","last_fullsized":"2024-09-10T07:52:01.577180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:40.800624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571582+0000","last_change":"2024-09-10T07:52:00.571582+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571582+0000","last_undegraded":"2024-09-10T07:52:00.571582+0000","last_fullsized":"2024-09-10T07:52:00.571582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.616068+0000","last_change":"2024-09-10T07:52:00.592703+0000","last_active":"2024-09-10T07:52:01.616068+0000","last_peered":"2024-09-10T07:52:01.616068+0000","last_clean":"2024-09-10T07:52:01.616068+0000","last_became_active":"2024-09-10T07:52:00.592405+0000","last_became_peered":"2024-09-10T07:52:00.592405+0000","last_unstale":"2024-09-10T07:52:01.616068+0000","last_undegraded":"2024-09-10T07:52:01.616068+0000","last_fullsized":"2024-09-10T07:52:01.616068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:50.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.579327+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:52:01.579327+0000","last_peered":"2024-09-10T07:52:01.579327+0000","last_clean":"2024-09-10T07:52:01.579327+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:52:01.579327+0000","last_undegraded":"2024-09-10T07:52:01.579327+0000","last_fullsized":"2024-09-10T07:52:01.579327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476808,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1069056,"data_stored":459078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":465853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1662976,"data_stored":1049446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476807,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":466525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1650688,"data_stored":1043171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1085440,"data_stored":465887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:52:06.628 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3c is not active+clean 2024-09-10T07:52:06.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.579141+0000', 'last_change': '2024-09-10T07:52:00.579141+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.579141+0000', 'last_undegraded': '2024-09-10T07:52:00.579141+0000', 'last_fullsized': '2024-09-10T07:52:00.579141+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:52:06.628 INFO:tasks.ceph.ceph_manager.ceph:PG 13.35 is not active+clean 2024-09-10T07:52:06.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.572168+0000', 'last_change': '2024-09-10T07:52:00.572168+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.572168+0000', 'last_undegraded': '2024-09-10T07:52:00.572168+0000', 'last_fullsized': '2024-09-10T07:52:00.572168+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.32 is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.578583+0000', 'last_change': '2024-09-10T07:52:00.578583+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.578583+0000', 'last_undegraded': '2024-09-10T07:52:00.578583+0000', 'last_fullsized': '2024-09-10T07:52:00.578583+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2e is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.572822+0000', 'last_change': '2024-09-10T07:52:00.572822+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.572822+0000', 'last_undegraded': '2024-09-10T07:52:00.572822+0000', 'last_fullsized': '2024-09-10T07:52:00.572822+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2d is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.577424+0000', 'last_change': '2024-09-10T07:52:00.577424+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.577424+0000', 'last_undegraded': '2024-09-10T07:52:00.577424+0000', 'last_fullsized': '2024-09-10T07:52:00.577424+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2c is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.578130+0000', 'last_change': '2024-09-10T07:52:00.578130+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.578130+0000', 'last_undegraded': '2024-09-10T07:52:00.578130+0000', 'last_fullsized': '2024-09-10T07:52:00.578130+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2b is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.579577+0000', 'last_change': '2024-09-10T07:52:00.579577+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.579577+0000', 'last_undegraded': '2024-09-10T07:52:00.579577+0000', 'last_fullsized': '2024-09-10T07:52:00.579577+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2a is not active+clean 2024-09-10T07:52:06.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.578055+0000', 'last_change': '2024-09-10T07:52:00.578055+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.578055+0000', 'last_undegraded': '2024-09-10T07:52:00.578055+0000', 'last_fullsized': '2024-09-10T07:52:00.578055+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.28 is not active+clean 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.573077+0000', 'last_change': '2024-09-10T07:52:00.573077+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.573077+0000', 'last_undegraded': '2024-09-10T07:52:00.573077+0000', 'last_fullsized': '2024-09-10T07:52:00.573077+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.27 is not active+clean 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.571664+0000', 'last_change': '2024-09-10T07:52:00.571664+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.571664+0000', 'last_undegraded': '2024-09-10T07:52:00.571664+0000', 'last_fullsized': '2024-09-10T07:52:00.571664+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.23 is not active+clean 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.579083+0000', 'last_change': '2024-09-10T07:52:00.579083+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.579083+0000', 'last_undegraded': '2024-09-10T07:52:00.579083+0000', 'last_fullsized': '2024-09-10T07:52:00.579083+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.22 is not active+clean 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.577512+0000', 'last_change': '2024-09-10T07:52:00.577512+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.577512+0000', 'last_undegraded': '2024-09-10T07:52:00.577512+0000', 'last_fullsized': '2024-09-10T07:52:00.577512+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1a is not active+clean 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.587100+0000', 'last_change': '2024-09-10T07:52:00.587100+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.587100+0000', 'last_undegraded': '2024-09-10T07:52:00.587100+0000', 'last_fullsized': '2024-09-10T07:52:00.587100+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:52:06.630 INFO:tasks.ceph.ceph_manager.ceph:PG 13.19 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.577375+0000', 'last_change': '2024-09-10T07:52:00.577375+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.577375+0000', 'last_undegraded': '2024-09-10T07:52:00.577375+0000', 'last_fullsized': '2024-09-10T07:52:00.577375+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:PG 13.16 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.579928+0000', 'last_change': '2024-09-10T07:52:00.579928+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.579928+0000', 'last_undegraded': '2024-09-10T07:52:00.579928+0000', 'last_fullsized': '2024-09-10T07:52:00.579928+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:PG 13.14 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.573300+0000', 'last_change': '2024-09-10T07:52:00.573300+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.573300+0000', 'last_undegraded': '2024-09-10T07:52:00.573300+0000', 'last_fullsized': '2024-09-10T07:52:00.573300+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:PG 13.12 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.572402+0000', 'last_change': '2024-09-10T07:52:00.572402+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.572402+0000', 'last_undegraded': '2024-09-10T07:52:00.572402+0000', 'last_fullsized': '2024-09-10T07:52:00.572402+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:PG 13.10 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.579130+0000', 'last_change': '2024-09-10T07:52:00.579130+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.579130+0000', 'last_undegraded': '2024-09-10T07:52:00.579130+0000', 'last_fullsized': '2024-09-10T07:52:00.579130+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:PG 13.11 is not active+clean 2024-09-10T07:52:06.631 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.571585+0000', 'last_change': '2024-09-10T07:52:00.571585+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.571585+0000', 'last_undegraded': '2024-09-10T07:52:00.571585+0000', 'last_fullsized': '2024-09-10T07:52:00.571585+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:PG 13.8 is not active+clean 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.578558+0000', 'last_change': '2024-09-10T07:52:00.578558+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.578558+0000', 'last_undegraded': '2024-09-10T07:52:00.578558+0000', 'last_fullsized': '2024-09-10T07:52:00.578558+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3 is not active+clean 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.572823+0000', 'last_change': '2024-09-10T07:52:00.572823+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.572823+0000', 'last_undegraded': '2024-09-10T07:52:00.572823+0000', 'last_fullsized': '2024-09-10T07:52:00.572823+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:PG 13.c is not active+clean 2024-09-10T07:52:06.632 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:52:00.571582+0000', 'last_change': '2024-09-10T07:52:00.571582+0000', 'last_active': '2024-09-10T07:52:00.561176+0000', 'last_peered': '2024-09-10T07:52:00.561176+0000', 'last_clean': '2024-09-10T07:52:00.561176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:52:00.571582+0000', 'last_undegraded': '2024-09-10T07:52:00.571582+0000', 'last_fullsized': '2024-09-10T07:52:00.571582+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'last_clean_scrub_stamp': '2024-09-10T07:52:00.561176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:52:06.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:06.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.821+0000 7fe62b200640 1 Processor -- start 2024-09-10T07:52:06.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.821+0000 7fe62b200640 1 -- start start 2024-09-10T07:52:06.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe62415c720 0x7fe624071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 0x7fe624073610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 0x7fe624073b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe624072210 con 0x7fe62415c720 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe624074090 con 0x7fe624070f30 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.822+0000 7fe62b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240cd7f0 con 0x7fe6240cfdc0 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe62415c720 0x7fe624071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe62415c720 0x7fe624071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59792/0 (socket says 172.21.15.3:59792) 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 -- 172.21.15.3:0/1894199495 learned_addr learned my addr 172.21.15.3:0/1894199495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe629200640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 0x7fe624073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623400640 1 --2- 172.21.15.3:0/1894199495 >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 0x7fe624073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 -- 172.21.15.3:0/1894199495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 msgr2=0x7fe624073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 0x7fe624073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 -- 172.21.15.3:0/1894199495 >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 msgr2=0x7fe624073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 --2- 172.21.15.3:0/1894199495 >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 0x7fe624073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623e00640 1 -- 172.21.15.3:0/1894199495 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6240cf880 con 0x7fe62415c720 2024-09-10T07:52:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe629200640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 0x7fe624073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:06.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.823+0000 7fe623400640 1 --2- 172.21.15.3:0/1894199495 >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 0x7fe624073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.824+0000 7fe623e00640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe62415c720 0x7fe624071cd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe624071550 tx=0x7fe61402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603fc83cf5e881f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.824+0000 7fe622a00640 1 -- 172.21.15.3:0/1894199495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614037070 con 0x7fe62415c720 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.824+0000 7fe622a00640 1 -- 172.21.15.3:0/1894199495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614032440 con 0x7fe62415c720 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.824+0000 7fe622a00640 1 -- 172.21.15.3:0/1894199495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614032810 con 0x7fe62415c720 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 -- 172.21.15.3:0/1894199495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe62415c720 msgr2=0x7fe624071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe62415c720 0x7fe624071cd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe624071550 tx=0x7fe61402a1d0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 -- 172.21.15.3:0/1894199495 shutdown_connections 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6240cfdc0 0x7fe624073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 --2- 172.21.15.3:0/1894199495 >> v2:172.21.15.70:3300/0 conn(0x7fe624070f30 0x7fe624073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 --2- 172.21.15.3:0/1894199495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe62415c720 0x7fe624071cd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 -- 172.21.15.3:0/1894199495 >> 172.21.15.3:0/1894199495 conn(0x7fe62406cfc0 msgr2=0x7fe62406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 -- 172.21.15.3:0/1894199495 shutdown_connections 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.825+0000 7fe62b200640 1 -- 172.21.15.3:0/1894199495 wait complete. 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.827+0000 7fe62b200640 1 Processor -- start 2024-09-10T07:52:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.827+0000 7fe62b200640 1 -- start start 2024-09-10T07:52:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe624070f30 0x7fe6241590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe62415c720 0x7fe62414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe62406ed10 con 0x7fe6240cfdc0 2024-09-10T07:52:06.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6240cdd70 con 0x7fe624070f30 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe62b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240ce2a0 con 0x7fe62415c720 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe623400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe623400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59802/0 (socket says 172.21.15.3:59802) 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe623400640 1 -- 172.21.15.3:0/450828945 learned_addr learned my addr 172.21.15.3:0/450828945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe623e00640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe624070f30 0x7fe6241590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.828+0000 7fe629200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe62415c720 0x7fe62414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 -- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe62415c720 msgr2=0x7fe62414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe62415c720 0x7fe62414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 -- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe624070f30 msgr2=0x7fe6241590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe624070f30 0x7fe6241590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 -- 172.21.15.3:0/450828945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6240b28a0 con 0x7fe6240cfdc0 2024-09-10T07:52:06.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe623400640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e810 tx=0x7fe61000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000f040 con 0x7fe6240cfdc0 2024-09-10T07:52:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610004030 con 0x7fe6240cfdc0 2024-09-10T07:52:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610004400 con 0x7fe6240cfdc0 2024-09-10T07:52:06.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6240b2b80 con 0x7fe6240cfdc0 2024-09-10T07:52:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.829+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241e6010 con 0x7fe6240cfdc0 2024-09-10T07:52:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.831+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6240cf2b0 con 0x7fe6240cfdc0 2024-09-10T07:52:06.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.831+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe61000c6d0 con 0x7fe6240cfdc0 2024-09-10T07:52:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.832+0000 7fe61be00640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.832+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fe61008ab20 con 0x7fe6240cfdc0 2024-09-10T07:52:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.833+0000 7fe623e00640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:06.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.833+0000 7fe623e00640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 0x7fe60406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe614034000 tx=0x7fe61402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:06.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:06.837+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe610017020 con 0x7fe6240cfdc0 2024-09-10T07:52:07.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.087+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe624071cb0 con 0x7fe6240cfdc0 2024-09-10T07:52:07.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.088+0000 7fe61be00640 1 -- 172.21.15.3:0/450828945 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1303 (secure 0 0 0) 0x7fe610058de0 con 0x7fe6240cfdc0 2024-09-10T07:52:07.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:07.088 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":360,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":75},{"state_name":"creating+peering","count":22}],"num_pgs":97,"num_pools":3,"num_objects":12,"data_bytes":592744,"bytes_used":269520896,"bytes_avail":772824592384,"bytes_total":773094113280,"inactive_pgs_ratio":0.22680412232875824,"write_bytes_sec":1194,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":85,"btime":"2024-09-10T07:52:04:593834+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-09-10T07:51:49.561868+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 msgr2=0x7fe60406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 0x7fe60406e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe614034000 tx=0x7fe61402e000 comp rx=0 tx=0).stop 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 msgr2=0x7fe6240b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e810 tx=0x7fe61000ecd0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 shutdown_connections 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe60406be30 0x7fe60406e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe62415c720 0x7fe62414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6240cfdc0 0x7fe6240b2070 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 --2- 172.21.15.3:0/450828945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe624070f30 0x7fe6241590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 >> 172.21.15.3:0/450828945 conn(0x7fe62406cfc0 msgr2=0x7fe62415e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 shutdown_connections 2024-09-10T07:52:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.090+0000 7fe62b200640 1 -- 172.21.15.3:0/450828945 wait complete. 2024-09-10T07:52:07.106 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:52:07.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:52:07.169 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:52:07.169+0000 7f8caa000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:52:07.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.262+0000 7f53fa400640 1 Processor -- start 2024-09-10T07:52:07.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.262+0000 7f53fa400640 1 -- start start 2024-09-10T07:52:07.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.263+0000 7f53fa400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53f40cfba0 0x7f53f415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:07.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53fa400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f415bfb0 con 0x7f53f40ccf60 2024-09-10T07:52:07.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53fa400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f40b4140 con 0x7f53f415abc0 2024-09-10T07:52:07.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53fa400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f53f415ec40 con 0x7f53f40cfba0 2024-09-10T07:52:07.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53f3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f53f40ccf60 0x7f53f40b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57342/0 (socket says 172.21.15.3:57342) 2024-09-10T07:52:07.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53f3e00640 1 -- 172.21.15.3:0/1036425908 learned_addr learned my addr 172.21.15.3:0/1036425908 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:07.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53ebe00640 1 --2- 172.21.15.3:0/1036425908 >> v2:172.21.15.70:3300/0 conn(0x7f53f40cfba0 0x7f53f415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:07.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53f2a00640 1 -- 172.21.15.3:0/1036425908 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664105247 0 0) 0x7f53f415bfb0 con 0x7f53f40ccf60 2024-09-10T07:52:07.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53f2a00640 1 -- 172.21.15.3:0/1036425908 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d8003520 con 0x7f53f40ccf60 2024-09-10T07:52:07.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53ebe00640 1 -- 172.21.15.3:0/1036425908 >> v1:172.21.15.70:6789/0 conn(0x7f53f415abc0 legacy=0x7f53f40b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53ebe00640 1 -- 172.21.15.3:0/1036425908 >> v1:172.21.15.3:6789/0 conn(0x7f53f40ccf60 legacy=0x7f53f40b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.264+0000 7f53ebe00640 1 -- 172.21.15.3:0/1036425908 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f415ee70 con 0x7f53f40cfba0 2024-09-10T07:52:07.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53ebe00640 1 --2- 172.21.15.3:0/1036425908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cfba0 0x7f53f415e700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00a940 tx=0x7f53dc02b550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1710d5dc484f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:07.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53f2a00640 1 -- 172.21.15.3:0/1036425908 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00b040 con 0x7f53f40cfba0 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53f2a00640 1 -- 172.21.15.3:0/1036425908 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc030710 con 0x7f53f40cfba0 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53f2a00640 1 -- 172.21.15.3:0/1036425908 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc030ae0 con 0x7f53f40cfba0 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53fa400640 1 -- 172.21.15.3:0/1036425908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cfba0 msgr2=0x7f53f415e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53fa400640 1 --2- 172.21.15.3:0/1036425908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cfba0 0x7f53f415e700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00a940 tx=0x7f53dc02b550 comp rx=0 tx=0).stop 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53fa400640 1 -- 172.21.15.3:0/1036425908 shutdown_connections 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53fa400640 1 --2- 172.21.15.3:0/1036425908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cfba0 0x7f53f415e700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.266+0000 7f53fa400640 1 -- 172.21.15.3:0/1036425908 >> 172.21.15.3:0/1036425908 conn(0x7f53f406d000 msgr2=0x7f53f406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.267+0000 7f53fa400640 1 -- 172.21.15.3:0/1036425908 shutdown_connections 2024-09-10T07:52:07.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.267+0000 7f53fa400640 1 -- 172.21.15.3:0/1036425908 wait complete. 2024-09-10T07:52:07.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.268+0000 7f53fa400640 1 Processor -- start 2024-09-10T07:52:07.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.268+0000 7f53fa400640 1 -- start start 2024-09-10T07:52:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53f40cfba0 0x7f53f4073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 0x7f53f40ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40b4ff0 con 0x7f53f40ccf60 2024-09-10T07:52:07.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f53f415f430 con 0x7f53f415abc0 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53fa400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f53f40b5170 con 0x7f53f40cfba0 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59810/0 (socket says 172.21.15.3:59810) 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.269+0000 7f53f3400640 1 -- 172.21.15.3:0/2272442271 learned_addr learned my addr 172.21.15.3:0/2272442271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 -- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53f40cfba0 msgr2=0x7f53f4073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53f40cfba0 0x7f53f4073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 -- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 msgr2=0x7f53f40ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3e00640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 0x7f53f40ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 0x7f53f40ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 -- 172.21.15.3:0/2272442271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f40719f0 con 0x7f53f40ccf60 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3e00640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 0x7f53f40ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53f3400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c430 tx=0x7f53e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017070 con 0x7f53f40ccf60 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e40052d0 con 0x7f53f40ccf60 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4003830 con 0x7f53f40ccf60 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.270+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f4071cd0 con 0x7f53f40ccf60 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.271+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f4072290 con 0x7f53f40ccf60 2024-09-10T07:52:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.271+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f40d1380 con 0x7f53f40ccf60 2024-09-10T07:52:07.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.272+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53e401a070 con 0x7f53f40ccf60 2024-09-10T07:52:07.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.273+0000 7f53eb400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:07.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.273+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f53e408a290 con 0x7f53f40ccf60 2024-09-10T07:52:07.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.274+0000 7f53ebe00640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:07.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.274+0000 7f53ebe00640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 0x7f53d006e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:07.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.278+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53f40d1380 con 0x7f53f40ccf60 2024-09-10T07:52:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.457+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53f415e700 con 0x7f53d006be30 2024-09-10T07:52:07.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.464+0000 7f53eb400640 1 -- 172.21.15.3:0/2272442271 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250215 (secure 0 0 0) 0x7f53f415e700 con 0x7f53d006be30 2024-09-10T07:52:07.466 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 msgr2=0x7f53d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 0x7f53d006e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53dc035000 tx=0x7f53dc02f000 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 msgr2=0x7f53f4148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c430 tx=0x7f53e400c8f0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 shutdown_connections 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f415abc0 0x7f53f40ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53f40cfba0 0x7f53f4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 --2- 172.21.15.3:0/2272442271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53f40ccf60 0x7f53f4148e80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.467+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 >> 172.21.15.3:0/2272442271 conn(0x7f53f406d000 msgr2=0x7f53f40aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.468+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 shutdown_connections 2024-09-10T07:52:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:07.468+0000 7f53fa400640 1 -- 172.21.15.3:0/2272442271 wait complete. 2024-09-10T07:52:07.482 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T07:52:05.564870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263204,"kb_used_data":9524,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711516,"statfs":{"total":773094113280,"available":772824592384,"internally_reserved":0,"allocated":9752576,"data_stored":4855566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000389"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597406+0000","last_change":"2024-09-10T07:52:01.591976+0000","last_active":"2024-09-10T07:52:01.597406+0000","last_peered":"2024-09-10T07:52:01.597406+0000","last_clean":"2024-09-10T07:52:01.597406+0000","last_became_active":"2024-09-10T07:52:01.591474+0000","last_became_peered":"2024-09-10T07:52:01.591474+0000","last_unstale":"2024-09-10T07:52:01.597406+0000","last_undegraded":"2024-09-10T07:52:01.597406+0000","last_fullsized":"2024-09-10T07:52:01.597406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:23.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586917+0000","last_change":"2024-09-10T07:52:01.588364+0000","last_active":"2024-09-10T07:52:02.586917+0000","last_peered":"2024-09-10T07:52:02.586917+0000","last_clean":"2024-09-10T07:52:02.586917+0000","last_became_active":"2024-09-10T07:52:01.585650+0000","last_became_peered":"2024-09-10T07:52:01.585650+0000","last_unstale":"2024-09-10T07:52:02.586917+0000","last_undegraded":"2024-09-10T07:52:02.586917+0000","last_fullsized":"2024-09-10T07:52:02.586917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:00.061836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607780+0000","last_change":"2024-09-10T07:52:01.590826+0000","last_active":"2024-09-10T07:52:01.607780+0000","last_peered":"2024-09-10T07:52:01.607780+0000","last_clean":"2024-09-10T07:52:01.607780+0000","last_became_active":"2024-09-10T07:52:01.590520+0000","last_became_peered":"2024-09-10T07:52:01.590520+0000","last_unstale":"2024-09-10T07:52:01.607780+0000","last_undegraded":"2024-09-10T07:52:01.607780+0000","last_fullsized":"2024-09-10T07:52:01.607780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:09.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579141+0000","last_change":"2024-09-10T07:52:00.579141+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579141+0000","last_undegraded":"2024-09-10T07:52:00.579141+0000","last_fullsized":"2024-09-10T07:52:00.579141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597009+0000","last_change":"2024-09-10T07:52:01.588987+0000","last_active":"2024-09-10T07:52:01.597009+0000","last_peered":"2024-09-10T07:52:01.597009+0000","last_clean":"2024-09-10T07:52:01.597009+0000","last_became_active":"2024-09-10T07:52:01.588704+0000","last_became_peered":"2024-09-10T07:52:01.588704+0000","last_unstale":"2024-09-10T07:52:01.597009+0000","last_undegraded":"2024-09-10T07:52:01.597009+0000","last_fullsized":"2024-09-10T07:52:01.597009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.917124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607788+0000","last_change":"2024-09-10T07:52:01.590162+0000","last_active":"2024-09-10T07:52:01.607788+0000","last_peered":"2024-09-10T07:52:01.607788+0000","last_clean":"2024-09-10T07:52:01.607788+0000","last_became_active":"2024-09-10T07:52:01.585077+0000","last_became_peered":"2024-09-10T07:52:01.585077+0000","last_unstale":"2024-09-10T07:52:01.607788+0000","last_undegraded":"2024-09-10T07:52:01.607788+0000","last_fullsized":"2024-09-10T07:52:01.607788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586493+0000","last_change":"2024-09-10T07:52:01.592098+0000","last_active":"2024-09-10T07:52:02.586493+0000","last_peered":"2024-09-10T07:52:02.586493+0000","last_clean":"2024-09-10T07:52:02.586493+0000","last_became_active":"2024-09-10T07:52:01.591441+0000","last_became_peered":"2024-09-10T07:52:01.591441+0000","last_unstale":"2024-09-10T07:52:02.586493+0000","last_undegraded":"2024-09-10T07:52:02.586493+0000","last_fullsized":"2024-09-10T07:52:02.586493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:03.328092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604052+0000","last_change":"2024-09-10T07:52:01.597132+0000","last_active":"2024-09-10T07:52:01.604052+0000","last_peered":"2024-09-10T07:52:01.604052+0000","last_clean":"2024-09-10T07:52:01.604052+0000","last_became_active":"2024-09-10T07:52:01.596923+0000","last_became_peered":"2024-09-10T07:52:01.596923+0000","last_unstale":"2024-09-10T07:52:01.604052+0000","last_undegraded":"2024-09-10T07:52:01.604052+0000","last_fullsized":"2024-09-10T07:52:01.604052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597340+0000","last_change":"2024-09-10T07:52:01.591154+0000","last_active":"2024-09-10T07:52:01.597340+0000","last_peered":"2024-09-10T07:52:01.597340+0000","last_clean":"2024-09-10T07:52:01.597340+0000","last_became_active":"2024-09-10T07:52:01.589021+0000","last_became_peered":"2024-09-10T07:52:01.589021+0000","last_unstale":"2024-09-10T07:52:01.597340+0000","last_undegraded":"2024-09-10T07:52:01.597340+0000","last_fullsized":"2024-09-10T07:52:01.597340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:27.551937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608406+0000","last_change":"2024-09-10T07:52:01.588436+0000","last_active":"2024-09-10T07:52:01.608406+0000","last_peered":"2024-09-10T07:52:01.608406+0000","last_clean":"2024-09-10T07:52:01.608406+0000","last_became_active":"2024-09-10T07:52:01.583416+0000","last_became_peered":"2024-09-10T07:52:01.583416+0000","last_unstale":"2024-09-10T07:52:01.608406+0000","last_undegraded":"2024-09-10T07:52:01.608406+0000","last_fullsized":"2024-09-10T07:52:01.608406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572168+0000","last_change":"2024-09-10T07:52:00.572168+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572168+0000","last_undegraded":"2024-09-10T07:52:00.572168+0000","last_fullsized":"2024-09-10T07:52:00.572168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604254+0000","last_change":"2024-09-10T07:52:01.596178+0000","last_active":"2024-09-10T07:52:01.604254+0000","last_peered":"2024-09-10T07:52:01.604254+0000","last_clean":"2024-09-10T07:52:01.604254+0000","last_became_active":"2024-09-10T07:52:01.595875+0000","last_became_peered":"2024-09-10T07:52:01.595875+0000","last_unstale":"2024-09-10T07:52:01.604254+0000","last_undegraded":"2024-09-10T07:52:01.604254+0000","last_fullsized":"2024-09-10T07:52:01.604254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:29.458387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.613750+0000","last_change":"2024-09-10T07:52:01.610337+0000","last_active":"2024-09-10T07:52:01.613750+0000","last_peered":"2024-09-10T07:52:01.613750+0000","last_clean":"2024-09-10T07:52:01.613750+0000","last_became_active":"2024-09-10T07:52:01.609110+0000","last_became_peered":"2024-09-10T07:52:01.609110+0000","last_unstale":"2024-09-10T07:52:01.613750+0000","last_undegraded":"2024-09-10T07:52:01.613750+0000","last_fullsized":"2024-09-10T07:52:01.613750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:46.374455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578583+0000","last_change":"2024-09-10T07:52:00.578583+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578583+0000","last_undegraded":"2024-09-10T07:52:00.578583+0000","last_fullsized":"2024-09-10T07:52:00.578583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586567+0000","last_change":"2024-09-10T07:52:01.591883+0000","last_active":"2024-09-10T07:52:02.586567+0000","last_peered":"2024-09-10T07:52:02.586567+0000","last_clean":"2024-09-10T07:52:02.586567+0000","last_became_active":"2024-09-10T07:52:01.591017+0000","last_became_peered":"2024-09-10T07:52:01.591017+0000","last_unstale":"2024-09-10T07:52:02.586567+0000","last_undegraded":"2024-09-10T07:52:02.586567+0000","last_fullsized":"2024-09-10T07:52:02.586567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:38.130494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607854+0000","last_change":"2024-09-10T07:52:01.586075+0000","last_active":"2024-09-10T07:52:01.607854+0000","last_peered":"2024-09-10T07:52:01.607854+0000","last_clean":"2024-09-10T07:52:01.607854+0000","last_became_active":"2024-09-10T07:52:01.585724+0000","last_became_peered":"2024-09-10T07:52:01.585724+0000","last_unstale":"2024-09-10T07:52:01.607854+0000","last_undegraded":"2024-09-10T07:52:01.607854+0000","last_fullsized":"2024-09-10T07:52:01.607854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604084+0000","last_change":"2024-09-10T07:52:01.598605+0000","last_active":"2024-09-10T07:52:01.604084+0000","last_peered":"2024-09-10T07:52:01.604084+0000","last_clean":"2024-09-10T07:52:01.604084+0000","last_became_active":"2024-09-10T07:52:01.596593+0000","last_became_peered":"2024-09-10T07:52:01.596593+0000","last_unstale":"2024-09-10T07:52:01.604084+0000","last_undegraded":"2024-09-10T07:52:01.604084+0000","last_fullsized":"2024-09-10T07:52:01.604084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:15.935044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572822+0000","last_change":"2024-09-10T07:52:00.572822+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572822+0000","last_undegraded":"2024-09-10T07:52:00.572822+0000","last_fullsized":"2024-09-10T07:52:00.572822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577424+0000","last_change":"2024-09-10T07:52:00.577424+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577424+0000","last_undegraded":"2024-09-10T07:52:00.577424+0000","last_fullsized":"2024-09-10T07:52:00.577424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578130+0000","last_change":"2024-09-10T07:52:00.578130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578130+0000","last_undegraded":"2024-09-10T07:52:00.578130+0000","last_fullsized":"2024-09-10T07:52:00.578130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579577+0000","last_change":"2024-09-10T07:52:00.579577+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579577+0000","last_undegraded":"2024-09-10T07:52:00.579577+0000","last_fullsized":"2024-09-10T07:52:00.579577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578055+0000","last_change":"2024-09-10T07:52:00.578055+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578055+0000","last_undegraded":"2024-09-10T07:52:00.578055+0000","last_fullsized":"2024-09-10T07:52:00.578055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603961+0000","last_change":"2024-09-10T07:52:01.597923+0000","last_active":"2024-09-10T07:52:01.603961+0000","last_peered":"2024-09-10T07:52:01.603961+0000","last_clean":"2024-09-10T07:52:01.603961+0000","last_became_active":"2024-09-10T07:52:01.597283+0000","last_became_peered":"2024-09-10T07:52:01.597283+0000","last_unstale":"2024-09-10T07:52:01.603961+0000","last_undegraded":"2024-09-10T07:52:01.603961+0000","last_fullsized":"2024-09-10T07:52:01.603961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:45.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573077+0000","last_change":"2024-09-10T07:52:00.573077+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573077+0000","last_undegraded":"2024-09-10T07:52:00.573077+0000","last_fullsized":"2024-09-10T07:52:00.573077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571664+0000","last_change":"2024-09-10T07:52:00.571664+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571664+0000","last_undegraded":"2024-09-10T07:52:00.571664+0000","last_fullsized":"2024-09-10T07:52:00.571664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597303+0000","last_change":"2024-09-10T07:52:01.594422+0000","last_active":"2024-09-10T07:52:01.597303+0000","last_peered":"2024-09-10T07:52:01.597303+0000","last_clean":"2024-09-10T07:52:01.597303+0000","last_became_active":"2024-09-10T07:52:01.593648+0000","last_became_peered":"2024-09-10T07:52:01.593648+0000","last_unstale":"2024-09-10T07:52:01.597303+0000","last_undegraded":"2024-09-10T07:52:01.597303+0000","last_fullsized":"2024-09-10T07:52:01.597303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:56.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.592573+0000","last_change":"2024-09-10T07:52:01.588890+0000","last_active":"2024-09-10T07:52:01.592573+0000","last_peered":"2024-09-10T07:52:01.592573+0000","last_clean":"2024-09-10T07:52:01.592573+0000","last_became_active":"2024-09-10T07:52:01.588642+0000","last_became_peered":"2024-09-10T07:52:01.588642+0000","last_unstale":"2024-09-10T07:52:01.592573+0000","last_undegraded":"2024-09-10T07:52:01.592573+0000","last_fullsized":"2024-09-10T07:52:01.592573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:55.090432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604635+0000","last_change":"2024-09-10T07:52:01.598808+0000","last_active":"2024-09-10T07:52:01.604635+0000","last_peered":"2024-09-10T07:52:01.604635+0000","last_clean":"2024-09-10T07:52:01.604635+0000","last_became_active":"2024-09-10T07:52:01.598556+0000","last_became_peered":"2024-09-10T07:52:01.598556+0000","last_unstale":"2024-09-10T07:52:01.604635+0000","last_undegraded":"2024-09-10T07:52:01.604635+0000","last_fullsized":"2024-09-10T07:52:01.604635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:53.560109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579083+0000","last_change":"2024-09-10T07:52:00.579083+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579083+0000","last_undegraded":"2024-09-10T07:52:00.579083+0000","last_fullsized":"2024-09-10T07:52:00.579083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577512+0000","last_change":"2024-09-10T07:52:00.577512+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577512+0000","last_undegraded":"2024-09-10T07:52:00.577512+0000","last_fullsized":"2024-09-10T07:52:00.577512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607922+0000","last_change":"2024-09-10T07:52:01.590994+0000","last_active":"2024-09-10T07:52:01.607922+0000","last_peered":"2024-09-10T07:52:01.607922+0000","last_clean":"2024-09-10T07:52:01.607922+0000","last_became_active":"2024-09-10T07:52:01.588304+0000","last_became_peered":"2024-09-10T07:52:01.588304+0000","last_unstale":"2024-09-10T07:52:01.607922+0000","last_undegraded":"2024-09-10T07:52:01.607922+0000","last_fullsized":"2024-09-10T07:52:01.607922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:48.624562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609278+0000","last_change":"2024-09-10T07:52:01.598304+0000","last_active":"2024-09-10T07:52:01.609278+0000","last_peered":"2024-09-10T07:52:01.609278+0000","last_clean":"2024-09-10T07:52:01.609278+0000","last_became_active":"2024-09-10T07:52:01.597760+0000","last_became_peered":"2024-09-10T07:52:01.597760+0000","last_unstale":"2024-09-10T07:52:01.609278+0000","last_undegraded":"2024-09-10T07:52:01.609278+0000","last_fullsized":"2024-09-10T07:52:01.609278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:36.143408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606093+0000","last_change":"2024-09-10T07:52:01.602531+0000","last_active":"2024-09-10T07:52:01.606093+0000","last_peered":"2024-09-10T07:52:01.606093+0000","last_clean":"2024-09-10T07:52:01.606093+0000","last_became_active":"2024-09-10T07:52:01.599463+0000","last_became_peered":"2024-09-10T07:52:01.599463+0000","last_unstale":"2024-09-10T07:52:01.606093+0000","last_undegraded":"2024-09-10T07:52:01.606093+0000","last_fullsized":"2024-09-10T07:52:01.606093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:14.042212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617284+0000","last_change":"2024-09-10T07:52:00.595443+0000","last_active":"2024-09-10T07:52:01.617284+0000","last_peered":"2024-09-10T07:52:01.617284+0000","last_clean":"2024-09-10T07:52:01.617284+0000","last_became_active":"2024-09-10T07:52:00.595138+0000","last_became_peered":"2024-09-10T07:52:00.595138+0000","last_unstale":"2024-09-10T07:52:01.617284+0000","last_undegraded":"2024-09-10T07:52:01.617284+0000","last_fullsized":"2024-09-10T07:52:01.617284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:11.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597386+0000","last_change":"2024-09-10T07:52:01.591760+0000","last_active":"2024-09-10T07:52:01.597386+0000","last_peered":"2024-09-10T07:52:01.597386+0000","last_clean":"2024-09-10T07:52:01.597386+0000","last_became_active":"2024-09-10T07:52:01.591343+0000","last_became_peered":"2024-09-10T07:52:01.591343+0000","last_unstale":"2024-09-10T07:52:01.597386+0000","last_undegraded":"2024-09-10T07:52:01.597386+0000","last_fullsized":"2024-09-10T07:52:01.597386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:52.869545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.612630+0000","last_change":"2024-09-10T07:52:00.586319+0000","last_active":"2024-09-10T07:52:01.612630+0000","last_peered":"2024-09-10T07:52:01.612630+0000","last_clean":"2024-09-10T07:52:01.612630+0000","last_became_active":"2024-09-10T07:52:00.586031+0000","last_became_peered":"2024-09-10T07:52:00.586031+0000","last_unstale":"2024-09-10T07:52:01.612630+0000","last_undegraded":"2024-09-10T07:52:01.612630+0000","last_fullsized":"2024-09-10T07:52:01.612630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.455287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604296+0000","last_change":"2024-09-10T07:52:01.596781+0000","last_active":"2024-09-10T07:52:01.604296+0000","last_peered":"2024-09-10T07:52:01.604296+0000","last_clean":"2024-09-10T07:52:01.604296+0000","last_became_active":"2024-09-10T07:52:01.596506+0000","last_became_peered":"2024-09-10T07:52:01.596506+0000","last_unstale":"2024-09-10T07:52:01.604296+0000","last_undegraded":"2024-09-10T07:52:01.604296+0000","last_fullsized":"2024-09-10T07:52:01.604296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:01.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586616+0000","last_change":"2024-09-10T07:52:00.583729+0000","last_active":"2024-09-10T07:52:00.586616+0000","last_peered":"2024-09-10T07:52:00.586616+0000","last_clean":"2024-09-10T07:52:00.586616+0000","last_became_active":"2024-09-10T07:52:00.582640+0000","last_became_peered":"2024-09-10T07:52:00.582640+0000","last_unstale":"2024-09-10T07:52:00.586616+0000","last_undegraded":"2024-09-10T07:52:00.586616+0000","last_fullsized":"2024-09-10T07:52:00.586616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597622+0000","last_change":"2024-09-10T07:52:01.590956+0000","last_active":"2024-09-10T07:52:01.597622+0000","last_peered":"2024-09-10T07:52:01.597622+0000","last_clean":"2024-09-10T07:52:01.597622+0000","last_became_active":"2024-09-10T07:52:01.590681+0000","last_became_peered":"2024-09-10T07:52:01.590681+0000","last_unstale":"2024-09-10T07:52:01.597622+0000","last_undegraded":"2024-09-10T07:52:01.597622+0000","last_fullsized":"2024-09-10T07:52:01.597622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:08.691480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.598631+0000","last_change":"2024-09-10T07:52:00.597568+0000","last_active":"2024-09-10T07:52:00.598631+0000","last_peered":"2024-09-10T07:52:00.598631+0000","last_clean":"2024-09-10T07:52:00.598631+0000","last_became_active":"2024-09-10T07:52:00.597236+0000","last_became_peered":"2024-09-10T07:52:00.597236+0000","last_unstale":"2024-09-10T07:52:00.598631+0000","last_undegraded":"2024-09-10T07:52:00.598631+0000","last_fullsized":"2024-09-10T07:52:00.598631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:59.599432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.587100+0000","last_change":"2024-09-10T07:52:00.587100+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.587100+0000","last_undegraded":"2024-09-10T07:52:00.587100+0000","last_fullsized":"2024-09-10T07:52:00.587100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.611712+0000","last_change":"2024-09-10T07:52:00.590175+0000","last_active":"2024-09-10T07:52:01.611712+0000","last_peered":"2024-09-10T07:52:01.611712+0000","last_clean":"2024-09-10T07:52:01.611712+0000","last_became_active":"2024-09-10T07:52:00.589851+0000","last_became_peered":"2024-09-10T07:52:00.589851+0000","last_unstale":"2024-09-10T07:52:01.611712+0000","last_undegraded":"2024-09-10T07:52:01.611712+0000","last_fullsized":"2024-09-10T07:52:01.611712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:05.625972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.606112+0000","last_change":"2024-09-10T07:52:01.587513+0000","last_active":"2024-09-10T07:52:01.606112+0000","last_peered":"2024-09-10T07:52:01.606112+0000","last_clean":"2024-09-10T07:52:01.606112+0000","last_became_active":"2024-09-10T07:52:01.585495+0000","last_became_peered":"2024-09-10T07:52:01.585495+0000","last_unstale":"2024-09-10T07:52:01.606112+0000","last_undegraded":"2024-09-10T07:52:01.606112+0000","last_fullsized":"2024-09-10T07:52:01.606112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609541+0000","last_change":"2024-09-10T07:52:00.593997+0000","last_active":"2024-09-10T07:52:01.609541+0000","last_peered":"2024-09-10T07:52:01.609541+0000","last_clean":"2024-09-10T07:52:01.609541+0000","last_became_active":"2024-09-10T07:52:00.593699+0000","last_became_peered":"2024-09-10T07:52:00.593699+0000","last_unstale":"2024-09-10T07:52:01.609541+0000","last_undegraded":"2024-09-10T07:52:01.609541+0000","last_fullsized":"2024-09-10T07:52:01.609541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.601520+0000","last_change":"2024-09-10T07:52:01.597393+0000","last_active":"2024-09-10T07:52:01.601520+0000","last_peered":"2024-09-10T07:52:01.601520+0000","last_clean":"2024-09-10T07:52:01.601520+0000","last_became_active":"2024-09-10T07:52:01.596883+0000","last_became_peered":"2024-09-10T07:52:01.596883+0000","last_unstale":"2024-09-10T07:52:01.601520+0000","last_undegraded":"2024-09-10T07:52:01.601520+0000","last_fullsized":"2024-09-10T07:52:01.601520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:25.530868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594622+0000","last_change":"2024-09-10T07:52:00.592555+0000","last_active":"2024-09-10T07:52:00.594622+0000","last_peered":"2024-09-10T07:52:00.594622+0000","last_clean":"2024-09-10T07:52:00.594622+0000","last_became_active":"2024-09-10T07:52:00.592274+0000","last_became_peered":"2024-09-10T07:52:00.592274+0000","last_unstale":"2024-09-10T07:52:00.594622+0000","last_undegraded":"2024-09-10T07:52:00.594622+0000","last_fullsized":"2024-09-10T07:52:00.594622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:20.328875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.577375+0000","last_change":"2024-09-10T07:52:00.577375+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.577375+0000","last_undegraded":"2024-09-10T07:52:00.577375+0000","last_fullsized":"2024-09-10T07:52:00.577375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594746+0000","last_change":"2024-09-10T07:52:00.592915+0000","last_active":"2024-09-10T07:52:00.594746+0000","last_peered":"2024-09-10T07:52:00.594746+0000","last_clean":"2024-09-10T07:52:00.594746+0000","last_became_active":"2024-09-10T07:52:00.592202+0000","last_became_peered":"2024-09-10T07:52:00.592202+0000","last_unstale":"2024-09-10T07:52:00.594746+0000","last_undegraded":"2024-09-10T07:52:00.594746+0000","last_fullsized":"2024-09-10T07:52:00.594746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:52.317515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579928+0000","last_change":"2024-09-10T07:52:00.579928+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579928+0000","last_undegraded":"2024-09-10T07:52:00.579928+0000","last_fullsized":"2024-09-10T07:52:00.579928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582318+0000","last_change":"2024-09-10T07:52:00.589330+0000","last_active":"2024-09-10T07:52:01.582318+0000","last_peered":"2024-09-10T07:52:01.582318+0000","last_clean":"2024-09-10T07:52:01.582318+0000","last_became_active":"2024-09-10T07:52:00.589030+0000","last_became_peered":"2024-09-10T07:52:00.589030+0000","last_unstale":"2024-09-10T07:52:01.582318+0000","last_undegraded":"2024-09-10T07:52:01.582318+0000","last_fullsized":"2024-09-10T07:52:01.582318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597425+0000","last_change":"2024-09-10T07:52:01.591984+0000","last_active":"2024-09-10T07:52:01.597425+0000","last_peered":"2024-09-10T07:52:01.597425+0000","last_clean":"2024-09-10T07:52:01.597425+0000","last_became_active":"2024-09-10T07:52:01.591705+0000","last_became_peered":"2024-09-10T07:52:01.591705+0000","last_unstale":"2024-09-10T07:52:01.597425+0000","last_undegraded":"2024-09-10T07:52:01.597425+0000","last_fullsized":"2024-09-10T07:52:01.597425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:34.732740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573495+0000","last_change":"2024-09-10T07:52:00.593909+0000","last_active":"2024-09-10T07:52:01.573495+0000","last_peered":"2024-09-10T07:52:01.573495+0000","last_clean":"2024-09-10T07:52:01.573495+0000","last_became_active":"2024-09-10T07:52:00.593481+0000","last_became_peered":"2024-09-10T07:52:00.593481+0000","last_unstale":"2024-09-10T07:52:01.573495+0000","last_undegraded":"2024-09-10T07:52:01.573495+0000","last_fullsized":"2024-09-10T07:52:01.573495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:00.917860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.573300+0000","last_change":"2024-09-10T07:52:00.573300+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.573300+0000","last_undegraded":"2024-09-10T07:52:00.573300+0000","last_fullsized":"2024-09-10T07:52:00.573300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.615265+0000","last_change":"2024-09-10T07:52:00.581436+0000","last_active":"2024-09-10T07:52:01.615265+0000","last_peered":"2024-09-10T07:52:01.615265+0000","last_clean":"2024-09-10T07:52:01.615265+0000","last_became_active":"2024-09-10T07:52:00.581184+0000","last_became_peered":"2024-09-10T07:52:00.581184+0000","last_unstale":"2024-09-10T07:52:01.615265+0000","last_undegraded":"2024-09-10T07:52:01.615265+0000","last_fullsized":"2024-09-10T07:52:01.615265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608252+0000","last_change":"2024-09-10T07:52:01.598803+0000","last_active":"2024-09-10T07:52:01.608252+0000","last_peered":"2024-09-10T07:52:01.608252+0000","last_clean":"2024-09-10T07:52:01.608252+0000","last_became_active":"2024-09-10T07:52:01.598546+0000","last_became_peered":"2024-09-10T07:52:01.598546+0000","last_unstale":"2024-09-10T07:52:01.608252+0000","last_undegraded":"2024-09-10T07:52:01.608252+0000","last_fullsized":"2024-09-10T07:52:01.608252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.868625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617578+0000","last_change":"2024-09-10T07:52:00.593008+0000","last_active":"2024-09-10T07:52:01.617578+0000","last_peered":"2024-09-10T07:52:01.617578+0000","last_clean":"2024-09-10T07:52:01.617578+0000","last_became_active":"2024-09-10T07:52:00.592734+0000","last_became_peered":"2024-09-10T07:52:00.592734+0000","last_unstale":"2024-09-10T07:52:01.617578+0000","last_undegraded":"2024-09-10T07:52:01.617578+0000","last_fullsized":"2024-09-10T07:52:01.617578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572402+0000","last_change":"2024-09-10T07:52:00.572402+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572402+0000","last_undegraded":"2024-09-10T07:52:00.572402+0000","last_fullsized":"2024-09-10T07:52:00.572402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.594717+0000","last_change":"2024-09-10T07:52:00.591803+0000","last_active":"2024-09-10T07:52:00.594717+0000","last_peered":"2024-09-10T07:52:00.594717+0000","last_clean":"2024-09-10T07:52:00.594717+0000","last_became_active":"2024-09-10T07:52:00.591495+0000","last_became_peered":"2024-09-10T07:52:00.591495+0000","last_unstale":"2024-09-10T07:52:00.594717+0000","last_undegraded":"2024-09-10T07:52:00.594717+0000","last_fullsized":"2024-09-10T07:52:00.594717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:07.100330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602204+0000","last_change":"2024-09-10T07:52:01.597959+0000","last_active":"2024-09-10T07:52:01.602204+0000","last_peered":"2024-09-10T07:52:01.602204+0000","last_clean":"2024-09-10T07:52:01.602204+0000","last_became_active":"2024-09-10T07:52:01.597689+0000","last_became_peered":"2024-09-10T07:52:01.597689+0000","last_unstale":"2024-09-10T07:52:01.602204+0000","last_undegraded":"2024-09-10T07:52:01.602204+0000","last_fullsized":"2024-09-10T07:52:01.602204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:21.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586638+0000","last_change":"2024-09-10T07:52:00.596444+0000","last_active":"2024-09-10T07:52:02.586638+0000","last_peered":"2024-09-10T07:52:02.586638+0000","last_clean":"2024-09-10T07:52:02.586638+0000","last_became_active":"2024-09-10T07:52:00.596005+0000","last_became_peered":"2024-09-10T07:52:00.596005+0000","last_unstale":"2024-09-10T07:52:02.586638+0000","last_undegraded":"2024-09-10T07:52:02.586638+0000","last_fullsized":"2024-09-10T07:52:02.586638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:45.516907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.579130+0000","last_change":"2024-09-10T07:52:00.579130+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.579130+0000","last_undegraded":"2024-09-10T07:52:00.579130+0000","last_fullsized":"2024-09-10T07:52:00.579130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572551+0000","last_change":"2024-09-10T07:52:00.596629+0000","last_active":"2024-09-10T07:52:01.572551+0000","last_peered":"2024-09-10T07:52:01.572551+0000","last_clean":"2024-09-10T07:52:01.572551+0000","last_became_active":"2024-09-10T07:52:00.595725+0000","last_became_peered":"2024-09-10T07:52:00.595725+0000","last_unstale":"2024-09-10T07:52:01.572551+0000","last_undegraded":"2024-09-10T07:52:01.572551+0000","last_fullsized":"2024-09-10T07:52:01.572551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571585+0000","last_change":"2024-09-10T07:52:00.571585+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571585+0000","last_undegraded":"2024-09-10T07:52:00.571585+0000","last_fullsized":"2024-09-10T07:52:00.571585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597120+0000","last_change":"2024-09-10T07:52:00.595747+0000","last_active":"2024-09-10T07:52:00.597120+0000","last_peered":"2024-09-10T07:52:00.597120+0000","last_clean":"2024-09-10T07:52:00.597120+0000","last_became_active":"2024-09-10T07:52:00.595362+0000","last_became_peered":"2024-09-10T07:52:00.595362+0000","last_unstale":"2024-09-10T07:52:00.597120+0000","last_undegraded":"2024-09-10T07:52:00.597120+0000","last_fullsized":"2024-09-10T07:52:00.597120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.604031+0000","last_change":"2024-09-10T07:52:01.595164+0000","last_active":"2024-09-10T07:52:01.604031+0000","last_peered":"2024-09-10T07:52:01.604031+0000","last_clean":"2024-09-10T07:52:01.604031+0000","last_became_active":"2024-09-10T07:52:01.594830+0000","last_became_peered":"2024-09-10T07:52:01.594830+0000","last_unstale":"2024-09-10T07:52:01.604031+0000","last_undegraded":"2024-09-10T07:52:01.604031+0000","last_fullsized":"2024-09-10T07:52:01.604031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:42.789252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.617396+0000","last_change":"2024-09-10T07:52:00.596061+0000","last_active":"2024-09-10T07:52:01.617396+0000","last_peered":"2024-09-10T07:52:01.617396+0000","last_clean":"2024-09-10T07:52:01.617396+0000","last_became_active":"2024-09-10T07:52:00.595743+0000","last_became_peered":"2024-09-10T07:52:00.595743+0000","last_unstale":"2024-09-10T07:52:01.617396+0000","last_undegraded":"2024-09-10T07:52:01.617396+0000","last_fullsized":"2024-09-10T07:52:01.617396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:29.669564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.608352+0000","last_change":"2024-09-10T07:52:01.594958+0000","last_active":"2024-09-10T07:52:01.608352+0000","last_peered":"2024-09-10T07:52:01.608352+0000","last_clean":"2024-09-10T07:52:01.608352+0000","last_became_active":"2024-09-10T07:52:01.594693+0000","last_became_peered":"2024-09-10T07:52:01.594693+0000","last_unstale":"2024-09-10T07:52:01.608352+0000","last_undegraded":"2024-09-10T07:52:01.608352+0000","last_fullsized":"2024-09-10T07:52:01.608352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:22.172497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.599721+0000","last_change":"2024-09-10T07:52:00.598950+0000","last_active":"2024-09-10T07:52:00.599721+0000","last_peered":"2024-09-10T07:52:00.599721+0000","last_clean":"2024-09-10T07:52:00.599721+0000","last_became_active":"2024-09-10T07:52:00.598731+0000","last_became_peered":"2024-09-10T07:52:00.598731+0000","last_unstale":"2024-09-10T07:52:00.599721+0000","last_undegraded":"2024-09-10T07:52:00.599721+0000","last_fullsized":"2024-09-10T07:52:00.599721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600611+0000","last_change":"2024-09-10T07:52:01.592487+0000","last_active":"2024-09-10T07:52:01.600611+0000","last_peered":"2024-09-10T07:52:01.600611+0000","last_clean":"2024-09-10T07:52:01.600611+0000","last_became_active":"2024-09-10T07:52:01.590383+0000","last_became_peered":"2024-09-10T07:52:01.590383+0000","last_unstale":"2024-09-10T07:52:01.600611+0000","last_undegraded":"2024-09-10T07:52:01.600611+0000","last_fullsized":"2024-09-10T07:52:01.600611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.597893+0000","last_change":"2024-09-10T07:52:00.595658+0000","last_active":"2024-09-10T07:52:00.597893+0000","last_peered":"2024-09-10T07:52:00.597893+0000","last_clean":"2024-09-10T07:52:00.597893+0000","last_became_active":"2024-09-10T07:52:00.595374+0000","last_became_peered":"2024-09-10T07:52:00.595374+0000","last_unstale":"2024-09-10T07:52:00.597893+0000","last_undegraded":"2024-09-10T07:52:00.597893+0000","last_fullsized":"2024-09-10T07:52:00.597893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:36:08.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586631+0000","last_change":"2024-09-10T07:52:01.588191+0000","last_active":"2024-09-10T07:52:02.586631+0000","last_peered":"2024-09-10T07:52:02.586631+0000","last_clean":"2024-09-10T07:52:02.586631+0000","last_became_active":"2024-09-10T07:52:01.587568+0000","last_became_peered":"2024-09-10T07:52:01.587568+0000","last_unstale":"2024-09-10T07:52:02.586631+0000","last_undegraded":"2024-09-10T07:52:02.586631+0000","last_fullsized":"2024-09-10T07:52:02.586631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:28.983805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.590896+0000","last_change":"2024-09-10T07:52:00.586945+0000","last_active":"2024-09-10T07:52:00.590896+0000","last_peered":"2024-09-10T07:52:00.590896+0000","last_clean":"2024-09-10T07:52:00.590896+0000","last_became_active":"2024-09-10T07:52:00.586368+0000","last_became_peered":"2024-09-10T07:52:00.586368+0000","last_unstale":"2024-09-10T07:52:00.590896+0000","last_undegraded":"2024-09-10T07:52:00.590896+0000","last_fullsized":"2024-09-10T07:52:00.590896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:31.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.607474+0000","last_change":"2024-09-10T07:52:01.596817+0000","last_active":"2024-09-10T07:52:01.607474+0000","last_peered":"2024-09-10T07:52:01.607474+0000","last_clean":"2024-09-10T07:52:01.607474+0000","last_became_active":"2024-09-10T07:52:01.595480+0000","last_became_peered":"2024-09-10T07:52:01.595480+0000","last_unstale":"2024-09-10T07:52:01.607474+0000","last_undegraded":"2024-09-10T07:52:01.607474+0000","last_fullsized":"2024-09-10T07:52:01.607474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:38.832201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586749+0000","last_change":"2024-09-10T07:52:00.590479+0000","last_active":"2024-09-10T07:52:02.586749+0000","last_peered":"2024-09-10T07:52:02.586749+0000","last_clean":"2024-09-10T07:52:02.586749+0000","last_became_active":"2024-09-10T07:52:00.589855+0000","last_became_peered":"2024-09-10T07:52:00.589855+0000","last_unstale":"2024-09-10T07:52:02.586749+0000","last_undegraded":"2024-09-10T07:52:02.586749+0000","last_fullsized":"2024-09-10T07:52:02.586749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:34.433999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.578558+0000","last_change":"2024-09-10T07:52:00.578558+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.578558+0000","last_undegraded":"2024-09-10T07:52:00.578558+0000","last_fullsized":"2024-09-10T07:52:00.578558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.586935+0000","last_change":"2024-09-10T07:52:00.584260+0000","last_active":"2024-09-10T07:52:00.586935+0000","last_peered":"2024-09-10T07:52:00.586935+0000","last_clean":"2024-09-10T07:52:00.586935+0000","last_became_active":"2024-09-10T07:52:00.583375+0000","last_became_peered":"2024-09-10T07:52:00.583375+0000","last_unstale":"2024-09-10T07:52:00.586935+0000","last_undegraded":"2024-09-10T07:52:00.586935+0000","last_fullsized":"2024-09-10T07:52:00.586935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.603982+0000","last_change":"2024-09-10T07:52:01.598146+0000","last_active":"2024-09-10T07:52:01.603982+0000","last_peered":"2024-09-10T07:52:01.603982+0000","last_clean":"2024-09-10T07:52:01.603982+0000","last_became_active":"2024-09-10T07:52:01.597673+0000","last_became_peered":"2024-09-10T07:52:01.597673+0000","last_unstale":"2024-09-10T07:52:01.603982+0000","last_undegraded":"2024-09-10T07:52:01.603982+0000","last_fullsized":"2024-09-10T07:52:01.603982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.593922+0000","last_change":"2024-09-10T07:52:00.592001+0000","last_active":"2024-09-10T07:52:00.593922+0000","last_peered":"2024-09-10T07:52:00.593922+0000","last_clean":"2024-09-10T07:52:00.593922+0000","last_became_active":"2024-09-10T07:52:00.590320+0000","last_became_peered":"2024-09-10T07:52:00.590320+0000","last_unstale":"2024-09-10T07:52:00.593922+0000","last_undegraded":"2024-09-10T07:52:00.593922+0000","last_fullsized":"2024-09-10T07:52:00.593922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:37.783344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.597324+0000","last_change":"2024-09-10T07:52:01.594864+0000","last_active":"2024-09-10T07:52:01.597324+0000","last_peered":"2024-09-10T07:52:01.597324+0000","last_clean":"2024-09-10T07:52:01.597324+0000","last_became_active":"2024-09-10T07:52:01.594608+0000","last_became_peered":"2024-09-10T07:52:01.594608+0000","last_unstale":"2024-09-10T07:52:01.597324+0000","last_undegraded":"2024-09-10T07:52:01.597324+0000","last_fullsized":"2024-09-10T07:52:01.597324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.077924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.595080+0000","last_change":"2024-09-10T07:52:00.594080+0000","last_active":"2024-09-10T07:52:00.595080+0000","last_peered":"2024-09-10T07:52:00.595080+0000","last_clean":"2024-09-10T07:52:00.595080+0000","last_became_active":"2024-09-10T07:52:00.593703+0000","last_became_peered":"2024-09-10T07:52:00.593703+0000","last_unstale":"2024-09-10T07:52:00.595080+0000","last_undegraded":"2024-09-10T07:52:00.595080+0000","last_fullsized":"2024-09-10T07:52:00.595080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.614133+0000","last_change":"2024-09-10T07:52:01.606931+0000","last_active":"2024-09-10T07:52:01.614133+0000","last_peered":"2024-09-10T07:52:01.614133+0000","last_clean":"2024-09-10T07:52:01.614133+0000","last_became_active":"2024-09-10T07:52:01.606658+0000","last_became_peered":"2024-09-10T07:52:01.606658+0000","last_unstale":"2024-09-10T07:52:01.614133+0000","last_undegraded":"2024-09-10T07:52:01.614133+0000","last_fullsized":"2024-09-10T07:52:01.614133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:48.106660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T07:52:00.582768+0000","last_change":"2024-09-10T07:52:00.580977+0000","last_active":"2024-09-10T07:52:00.582768+0000","last_peered":"2024-09-10T07:52:00.582768+0000","last_clean":"2024-09-10T07:52:00.582768+0000","last_became_active":"2024-09-10T07:52:00.580503+0000","last_became_peered":"2024-09-10T07:52:00.580503+0000","last_unstale":"2024-09-10T07:52:00.582768+0000","last_undegraded":"2024-09-10T07:52:00.582768+0000","last_fullsized":"2024-09-10T07:52:00.582768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.886233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.602646+0000","last_change":"2024-09-10T07:52:01.591171+0000","last_active":"2024-09-10T07:52:01.602646+0000","last_peered":"2024-09-10T07:52:01.602646+0000","last_clean":"2024-09-10T07:52:01.602646+0000","last_became_active":"2024-09-10T07:52:01.590850+0000","last_became_peered":"2024-09-10T07:52:01.590850+0000","last_unstale":"2024-09-10T07:52:01.602646+0000","last_undegraded":"2024-09-10T07:52:01.602646+0000","last_fullsized":"2024-09-10T07:52:01.602646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:24.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.582479+0000","last_change":"2024-09-10T07:52:00.588158+0000","last_active":"2024-09-10T07:52:01.582479+0000","last_peered":"2024-09-10T07:52:01.582479+0000","last_clean":"2024-09-10T07:52:01.582479+0000","last_became_active":"2024-09-10T07:52:00.587861+0000","last_became_peered":"2024-09-10T07:52:00.587861+0000","last_unstale":"2024-09-10T07:52:01.582479+0000","last_undegraded":"2024-09-10T07:52:01.582479+0000","last_fullsized":"2024-09-10T07:52:01.582479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.605798+0000","last_change":"2024-09-10T07:52:01.585615+0000","last_active":"2024-09-10T07:52:01.605798+0000","last_peered":"2024-09-10T07:52:01.605798+0000","last_clean":"2024-09-10T07:52:01.605798+0000","last_became_active":"2024-09-10T07:52:01.585372+0000","last_became_peered":"2024-09-10T07:52:01.585372+0000","last_unstale":"2024-09-10T07:52:01.605798+0000","last_undegraded":"2024-09-10T07:52:01.605798+0000","last_fullsized":"2024-09-10T07:52:01.605798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:35.548692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.573044+0000","last_change":"2024-09-10T07:52:00.586049+0000","last_active":"2024-09-10T07:52:01.573044+0000","last_peered":"2024-09-10T07:52:01.573044+0000","last_clean":"2024-09-10T07:52:01.573044+0000","last_became_active":"2024-09-10T07:52:00.585765+0000","last_became_peered":"2024-09-10T07:52:00.585765+0000","last_unstale":"2024-09-10T07:52:01.573044+0000","last_undegraded":"2024-09-10T07:52:01.573044+0000","last_fullsized":"2024-09-10T07:52:01.573044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:41.786152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.609802+0000","last_change":"2024-09-10T07:52:01.605802+0000","last_active":"2024-09-10T07:52:01.609802+0000","last_peered":"2024-09-10T07:52:01.609802+0000","last_clean":"2024-09-10T07:52:01.609802+0000","last_became_active":"2024-09-10T07:52:01.605500+0000","last_became_peered":"2024-09-10T07:52:01.605500+0000","last_unstale":"2024-09-10T07:52:01.609802+0000","last_undegraded":"2024-09-10T07:52:01.609802+0000","last_fullsized":"2024-09-10T07:52:01.609802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:51.503646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586882+0000","last_change":"2024-09-10T07:52:00.587749+0000","last_active":"2024-09-10T07:52:02.586882+0000","last_peered":"2024-09-10T07:52:02.586882+0000","last_clean":"2024-09-10T07:52:02.586882+0000","last_became_active":"2024-09-10T07:52:00.587486+0000","last_became_peered":"2024-09-10T07:52:00.587486+0000","last_unstale":"2024-09-10T07:52:02.586882+0000","last_undegraded":"2024-09-10T07:52:02.586882+0000","last_fullsized":"2024-09-10T07:52:02.586882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:47.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.572823+0000","last_change":"2024-09-10T07:52:00.572823+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.572823+0000","last_undegraded":"2024-09-10T07:52:00.572823+0000","last_fullsized":"2024-09-10T07:52:00.572823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.572808+0000","last_change":"2024-09-10T07:52:00.593914+0000","last_active":"2024-09-10T07:52:01.572808+0000","last_peered":"2024-09-10T07:52:01.572808+0000","last_clean":"2024-09-10T07:52:01.572808+0000","last_became_active":"2024-09-10T07:52:00.591741+0000","last_became_peered":"2024-09-10T07:52:00.591741+0000","last_unstale":"2024-09-10T07:52:01.572808+0000","last_undegraded":"2024-09-10T07:52:01.572808+0000","last_fullsized":"2024-09-10T07:52:01.572808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:40.147081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.598161+0000","last_change":"2024-09-10T07:52:01.587331+0000","last_active":"2024-09-10T07:52:01.598161+0000","last_peered":"2024-09-10T07:52:01.598161+0000","last_clean":"2024-09-10T07:52:01.598161+0000","last_became_active":"2024-09-10T07:52:01.587019+0000","last_became_peered":"2024-09-10T07:52:01.587019+0000","last_unstale":"2024-09-10T07:52:01.598161+0000","last_undegraded":"2024-09-10T07:52:01.598161+0000","last_fullsized":"2024-09-10T07:52:01.598161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:33.250694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.580356+0000","last_change":"2024-09-10T07:52:00.580768+0000","last_active":"2024-09-10T07:52:01.580356+0000","last_peered":"2024-09-10T07:52:01.580356+0000","last_clean":"2024-09-10T07:52:01.580356+0000","last_became_active":"2024-09-10T07:52:00.580422+0000","last_became_peered":"2024-09-10T07:52:00.580422+0000","last_unstale":"2024-09-10T07:52:01.580356+0000","last_undegraded":"2024-09-10T07:52:01.580356+0000","last_fullsized":"2024-09-10T07:52:01.580356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.600973+0000","last_change":"2024-09-10T07:52:01.593290+0000","last_active":"2024-09-10T07:52:01.600973+0000","last_peered":"2024-09-10T07:52:01.600973+0000","last_clean":"2024-09-10T07:52:01.600973+0000","last_became_active":"2024-09-10T07:52:01.593023+0000","last_became_peered":"2024-09-10T07:52:01.593023+0000","last_unstale":"2024-09-10T07:52:01.600973+0000","last_undegraded":"2024-09-10T07:52:01.600973+0000","last_fullsized":"2024-09-10T07:52:01.600973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:53.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.577180+0000","last_change":"2024-09-10T07:52:00.592995+0000","last_active":"2024-09-10T07:52:01.577180+0000","last_peered":"2024-09-10T07:52:01.577180+0000","last_clean":"2024-09-10T07:52:01.577180+0000","last_became_active":"2024-09-10T07:52:00.592712+0000","last_became_peered":"2024-09-10T07:52:00.592712+0000","last_unstale":"2024-09-10T07:52:01.577180+0000","last_undegraded":"2024-09-10T07:52:01.577180+0000","last_fullsized":"2024-09-10T07:52:01.577180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:40.800624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-09-10T07:52:00.571582+0000","last_change":"2024-09-10T07:52:00.571582+0000","last_active":"2024-09-10T07:52:00.561176+0000","last_peered":"2024-09-10T07:52:00.561176+0000","last_clean":"2024-09-10T07:52:00.561176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:52:00.571582+0000","last_undegraded":"2024-09-10T07:52:00.571582+0000","last_fullsized":"2024-09-10T07:52:00.571582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.616068+0000","last_change":"2024-09-10T07:52:00.592703+0000","last_active":"2024-09-10T07:52:01.616068+0000","last_peered":"2024-09-10T07:52:01.616068+0000","last_clean":"2024-09-10T07:52:01.616068+0000","last_became_active":"2024-09-10T07:52:00.592405+0000","last_became_peered":"2024-09-10T07:52:00.592405+0000","last_unstale":"2024-09-10T07:52:01.616068+0000","last_undegraded":"2024-09-10T07:52:01.616068+0000","last_fullsized":"2024-09-10T07:52:01.616068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:50.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T07:52:01.579327+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:52:01.579327+0000","last_peered":"2024-09-10T07:52:01.579327+0000","last_clean":"2024-09-10T07:52:01.579327+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:52:01.579327+0000","last_undegraded":"2024-09-10T07:52:01.579327+0000","last_fullsized":"2024-09-10T07:52:01.579327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476808,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1069056,"data_stored":459078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":465853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1662976,"data_stored":1049446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476807,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":466525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":452803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1650688,"data_stored":1043171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1085440,"data_stored":465887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:52:10.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:52:10.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.657+0000 7f1066c00640 1 Processor -- start 2024-09-10T07:52:10.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.657+0000 7f1066c00640 1 -- start start 2024-09-10T07:52:10.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.658+0000 7f1066c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.658+0000 7f1066c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.658+0000 7f1066c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600b4250 con 0x7f106015b4f0 2024-09-10T07:52:10.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.658+0000 7f1066c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f10600b44f0 con 0x7f10600d08f0 2024-09-10T07:52:10.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.658+0000 7f1066c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f10600ccd80 con 0x7f106015b110 2024-09-10T07:52:10.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f105fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f1064c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f1064c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:44796/0 (socket says 172.21.15.3:44796) 2024-09-10T07:52:10.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f105fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:57278/0 (socket says 172.21.15.3:57278) 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f1064c00640 1 -- 172.21.15.3:0/2151425311 learned_addr learned my addr 172.21.15.3:0/2151425311 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f105fe00640 1 -- 172.21.15.3:0/2151425311 >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 msgr2=0x7f10600cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f105fe00640 1 --2- 172.21.15.3:0/2151425311 >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.659+0000 7f105fe00640 1 -- 172.21.15.3:0/2151425311 >> v1:172.21.15.70:6789/0 conn(0x7f106015b4f0 legacy=0x7f10600d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.660+0000 7f105fe00640 1 -- 172.21.15.3:0/2151425311 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600cd060 con 0x7f106015b110 2024-09-10T07:52:10.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.660+0000 7f1064c00640 1 --2- 172.21.15.3:0/2151425311 >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f105fe00640 1 --2- 172.21.15.3:0/2151425311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f105000ba40 tx=0x7f105002c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2613d03225446ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f105f400640 1 -- 172.21.15.3:0/2151425311 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050032750 con 0x7f106015b110 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f105f400640 1 -- 172.21.15.3:0/2151425311 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050030050 con 0x7f106015b110 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f105f400640 1 -- 172.21.15.3:0/2151425311 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050032b10 con 0x7f106015b110 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f1066c00640 1 -- 172.21.15.3:0/2151425311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 msgr2=0x7f10600b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f1066c00640 1 --2- 172.21.15.3:0/2151425311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f105000ba40 tx=0x7f105002c850 comp rx=0 tx=0).stop 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f1066c00640 1 -- 172.21.15.3:0/2151425311 shutdown_connections 2024-09-10T07:52:10.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.661+0000 7f1066c00640 1 --2- 172.21.15.3:0/2151425311 >> v2:172.21.15.70:3300/0 conn(0x7f10600d08f0 0x7f10600cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.662+0000 7f1066c00640 1 --2- 172.21.15.3:0/2151425311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f10600b3d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.662+0000 7f1066c00640 1 -- 172.21.15.3:0/2151425311 >> 172.21.15.3:0/2151425311 conn(0x7f106006d000 msgr2=0x7f106006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:10.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.662+0000 7f1066c00640 1 -- 172.21.15.3:0/2151425311 shutdown_connections 2024-09-10T07:52:10.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.662+0000 7f1066c00640 1 -- 172.21.15.3:0/2151425311 wait complete. 2024-09-10T07:52:10.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.663+0000 7f1066c00640 1 Processor -- start 2024-09-10T07:52:10.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.663+0000 7f1066c00640 1 -- start start 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10600d08f0 0x7f106015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f1060159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1060073d10 con 0x7f106015b4f0 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10600cd620 con 0x7f10600d08f0 2024-09-10T07:52:10.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1066c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1060073e90 con 0x7f106015b110 2024-09-10T07:52:10.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.664+0000 7f1064c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59816/0 (socket says 172.21.15.3:59816) 2024-09-10T07:52:10.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 -- 172.21.15.3:0/1686835359 learned_addr learned my addr 172.21.15.3:0/1686835359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:10.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f105fe00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10600d08f0 0x7f106015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1057400640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f1060159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 -- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 msgr2=0x7f1060159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f1060159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 -- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10600d08f0 msgr2=0x7f106015ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10600d08f0 0x7f106015ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 -- 172.21.15.3:0/1686835359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106015d100 con 0x7f106015b4f0 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.665+0000 7f1064c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f105800efc0 tx=0x7f105800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:10.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.666+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105800cbe0 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.666+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105800cd80 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.666+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058003bb0 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.666+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106015d3e0 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.666+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106015d9a0 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.667+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10600b3d10 con 0x7f106015b4f0 2024-09-10T07:52:10.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.668+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1058005d10 con 0x7f106015b4f0 2024-09-10T07:52:10.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.669+0000 7f105cc00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 0x7f103c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:10.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.669+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f105808afe0 con 0x7f106015b4f0 2024-09-10T07:52:10.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.669+0000 7f105fe00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 0x7f103c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:10.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.670+0000 7f105fe00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 0x7f103c06e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1050037000 tx=0x7f1050032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.673+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1058022020 con 0x7f106015b4f0 2024-09-10T07:52:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.855+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10600af6c0 con 0x7f103c06be30 2024-09-10T07:52:10.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.862+0000 7f105cc00640 1 -- 172.21.15.3:0/1686835359 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f10600af6c0 con 0x7f103c06be30 2024-09-10T07:52:10.863 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:10.864 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 msgr2=0x7f103c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 0x7f103c06e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1050037000 tx=0x7f1050032000 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 msgr2=0x7f1060145a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f105800efc0 tx=0x7f105800c490 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 shutdown_connections 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f103c06be30 0x7f103c06e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106015b4f0 0x7f1060145a70 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106015b110 0x7f1060159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 --2- 172.21.15.3:0/1686835359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10600d08f0 0x7f106015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 >> 172.21.15.3:0/1686835359 conn(0x7f106006d000 msgr2=0x7f10600cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 shutdown_connections 2024-09-10T07:52:10.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:10.866+0000 7f1066c00640 1 -- 172.21.15.3:0/1686835359 wait complete. 2024-09-10T07:52:10.881 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T07:52:09.565799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263448,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711272,"statfs":{"total":773094113280,"available":772824342528,"internally_reserved":0,"allocated":10002432,"data_stored":4962780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732575+0000","last_change":"2024-09-10T07:52:01.591976+0000","last_active":"2024-09-10T07:52:03.732575+0000","last_peered":"2024-09-10T07:52:03.732575+0000","last_clean":"2024-09-10T07:52:03.732575+0000","last_became_active":"2024-09-10T07:52:01.591474+0000","last_became_peered":"2024-09-10T07:52:01.591474+0000","last_unstale":"2024-09-10T07:52:03.732575+0000","last_undegraded":"2024-09-10T07:52:03.732575+0000","last_fullsized":"2024-09-10T07:52:03.732575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586917+0000","last_change":"2024-09-10T07:52:01.588364+0000","last_active":"2024-09-10T07:52:02.586917+0000","last_peered":"2024-09-10T07:52:02.586917+0000","last_clean":"2024-09-10T07:52:02.586917+0000","last_became_active":"2024-09-10T07:52:01.585650+0000","last_became_peered":"2024-09-10T07:52:01.585650+0000","last_unstale":"2024-09-10T07:52:02.586917+0000","last_undegraded":"2024-09-10T07:52:02.586917+0000","last_fullsized":"2024-09-10T07:52:02.586917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:00.061836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731509+0000","last_change":"2024-09-10T07:52:01.590826+0000","last_active":"2024-09-10T07:52:03.731509+0000","last_peered":"2024-09-10T07:52:03.731509+0000","last_clean":"2024-09-10T07:52:03.731509+0000","last_became_active":"2024-09-10T07:52:01.590520+0000","last_became_peered":"2024-09-10T07:52:01.590520+0000","last_unstale":"2024-09-10T07:52:03.731509+0000","last_undegraded":"2024-09-10T07:52:03.731509+0000","last_fullsized":"2024-09-10T07:52:03.731509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:09.660081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731824+0000","last_change":"2024-09-10T07:52:01.607606+0000","last_active":"2024-09-10T07:52:03.731824+0000","last_peered":"2024-09-10T07:52:03.731824+0000","last_clean":"2024-09-10T07:52:03.731824+0000","last_became_active":"2024-09-10T07:52:01.606916+0000","last_became_peered":"2024-09-10T07:52:01.606916+0000","last_unstale":"2024-09-10T07:52:03.731824+0000","last_undegraded":"2024-09-10T07:52:03.731824+0000","last_fullsized":"2024-09-10T07:52:03.731824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:23.425588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586244+0000","last_change":"2024-09-10T07:52:01.588987+0000","last_active":"2024-09-10T07:52:02.586244+0000","last_peered":"2024-09-10T07:52:02.586244+0000","last_clean":"2024-09-10T07:52:02.586244+0000","last_became_active":"2024-09-10T07:52:01.588704+0000","last_became_peered":"2024-09-10T07:52:01.588704+0000","last_unstale":"2024-09-10T07:52:02.586244+0000","last_undegraded":"2024-09-10T07:52:02.586244+0000","last_fullsized":"2024-09-10T07:52:02.586244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:05.909981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731239+0000","last_change":"2024-09-10T07:52:01.590162+0000","last_active":"2024-09-10T07:52:03.731239+0000","last_peered":"2024-09-10T07:52:03.731239+0000","last_clean":"2024-09-10T07:52:03.731239+0000","last_became_active":"2024-09-10T07:52:01.585077+0000","last_became_peered":"2024-09-10T07:52:01.585077+0000","last_unstale":"2024-09-10T07:52:03.731239+0000","last_undegraded":"2024-09-10T07:52:03.731239+0000","last_fullsized":"2024-09-10T07:52:03.731239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:15.104496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586493+0000","last_change":"2024-09-10T07:52:01.592098+0000","last_active":"2024-09-10T07:52:02.586493+0000","last_peered":"2024-09-10T07:52:02.586493+0000","last_clean":"2024-09-10T07:52:02.586493+0000","last_became_active":"2024-09-10T07:52:01.591441+0000","last_became_peered":"2024-09-10T07:52:01.591441+0000","last_unstale":"2024-09-10T07:52:02.586493+0000","last_undegraded":"2024-09-10T07:52:02.586493+0000","last_fullsized":"2024-09-10T07:52:02.586493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:03.328092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587494+0000","last_change":"2024-09-10T07:52:01.597132+0000","last_active":"2024-09-10T07:52:02.587494+0000","last_peered":"2024-09-10T07:52:02.587494+0000","last_clean":"2024-09-10T07:52:02.587494+0000","last_became_active":"2024-09-10T07:52:01.596923+0000","last_became_peered":"2024-09-10T07:52:01.596923+0000","last_unstale":"2024-09-10T07:52:02.587494+0000","last_undegraded":"2024-09-10T07:52:02.587494+0000","last_fullsized":"2024-09-10T07:52:02.587494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:09.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732119+0000","last_change":"2024-09-10T07:52:01.591154+0000","last_active":"2024-09-10T07:52:03.732119+0000","last_peered":"2024-09-10T07:52:03.732119+0000","last_clean":"2024-09-10T07:52:03.732119+0000","last_became_active":"2024-09-10T07:52:01.589021+0000","last_became_peered":"2024-09-10T07:52:01.589021+0000","last_unstale":"2024-09-10T07:52:03.732119+0000","last_undegraded":"2024-09-10T07:52:03.732119+0000","last_fullsized":"2024-09-10T07:52:03.732119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:41.803724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731576+0000","last_change":"2024-09-10T07:52:01.588436+0000","last_active":"2024-09-10T07:52:03.731576+0000","last_peered":"2024-09-10T07:52:03.731576+0000","last_clean":"2024-09-10T07:52:03.731576+0000","last_became_active":"2024-09-10T07:52:01.583416+0000","last_became_peered":"2024-09-10T07:52:01.583416+0000","last_unstale":"2024-09-10T07:52:03.731576+0000","last_undegraded":"2024-09-10T07:52:03.731576+0000","last_fullsized":"2024-09-10T07:52:03.731576+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:08.103430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594944+0000","last_change":"2024-09-10T07:52:01.598284+0000","last_active":"2024-09-10T07:52:02.594944+0000","last_peered":"2024-09-10T07:52:02.594944+0000","last_clean":"2024-09-10T07:52:02.594944+0000","last_became_active":"2024-09-10T07:52:01.597991+0000","last_became_peered":"2024-09-10T07:52:01.597991+0000","last_unstale":"2024-09-10T07:52:02.594944+0000","last_undegraded":"2024-09-10T07:52:02.594944+0000","last_fullsized":"2024-09-10T07:52:02.594944+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:51.233119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587724+0000","last_change":"2024-09-10T07:52:01.596178+0000","last_active":"2024-09-10T07:52:02.587724+0000","last_peered":"2024-09-10T07:52:02.587724+0000","last_clean":"2024-09-10T07:52:02.587724+0000","last_became_active":"2024-09-10T07:52:01.595875+0000","last_became_peered":"2024-09-10T07:52:01.595875+0000","last_unstale":"2024-09-10T07:52:02.587724+0000","last_undegraded":"2024-09-10T07:52:02.587724+0000","last_fullsized":"2024-09-10T07:52:02.587724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:36.143408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586402+0000","last_change":"2024-09-10T07:52:01.610337+0000","last_active":"2024-09-10T07:52:02.586402+0000","last_peered":"2024-09-10T07:52:02.586402+0000","last_clean":"2024-09-10T07:52:02.586402+0000","last_became_active":"2024-09-10T07:52:01.609110+0000","last_became_peered":"2024-09-10T07:52:01.609110+0000","last_unstale":"2024-09-10T07:52:02.586402+0000","last_undegraded":"2024-09-10T07:52:02.586402+0000","last_fullsized":"2024-09-10T07:52:02.586402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:07.855161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731422+0000","last_change":"2024-09-10T07:52:01.590435+0000","last_active":"2024-09-10T07:52:03.731422+0000","last_peered":"2024-09-10T07:52:03.731422+0000","last_clean":"2024-09-10T07:52:03.731422+0000","last_became_active":"2024-09-10T07:52:01.589737+0000","last_became_peered":"2024-09-10T07:52:01.589737+0000","last_unstale":"2024-09-10T07:52:03.731422+0000","last_undegraded":"2024-09-10T07:52:03.731422+0000","last_fullsized":"2024-09-10T07:52:03.731422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:35.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586567+0000","last_change":"2024-09-10T07:52:01.591883+0000","last_active":"2024-09-10T07:52:02.586567+0000","last_peered":"2024-09-10T07:52:02.586567+0000","last_clean":"2024-09-10T07:52:02.586567+0000","last_became_active":"2024-09-10T07:52:01.591017+0000","last_became_peered":"2024-09-10T07:52:01.591017+0000","last_unstale":"2024-09-10T07:52:02.586567+0000","last_undegraded":"2024-09-10T07:52:02.586567+0000","last_fullsized":"2024-09-10T07:52:02.586567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:38.130494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730854+0000","last_change":"2024-09-10T07:52:01.586075+0000","last_active":"2024-09-10T07:52:03.730854+0000","last_peered":"2024-09-10T07:52:03.730854+0000","last_clean":"2024-09-10T07:52:03.730854+0000","last_became_active":"2024-09-10T07:52:01.585724+0000","last_became_peered":"2024-09-10T07:52:01.585724+0000","last_unstale":"2024-09-10T07:52:03.730854+0000","last_undegraded":"2024-09-10T07:52:03.730854+0000","last_fullsized":"2024-09-10T07:52:03.730854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:44.690389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587795+0000","last_change":"2024-09-10T07:52:01.598605+0000","last_active":"2024-09-10T07:52:02.587795+0000","last_peered":"2024-09-10T07:52:02.587795+0000","last_clean":"2024-09-10T07:52:02.587795+0000","last_became_active":"2024-09-10T07:52:01.596593+0000","last_became_peered":"2024-09-10T07:52:01.596593+0000","last_unstale":"2024-09-10T07:52:02.587795+0000","last_undegraded":"2024-09-10T07:52:02.587795+0000","last_fullsized":"2024-09-10T07:52:02.587795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:44.690389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594569+0000","last_change":"2024-09-10T07:52:01.595930+0000","last_active":"2024-09-10T07:52:02.594569+0000","last_peered":"2024-09-10T07:52:02.594569+0000","last_clean":"2024-09-10T07:52:02.594569+0000","last_became_active":"2024-09-10T07:52:01.595661+0000","last_became_peered":"2024-09-10T07:52:01.595661+0000","last_unstale":"2024-09-10T07:52:02.594569+0000","last_undegraded":"2024-09-10T07:52:02.594569+0000","last_fullsized":"2024-09-10T07:52:02.594569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:06.937269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732290+0000","last_change":"2024-09-10T07:52:01.590280+0000","last_active":"2024-09-10T07:52:03.732290+0000","last_peered":"2024-09-10T07:52:03.732290+0000","last_clean":"2024-09-10T07:52:03.732290+0000","last_became_active":"2024-09-10T07:52:01.589937+0000","last_became_peered":"2024-09-10T07:52:01.589937+0000","last_unstale":"2024-09-10T07:52:03.732290+0000","last_undegraded":"2024-09-10T07:52:03.732290+0000","last_fullsized":"2024-09-10T07:52:03.732290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:42.789252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732168+0000","last_change":"2024-09-10T07:52:01.591761+0000","last_active":"2024-09-10T07:52:03.732168+0000","last_peered":"2024-09-10T07:52:03.732168+0000","last_clean":"2024-09-10T07:52:03.732168+0000","last_became_active":"2024-09-10T07:52:01.591077+0000","last_became_peered":"2024-09-10T07:52:01.591077+0000","last_unstale":"2024-09-10T07:52:03.732168+0000","last_undegraded":"2024-09-10T07:52:03.732168+0000","last_fullsized":"2024-09-10T07:52:03.732168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:41.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732014+0000","last_change":"2024-09-10T07:52:01.592547+0000","last_active":"2024-09-10T07:52:03.732014+0000","last_peered":"2024-09-10T07:52:03.732014+0000","last_clean":"2024-09-10T07:52:03.732014+0000","last_became_active":"2024-09-10T07:52:01.592302+0000","last_became_peered":"2024-09-10T07:52:01.592302+0000","last_unstale":"2024-09-10T07:52:03.732014+0000","last_undegraded":"2024-09-10T07:52:03.732014+0000","last_fullsized":"2024-09-10T07:52:03.732014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:05.492373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731595+0000","last_change":"2024-09-10T07:52:01.592194+0000","last_active":"2024-09-10T07:52:03.731595+0000","last_peered":"2024-09-10T07:52:03.731595+0000","last_clean":"2024-09-10T07:52:03.731595+0000","last_became_active":"2024-09-10T07:52:01.591925+0000","last_became_peered":"2024-09-10T07:52:01.591925+0000","last_unstale":"2024-09-10T07:52:03.731595+0000","last_undegraded":"2024-09-10T07:52:03.731595+0000","last_fullsized":"2024-09-10T07:52:03.731595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:41.803724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587501+0000","last_change":"2024-09-10T07:52:01.597923+0000","last_active":"2024-09-10T07:52:02.587501+0000","last_peered":"2024-09-10T07:52:02.587501+0000","last_clean":"2024-09-10T07:52:02.587501+0000","last_became_active":"2024-09-10T07:52:01.597283+0000","last_became_peered":"2024-09-10T07:52:01.597283+0000","last_unstale":"2024-09-10T07:52:02.587501+0000","last_undegraded":"2024-09-10T07:52:02.587501+0000","last_fullsized":"2024-09-10T07:52:02.587501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:48.624562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594724+0000","last_change":"2024-09-10T07:52:01.585168+0000","last_active":"2024-09-10T07:52:02.594724+0000","last_peered":"2024-09-10T07:52:02.594724+0000","last_clean":"2024-09-10T07:52:02.594724+0000","last_became_active":"2024-09-10T07:52:01.584898+0000","last_became_peered":"2024-09-10T07:52:01.584898+0000","last_unstale":"2024-09-10T07:52:02.594724+0000","last_undegraded":"2024-09-10T07:52:02.594724+0000","last_fullsized":"2024-09-10T07:52:02.594724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:01.182595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594810+0000","last_change":"2024-09-10T07:52:01.591367+0000","last_active":"2024-09-10T07:52:02.594810+0000","last_peered":"2024-09-10T07:52:02.594810+0000","last_clean":"2024-09-10T07:52:02.594810+0000","last_became_active":"2024-09-10T07:52:01.591132+0000","last_became_peered":"2024-09-10T07:52:01.591132+0000","last_unstale":"2024-09-10T07:52:02.594810+0000","last_undegraded":"2024-09-10T07:52:02.594810+0000","last_fullsized":"2024-09-10T07:52:02.594810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:10.690640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586810+0000","last_change":"2024-09-10T07:52:01.594422+0000","last_active":"2024-09-10T07:52:02.586810+0000","last_peered":"2024-09-10T07:52:02.586810+0000","last_clean":"2024-09-10T07:52:02.586810+0000","last_became_active":"2024-09-10T07:52:01.593648+0000","last_became_peered":"2024-09-10T07:52:01.593648+0000","last_unstale":"2024-09-10T07:52:02.586810+0000","last_undegraded":"2024-09-10T07:52:02.586810+0000","last_fullsized":"2024-09-10T07:52:02.586810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:56.376585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586643+0000","last_change":"2024-09-10T07:52:01.588890+0000","last_active":"2024-09-10T07:52:02.586643+0000","last_peered":"2024-09-10T07:52:02.586643+0000","last_clean":"2024-09-10T07:52:02.586643+0000","last_became_active":"2024-09-10T07:52:01.588642+0000","last_became_peered":"2024-09-10T07:52:01.588642+0000","last_unstale":"2024-09-10T07:52:02.586643+0000","last_undegraded":"2024-09-10T07:52:02.586643+0000","last_fullsized":"2024-09-10T07:52:02.586643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:33.979734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731855+0000","last_change":"2024-09-10T07:52:01.598808+0000","last_active":"2024-09-10T07:52:03.731855+0000","last_peered":"2024-09-10T07:52:03.731855+0000","last_clean":"2024-09-10T07:52:03.731855+0000","last_became_active":"2024-09-10T07:52:01.598556+0000","last_became_peered":"2024-09-10T07:52:01.598556+0000","last_unstale":"2024-09-10T07:52:03.731855+0000","last_undegraded":"2024-09-10T07:52:03.731855+0000","last_fullsized":"2024-09-10T07:52:03.731855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:06.629072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731620+0000","last_change":"2024-09-10T07:52:01.592516+0000","last_active":"2024-09-10T07:52:03.731620+0000","last_peered":"2024-09-10T07:52:03.731620+0000","last_clean":"2024-09-10T07:52:03.731620+0000","last_became_active":"2024-09-10T07:52:01.592222+0000","last_became_peered":"2024-09-10T07:52:01.592222+0000","last_unstale":"2024-09-10T07:52:03.731620+0000","last_undegraded":"2024-09-10T07:52:03.731620+0000","last_fullsized":"2024-09-10T07:52:03.731620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:53.879131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731754+0000","last_change":"2024-09-10T07:52:01.590661+0000","last_active":"2024-09-10T07:52:03.731754+0000","last_peered":"2024-09-10T07:52:03.731754+0000","last_clean":"2024-09-10T07:52:03.731754+0000","last_became_active":"2024-09-10T07:52:01.590149+0000","last_became_peered":"2024-09-10T07:52:01.590149+0000","last_unstale":"2024-09-10T07:52:03.731754+0000","last_undegraded":"2024-09-10T07:52:03.731754+0000","last_fullsized":"2024-09-10T07:52:03.731754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:51.233119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730838+0000","last_change":"2024-09-10T07:52:01.590994+0000","last_active":"2024-09-10T07:52:03.730838+0000","last_peered":"2024-09-10T07:52:03.730838+0000","last_clean":"2024-09-10T07:52:03.730838+0000","last_became_active":"2024-09-10T07:52:01.588304+0000","last_became_peered":"2024-09-10T07:52:01.588304+0000","last_unstale":"2024-09-10T07:52:03.730838+0000","last_undegraded":"2024-09-10T07:52:03.730838+0000","last_fullsized":"2024-09-10T07:52:03.730838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:40.932952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731000+0000","last_change":"2024-09-10T07:52:01.598304+0000","last_active":"2024-09-10T07:52:03.731000+0000","last_peered":"2024-09-10T07:52:03.731000+0000","last_clean":"2024-09-10T07:52:03.731000+0000","last_became_active":"2024-09-10T07:52:01.597760+0000","last_became_peered":"2024-09-10T07:52:01.597760+0000","last_unstale":"2024-09-10T07:52:03.731000+0000","last_undegraded":"2024-09-10T07:52:03.731000+0000","last_fullsized":"2024-09-10T07:52:03.731000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:28.966456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587728+0000","last_change":"2024-09-10T07:52:01.602531+0000","last_active":"2024-09-10T07:52:02.587728+0000","last_peered":"2024-09-10T07:52:02.587728+0000","last_clean":"2024-09-10T07:52:02.587728+0000","last_became_active":"2024-09-10T07:52:01.599463+0000","last_became_peered":"2024-09-10T07:52:01.599463+0000","last_unstale":"2024-09-10T07:52:02.587728+0000","last_undegraded":"2024-09-10T07:52:02.587728+0000","last_fullsized":"2024-09-10T07:52:02.587728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:40.932952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587128+0000","last_change":"2024-09-10T07:52:00.595443+0000","last_active":"2024-09-10T07:52:02.587128+0000","last_peered":"2024-09-10T07:52:02.587128+0000","last_clean":"2024-09-10T07:52:02.587128+0000","last_became_active":"2024-09-10T07:52:00.595138+0000","last_became_peered":"2024-09-10T07:52:00.595138+0000","last_unstale":"2024-09-10T07:52:02.587128+0000","last_undegraded":"2024-09-10T07:52:02.587128+0000","last_fullsized":"2024-09-10T07:52:02.587128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:11.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732267+0000","last_change":"2024-09-10T07:52:01.591760+0000","last_active":"2024-09-10T07:52:03.732267+0000","last_peered":"2024-09-10T07:52:03.732267+0000","last_clean":"2024-09-10T07:52:03.732267+0000","last_became_active":"2024-09-10T07:52:01.591343+0000","last_became_peered":"2024-09-10T07:52:01.591343+0000","last_unstale":"2024-09-10T07:52:03.732267+0000","last_undegraded":"2024-09-10T07:52:03.732267+0000","last_fullsized":"2024-09-10T07:52:03.732267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:10.690640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731695+0000","last_change":"2024-09-10T07:52:00.586319+0000","last_active":"2024-09-10T07:52:03.731695+0000","last_peered":"2024-09-10T07:52:03.731695+0000","last_clean":"2024-09-10T07:52:03.731695+0000","last_became_active":"2024-09-10T07:52:00.586031+0000","last_became_peered":"2024-09-10T07:52:00.586031+0000","last_unstale":"2024-09-10T07:52:03.731695+0000","last_undegraded":"2024-09-10T07:52:03.731695+0000","last_fullsized":"2024-09-10T07:52:03.731695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.455287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587830+0000","last_change":"2024-09-10T07:52:01.596781+0000","last_active":"2024-09-10T07:52:02.587830+0000","last_peered":"2024-09-10T07:52:02.587830+0000","last_clean":"2024-09-10T07:52:02.587830+0000","last_became_active":"2024-09-10T07:52:01.596506+0000","last_became_peered":"2024-09-10T07:52:01.596506+0000","last_unstale":"2024-09-10T07:52:02.587830+0000","last_undegraded":"2024-09-10T07:52:02.587830+0000","last_fullsized":"2024-09-10T07:52:02.587830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:28.966456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594906+0000","last_change":"2024-09-10T07:52:00.583729+0000","last_active":"2024-09-10T07:52:02.594906+0000","last_peered":"2024-09-10T07:52:02.594906+0000","last_clean":"2024-09-10T07:52:02.594906+0000","last_became_active":"2024-09-10T07:52:00.582640+0000","last_became_peered":"2024-09-10T07:52:00.582640+0000","last_unstale":"2024-09-10T07:52:02.594906+0000","last_undegraded":"2024-09-10T07:52:02.594906+0000","last_fullsized":"2024-09-10T07:52:02.594906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:32.247594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731605+0000","last_change":"2024-09-10T07:52:01.590956+0000","last_active":"2024-09-10T07:52:03.731605+0000","last_peered":"2024-09-10T07:52:03.731605+0000","last_clean":"2024-09-10T07:52:03.731605+0000","last_became_active":"2024-09-10T07:52:01.590681+0000","last_became_peered":"2024-09-10T07:52:01.590681+0000","last_unstale":"2024-09-10T07:52:03.731605+0000","last_undegraded":"2024-09-10T07:52:03.731605+0000","last_fullsized":"2024-09-10T07:52:03.731605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:35.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731946+0000","last_change":"2024-09-10T07:52:00.597568+0000","last_active":"2024-09-10T07:52:03.731946+0000","last_peered":"2024-09-10T07:52:03.731946+0000","last_clean":"2024-09-10T07:52:03.731946+0000","last_became_active":"2024-09-10T07:52:00.597236+0000","last_became_peered":"2024-09-10T07:52:00.597236+0000","last_unstale":"2024-09-10T07:52:03.731946+0000","last_undegraded":"2024-09-10T07:52:03.731946+0000","last_fullsized":"2024-09-10T07:52:03.731946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:19.306576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731701+0000","last_change":"2024-09-10T07:52:01.589014+0000","last_active":"2024-09-10T07:52:03.731701+0000","last_peered":"2024-09-10T07:52:03.731701+0000","last_clean":"2024-09-10T07:52:03.731701+0000","last_became_active":"2024-09-10T07:52:01.588616+0000","last_became_peered":"2024-09-10T07:52:01.588616+0000","last_unstale":"2024-09-10T07:52:03.731701+0000","last_undegraded":"2024-09-10T07:52:03.731701+0000","last_fullsized":"2024-09-10T07:52:03.731701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:23.264296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587513+0000","last_change":"2024-09-10T07:52:00.590175+0000","last_active":"2024-09-10T07:52:02.587513+0000","last_peered":"2024-09-10T07:52:02.587513+0000","last_clean":"2024-09-10T07:52:02.587513+0000","last_became_active":"2024-09-10T07:52:00.589851+0000","last_became_peered":"2024-09-10T07:52:00.589851+0000","last_unstale":"2024-09-10T07:52:02.587513+0000","last_undegraded":"2024-09-10T07:52:02.587513+0000","last_fullsized":"2024-09-10T07:52:02.587513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:05.625972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731523+0000","last_change":"2024-09-10T07:52:01.587513+0000","last_active":"2024-09-10T07:52:03.731523+0000","last_peered":"2024-09-10T07:52:03.731523+0000","last_clean":"2024-09-10T07:52:03.731523+0000","last_became_active":"2024-09-10T07:52:01.585495+0000","last_became_peered":"2024-09-10T07:52:01.585495+0000","last_unstale":"2024-09-10T07:52:03.731523+0000","last_undegraded":"2024-09-10T07:52:03.731523+0000","last_fullsized":"2024-09-10T07:52:03.731523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:49.589912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731268+0000","last_change":"2024-09-10T07:52:00.593997+0000","last_active":"2024-09-10T07:52:03.731268+0000","last_peered":"2024-09-10T07:52:03.731268+0000","last_clean":"2024-09-10T07:52:03.731268+0000","last_became_active":"2024-09-10T07:52:00.593699+0000","last_became_peered":"2024-09-10T07:52:00.593699+0000","last_unstale":"2024-09-10T07:52:03.731268+0000","last_undegraded":"2024-09-10T07:52:03.731268+0000","last_fullsized":"2024-09-10T07:52:03.731268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730781+0000","last_change":"2024-09-10T07:52:01.597393+0000","last_active":"2024-09-10T07:52:03.730781+0000","last_peered":"2024-09-10T07:52:03.730781+0000","last_clean":"2024-09-10T07:52:03.730781+0000","last_became_active":"2024-09-10T07:52:01.596883+0000","last_became_peered":"2024-09-10T07:52:01.596883+0000","last_unstale":"2024-09-10T07:52:03.730781+0000","last_undegraded":"2024-09-10T07:52:03.730781+0000","last_fullsized":"2024-09-10T07:52:03.730781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:31.280960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731822+0000","last_change":"2024-09-10T07:52:00.592555+0000","last_active":"2024-09-10T07:52:03.731822+0000","last_peered":"2024-09-10T07:52:03.731822+0000","last_clean":"2024-09-10T07:52:03.731822+0000","last_became_active":"2024-09-10T07:52:00.592274+0000","last_became_peered":"2024-09-10T07:52:00.592274+0000","last_unstale":"2024-09-10T07:52:03.731822+0000","last_undegraded":"2024-09-10T07:52:03.731822+0000","last_fullsized":"2024-09-10T07:52:03.731822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:07.688380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731717+0000","last_change":"2024-09-10T07:52:01.591808+0000","last_active":"2024-09-10T07:52:03.731717+0000","last_peered":"2024-09-10T07:52:03.731717+0000","last_clean":"2024-09-10T07:52:03.731717+0000","last_became_active":"2024-09-10T07:52:01.591373+0000","last_became_peered":"2024-09-10T07:52:01.591373+0000","last_unstale":"2024-09-10T07:52:03.731717+0000","last_undegraded":"2024-09-10T07:52:03.731717+0000","last_fullsized":"2024-09-10T07:52:03.731717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:10.690640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731175+0000","last_change":"2024-09-10T07:52:00.592915+0000","last_active":"2024-09-10T07:52:03.731175+0000","last_peered":"2024-09-10T07:52:03.731175+0000","last_clean":"2024-09-10T07:52:03.731175+0000","last_became_active":"2024-09-10T07:52:00.592202+0000","last_became_peered":"2024-09-10T07:52:00.592202+0000","last_unstale":"2024-09-10T07:52:03.731175+0000","last_undegraded":"2024-09-10T07:52:03.731175+0000","last_fullsized":"2024-09-10T07:52:03.731175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731507+0000","last_change":"2024-09-10T07:52:01.596550+0000","last_active":"2024-09-10T07:52:03.731507+0000","last_peered":"2024-09-10T07:52:03.731507+0000","last_clean":"2024-09-10T07:52:03.731507+0000","last_became_active":"2024-09-10T07:52:01.596262+0000","last_became_peered":"2024-09-10T07:52:01.596262+0000","last_unstale":"2024-09-10T07:52:03.731507+0000","last_undegraded":"2024-09-10T07:52:03.731507+0000","last_fullsized":"2024-09-10T07:52:03.731507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:06.629072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732494+0000","last_change":"2024-09-10T07:52:00.589330+0000","last_active":"2024-09-10T07:52:03.732494+0000","last_peered":"2024-09-10T07:52:03.732494+0000","last_clean":"2024-09-10T07:52:03.732494+0000","last_became_active":"2024-09-10T07:52:00.589030+0000","last_became_peered":"2024-09-10T07:52:00.589030+0000","last_unstale":"2024-09-10T07:52:03.732494+0000","last_undegraded":"2024-09-10T07:52:03.732494+0000","last_fullsized":"2024-09-10T07:52:03.732494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732405+0000","last_change":"2024-09-10T07:52:01.591984+0000","last_active":"2024-09-10T07:52:03.732405+0000","last_peered":"2024-09-10T07:52:03.732405+0000","last_clean":"2024-09-10T07:52:03.732405+0000","last_became_active":"2024-09-10T07:52:01.591705+0000","last_became_peered":"2024-09-10T07:52:01.591705+0000","last_unstale":"2024-09-10T07:52:03.732405+0000","last_undegraded":"2024-09-10T07:52:03.732405+0000","last_fullsized":"2024-09-10T07:52:03.732405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:51.233119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731671+0000","last_change":"2024-09-10T07:52:00.593909+0000","last_active":"2024-09-10T07:52:03.731671+0000","last_peered":"2024-09-10T07:52:03.731671+0000","last_clean":"2024-09-10T07:52:03.731671+0000","last_became_active":"2024-09-10T07:52:00.593481+0000","last_became_peered":"2024-09-10T07:52:00.593481+0000","last_unstale":"2024-09-10T07:52:03.731671+0000","last_undegraded":"2024-09-10T07:52:03.731671+0000","last_fullsized":"2024-09-10T07:52:03.731671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:00.917860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594505+0000","last_change":"2024-09-10T07:52:01.593636+0000","last_active":"2024-09-10T07:52:02.594505+0000","last_peered":"2024-09-10T07:52:02.594505+0000","last_clean":"2024-09-10T07:52:02.594505+0000","last_became_active":"2024-09-10T07:52:01.593360+0000","last_became_peered":"2024-09-10T07:52:01.593360+0000","last_unstale":"2024-09-10T07:52:02.594505+0000","last_undegraded":"2024-09-10T07:52:02.594505+0000","last_fullsized":"2024-09-10T07:52:02.594505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:41.803724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731931+0000","last_change":"2024-09-10T07:52:00.581436+0000","last_active":"2024-09-10T07:52:03.731931+0000","last_peered":"2024-09-10T07:52:03.731931+0000","last_clean":"2024-09-10T07:52:03.731931+0000","last_became_active":"2024-09-10T07:52:00.581184+0000","last_became_peered":"2024-09-10T07:52:00.581184+0000","last_unstale":"2024-09-10T07:52:03.731931+0000","last_undegraded":"2024-09-10T07:52:03.731931+0000","last_fullsized":"2024-09-10T07:52:03.731931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:39.944448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586641+0000","last_change":"2024-09-10T07:52:01.598803+0000","last_active":"2024-09-10T07:52:02.586641+0000","last_peered":"2024-09-10T07:52:02.586641+0000","last_clean":"2024-09-10T07:52:02.586641+0000","last_became_active":"2024-09-10T07:52:01.598546+0000","last_became_peered":"2024-09-10T07:52:01.598546+0000","last_unstale":"2024-09-10T07:52:02.586641+0000","last_undegraded":"2024-09-10T07:52:02.586641+0000","last_fullsized":"2024-09-10T07:52:02.586641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:43.802928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587933+0000","last_change":"2024-09-10T07:52:00.593008+0000","last_active":"2024-09-10T07:52:02.587933+0000","last_peered":"2024-09-10T07:52:02.587933+0000","last_clean":"2024-09-10T07:52:02.587933+0000","last_became_active":"2024-09-10T07:52:00.592734+0000","last_became_peered":"2024-09-10T07:52:00.592734+0000","last_unstale":"2024-09-10T07:52:02.587933+0000","last_undegraded":"2024-09-10T07:52:02.587933+0000","last_fullsized":"2024-09-10T07:52:02.587933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594477+0000","last_change":"2024-09-10T07:52:01.597840+0000","last_active":"2024-09-10T07:52:02.594477+0000","last_peered":"2024-09-10T07:52:02.594477+0000","last_clean":"2024-09-10T07:52:02.594477+0000","last_became_active":"2024-09-10T07:52:01.597536+0000","last_became_peered":"2024-09-10T07:52:01.597536+0000","last_unstale":"2024-09-10T07:52:02.594477+0000","last_undegraded":"2024-09-10T07:52:02.594477+0000","last_fullsized":"2024-09-10T07:52:02.594477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:04:38.412767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731739+0000","last_change":"2024-09-10T07:52:00.591803+0000","last_active":"2024-09-10T07:52:03.731739+0000","last_peered":"2024-09-10T07:52:03.731739+0000","last_clean":"2024-09-10T07:52:03.731739+0000","last_became_active":"2024-09-10T07:52:00.591495+0000","last_became_peered":"2024-09-10T07:52:00.591495+0000","last_unstale":"2024-09-10T07:52:03.731739+0000","last_undegraded":"2024-09-10T07:52:03.731739+0000","last_fullsized":"2024-09-10T07:52:03.731739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:11.012083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731065+0000","last_change":"2024-09-10T07:52:01.597959+0000","last_active":"2024-09-10T07:52:03.731065+0000","last_peered":"2024-09-10T07:52:03.731065+0000","last_clean":"2024-09-10T07:52:03.731065+0000","last_became_active":"2024-09-10T07:52:01.597689+0000","last_became_peered":"2024-09-10T07:52:01.597689+0000","last_unstale":"2024-09-10T07:52:03.731065+0000","last_undegraded":"2024-09-10T07:52:03.731065+0000","last_fullsized":"2024-09-10T07:52:03.731065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:04:38.412767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586638+0000","last_change":"2024-09-10T07:52:00.596444+0000","last_active":"2024-09-10T07:52:02.586638+0000","last_peered":"2024-09-10T07:52:02.586638+0000","last_clean":"2024-09-10T07:52:02.586638+0000","last_became_active":"2024-09-10T07:52:00.596005+0000","last_became_peered":"2024-09-10T07:52:00.596005+0000","last_unstale":"2024-09-10T07:52:02.586638+0000","last_undegraded":"2024-09-10T07:52:02.586638+0000","last_fullsized":"2024-09-10T07:52:02.586638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:45.516907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731280+0000","last_change":"2024-09-10T07:52:01.591043+0000","last_active":"2024-09-10T07:52:03.731280+0000","last_peered":"2024-09-10T07:52:03.731280+0000","last_clean":"2024-09-10T07:52:03.731280+0000","last_became_active":"2024-09-10T07:52:01.590704+0000","last_became_peered":"2024-09-10T07:52:01.590704+0000","last_unstale":"2024-09-10T07:52:03.731280+0000","last_undegraded":"2024-09-10T07:52:03.731280+0000","last_fullsized":"2024-09-10T07:52:03.731280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:01.182595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731069+0000","last_change":"2024-09-10T07:52:00.596629+0000","last_active":"2024-09-10T07:52:03.731069+0000","last_peered":"2024-09-10T07:52:03.731069+0000","last_clean":"2024-09-10T07:52:03.731069+0000","last_became_active":"2024-09-10T07:52:00.595725+0000","last_became_peered":"2024-09-10T07:52:00.595725+0000","last_unstale":"2024-09-10T07:52:03.731069+0000","last_undegraded":"2024-09-10T07:52:03.731069+0000","last_fullsized":"2024-09-10T07:52:03.731069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:09.687540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594354+0000","last_change":"2024-09-10T07:52:01.586968+0000","last_active":"2024-09-10T07:52:02.594354+0000","last_peered":"2024-09-10T07:52:02.594354+0000","last_clean":"2024-09-10T07:52:02.594354+0000","last_became_active":"2024-09-10T07:52:01.585642+0000","last_became_peered":"2024-09-10T07:52:01.585642+0000","last_unstale":"2024-09-10T07:52:02.594354+0000","last_undegraded":"2024-09-10T07:52:02.594354+0000","last_fullsized":"2024-09-10T07:52:02.594354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:04.472929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594638+0000","last_change":"2024-09-10T07:52:00.595747+0000","last_active":"2024-09-10T07:52:02.594638+0000","last_peered":"2024-09-10T07:52:02.594638+0000","last_clean":"2024-09-10T07:52:02.594638+0000","last_became_active":"2024-09-10T07:52:00.595362+0000","last_became_peered":"2024-09-10T07:52:00.595362+0000","last_unstale":"2024-09-10T07:52:02.594638+0000","last_undegraded":"2024-09-10T07:52:02.594638+0000","last_fullsized":"2024-09-10T07:52:02.594638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:26.548837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587871+0000","last_change":"2024-09-10T07:52:01.595164+0000","last_active":"2024-09-10T07:52:02.587871+0000","last_peered":"2024-09-10T07:52:02.587871+0000","last_clean":"2024-09-10T07:52:02.587871+0000","last_became_active":"2024-09-10T07:52:01.594830+0000","last_became_peered":"2024-09-10T07:52:01.594830+0000","last_unstale":"2024-09-10T07:52:02.587871+0000","last_undegraded":"2024-09-10T07:52:02.587871+0000","last_fullsized":"2024-09-10T07:52:02.587871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:15.104496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587209+0000","last_change":"2024-09-10T07:52:00.596061+0000","last_active":"2024-09-10T07:52:02.587209+0000","last_peered":"2024-09-10T07:52:02.587209+0000","last_clean":"2024-09-10T07:52:02.587209+0000","last_became_active":"2024-09-10T07:52:00.595743+0000","last_became_peered":"2024-09-10T07:52:00.595743+0000","last_unstale":"2024-09-10T07:52:02.587209+0000","last_undegraded":"2024-09-10T07:52:02.587209+0000","last_fullsized":"2024-09-10T07:52:02.587209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:29.669564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731781+0000","last_change":"2024-09-10T07:52:01.594958+0000","last_active":"2024-09-10T07:52:03.731781+0000","last_peered":"2024-09-10T07:52:03.731781+0000","last_clean":"2024-09-10T07:52:03.731781+0000","last_became_active":"2024-09-10T07:52:01.594693+0000","last_became_peered":"2024-09-10T07:52:01.594693+0000","last_unstale":"2024-09-10T07:52:03.731781+0000","last_undegraded":"2024-09-10T07:52:03.731781+0000","last_fullsized":"2024-09-10T07:52:03.731781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:20.309676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732288+0000","last_change":"2024-09-10T07:52:00.598950+0000","last_active":"2024-09-10T07:52:03.732288+0000","last_peered":"2024-09-10T07:52:03.732288+0000","last_clean":"2024-09-10T07:52:03.732288+0000","last_became_active":"2024-09-10T07:52:00.598731+0000","last_became_peered":"2024-09-10T07:52:00.598731+0000","last_unstale":"2024-09-10T07:52:03.732288+0000","last_undegraded":"2024-09-10T07:52:03.732288+0000","last_fullsized":"2024-09-10T07:52:03.732288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:52.381143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.588160+0000","last_change":"2024-09-10T07:52:01.592487+0000","last_active":"2024-09-10T07:52:02.588160+0000","last_peered":"2024-09-10T07:52:02.588160+0000","last_clean":"2024-09-10T07:52:02.588160+0000","last_became_active":"2024-09-10T07:52:01.590383+0000","last_became_peered":"2024-09-10T07:52:01.590383+0000","last_unstale":"2024-09-10T07:52:02.588160+0000","last_undegraded":"2024-09-10T07:52:02.588160+0000","last_fullsized":"2024-09-10T07:52:02.588160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:09.660081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732140+0000","last_change":"2024-09-10T07:52:00.595658+0000","last_active":"2024-09-10T07:52:03.732140+0000","last_peered":"2024-09-10T07:52:03.732140+0000","last_clean":"2024-09-10T07:52:03.732140+0000","last_became_active":"2024-09-10T07:52:00.595374+0000","last_became_peered":"2024-09-10T07:52:00.595374+0000","last_unstale":"2024-09-10T07:52:03.732140+0000","last_undegraded":"2024-09-10T07:52:03.732140+0000","last_fullsized":"2024-09-10T07:52:03.732140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:22.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586631+0000","last_change":"2024-09-10T07:52:01.588191+0000","last_active":"2024-09-10T07:52:02.586631+0000","last_peered":"2024-09-10T07:52:02.586631+0000","last_clean":"2024-09-10T07:52:02.586631+0000","last_became_active":"2024-09-10T07:52:01.587568+0000","last_became_peered":"2024-09-10T07:52:01.587568+0000","last_unstale":"2024-09-10T07:52:02.586631+0000","last_undegraded":"2024-09-10T07:52:02.586631+0000","last_fullsized":"2024-09-10T07:52:02.586631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:28.983805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732071+0000","last_change":"2024-09-10T07:52:00.586945+0000","last_active":"2024-09-10T07:52:03.732071+0000","last_peered":"2024-09-10T07:52:03.732071+0000","last_clean":"2024-09-10T07:52:03.732071+0000","last_became_active":"2024-09-10T07:52:00.586368+0000","last_became_peered":"2024-09-10T07:52:00.586368+0000","last_unstale":"2024-09-10T07:52:03.732071+0000","last_undegraded":"2024-09-10T07:52:03.732071+0000","last_fullsized":"2024-09-10T07:52:03.732071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:26.548837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586772+0000","last_change":"2024-09-10T07:52:01.596817+0000","last_active":"2024-09-10T07:52:02.586772+0000","last_peered":"2024-09-10T07:52:02.586772+0000","last_clean":"2024-09-10T07:52:02.586772+0000","last_became_active":"2024-09-10T07:52:01.595480+0000","last_became_peered":"2024-09-10T07:52:01.595480+0000","last_unstale":"2024-09-10T07:52:02.586772+0000","last_undegraded":"2024-09-10T07:52:02.586772+0000","last_fullsized":"2024-09-10T07:52:02.586772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:04:09.903849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586749+0000","last_change":"2024-09-10T07:52:00.590479+0000","last_active":"2024-09-10T07:52:02.586749+0000","last_peered":"2024-09-10T07:52:02.586749+0000","last_clean":"2024-09-10T07:52:02.586749+0000","last_became_active":"2024-09-10T07:52:00.589855+0000","last_became_peered":"2024-09-10T07:52:00.589855+0000","last_unstale":"2024-09-10T07:52:02.586749+0000","last_undegraded":"2024-09-10T07:52:02.586749+0000","last_fullsized":"2024-09-10T07:52:02.586749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:34.433999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731923+0000","last_change":"2024-09-10T07:52:01.594357+0000","last_active":"2024-09-10T07:52:03.731923+0000","last_peered":"2024-09-10T07:52:03.731923+0000","last_clean":"2024-09-10T07:52:03.731923+0000","last_became_active":"2024-09-10T07:52:01.594021+0000","last_became_peered":"2024-09-10T07:52:01.594021+0000","last_unstale":"2024-09-10T07:52:03.731923+0000","last_undegraded":"2024-09-10T07:52:03.731923+0000","last_fullsized":"2024-09-10T07:52:03.731923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:34.468115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594345+0000","last_change":"2024-09-10T07:52:00.584260+0000","last_active":"2024-09-10T07:52:02.594345+0000","last_peered":"2024-09-10T07:52:02.594345+0000","last_clean":"2024-09-10T07:52:02.594345+0000","last_became_active":"2024-09-10T07:52:00.583375+0000","last_became_peered":"2024-09-10T07:52:00.583375+0000","last_unstale":"2024-09-10T07:52:02.594345+0000","last_undegraded":"2024-09-10T07:52:02.594345+0000","last_fullsized":"2024-09-10T07:52:02.594345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:57.478505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587657+0000","last_change":"2024-09-10T07:52:01.598146+0000","last_active":"2024-09-10T07:52:02.587657+0000","last_peered":"2024-09-10T07:52:02.587657+0000","last_clean":"2024-09-10T07:52:02.587657+0000","last_became_active":"2024-09-10T07:52:01.597673+0000","last_became_peered":"2024-09-10T07:52:01.597673+0000","last_unstale":"2024-09-10T07:52:02.587657+0000","last_undegraded":"2024-09-10T07:52:02.587657+0000","last_fullsized":"2024-09-10T07:52:02.587657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:22.172497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731401+0000","last_change":"2024-09-10T07:52:00.592001+0000","last_active":"2024-09-10T07:52:03.731401+0000","last_peered":"2024-09-10T07:52:03.731401+0000","last_clean":"2024-09-10T07:52:03.731401+0000","last_became_active":"2024-09-10T07:52:00.590320+0000","last_became_peered":"2024-09-10T07:52:00.590320+0000","last_unstale":"2024-09-10T07:52:03.731401+0000","last_undegraded":"2024-09-10T07:52:03.731401+0000","last_fullsized":"2024-09-10T07:52:03.731401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:32.247594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586988+0000","last_change":"2024-09-10T07:52:01.594864+0000","last_active":"2024-09-10T07:52:02.586988+0000","last_peered":"2024-09-10T07:52:02.586988+0000","last_clean":"2024-09-10T07:52:02.586988+0000","last_became_active":"2024-09-10T07:52:01.594608+0000","last_became_peered":"2024-09-10T07:52:01.594608+0000","last_unstale":"2024-09-10T07:52:02.586988+0000","last_undegraded":"2024-09-10T07:52:02.586988+0000","last_fullsized":"2024-09-10T07:52:02.586988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:06.172224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.732367+0000","last_change":"2024-09-10T07:52:00.594080+0000","last_active":"2024-09-10T07:52:03.732367+0000","last_peered":"2024-09-10T07:52:03.732367+0000","last_clean":"2024-09-10T07:52:03.732367+0000","last_became_active":"2024-09-10T07:52:00.593703+0000","last_became_peered":"2024-09-10T07:52:00.593703+0000","last_unstale":"2024-09-10T07:52:03.732367+0000","last_undegraded":"2024-09-10T07:52:03.732367+0000","last_fullsized":"2024-09-10T07:52:03.732367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:51.866445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587003+0000","last_change":"2024-09-10T07:52:01.606931+0000","last_active":"2024-09-10T07:52:02.587003+0000","last_peered":"2024-09-10T07:52:02.587003+0000","last_clean":"2024-09-10T07:52:02.587003+0000","last_became_active":"2024-09-10T07:52:01.606658+0000","last_became_peered":"2024-09-10T07:52:01.606658+0000","last_unstale":"2024-09-10T07:52:02.587003+0000","last_undegraded":"2024-09-10T07:52:02.587003+0000","last_fullsized":"2024-09-10T07:52:02.587003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:03.221784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594642+0000","last_change":"2024-09-10T07:52:00.580977+0000","last_active":"2024-09-10T07:52:02.594642+0000","last_peered":"2024-09-10T07:52:02.594642+0000","last_clean":"2024-09-10T07:52:02.594642+0000","last_became_active":"2024-09-10T07:52:00.580503+0000","last_became_peered":"2024-09-10T07:52:00.580503+0000","last_unstale":"2024-09-10T07:52:02.594642+0000","last_undegraded":"2024-09-10T07:52:02.594642+0000","last_fullsized":"2024-09-10T07:52:02.594642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:07.688380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586511+0000","last_change":"2024-09-10T07:52:01.591171+0000","last_active":"2024-09-10T07:52:02.586511+0000","last_peered":"2024-09-10T07:52:02.586511+0000","last_clean":"2024-09-10T07:52:02.586511+0000","last_became_active":"2024-09-10T07:52:01.590850+0000","last_became_peered":"2024-09-10T07:52:01.590850+0000","last_unstale":"2024-09-10T07:52:02.586511+0000","last_undegraded":"2024-09-10T07:52:02.586511+0000","last_fullsized":"2024-09-10T07:52:02.586511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:57.828442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.588207+0000","last_change":"2024-09-10T07:52:00.588158+0000","last_active":"2024-09-10T07:52:02.588207+0000","last_peered":"2024-09-10T07:52:02.588207+0000","last_clean":"2024-09-10T07:52:02.588207+0000","last_became_active":"2024-09-10T07:52:00.587861+0000","last_became_peered":"2024-09-10T07:52:00.587861+0000","last_unstale":"2024-09-10T07:52:02.588207+0000","last_undegraded":"2024-09-10T07:52:02.588207+0000","last_fullsized":"2024-09-10T07:52:02.588207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:50.230019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731671+0000","last_change":"2024-09-10T07:52:01.585615+0000","last_active":"2024-09-10T07:52:03.731671+0000","last_peered":"2024-09-10T07:52:03.731671+0000","last_clean":"2024-09-10T07:52:03.731671+0000","last_became_active":"2024-09-10T07:52:01.585372+0000","last_became_peered":"2024-09-10T07:52:01.585372+0000","last_unstale":"2024-09-10T07:52:03.731671+0000","last_undegraded":"2024-09-10T07:52:03.731671+0000","last_fullsized":"2024-09-10T07:52:03.731671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:27:00.602532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731443+0000","last_change":"2024-09-10T07:52:00.586049+0000","last_active":"2024-09-10T07:52:03.731443+0000","last_peered":"2024-09-10T07:52:03.731443+0000","last_clean":"2024-09-10T07:52:03.731443+0000","last_became_active":"2024-09-10T07:52:00.585765+0000","last_became_peered":"2024-09-10T07:52:00.585765+0000","last_unstale":"2024-09-10T07:52:03.731443+0000","last_undegraded":"2024-09-10T07:52:03.731443+0000","last_fullsized":"2024-09-10T07:52:03.731443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:41.786152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731182+0000","last_change":"2024-09-10T07:52:01.605802+0000","last_active":"2024-09-10T07:52:03.731182+0000","last_peered":"2024-09-10T07:52:03.731182+0000","last_clean":"2024-09-10T07:52:03.731182+0000","last_became_active":"2024-09-10T07:52:01.605500+0000","last_became_peered":"2024-09-10T07:52:01.605500+0000","last_unstale":"2024-09-10T07:52:03.731182+0000","last_undegraded":"2024-09-10T07:52:03.731182+0000","last_fullsized":"2024-09-10T07:52:03.731182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:06.937269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586882+0000","last_change":"2024-09-10T07:52:00.587749+0000","last_active":"2024-09-10T07:52:02.586882+0000","last_peered":"2024-09-10T07:52:02.586882+0000","last_clean":"2024-09-10T07:52:02.586882+0000","last_became_active":"2024-09-10T07:52:00.587486+0000","last_became_peered":"2024-09-10T07:52:00.587486+0000","last_unstale":"2024-09-10T07:52:02.586882+0000","last_undegraded":"2024-09-10T07:52:02.586882+0000","last_fullsized":"2024-09-10T07:52:02.586882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:47.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594820+0000","last_change":"2024-09-10T07:52:01.589454+0000","last_active":"2024-09-10T07:52:02.594820+0000","last_peered":"2024-09-10T07:52:02.594820+0000","last_clean":"2024-09-10T07:52:02.594820+0000","last_became_active":"2024-09-10T07:52:01.589153+0000","last_became_peered":"2024-09-10T07:52:01.589153+0000","last_unstale":"2024-09-10T07:52:02.594820+0000","last_undegraded":"2024-09-10T07:52:02.594820+0000","last_fullsized":"2024-09-10T07:52:02.594820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:59:06.629072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731333+0000","last_change":"2024-09-10T07:52:00.593914+0000","last_active":"2024-09-10T07:52:03.731333+0000","last_peered":"2024-09-10T07:52:03.731333+0000","last_clean":"2024-09-10T07:52:03.731333+0000","last_became_active":"2024-09-10T07:52:00.591741+0000","last_became_peered":"2024-09-10T07:52:00.591741+0000","last_unstale":"2024-09-10T07:52:03.731333+0000","last_undegraded":"2024-09-10T07:52:03.731333+0000","last_fullsized":"2024-09-10T07:52:03.731333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:40.147081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730897+0000","last_change":"2024-09-10T07:52:01.587331+0000","last_active":"2024-09-10T07:52:03.730897+0000","last_peered":"2024-09-10T07:52:03.730897+0000","last_clean":"2024-09-10T07:52:03.730897+0000","last_became_active":"2024-09-10T07:52:01.587019+0000","last_became_peered":"2024-09-10T07:52:01.587019+0000","last_unstale":"2024-09-10T07:52:03.730897+0000","last_undegraded":"2024-09-10T07:52:03.730897+0000","last_fullsized":"2024-09-10T07:52:03.730897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:01.182595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.731001+0000","last_change":"2024-09-10T07:52:00.580768+0000","last_active":"2024-09-10T07:52:03.731001+0000","last_peered":"2024-09-10T07:52:03.731001+0000","last_clean":"2024-09-10T07:52:03.731001+0000","last_became_active":"2024-09-10T07:52:00.580422+0000","last_became_peered":"2024-09-10T07:52:00.580422+0000","last_unstale":"2024-09-10T07:52:03.731001+0000","last_undegraded":"2024-09-10T07:52:03.731001+0000","last_fullsized":"2024-09-10T07:52:03.731001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:27.813784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730907+0000","last_change":"2024-09-10T07:52:01.593290+0000","last_active":"2024-09-10T07:52:03.730907+0000","last_peered":"2024-09-10T07:52:03.730907+0000","last_clean":"2024-09-10T07:52:03.730907+0000","last_became_active":"2024-09-10T07:52:01.593023+0000","last_became_peered":"2024-09-10T07:52:01.593023+0000","last_unstale":"2024-09-10T07:52:03.730907+0000","last_undegraded":"2024-09-10T07:52:03.730907+0000","last_fullsized":"2024-09-10T07:52:03.730907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:04.472929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.587506+0000","last_change":"2024-09-10T07:52:00.592995+0000","last_active":"2024-09-10T07:52:02.587506+0000","last_peered":"2024-09-10T07:52:02.587506+0000","last_clean":"2024-09-10T07:52:02.587506+0000","last_became_active":"2024-09-10T07:52:00.592712+0000","last_became_peered":"2024-09-10T07:52:00.592712+0000","last_unstale":"2024-09-10T07:52:02.587506+0000","last_undegraded":"2024-09-10T07:52:02.587506+0000","last_fullsized":"2024-09-10T07:52:02.587506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:40.800624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.594785+0000","last_change":"2024-09-10T07:52:01.592738+0000","last_active":"2024-09-10T07:52:02.594785+0000","last_peered":"2024-09-10T07:52:02.594785+0000","last_clean":"2024-09-10T07:52:02.594785+0000","last_became_active":"2024-09-10T07:52:01.592461+0000","last_became_peered":"2024-09-10T07:52:01.592461+0000","last_unstale":"2024-09-10T07:52:02.594785+0000","last_undegraded":"2024-09-10T07:52:02.594785+0000","last_fullsized":"2024-09-10T07:52:02.594785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:52:00.561176+0000","last_clean_scrub_stamp":"2024-09-10T07:52:00.561176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:35.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:02.586741+0000","last_change":"2024-09-10T07:52:00.592703+0000","last_active":"2024-09-10T07:52:02.586741+0000","last_peered":"2024-09-10T07:52:02.586741+0000","last_clean":"2024-09-10T07:52:02.586741+0000","last_became_active":"2024-09-10T07:52:00.592405+0000","last_became_peered":"2024-09-10T07:52:00.592405+0000","last_unstale":"2024-09-10T07:52:02.586741+0000","last_undegraded":"2024-09-10T07:52:02.586741+0000","last_fullsized":"2024-09-10T07:52:02.586741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:51:59.558076+0000","last_clean_scrub_stamp":"2024-09-10T07:51:59.558076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:50.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T07:52:03.730889+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:52:03.730889+0000","last_peered":"2024-09-10T07:52:03.730889+0000","last_clean":"2024-09-10T07:52:03.730889+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:52:03.730889+0000","last_undegraded":"2024-09-10T07:52:03.730889+0000","last_fullsized":"2024-09-10T07:52:03.730889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476809,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":472827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476809,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1093632,"data_stored":472153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476807,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1691648,"data_stored":1062611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476808,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":473787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1097728,"data_stored":472223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":472825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1703936,"data_stored":1064167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1097728,"data_stored":472187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:52:10.885 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:52:10.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:52:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.086+0000 7f8c12600640 1 Processor -- start 2024-09-10T07:52:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.086+0000 7f8c12600640 1 -- start start 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c12600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c0c1451d0 0x7f8c0c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c12600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c0c145730 0x7f8c0c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c12600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0cd900 con 0x7f8c0c0cf270 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c12600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c06d7c0 con 0x7f8c0c145730 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c12600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0cdbe0 con 0x7f8c0c1451d0 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c0aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c0c145730 0x7f8c0c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c0aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c0c145730 0x7f8c0c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59818/0 (socket says 172.21.15.3:59818) 2024-09-10T07:52:11.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.087+0000 7f8c0aa00640 1 -- 172.21.15.3:0/3796127552 learned_addr learned my addr 172.21.15.3:0/3796127552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:11.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.088+0000 7f8c0b400640 1 --2- 172.21.15.3:0/3796127552 >> v2:172.21.15.70:3300/0 conn(0x7f8c0c1451d0 0x7f8c0c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.088+0000 7f8c0aa00640 1 -- 172.21.15.3:0/3796127552 >> v2:172.21.15.70:3300/0 conn(0x7f8c0c1451d0 msgr2=0x7f8c0c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.088+0000 7f8c0aa00640 1 --2- 172.21.15.3:0/3796127552 >> v2:172.21.15.70:3300/0 conn(0x7f8c0c1451d0 0x7f8c0c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.088+0000 7f8c0aa00640 1 -- 172.21.15.3:0/3796127552 >> v1:172.21.15.70:6789/0 conn(0x7f8c0c0cf270 legacy=0x7f8c0c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.088+0000 7f8c0aa00640 1 -- 172.21.15.3:0/3796127552 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c06da10 con 0x7f8c0c145730 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.089+0000 7f8c0aa00640 1 --2- 172.21.15.3:0/3796127552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c145730 0x7f8c0c0b3370 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0732d0 tx=0x7f8bf802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a5a1613975e184 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.089+0000 7f8c0a000640 1 -- 172.21.15.3:0/3796127552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8037070 con 0x7f8c0c145730 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.089+0000 7f8c0a000640 1 -- 172.21.15.3:0/3796127552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf8032440 con 0x7f8c0c145730 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.089+0000 7f8c0a000640 1 -- 172.21.15.3:0/3796127552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8032810 con 0x7f8c0c145730 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 -- 172.21.15.3:0/3796127552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c145730 msgr2=0x7f8c0c0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 --2- 172.21.15.3:0/3796127552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c145730 0x7f8c0c0b3370 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0732d0 tx=0x7f8bf802a1f0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 -- 172.21.15.3:0/3796127552 shutdown_connections 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 --2- 172.21.15.3:0/3796127552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c145730 0x7f8c0c0b3370 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 --2- 172.21.15.3:0/3796127552 >> v2:172.21.15.70:3300/0 conn(0x7f8c0c1451d0 0x7f8c0c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 -- 172.21.15.3:0/3796127552 >> 172.21.15.3:0/3796127552 conn(0x7f8c0c0b5cb0 msgr2=0x7f8c0c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 -- 172.21.15.3:0/3796127552 shutdown_connections 2024-09-10T07:52:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.090+0000 7f8c12600640 1 -- 172.21.15.3:0/3796127552 wait complete. 2024-09-10T07:52:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.091+0000 7f8c12600640 1 Processor -- start 2024-09-10T07:52:11.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 -- start start 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 0x7f8c0c12f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c0c145730 0x7f8c0c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c06df30 con 0x7f8c0c1451d0 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.092+0000 7f8c12600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0b38b0 con 0x7f8c0c0cf270 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c12600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c06e0b0 con 0x7f8c0c145730 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59834/0 (socket says 172.21.15.3:59834) 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 -- 172.21.15.3:0/1895843983 learned_addr learned my addr 172.21.15.3:0/1895843983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:11.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0be00640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c0c145730 0x7f8c0c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0b400640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 0x7f8c0c12f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 -- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c0c145730 msgr2=0x7f8c0c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c0c145730 0x7f8c0c071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 -- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 msgr2=0x7f8c0c12f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 0x7f8c0c12f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0aa00640 1 -- 172.21.15.3:0/1895843983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c072450 con 0x7f8c0c1451d0 2024-09-10T07:52:11.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.093+0000 7f8c0b400640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 0x7f8c0c12f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c0aa00640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8000c00 tx=0x7f8bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8041020 con 0x7f8c0c1451d0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf80306c0 con 0x7f8c0c1451d0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8037040 con 0x7f8c0c1451d0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c072730 con 0x7f8c0c1451d0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.094+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c12fd70 con 0x7f8c0c1451d0 2024-09-10T07:52:11.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.095+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c06cfc0 con 0x7f8c0c1451d0 2024-09-10T07:52:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.096+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bf8044020 con 0x7f8c0c1451d0 2024-09-10T07:52:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.097+0000 7f8c03400640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.097+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf80ab5c0 con 0x7f8c0c1451d0 2024-09-10T07:52:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.097+0000 7f8c0b400640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.098+0000 7f8c0b400640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc006fd0 tx=0x7f8bfc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.101+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bf804b030 con 0x7f8c0c1451d0 2024-09-10T07:52:11.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.357+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8c0c0b2710 con 0x7f8c0c1451d0 2024-09-10T07:52:11.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.359+0000 7f8c03400640 1 -- 172.21.15.3:0/1895843983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f8bf8079f60 con 0x7f8c0c1451d0 2024-09-10T07:52:11.358 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:52:11.358 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:52:11.358 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:52:11.358 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:52:11.359 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:52:11.359 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 msgr2=0x7f8bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc006fd0 tx=0x7f8bfc008040 comp rx=0 tx=0).stop 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 msgr2=0x7f8c0c071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8000c00 tx=0x7f8bf80046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 shutdown_connections 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8c0c145730 0x7f8c0c071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c0c1451d0 0x7f8c0c071920 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 --2- 172.21.15.3:0/1895843983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c0c0cf270 0x7f8c0c12f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 >> 172.21.15.3:0/1895843983 conn(0x7f8c0c0b5cb0 msgr2=0x7f8c0c1442d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 shutdown_connections 2024-09-10T07:52:11.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.362+0000 7f8c12600640 1 -- 172.21.15.3:0/1895843983 wait complete. 2024-09-10T07:52:11.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:11.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.535+0000 7fc72f200640 1 Processor -- start 2024-09-10T07:52:11.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.535+0000 7fc72f200640 1 -- start start 2024-09-10T07:52:11.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.536+0000 7fc72f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc73015e860 0x7fc7300cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.536+0000 7fc72f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.536+0000 7fc72f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7300cee40 con 0x7fc7300cfbf0 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.536+0000 7fc72f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc7300d3780 con 0x7fc73015e860 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.536+0000 7fc72f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc7300cf120 con 0x7fc730073150 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc73015e860 0x7fc7300cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc73015e860 0x7fc7300cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59836/0 (socket says 172.21.15.3:59836) 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 -- 172.21.15.3:0/3319657748 learned_addr learned my addr 172.21.15.3:0/3319657748 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc727e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc7300cfbf0 0x7fc7300d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57350/0 (socket says 172.21.15.3:57350) 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72e800640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 -- 172.21.15.3:0/3319657748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 msgr2=0x7fc730073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 -- 172.21.15.3:0/3319657748 >> v1:172.21.15.3:6789/0 conn(0x7fc7300cfbf0 legacy=0x7fc7300d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.537+0000 7fc72de00640 1 -- 172.21.15.3:0/3319657748 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7300d3410 con 0x7fc73015e860 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.538+0000 7fc72de00640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73015e860 0x7fc7300cf6b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc720009870 tx=0x7fc72002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c7d16275ebacd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.538+0000 7fc72d400640 1 -- 172.21.15.3:0/3319657748 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720037070 con 0x7fc73015e860 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.538+0000 7fc72d400640 1 -- 172.21.15.3:0/3319657748 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc720032440 con 0x7fc73015e860 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.538+0000 7fc72d400640 1 -- 172.21.15.3:0/3319657748 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720032810 con 0x7fc73015e860 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 -- 172.21.15.3:0/3319657748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73015e860 msgr2=0x7fc7300cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73015e860 0x7fc7300cf6b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc720009870 tx=0x7fc72002a250 comp rx=0 tx=0).stop 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 -- 172.21.15.3:0/3319657748 shutdown_connections 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 --2- 172.21.15.3:0/3319657748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc73015e860 0x7fc7300cf6b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 -- 172.21.15.3:0/3319657748 >> 172.21.15.3:0/3319657748 conn(0x7fc7300d4650 msgr2=0x7fc7300d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 -- 172.21.15.3:0/3319657748 shutdown_connections 2024-09-10T07:52:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.539+0000 7fc72f200640 1 -- 172.21.15.3:0/3319657748 wait complete. 2024-09-10T07:52:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.541+0000 7fc72f200640 1 Processor -- start 2024-09-10T07:52:11.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.541+0000 7fc72f200640 1 -- start start 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc73015e860 0x7fc7300acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300d2460 con 0x7fc7300cfbf0 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300cf400 con 0x7fc73015e860 2024-09-10T07:52:11.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc7300b7880 con 0x7fc730073150 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc727e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc727e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59850/0 (socket says 172.21.15.3:59850) 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc727e00640 1 -- 172.21.15.3:0/4269116867 learned_addr learned my addr 172.21.15.3:0/4269116867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72de00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.542+0000 7fc72e800640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc73015e860 0x7fc7300acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 -- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 msgr2=0x7fc730152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 -- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc73015e860 msgr2=0x7fc7300acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc73015e860 0x7fc7300acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 -- 172.21.15.3:0/4269116867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7300ab070 con 0x7fc7300cfbf0 2024-09-10T07:52:11.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc727e00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c400 tx=0x7fc71800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7180074e0 con 0x7fc7300cfbf0 2024-09-10T07:52:11.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc718007680 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718004a00 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7300ab350 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.543+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7300ab910 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.544+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7300cf6b0 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.545+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc71801a020 con 0x7fc7300cfbf0 2024-09-10T07:52:11.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.546+0000 7fc726000640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 0x7fc70c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.546+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc71808a530 con 0x7fc7300cfbf0 2024-09-10T07:52:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.546+0000 7fc72de00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 0x7fc70c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:11.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.547+0000 7fc72de00640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 0x7fc70c06e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc720034000 tx=0x7fc72002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.551+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc718058880 con 0x7fc7300cfbf0 2024-09-10T07:52:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.773+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc730152f80 con 0x7fc7300cfbf0 2024-09-10T07:52:11.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.774+0000 7fc726000640 1 -- 172.21.15.3:0/4269116867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5216 (secure 0 0 0) 0x7fc71805cb80 con 0x7fc7300cfbf0 2024-09-10T07:52:11.773 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:11.773 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":85,"btime":"2024-09-10T07:52:04:593834+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:04.020861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[7239]},"id":6}]} 2024-09-10T07:52:11.773 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 85 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 msgr2=0x7fc70c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 0x7fc70c06e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc720034000 tx=0x7fc72002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 msgr2=0x7fc7300cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc71800c400 tx=0x7fc71800c8c0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 shutdown_connections 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc70c06be30 0x7fc70c06e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc73015e860 0x7fc7300acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7300cfbf0 0x7fc7300cdf20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 --2- 172.21.15.3:0/4269116867 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc730073150 0x7fc730152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.776+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 >> 172.21.15.3:0/4269116867 conn(0x7fc7300d4650 msgr2=0x7fc7300d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.777+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 shutdown_connections 2024-09-10T07:52:11.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:11.777+0000 7fc72f200640 1 -- 172.21.15.3:0/4269116867 wait complete. 2024-09-10T07:52:12.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:52:00.997662+0000', 'modified': '2024-09-10T07:52:04.020861+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7239': {'gid': 7239, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2433616379', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 2433616379}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2433616379}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7239, 'qdb_cluster': [7239]} 2024-09-10T07:52:12.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:52:12.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:52:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.356+0000 7ff078800640 1 Processor -- start 2024-09-10T07:52:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.356+0000 7ff078800640 1 -- start start 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff078800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff07015e4f0 0x7ff07015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff078800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0701603c0 con 0x7ff07006d160 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff078800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07006c0c0 con 0x7ff07006bb20 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff078800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff07006ccd0 con 0x7ff07015e4f0 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff075e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff07015e4f0 0x7ff07015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff075e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff07015e4f0 0x7ff07015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59864/0 (socket says 172.21.15.3:59864) 2024-09-10T07:52:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.357+0000 7ff075e00640 1 -- 172.21.15.3:0/2385431547 learned_addr learned my addr 172.21.15.3:0/2385431547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.358+0000 7ff075e00640 1 -- 172.21.15.3:0/2385431547 >> v1:172.21.15.70:6789/0 conn(0x7ff07006bb20 legacy=0x7ff07015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.358+0000 7ff075e00640 1 -- 172.21.15.3:0/2385431547 >> v1:172.21.15.3:6789/0 conn(0x7ff07006d160 legacy=0x7ff07006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.358+0000 7ff075e00640 1 -- 172.21.15.3:0/2385431547 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07006c3a0 con 0x7ff07015e4f0 2024-09-10T07:52:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.359+0000 7ff075e00640 1 --2- 172.21.15.3:0/2385431547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff07015f570 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff064009870 tx=0x7ff06402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11204b1779c30e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.359+0000 7ff074a00640 1 -- 172.21.15.3:0/2385431547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064037070 con 0x7ff07015e4f0 2024-09-10T07:52:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.359+0000 7ff074a00640 1 -- 172.21.15.3:0/2385431547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff064032440 con 0x7ff07015e4f0 2024-09-10T07:52:12.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.359+0000 7ff074a00640 1 -- 172.21.15.3:0/2385431547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064032810 con 0x7ff07015e4f0 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 -- 172.21.15.3:0/2385431547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 msgr2=0x7ff07015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 --2- 172.21.15.3:0/2385431547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff07015f570 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff064009870 tx=0x7ff06402a2a0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 -- 172.21.15.3:0/2385431547 shutdown_connections 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 --2- 172.21.15.3:0/2385431547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff07015f570 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 -- 172.21.15.3:0/2385431547 >> 172.21.15.3:0/2385431547 conn(0x7ff07006e5b0 msgr2=0x7ff07006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 -- 172.21.15.3:0/2385431547 shutdown_connections 2024-09-10T07:52:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.360+0000 7ff078800640 1 -- 172.21.15.3:0/2385431547 wait complete. 2024-09-10T07:52:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.362+0000 7ff078800640 1 Processor -- start 2024-09-10T07:52:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.362+0000 7ff078800640 1 -- start start 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 0x7ff0700b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff07006d160 0x7ff0700aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff07015e8d0 con 0x7ff07015e4f0 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff07006c960 con 0x7ff07006d160 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff078800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff0700b6dc0 con 0x7ff07006bb20 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59880/0 (socket says 172.21.15.3:59880) 2024-09-10T07:52:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff075e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 0x7ff0700b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff076800640 1 -- 172.21.15.3:0/2436156304 learned_addr learned my addr 172.21.15.3:0/2436156304 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.363+0000 7ff075400640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff07006d160 0x7ff0700aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 -- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 msgr2=0x7ff0700b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 0x7ff0700b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 -- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff07006d160 msgr2=0x7ff0700aa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff07006d160 0x7ff0700aa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 -- 172.21.15.3:0/2436156304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701f3eb0 con 0x7ff07015e4f0 2024-09-10T07:52:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff075e00640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 0x7ff0700b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff076800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c7a0 tx=0x7ff06c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff06a000640 1 -- 172.21.15.3:0/2436156304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c003b70 con 0x7ff07015e4f0 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff06a000640 1 -- 172.21.15.3:0/2436156304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c003d30 con 0x7ff07015e4f0 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0700cee40 con 0x7ff07015e4f0 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff06a000640 1 -- 172.21.15.3:0/2436156304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00aa90 con 0x7ff07015e4f0 2024-09-10T07:52:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.364+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0700cf3e0 con 0x7ff07015e4f0 2024-09-10T07:52:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.366+0000 7ff06a000640 1 -- 172.21.15.3:0/2436156304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff06c00ac30 con 0x7ff07015e4f0 2024-09-10T07:52:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.367+0000 7ff06a000640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.367+0000 7ff06a000640 1 -- 172.21.15.3:0/2436156304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7ff06c08ac80 con 0x7ff07015e4f0 2024-09-10T07:52:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.367+0000 7ff075e00640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.368+0000 7ff075e00640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 0x7ff05406e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff064034000 tx=0x7ff06402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.368+0000 7ff07b6d8740 1 Processor -- start 2024-09-10T07:52:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.368+0000 7ff07b6d8740 1 -- start start 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f744a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f74024b0 0x5588f74050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5588f73633b0 con 0x5588f7406480 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5588f7368410 con 0x5588f7404cf0 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5588f7310620 con 0x5588f74024b0 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59896/0 (socket says 172.21.15.3:59896) 2024-09-10T07:52:12.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 -- 172.21.15.3:0/4197120686 learned_addr learned my addr 172.21.15.3:0/4197120686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 -- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f74024b0 msgr2=0x5588f74050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f74024b0 0x5588f74050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 -- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 msgr2=0x5588f744a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.369+0000 7ff076800640 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f744a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.370+0000 7ff076800640 1 -- 172.21.15.3:0/4197120686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f707b2d0 con 0x5588f7406480 2024-09-10T07:52:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.370+0000 7ff076800640 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0890e0 tx=0x7ff06c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819a87b38a1e64df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.370+0000 7ff05e600640 1 -- 172.21.15.3:0/4197120686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c05d2d0 con 0x5588f7406480 2024-09-10T07:52:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.370+0000 7ff05e600640 1 -- 172.21.15.3:0/4197120686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c061b00 con 0x5588f7406480 2024-09-10T07:52:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.370+0000 7ff05e600640 1 -- 172.21.15.3:0/4197120686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c055640 con 0x5588f7406480 2024-09-10T07:52:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- 172.21.15.3:0/4197120686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 msgr2=0x5588f73ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0890e0 tx=0x7ff06c011000 comp rx=0 tx=0).stop 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- 172.21.15.3:0/4197120686 shutdown_connections 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f73ff570 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f74024b0 0x5588f74050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/4197120686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f744a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- 172.21.15.3:0/4197120686 >> 172.21.15.3:0/4197120686 conn(0x5588f7401740 msgr2=0x5588f7449960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- 172.21.15.3:0/4197120686 shutdown_connections 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- 172.21.15.3:0/4197120686 wait complete. 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.371+0000 7ff07b6d8740 1 -- start start 2024-09-10T07:52:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.372+0000 7ff07b6d8740 1 Processor -- start 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.372+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f742f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7428440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5588f7323780 con 0x5588f7404cf0 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5588f73643d0 con 0x5588f7406480 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5588f7400be0 con 0x5588f7446f10 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff07b6d8740 10 client.0 _ll_register_callbacks cb 0x5588f73615d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:12.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59904/0 (socket says 172.21.15.3:59904) 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 -- 172.21.15.3:0/1937865742 learned_addr learned my addr 172.21.15.3:0/1937865742 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff076800640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7428440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075400640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f742f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 -- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 msgr2=0x5588f7428440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7428440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 -- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 msgr2=0x5588f742f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f742f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075e00640 1 -- 172.21.15.3:0/1937865742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f71456b0 con 0x5588f7404cf0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.373+0000 7ff075400640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f742f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff075e00640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0640090b0 tx=0x7ff0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e3f33c775eff6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff052a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:12.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064037040 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7ff06403c640 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c061b00 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff06403c840 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff07b6d8740 10 client.7377 fetch_fsmap learned FSMap version 85 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5588f742e010 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.374+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c061b00 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7ff064050030 con 0x5588f7404cf0 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 10 client.7377 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 10 client.7377 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 20 client.7377 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f745d530 0x5588f745a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 4 client.7377 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f7375b40 con 0x5588f745d530 2024-09-10T07:52:12.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff075400640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f745d530 0x5588f745a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff075400640 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f745d530 0x5588f745a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49e47ad1471fba6 server_cookie=4ddef13e17e5cc65 in_seq=0 out_seq=0 2024-09-10T07:52:12.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.375+0000 7ff052a00640 10 client.7377 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.376+0000 7ff052a00640 1 -- 172.21.15.3:0/1937865742 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588f7375b40 con 0x5588f745d530 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.376+0000 7ff052a00640 10 client.7377 handle_command_reply: tid=0 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.376+0000 7ff07b6d8740 1 client.7377 shutdown 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 msgr2=0x5588f742d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0640090b0 tx=0x7ff0640046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 shutdown_connections 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f745d530 0x5588f745a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7428440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f742f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1937865742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7404cf0 0x5588f742d200 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff052a00640 0 client.7377 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 >> 172.21.15.3:0/1937865742 conn(0x5588f74024b0 msgr2=0x5588f74479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 shutdown_connections 2024-09-10T07:52:12.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1937865742 wait complete. 2024-09-10T07:52:12.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.377+0000 7ff07b6d8740 20 client.7377 trim_cache size 0 max 16384 2024-09-10T07:52:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 Processor -- start 2024-09-10T07:52:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 -- start start 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 0x5588f7446810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ffc40 con 0x5588f7446f10 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff07014e570 con 0x5588f7406480 2024-09-10T07:52:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.396+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff0701ff5f0 con 0x5588f7404cf0 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59914/0 (socket says 172.21.15.3:59914) 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 -- 172.21.15.3:0/2239758004 learned_addr learned my addr 172.21.15.3:0/2239758004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff075e00640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 0x5588f7446810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff075400640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 -- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 msgr2=0x5588f7446810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 0x5588f7446810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 -- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 msgr2=0x5588f74479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 -- 172.21.15.3:0/2239758004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f73f1350 con 0x5588f7446f10 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff075e00640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 0x5588f7446810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff075400640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.397+0000 7ff076800640 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff06c011040 tx=0x7ff06c005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c345a88119c265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff05dc00640 1 -- 172.21.15.3:0/2239758004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c05d2d0 con 0x5588f7446f10 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff05dc00640 1 -- 172.21.15.3:0/2239758004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c072070 con 0x5588f7446f10 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff05dc00640 1 -- 172.21.15.3:0/2239758004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c061b00 con 0x5588f7446f10 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 -- 172.21.15.3:0/2239758004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 msgr2=0x5588f7447f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff06c011040 tx=0x7ff06c005070 comp rx=0 tx=0).stop 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 -- 172.21.15.3:0/2239758004 shutdown_connections 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/2239758004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7404cf0 0x5588f7446810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 -- 172.21.15.3:0/2239758004 >> 172.21.15.3:0/2239758004 conn(0x5588f74024b0 msgr2=0x5588f745cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 -- 172.21.15.3:0/2239758004 shutdown_connections 2024-09-10T07:52:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.398+0000 7ff07b6d8740 1 -- 172.21.15.3:0/2239758004 wait complete. 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.399+0000 7ff07b6d8740 1 -- start start 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.399+0000 7ff07b6d8740 1 Processor -- start 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f74ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5588f745d3b0 con 0x5588f7406480 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5588f745b2c0 con 0x5588f7404cf0 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5588f745d9d0 con 0x5588f7446f10 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff07b6d8740 10 client.0 _ll_register_callbacks cb 0x7ff0640096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff075400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff075400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59926/0 (socket says 172.21.15.3:59926) 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff075400640 1 -- 172.21.15.3:0/1444934394 learned_addr learned my addr 172.21.15.3:0/1444934394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075e00640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f74ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.400+0000 7ff076800640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 -- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 msgr2=0x5588f7447f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 -- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 msgr2=0x5588f74ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f74ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 -- 172.21.15.3:0/1444934394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f74b1610 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff075400640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff060004f80 tx=0x7ff0600054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af3e24cf74f2b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff052a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017a00 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7ff060017ba0 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c072070 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff06002a050 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff07b6d8740 10 client.7383 fetch_fsmap learned FSMap version 85 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.401+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5588f7449e00 con 0x5588f7406480 2024-09-10T07:52:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c072070 con 0x5588f7406480 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7ff06002d020 con 0x5588f7406480 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 10 client.7383 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 10 client.7383 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 20 client.7383 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f74cbd30 0x5588f746c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 4 client.7383 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "status"}) -- 0x5588f736fe90 con 0x5588f74cbd30 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.402+0000 7ff075e00640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f74cbd30 0x5588f746c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff075e00640 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f74cbd30 0x5588f746c570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a361580f0a35d61d server_cookie=3ba872137111b4a1 in_seq=0 out_seq=0 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff052a00640 10 client.7383 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff052a00640 1 -- 172.21.15.3:0/1444934394 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5588f736fe90 con 0x5588f74cbd30 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff052a00640 10 client.7383 handle_command_reply: tid=0 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 client.7383 shutdown 2024-09-10T07:52:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 msgr2=0x5588f74479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff060004f80 tx=0x7ff0600054d0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 shutdown_connections 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5588f74cbd30 0x5588f746c570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5588f7446f10 0x5588f7447f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5588f7406480 0x5588f74479e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 --2- 172.21.15.3:0/1444934394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5588f7404cf0 0x5588f74ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 >> 172.21.15.3:0/1444934394 conn(0x5588f74024b0 msgr2=0x5588f745cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff052a00640 0 client.7383 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 shutdown_connections 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 1 -- 172.21.15.3:0/1444934394 wait complete. 2024-09-10T07:52:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.403+0000 7ff07b6d8740 20 client.7383 trim_cache size 0 max 16384 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7239, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.825643103999999, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 85, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.891108159 2024-09-10T07:52:12.407 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 msgr2=0x7ff05406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 0x7ff05406e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff064034000 tx=0x7ff06402e000 comp rx=0 tx=0).stop 2024-09-10T07:52:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 msgr2=0x7ff0700aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c7a0 tx=0x7ff06c00cc60 comp rx=0 tx=0).stop 2024-09-10T07:52:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 shutdown_connections 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff05406be30 0x7ff05406e2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff07015e4f0 0x7ff0700aac00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff07006d160 0x7ff0700aa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 --2- 172.21.15.3:0/2436156304 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff07006bb20 0x7ff0700b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 >> 172.21.15.3:0/2436156304 conn(0x7ff07006e5b0 msgr2=0x7ff0700cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 shutdown_connections 2024-09-10T07:52:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.404+0000 7ff078800640 1 -- 172.21.15.3:0/2436156304 wait complete. 2024-09-10T07:52:12.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:52:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.575+0000 7fd367e00640 1 Processor -- start 2024-09-10T07:52:12.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.576+0000 7fd367e00640 1 -- start start 2024-09-10T07:52:12.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3680b8f00 0x7fd3680cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680d29d0 0x7fd3680d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3680733a0 con 0x7fd368161c00 2024-09-10T07:52:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd3680d4390 con 0x7fd3680b8f00 2024-09-10T07:52:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3680b77a0 con 0x7fd3680d29d0 2024-09-10T07:52:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd366a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd368161c00 0x7fd368072b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57360/0 (socket says 172.21.15.3:57360) 2024-09-10T07:52:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd366a00640 1 -- 172.21.15.3:0/2895297682 learned_addr learned my addr 172.21.15.3:0/2895297682 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd367400640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680d29d0 0x7fd3680d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.577+0000 7fd35fe00640 1 --2- 172.21.15.3:0/2895297682 >> v2:172.21.15.70:3300/0 conn(0x7fd3680b8f00 0x7fd3680cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1497903729 0 0) 0x7fd3680733a0 con 0x7fd368161c00 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd34c003520 con 0x7fd368161c00 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd35fe00640 1 -- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680d29d0 msgr2=0x7fd3680d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd35fe00640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680d29d0 0x7fd3680d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd35fe00640 1 -- 172.21.15.3:0/2895297682 >> v1:172.21.15.3:6789/0 conn(0x7fd368161c00 legacy=0x7fd368072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3999240086 0 0) 0x7fd34c003520 con 0x7fd368161c00 2024-09-10T07:52:12.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.578+0000 7fd35fe00640 1 -- 172.21.15.3:0/2895297682 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd368073680 con 0x7fd3680b8f00 2024-09-10T07:52:12.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.579+0000 7fd35fe00640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3680b8f00 0x7fd3680cfa10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd3500097d0 tx=0x7fd35002a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c6993f03d43fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.579+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350037070 con 0x7fd3680b8f00 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.579+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35002ab40 con 0x7fd3680b8f00 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.579+0000 7fd366000640 1 -- 172.21.15.3:0/2895297682 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358001670 con 0x7fd3680b8f00 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 -- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3680b8f00 msgr2=0x7fd3680cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3680b8f00 0x7fd3680cfa10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd3500097d0 tx=0x7fd35002a0b0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 -- 172.21.15.3:0/2895297682 shutdown_connections 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680d29d0 0x7fd3680d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 --2- 172.21.15.3:0/2895297682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3680b8f00 0x7fd3680cfa10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 -- 172.21.15.3:0/2895297682 >> 172.21.15.3:0/2895297682 conn(0x7fd3680d4600 msgr2=0x7fd3680d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 -- 172.21.15.3:0/2895297682 shutdown_connections 2024-09-10T07:52:12.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.580+0000 7fd367e00640 1 -- 172.21.15.3:0/2895297682 wait complete. 2024-09-10T07:52:12.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.582+0000 7fd367e00640 1 Processor -- start 2024-09-10T07:52:12.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.582+0000 7fd367e00640 1 -- start start 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680b8f00 0x7fd3681f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd368161c00 0x7fd3680ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3680b79d0 con 0x7fd3680d29d0 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd368073c40 con 0x7fd368161c00 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd368161fe0 con 0x7fd3680b8f00 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59936/0 (socket says 172.21.15.3:59936) 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 -- 172.21.15.3:0/1746549047 learned_addr learned my addr 172.21.15.3:0/1746549047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd366a00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680b8f00 0x7fd3681f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd367400640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd368161c00 0x7fd3680ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 -- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680b8f00 msgr2=0x7fd3681f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680b8f00 0x7fd3681f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 -- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd368161c00 msgr2=0x7fd3680ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd368161c00 0x7fd3680ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.583+0000 7fd35fe00640 1 -- 172.21.15.3:0/1746549047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3680b6940 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd35fe00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd350000c00 tx=0x7fd3500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350037040 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd350004270 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3680ab2b0 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350006e10 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.584+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3680ab870 con 0x7fd3680d29d0 2024-09-10T07:52:12.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.585+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3680cfa10 con 0x7fd3680d29d0 2024-09-10T07:52:12.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.586+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd350003610 con 0x7fd3680d29d0 2024-09-10T07:52:12.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.587+0000 7fd35ea00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:12.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.587+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fd3500ab930 con 0x7fd3680d29d0 2024-09-10T07:52:12.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.587+0000 7fd366a00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:12.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.588+0000 7fd366a00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd358005880 tx=0x7fd3580057f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:12.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.592+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd350079c80 con 0x7fd3680d29d0 2024-09-10T07:52:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.812+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3680d4b80 con 0x7fd3680d29d0 2024-09-10T07:52:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.812+0000 7fd35ea00640 1 -- 172.21.15.3:0/1746549047 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7fd35007df80 con 0x7fd3680d29d0 2024-09-10T07:52:12.811 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:52:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.814+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 msgr2=0x7fd34406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.814+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd358005880 tx=0x7fd3580057f0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.814+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 msgr2=0x7fd368156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.814+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd350000c00 tx=0x7fd3500046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 shutdown_connections 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd368161c00 0x7fd3680ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3680d29d0 0x7fd368156990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 --2- 172.21.15.3:0/1746549047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3680b8f00 0x7fd3681f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 >> 172.21.15.3:0/1746549047 conn(0x7fd3680d4600 msgr2=0x7fd3680c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 shutdown_connections 2024-09-10T07:52:12.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:12.815+0000 7fd367e00640 1 -- 172.21.15.3:0/1746549047 wait complete. 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:52:12.829 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:52:12.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:12.830 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:52:12.865 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42363sec preferred_lft 42363sec 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:52:12.866 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:52:12.867 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:52:12.868 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:52:12.868 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:52:12.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:12.868 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:52:12.959 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:52:12.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:12.962 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:52:12.998 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:52:12.998 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:52:12.999 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:52:12.999 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:52:12.999 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:52:12.999 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:52:13.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:52:13.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:13.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:13.196 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:52:13.231 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:52:13.231 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:52:13.232 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:52:13.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:13.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:52:13.329 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:52:13.329 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:52:13.364 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:52:13.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:52:13.364 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:52:13.439 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:52:13.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:13.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:52:13.569 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:52:13.569 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:52:13.569 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:52:13.585 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:52:13.585 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:52:13.585 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:52:13.585 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:52:13.585 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:52:13.606 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:52:13.606 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:52:13.606 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:52:13.606 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:52:13.607 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:52:13.607 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:52:13.607 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:52:13.607 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:52:13.608 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:52:13.608 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:52:13.608 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:52:13.608 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:52:13.608 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:52:13.611 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:52:13.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:13.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:52:13.668 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:52:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:52:13.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:client7392 2024-09-10T07:52:13.765 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:52:13.765 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:13.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:52:14.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.002+0000 7f8117200640 1 Processor -- start 2024-09-10T07:52:14.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.002+0000 7f8117200640 1 -- start start 2024-09-10T07:52:14.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8118161b70 0x7f81180d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f81180d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 0x7f81180d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8118161f50 con 0x7f8118161b70 2024-09-10T07:52:14.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f81180d3de0 con 0x7f81180d4ae0 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8117200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81180d40c0 con 0x7f81180d3860 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8115e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8118161b70 0x7f81180d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8115e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8118161b70 0x7f81180d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54212/0 (socket says 172.21.15.3:54212) 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.003+0000 7f8115e00640 1 -- 172.21.15.3:0/760951660 learned_addr learned my addr 172.21.15.3:0/760951660 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f810fe00640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f81180d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8116800640 1 --2- 172.21.15.3:0/760951660 >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 0x7f81180d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8115e00640 1 -- 172.21.15.3:0/760951660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 msgr2=0x7f81180d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8115e00640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f81180d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8115e00640 1 -- 172.21.15.3:0/760951660 >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 msgr2=0x7f81180d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8115e00640 1 --2- 172.21.15.3:0/760951660 >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 0x7f81180d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8115e00640 1 -- 172.21.15.3:0/760951660 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81180cf690 con 0x7f8118161b70 2024-09-10T07:52:14.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.004+0000 7f8116800640 1 --2- 172.21.15.3:0/760951660 >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 0x7f81180d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.005+0000 7f8115e00640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8118161b70 0x7f81180d3320 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f81180d28b0 tx=0x7f810802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619e31792b0ce45a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.005+0000 7f8115400640 1 -- 172.21.15.3:0/760951660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108037070 con 0x7f8118161b70 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.005+0000 7f8115400640 1 -- 172.21.15.3:0/760951660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108032440 con 0x7f8118161b70 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.005+0000 7f8115400640 1 -- 172.21.15.3:0/760951660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108032810 con 0x7f8118161b70 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 -- 172.21.15.3:0/760951660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8118161b70 msgr2=0x7f81180d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8118161b70 0x7f81180d3320 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f81180d28b0 tx=0x7f810802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 -- 172.21.15.3:0/760951660 shutdown_connections 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 --2- 172.21.15.3:0/760951660 >> v2:172.21.15.70:3300/0 conn(0x7f81180d4ae0 0x7f81180d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f81180d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 --2- 172.21.15.3:0/760951660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8118161b70 0x7f81180d3320 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 -- 172.21.15.3:0/760951660 >> 172.21.15.3:0/760951660 conn(0x7f8118062630 msgr2=0x7f8118062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 -- 172.21.15.3:0/760951660 shutdown_connections 2024-09-10T07:52:14.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.006+0000 7f8117200640 1 -- 172.21.15.3:0/760951660 wait complete. 2024-09-10T07:52:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.007+0000 7f8117200640 1 Processor -- start 2024-09-10T07:52:14.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.008+0000 7f8117200640 1 -- start start 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.008+0000 7f8117200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f811814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8117200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8117200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8118161b70 0x7f81180cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8117200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81180cf2d0 con 0x7f81180d4ae0 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8117200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81180c9fc0 con 0x7f8118161b70 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8117200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81180cf450 con 0x7f81180d3860 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54218/0 (socket says 172.21.15.3:54218) 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 -- 172.21.15.3:0/3116253829 learned_addr learned my addr 172.21.15.3:0/3116253829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8116800640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8118161b70 0x7f81180cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8115e00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f811814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 -- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 msgr2=0x7f811814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f811814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 -- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8118161b70 msgr2=0x7f81180cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8118161b70 0x7f81180cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f810fe00640 1 -- 172.21.15.3:0/3116253829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8118155e60 con 0x7f81180d4ae0 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.009+0000 7f8115e00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f811814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:14.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f810fe00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f810000d800 tx=0x7f810000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100018030 con 0x7f81180d4ae0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81000096a0 con 0x7f81180d4ae0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100009870 con 0x7f81180d4ae0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181f8cf0 con 0x7f81180d4ae0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.010+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181f8210 con 0x7f81180d4ae0 2024-09-10T07:52:14.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.011+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8118157260 con 0x7f81180d4ae0 2024-09-10T07:52:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.012+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f810001b070 con 0x7f81180d4ae0 2024-09-10T07:52:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.013+0000 7f810e000640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 0x7f80f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.013+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f810008b0f0 con 0x7f81180d4ae0 2024-09-10T07:52:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.013+0000 7f8115e00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 0x7f80f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.014+0000 7f8115e00640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 0x7f80f406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8108034000 tx=0x7f810802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.017+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8100059a90 con 0x7f81180d4ae0 2024-09-10T07:52:14.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.240+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f81180b7540 con 0x7f81180d4ae0 2024-09-10T07:52:14.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.626+0000 7f810e000640 1 -- 172.21.15.3:0/3116253829 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v86) ==== 105+0+0 (secure 0 0 0) 0x7f8100059390 con 0x7f81180d4ae0 2024-09-10T07:52:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 msgr2=0x7f80f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 0x7f80f406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8108034000 tx=0x7f810802e000 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 msgr2=0x7f81180ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f810000d800 tx=0x7f810000dcc0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 shutdown_connections 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f80f406be30 0x7f80f406e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8118161b70 0x7f81180cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81180d4ae0 0x7f81180ccdc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 --2- 172.21.15.3:0/3116253829 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81180d3860 0x7f811814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 >> 172.21.15.3:0/3116253829 conn(0x7f8118062630 msgr2=0x7f811815ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 shutdown_connections 2024-09-10T07:52:14.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.629+0000 7f8117200640 1 -- 172.21.15.3:0/3116253829 wait complete. 2024-09-10T07:52:14.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:14.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.808+0000 7f35c2c00640 1 Processor -- start 2024-09-10T07:52:14.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.808+0000 7f35c2c00640 1 -- start start 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c2c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35bc160710 0x7f35bc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c2c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35bc0d07c0 0x7f35bc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c2c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35bc0cfdd0 con 0x7f35bc160af0 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c2c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f35bc15cc90 con 0x7f35bc160710 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c2c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f35bc15cf70 con 0x7f35bc0d07c0 2024-09-10T07:52:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35bbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35bc160710 0x7f35bc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35bbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f35bc160710 0x7f35bc0cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54220/0 (socket says 172.21.15.3:54220) 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35bbe00640 1 -- 172.21.15.3:0/756717075 learned_addr learned my addr 172.21.15.3:0/756717075 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.809+0000 7f35c0c00640 1 --2- 172.21.15.3:0/756717075 >> v2:172.21.15.70:3300/0 conn(0x7f35bc0d07c0 0x7f35bc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.810+0000 7f35bbe00640 1 -- 172.21.15.3:0/756717075 >> v2:172.21.15.70:3300/0 conn(0x7f35bc0d07c0 msgr2=0x7f35bc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.810+0000 7f35bbe00640 1 --2- 172.21.15.3:0/756717075 >> v2:172.21.15.70:3300/0 conn(0x7f35bc0d07c0 0x7f35bc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.810+0000 7f35bbe00640 1 -- 172.21.15.3:0/756717075 >> v1:172.21.15.70:6789/0 conn(0x7f35bc160af0 legacy=0x7f35bc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.810+0000 7f35bbe00640 1 -- 172.21.15.3:0/756717075 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc15d250 con 0x7f35bc160710 2024-09-10T07:52:14.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35bbe00640 1 --2- 172.21.15.3:0/756717075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc160710 0x7f35bc0cf2c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0718d0 tx=0x7f35ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0b9dfc57343fef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35bb400640 1 -- 172.21.15.3:0/756717075 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac037070 con 0x7f35bc160710 2024-09-10T07:52:14.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35bb400640 1 -- 172.21.15.3:0/756717075 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ac032440 con 0x7f35bc160710 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35bb400640 1 -- 172.21.15.3:0/756717075 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac032810 con 0x7f35bc160710 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35c2c00640 1 -- 172.21.15.3:0/756717075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc160710 msgr2=0x7f35bc0cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.811+0000 7f35c2c00640 1 --2- 172.21.15.3:0/756717075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc160710 0x7f35bc0cf2c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0718d0 tx=0x7f35ac02a230 comp rx=0 tx=0).stop 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 -- 172.21.15.3:0/756717075 shutdown_connections 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 --2- 172.21.15.3:0/756717075 >> v2:172.21.15.70:3300/0 conn(0x7f35bc0d07c0 0x7f35bc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 --2- 172.21.15.3:0/756717075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc160710 0x7f35bc0cf2c0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 -- 172.21.15.3:0/756717075 >> 172.21.15.3:0/756717075 conn(0x7f35bc06d000 msgr2=0x7f35bc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 -- 172.21.15.3:0/756717075 shutdown_connections 2024-09-10T07:52:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.812+0000 7f35c2c00640 1 -- 172.21.15.3:0/756717075 wait complete. 2024-09-10T07:52:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.813+0000 7f35c2c00640 1 Processor -- start 2024-09-10T07:52:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.813+0000 7f35c2c00640 1 -- start start 2024-09-10T07:52:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35bc160710 0x7f35bc14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35bc160af0 0x7f35bc0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc15d770 con 0x7f35bc0d07c0 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc0cf800 con 0x7f35bc160af0 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35c2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f35bc14d500 con 0x7f35bc160710 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35bbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54228/0 (socket says 172.21.15.3:54228) 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.814+0000 7f35bbe00640 1 -- 172.21.15.3:0/3623628717 learned_addr learned my addr 172.21.15.3:0/3623628717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35c0c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35bc160af0 0x7f35bc0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35b3400640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35bc160710 0x7f35bc14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 -- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35bc160710 msgr2=0x7f35bc14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35bc160710 0x7f35bc14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 -- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35bc160af0 msgr2=0x7f35bc0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35bc160af0 0x7f35bc0b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 -- 172.21.15.3:0/3623628717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc14b400 con 0x7f35bc0d07c0 2024-09-10T07:52:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35bbe00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f35ac000c00 tx=0x7f35ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac041020 con 0x7f35bc0d07c0 2024-09-10T07:52:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ac0306c0 con 0x7f35bc0d07c0 2024-09-10T07:52:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac037040 con 0x7f35bc0d07c0 2024-09-10T07:52:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.815+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc151bb0 con 0x7f35bc0d07c0 2024-09-10T07:52:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.816+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc14ceb0 con 0x7f35bc0d07c0 2024-09-10T07:52:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.816+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35bc0afd50 con 0x7f35bc0d07c0 2024-09-10T07:52:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.817+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35ac044020 con 0x7f35bc0d07c0 2024-09-10T07:52:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.818+0000 7f35b8c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 0x7f359806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.818+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f35ac0ab740 con 0x7f35bc0d07c0 2024-09-10T07:52:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.819+0000 7f35b3400640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 0x7f359806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.819+0000 7f35b3400640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 0x7f359806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006fd0 tx=0x7f35a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:14.823+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35ac04b030 con 0x7f35bc0d07c0 2024-09-10T07:52:15.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.041+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35bc0cd210 con 0x7f35bc0d07c0 2024-09-10T07:52:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.042+0000 7f35b8c00640 1 -- 172.21.15.3:0/3623628717 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5232 (secure 0 0 0) 0x7f35ac07a050 con 0x7f35bc0d07c0 2024-09-10T07:52:15.041 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:15.041 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"btime":"2024-09-10T07:52:14:628695+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:14.628661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7239,"mds_1":7236},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[7239]},"id":6}]} 2024-09-10T07:52:15.041 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 87 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 msgr2=0x7f359806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 0x7f359806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006fd0 tx=0x7f35a4008040 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 msgr2=0x7f35bc1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f35ac000c00 tx=0x7f35ac0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 shutdown_connections 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f359806be30 0x7f359806e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35bc160af0 0x7f35bc0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35bc160710 0x7f35bc14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 --2- 172.21.15.3:0/3623628717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35bc0d07c0 0x7f35bc1515d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 >> 172.21.15.3:0/3623628717 conn(0x7f35bc06d000 msgr2=0x7f35bc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:15.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 shutdown_connections 2024-09-10T07:52:15.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:15.044+0000 7f35c2c00640 1 -- 172.21.15.3:0/3623628717 wait complete. 2024-09-10T07:52:15.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:52:00.997662+0000', 'modified': '2024-09-10T07:52:14.628661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7239, 'mds_1': 7236}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7236': {'gid': 7236, 'name': 'a', 'rank': 1, 'incarnation': 87, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6836/3872292817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 3872292817}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 3872292817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7239': {'gid': 7239, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2433616379', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 2433616379}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2433616379}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7239, 'qdb_cluster': [7239]} 2024-09-10T07:52:15.461 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7236:up:creating 2024-09-10T07:52:16.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.633+0000 7f13c3400640 1 Processor -- start 2024-09-10T07:52:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.633+0000 7f13c3400640 1 -- start start 2024-09-10T07:52:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.634+0000 7f13c3400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13bc160710 0x7f13bc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.634+0000 7f13c3400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13bc0d07c0 0x7f13bc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.634+0000 7f13c3400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc0cfdd0 con 0x7f13bc160af0 2024-09-10T07:52:16.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.634+0000 7f13c3400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f13bc15cc90 con 0x7f13bc0d07c0 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.634+0000 7f13c3400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f13bc15cf70 con 0x7f13bc160710 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c1400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13bc0d07c0 0x7f13bc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13bbe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f13bc160af0 0x7f13bc0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41768/0 (socket says 172.21.15.3:41768) 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13bbe00640 1 -- 172.21.15.3:0/3676099795 learned_addr learned my addr 172.21.15.3:0/3676099795 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c0a00640 1 --2- 172.21.15.3:0/3676099795 >> v2:172.21.15.70:3300/0 conn(0x7f13bc160710 0x7f13bc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c1400640 1 -- 172.21.15.3:0/3676099795 >> v2:172.21.15.70:3300/0 conn(0x7f13bc160710 msgr2=0x7f13bc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:16.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c1400640 1 --2- 172.21.15.3:0/3676099795 >> v2:172.21.15.70:3300/0 conn(0x7f13bc160710 0x7f13bc0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c1400640 1 -- 172.21.15.3:0/3676099795 >> v1:172.21.15.3:6789/0 conn(0x7f13bc160af0 legacy=0x7f13bc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13c1400640 1 -- 172.21.15.3:0/3676099795 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc15d250 con 0x7f13bc0d07c0 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.635+0000 7f13bb400640 1 -- 172.21.15.3:0/3676099795 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445733380 0 0) 0x7f13bc0cfdd0 con 0x7f13bc160af0 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.636+0000 7f13c1400640 1 --2- 172.21.15.3:0/3676099795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc0d07c0 0x7f13bc072070 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f13bc06ccc0 tx=0x7f13b402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5315a63d9854e219 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13bb400640 1 -- 172.21.15.3:0/3676099795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4007740 con 0x7f13bc0d07c0 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13bb400640 1 -- 172.21.15.3:0/3676099795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b40078e0 con 0x7f13bc0d07c0 2024-09-10T07:52:16.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13bb400640 1 -- 172.21.15.3:0/3676099795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4007cb0 con 0x7f13bc0d07c0 2024-09-10T07:52:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13c3400640 1 -- 172.21.15.3:0/3676099795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc0d07c0 msgr2=0x7f13bc072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13c3400640 1 --2- 172.21.15.3:0/3676099795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc0d07c0 0x7f13bc072070 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f13bc06ccc0 tx=0x7f13b402a680 comp rx=0 tx=0).stop 2024-09-10T07:52:16.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.637+0000 7f13c3400640 1 -- 172.21.15.3:0/3676099795 shutdown_connections 2024-09-10T07:52:16.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.638+0000 7f13c3400640 1 --2- 172.21.15.3:0/3676099795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc0d07c0 0x7f13bc072070 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.638+0000 7f13c3400640 1 --2- 172.21.15.3:0/3676099795 >> v2:172.21.15.70:3300/0 conn(0x7f13bc160710 0x7f13bc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.638+0000 7f13c3400640 1 -- 172.21.15.3:0/3676099795 >> 172.21.15.3:0/3676099795 conn(0x7f13bc06d000 msgr2=0x7f13bc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.638+0000 7f13c3400640 1 -- 172.21.15.3:0/3676099795 shutdown_connections 2024-09-10T07:52:16.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.638+0000 7f13c3400640 1 -- 172.21.15.3:0/3676099795 wait complete. 2024-09-10T07:52:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.639+0000 7f13c3400640 1 Processor -- start 2024-09-10T07:52:16.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.639+0000 7f13c3400640 1 -- start start 2024-09-10T07:52:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13bc0d07c0 0x7f13bc15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13bc160710 0x7f13bc15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc15d770 con 0x7f13bc160af0 2024-09-10T07:52:16.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc0cf800 con 0x7f13bc0d07c0 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.640+0000 7f13c3400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13bc15f600 con 0x7f13bc160710 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54244/0 (socket says 172.21.15.3:54244) 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 -- 172.21.15.3:0/1621896973 learned_addr learned my addr 172.21.15.3:0/1621896973 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13bbe00640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13bc160710 0x7f13bc15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c0a00640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13bc0d07c0 0x7f13bc15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 -- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13bc160710 msgr2=0x7f13bc15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13bc160710 0x7f13bc15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 -- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13bc0d07c0 msgr2=0x7f13bc15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13bc0d07c0 0x7f13bc15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.641+0000 7f13c1400640 1 -- 172.21.15.3:0/1621896973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc15e7d0 con 0x7f13bc160af0 2024-09-10T07:52:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.642+0000 7f13c1400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009e80 tx=0x7f13b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.644+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4040070 con 0x7f13bc160af0 2024-09-10T07:52:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.644+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b4031c40 con 0x7f13bc160af0 2024-09-10T07:52:16.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.644+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4009450 con 0x7f13bc160af0 2024-09-10T07:52:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.644+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc15eae0 con 0x7f13bc160af0 2024-09-10T07:52:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.644+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc15f320 con 0x7f13bc160af0 2024-09-10T07:52:16.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.645+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc0afd50 con 0x7f13bc160af0 2024-09-10T07:52:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.646+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13b4039040 con 0x7f13bc160af0 2024-09-10T07:52:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.647+0000 7f13b8c00640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 0x7f139c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.647+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f13b40abfe0 con 0x7f13bc160af0 2024-09-10T07:52:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.647+0000 7f13c0a00640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 0x7f139c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:16.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.648+0000 7f13c0a00640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 0x7f139c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f13bc153440 tx=0x7f13b0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:16.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.651+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f13bc15eae0 con 0x7f13bc160af0 2024-09-10T07:52:16.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.875+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13bc0729c0 con 0x7f13bc160af0 2024-09-10T07:52:16.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.876+0000 7f13b8c00640 1 -- 172.21.15.3:0/1621896973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5254 (secure 0 0 0) 0x7f13b407a2a0 con 0x7f13bc160af0 2024-09-10T07:52:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:16.875 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"btime":"2024-09-10T07:52:16:643731+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:16.643724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:16.875 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 90 2024-09-10T07:52:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 msgr2=0x7f139c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 0x7f139c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f13bc153440 tx=0x7f13b0009190 comp rx=0 tx=0).stop 2024-09-10T07:52:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 msgr2=0x7f13bc155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009e80 tx=0x7f13b4004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 shutdown_connections 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f139c06be30 0x7f139c06e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13bc160af0 0x7f13bc155640 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13bc160710 0x7f13bc15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 --2- 172.21.15.3:0/1621896973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13bc0d07c0 0x7f13bc15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.878+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 >> 172.21.15.3:0/1621896973 conn(0x7f13bc06d000 msgr2=0x7f13bc074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.879+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 shutdown_connections 2024-09-10T07:52:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:16.879+0000 7f13c3400640 1 -- 172.21.15.3:0/1621896973 wait complete. 2024-09-10T07:52:17.169 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:52:17.171+0000 7f8caa000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T07:52:17.169 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:52:17.171+0000 7f8caa000640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T07:52:17.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:52:00.997662+0000', 'modified': '2024-09-10T07:52:16.643724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7239, 'mds_1': 7236, 'mds_2': 6799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6799': {'gid': 6799, 'name': 'd', 'rank': 2, 'incarnation': 89, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.70:6835/3096818521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 3096818521}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3096818521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7236': {'gid': 7236, 'name': 'a', 'rank': 1, 'incarnation': 87, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.3:6836/3872292817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 3872292817}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 3872292817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7239': {'gid': 7239, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/2433616379', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 2433616379}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 2433616379}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7239, 'qdb_cluster': [6799, 7236, 7239]} 2024-09-10T07:52:17.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:52:17.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T07:52:17.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.455+0000 7f5eb3600640 1 Processor -- start 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.455+0000 7f5eb3600640 1 -- start start 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb3600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5eac161b30 0x7f5eac15f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb3600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 0x7f5eac06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb3600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac15fbb0 con 0x7f5eac1601f0 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb3600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac0b36a0 con 0x7f5eac161b30 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb3600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac0b3980 con 0x7f5eac15fdb0 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb0c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5eac161b30 0x7f5eac15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 0x7f5eac06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb0c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5eac161b30 0x7f5eac15f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:53620/0 (socket says 172.21.15.3:53620) 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 0x7f5eac06bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:37570/0 (socket says 172.21.15.3:37570) 2024-09-10T07:52:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb0c00640 1 -- 172.21.15.3:0/3625215400 learned_addr learned my addr 172.21.15.3:0/3625215400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.457+0000 7f5eb1600640 1 --1- 172.21.15.3:0/3625215400 >> v1:172.21.15.70:6789/0 conn(0x7f5eac1601f0 0x7f5eac0b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:38118/0 (socket says 172.21.15.3:38118) 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.458+0000 7f5eb0c00640 1 -- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 msgr2=0x7f5eac06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.458+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 0x7f5eac06bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.458+0000 7f5eb0c00640 1 -- 172.21.15.3:0/3625215400 >> v1:172.21.15.70:6789/0 conn(0x7f5eac1601f0 legacy=0x7f5eac0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.458+0000 7f5eb0c00640 1 -- 172.21.15.3:0/3625215400 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac0b3c60 con 0x7f5eac161b30 2024-09-10T07:52:17.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.458+0000 7f5eabe00640 1 -- 172.21.15.3:0/3625215400 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206019325 0 0) 0x7f5ea4001490 con 0x7f5eac1601f0 2024-09-10T07:52:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.459+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac161b30 0x7f5eac15f670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e815df7e47b3d49e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.459+0000 7f5eabe00640 1 -- 172.21.15.3:0/3625215400 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c037070 con 0x7f5eac161b30 2024-09-10T07:52:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.459+0000 7f5eabe00640 1 -- 172.21.15.3:0/3625215400 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c032440 con 0x7f5eac161b30 2024-09-10T07:52:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.459+0000 7f5eabe00640 1 -- 172.21.15.3:0/3625215400 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c032810 con 0x7f5eac161b30 2024-09-10T07:52:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 -- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac161b30 msgr2=0x7f5eac15f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 --2- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac161b30 0x7f5eac15f670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c02a270 comp rx=0 tx=0).stop 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 -- 172.21.15.3:0/3625215400 shutdown_connections 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 --2- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac15fdb0 0x7f5eac06bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 --2- 172.21.15.3:0/3625215400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac161b30 0x7f5eac15f670 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 -- 172.21.15.3:0/3625215400 >> 172.21.15.3:0/3625215400 conn(0x7f5eac06d000 msgr2=0x7f5eac06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 -- 172.21.15.3:0/3625215400 shutdown_connections 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.460+0000 7f5eb3600640 1 -- 172.21.15.3:0/3625215400 wait complete. 2024-09-10T07:52:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.462+0000 7f5eb3600640 1 Processor -- start 2024-09-10T07:52:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.462+0000 7f5eb3600640 1 -- start start 2024-09-10T07:52:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 0x7f5eac070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac161b30 0x7f5eac144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac161f10 con 0x7f5eac1601f0 2024-09-10T07:52:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0b4220 con 0x7f5eac15fdb0 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb3600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac0aefa0 con 0x7f5eac161b30 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 0x7f5eac070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54250/0 (socket says 172.21.15.3:54250) 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5ea3e00640 1 -- 172.21.15.3:0/68156228 learned_addr learned my addr 172.21.15.3:0/68156228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb1600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac161b30 0x7f5eac144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 -- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac161b30 msgr2=0x7f5eac144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac161b30 0x7f5eac144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 -- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 msgr2=0x7f5eac0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5eb0c00640 1 -- 172.21.15.3:0/68156228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac073110 con 0x7f5eac15fdb0 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.463+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 0x7f5eac070d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c031000 tx=0x7f5e9c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5ea9600640 1 -- 172.21.15.3:0/68156228 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c041020 con 0x7f5eac15fdb0 2024-09-10T07:52:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5ea9600640 1 -- 172.21.15.3:0/68156228 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c030640 con 0x7f5eac15fdb0 2024-09-10T07:52:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5ea9600640 1 -- 172.21.15.3:0/68156228 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c037040 con 0x7f5eac15fdb0 2024-09-10T07:52:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac0733d0 con 0x7f5eac15fdb0 2024-09-10T07:52:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.464+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac073990 con 0x7f5eac15fdb0 2024-09-10T07:52:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.467+0000 7f5ea9600640 1 -- 172.21.15.3:0/68156228 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e9c044020 con 0x7f5eac15fdb0 2024-09-10T07:52:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.468+0000 7f5ea9600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 0x7f5e8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.468+0000 7f5eb64cc740 1 Processor -- start 2024-09-10T07:52:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.468+0000 7f5ea9600640 1 -- 172.21.15.3:0/68156228 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f5e9c0ab3f0 con 0x7f5eac15fdb0 2024-09-10T07:52:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.468+0000 7f5eb64cc740 1 -- start start 2024-09-10T07:52:17.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.468+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 0x7f5e8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 0x7f5e8806e320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5eac14ea60 tx=0x7f5e94009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 0x55728104d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 0x557281047570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557280fab3b0 con 0x55728104ccf0 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557280fb0410 con 0x55728104e480 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.469+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557280f58620 con 0x55728104a4b0 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54260/0 (socket says 172.21.15.3:54260) 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 -- 172.21.15.3:0/1020102147 learned_addr learned my addr 172.21.15.3:0/1020102147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 0x55728104d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb1600640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 0x557281047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 -- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 msgr2=0x55728104d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 0x55728104d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 -- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 msgr2=0x557281047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 0x557281047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 -- 172.21.15.3:0/1020102147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557280cc32d0 con 0x55728104ccf0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 0x55728104d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb1600640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 0x557281047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.470+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0a97f0 tx=0x7f5e9c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363cc54e4688ef20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5ea1200640 1 -- 172.21.15.3:0/1020102147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0821e0 con 0x55728104ccf0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5ea1200640 1 -- 172.21.15.3:0/1020102147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0779b0 con 0x55728104ccf0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5ea1200640 1 -- 172.21.15.3:0/1020102147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c075d20 con 0x55728104ccf0 2024-09-10T07:52:17.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1020102147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 msgr2=0x557281092490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0a97f0 tx=0x7f5e9c009490 comp rx=0 tx=0).stop 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1020102147 shutdown_connections 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104e480 0x557281047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104a4b0 0x55728104d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1020102147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104ccf0 0x557281092490 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1020102147 >> 172.21.15.3:0/1020102147 conn(0x557281049740 msgr2=0x557281091960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1020102147 shutdown_connections 2024-09-10T07:52:17.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.471+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1020102147 wait complete. 2024-09-10T07:52:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.472+0000 7f5eb64cc740 1 -- start start 2024-09-10T07:52:17.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.472+0000 7f5eb64cc740 1 Processor -- start 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728106b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x55728104b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557280f6b780 con 0x55728104e480 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557280fac3d0 con 0x55728106dee0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557280fab3b0 con 0x55728104ccf0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5eb64cc740 10 client.0 _ll_register_callbacks cb 0x557280fa95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54268/0 (socket says 172.21.15.3:54268) 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.473+0000 7f5ea3e00640 1 -- 172.21.15.3:0/3651097650 learned_addr learned my addr 172.21.15.3:0/3651097650 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5eb1600640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x55728104b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728106b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 -- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 msgr2=0x55728106b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728106b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 -- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 msgr2=0x55728104b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x55728104b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 -- 172.21.15.3:0/3651097650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557280d8d6b0 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5e94014e60 tx=0x7f5e940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb04e7c41217c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5e99600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94026030 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f5e9400f040 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.474+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9c0779b0 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5e94032050 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 10 client.7422 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55728106bc70 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0779b0 con 0x55728104e480 2024-09-10T07:52:17.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f5e94035020 con 0x55728104e480 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 10 client.7422 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 10 client.7422 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 20 client.7422 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281076160 0x5572810af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 4 client.7422 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557280fbdb40 con 0x557281076160 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.475+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281076160 0x5572810af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.476+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281076160 0x5572810af890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12e4927e6d096919 server_cookie=fb9d58c53fec8dd2 in_seq=0 out_seq=0 2024-09-10T07:52:17.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.476+0000 7f5e99600640 10 client.7422 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:17.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5e99600640 1 -- 172.21.15.3:0/3651097650 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557280fbdb40 con 0x557281076160 2024-09-10T07:52:17.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5e99600640 10 client.7422 handle_command_reply: tid=0 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 client.7422 shutdown 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 msgr2=0x55728106d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5e94014e60 tx=0x7f5e940046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 shutdown_connections 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281076160 0x5572810af890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x55728104b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x55728106d9a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/3651097650 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728106b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5e99600640 0 client.7422 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.477+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 >> 172.21.15.3:0/3651097650 conn(0x55728104a4b0 msgr2=0x557281091080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.478+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 shutdown_connections 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.478+0000 7f5eb64cc740 1 -- 172.21.15.3:0/3651097650 wait complete. 2024-09-10T07:52:17.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.478+0000 7f5eb64cc740 20 client.7422 trim_cache size 0 max 16384 2024-09-10T07:52:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.496+0000 7f5eb64cc740 1 Processor -- start 2024-09-10T07:52:17.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.496+0000 7f5eb64cc740 1 -- start start 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104ccf0 0x55728104aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728106dee0 0x5572810936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac200030 con 0x55728104e480 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac200700 con 0x55728104ccf0 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.497+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac157b30 con 0x55728106dee0 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54270/0 (socket says 172.21.15.3:54270) 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 -- 172.21.15.3:0/1846189182 learned_addr learned my addr 172.21.15.3:0/1846189182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104ccf0 0x55728104aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5eb1600640 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728106dee0 0x5572810936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 -- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728106dee0 msgr2=0x5572810936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728106dee0 0x5572810936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 -- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104ccf0 msgr2=0x55728104aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104ccf0 0x55728104aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 -- 172.21.15.3:0/1846189182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557281047880 con 0x55728104e480 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5e940147d0 tx=0x7f5e9401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f5b657895a21b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5e9be00640 1 -- 172.21.15.3:0/1846189182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94026040 con 0x55728104e480 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5e9be00640 1 -- 172.21.15.3:0/1846189182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9403b070 con 0x55728104e480 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.498+0000 7f5e9be00640 1 -- 172.21.15.3:0/1846189182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9402b610 con 0x55728104e480 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1846189182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 msgr2=0x5572810a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5e940147d0 tx=0x7f5e9401d000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1846189182 shutdown_connections 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728106dee0 0x5572810936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810a0f90 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/1846189182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728104ccf0 0x55728104aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1846189182 >> 172.21.15.3:0/1846189182 conn(0x55728104a4b0 msgr2=0x557281074b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1846189182 shutdown_connections 2024-09-10T07:52:17.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.499+0000 7f5eb64cc740 1 -- 172.21.15.3:0/1846189182 wait complete. 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.500+0000 7f5eb64cc740 1 -- start start 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.500+0000 7f5eb64cc740 1 Processor -- start 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728110ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x557281093bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557281075fe0 con 0x55728104e480 2024-09-10T07:52:17.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55728106b720 con 0x55728106dee0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557281076600 con 0x55728104ccf0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb64cc740 10 client.0 _ll_register_callbacks cb 0x7f5e94009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54276/0 (socket says 172.21.15.3:54276) 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5ea3e00640 1 -- 172.21.15.3:0/781716276 learned_addr learned my addr 172.21.15.3:0/781716276 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728110ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.501+0000 7f5eb1600640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x557281093bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 -- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 msgr2=0x55728110ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728110ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 -- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 msgr2=0x557281093bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x557281093bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 -- 172.21.15.3:0/781716276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728108fb10 con 0x55728104e480 2024-09-10T07:52:17.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5eb1600640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x557281093bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5ea3e00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009430 tx=0x7f5e94019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b45d5027e64362 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5e99600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94026040 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f5e9402bc50 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.502+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9403b070 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5e94040400 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 10 client.7428 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x557281111a80 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9403b070 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f5e94035040 con 0x55728104e480 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 10 client.7428 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 10 client.7428 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:52:17.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 20 client.7428 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281070cc0 0x557281110a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 4 client.7428 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.503+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "status"}) -- 0x557281090bf0 con 0x557281070cc0 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281070cc0 0x557281110a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5eb0c00640 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281070cc0 0x557281110a00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45231191ee2217b8 server_cookie=5744b964e4bda64f in_seq=0 out_seq=0 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5e99600640 10 client.7428 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5e99600640 1 -- 172.21.15.3:0/781716276 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557281090bf0 con 0x557281070cc0 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5e99600640 10 client.7428 handle_command_reply: tid=0 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.504+0000 7f5eb64cc740 1 client.7428 shutdown 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 msgr2=0x5572810936b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009430 tx=0x7f5e94019000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 shutdown_connections 2024-09-10T07:52:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x557281070cc0 0x557281110a00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55728106dee0 0x557281093bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55728104e480 0x5572810936b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 --2- 172.21.15.3:0/781716276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55728104ccf0 0x55728110ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5e99600640 0 client.7428 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 >> 172.21.15.3:0/781716276 conn(0x55728104a4b0 msgr2=0x557281074a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 shutdown_connections 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 1 -- 172.21.15.3:0/781716276 wait complete. 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb64cc740 20 client.7428 trim_cache size 0 max 16384 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:17.509 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6799, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.862683895, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 90, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T07:52:17.510 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.002736551999998 2024-09-10T07:52:17.511 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:17.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 msgr2=0x7f5e8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 0x7f5e8806e320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5eac14ea60 tx=0x7f5e94009190 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 msgr2=0x7f5eac070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.505+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 0x7f5eac070d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c031000 tx=0x7f5e9c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 shutdown_connections 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5e8806be60 0x7f5e8806e320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5eac161b30 0x7f5eac144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5eac1601f0 0x7f5eac0d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 --2- 172.21.15.3:0/68156228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eac15fdb0 0x7f5eac070d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 >> 172.21.15.3:0/68156228 conn(0x7f5eac06d000 msgr2=0x7f5eac15dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 shutdown_connections 2024-09-10T07:52:17.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.506+0000 7f5eb3600640 1 -- 172.21.15.3:0/68156228 wait complete. 2024-09-10T07:52:17.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:52:17.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.684+0000 7f4fca000640 1 Processor -- start 2024-09-10T07:52:17.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.684+0000 7f4fca000640 1 -- start start 2024-09-10T07:52:17.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc415e8e0 0x7f4fc4162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fc40cf1f0 0x7f4fc40d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40d19d0 0x7f4fc40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc40b7920 con 0x7f4fc415e8e0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc40cf940 con 0x7f4fc40cf1f0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fca000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc40cfc20 con 0x7f4fc40d19d0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fc3400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc415e8e0 0x7f4fc4162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fc3400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fc415e8e0 0x7f4fc4162cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54286/0 (socket says 172.21.15.3:54286) 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.685+0000 7f4fc3400640 1 -- 172.21.15.3:0/2181777741 learned_addr learned my addr 172.21.15.3:0/2181777741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/2181777741 >> v2:172.21.15.70:3300/0 conn(0x7f4fc40cf1f0 0x7f4fc40d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40d19d0 0x7f4fc40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3400640 1 -- 172.21.15.3:0/2181777741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40d19d0 msgr2=0x7f4fc40b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3400640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40d19d0 0x7f4fc40b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3400640 1 -- 172.21.15.3:0/2181777741 >> v2:172.21.15.70:3300/0 conn(0x7f4fc40cf1f0 msgr2=0x7f4fc40d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3400640 1 --2- 172.21.15.3:0/2181777741 >> v2:172.21.15.70:3300/0 conn(0x7f4fc40cf1f0 0x7f4fc40d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.686+0000 7f4fc3400640 1 -- 172.21.15.3:0/2181777741 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc40d2ae0 con 0x7f4fc415e8e0 2024-09-10T07:52:17.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.687+0000 7f4fc3400640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc415e8e0 0x7f4fc4162cf0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4fc408d720 tx=0x7f4fb402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24c3ad5ec26f5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.687+0000 7f4fc2000640 1 -- 172.21.15.3:0/2181777741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4037070 con 0x7f4fc415e8e0 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.687+0000 7f4fc2000640 1 -- 172.21.15.3:0/2181777741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb4032440 con 0x7f4fc415e8e0 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.687+0000 7f4fc2000640 1 -- 172.21.15.3:0/2181777741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4032810 con 0x7f4fc415e8e0 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 -- 172.21.15.3:0/2181777741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc415e8e0 msgr2=0x7f4fc4162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc415e8e0 0x7f4fc4162cf0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4fc408d720 tx=0x7f4fb402a140 comp rx=0 tx=0).stop 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 -- 172.21.15.3:0/2181777741 shutdown_connections 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40d19d0 0x7f4fc40b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 --2- 172.21.15.3:0/2181777741 >> v2:172.21.15.70:3300/0 conn(0x7f4fc40cf1f0 0x7f4fc40d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 --2- 172.21.15.3:0/2181777741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc415e8e0 0x7f4fc4162cf0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 -- 172.21.15.3:0/2181777741 >> 172.21.15.3:0/2181777741 conn(0x7f4fc406bd00 msgr2=0x7f4fc406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 -- 172.21.15.3:0/2181777741 shutdown_connections 2024-09-10T07:52:17.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.688+0000 7f4fca000640 1 -- 172.21.15.3:0/2181777741 wait complete. 2024-09-10T07:52:17.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.690+0000 7f4fca000640 1 Processor -- start 2024-09-10T07:52:17.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.690+0000 7f4fca000640 1 -- start start 2024-09-10T07:52:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40cf1f0 0x7f4fc415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 0x7f4fc40d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc40b66b0 con 0x7f4fc40d19d0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc40cf630 con 0x7f4fc415e8e0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fca000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc40b6830 con 0x7f4fc40cf1f0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54290/0 (socket says 172.21.15.3:54290) 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40cf1f0 0x7f4fc415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 -- 172.21.15.3:0/1362789283 learned_addr learned my addr 172.21.15.3:0/1362789283 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 0x7f4fc40d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 -- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40cf1f0 msgr2=0x7f4fc415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40cf1f0 0x7f4fc415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 -- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 msgr2=0x7f4fc40d41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 0x7f4fc40d41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc2a00640 1 -- 172.21.15.3:0/1362789283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc415fcb0 con 0x7f4fc40d19d0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.691+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 0x7f4fc40d41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000d800 tx=0x7f4fb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fbb400640 1 -- 172.21.15.3:0/1362789283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0018020 con 0x7f4fc40d19d0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fbb400640 1 -- 172.21.15.3:0/1362789283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb0009640 con 0x7f4fc40d19d0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fbb400640 1 -- 172.21.15.3:0/1362789283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0009a10 con 0x7f4fc40d19d0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc415eef0 con 0x7f4fc40d19d0 2024-09-10T07:52:17.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.692+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc415f4b0 con 0x7f4fc40d19d0 2024-09-10T07:52:17.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.694+0000 7f4fbb400640 1 -- 172.21.15.3:0/1362789283 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fb001b020 con 0x7f4fc40d19d0 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.695+0000 7f4fbb400640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 0x7f4fa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.695+0000 7f4fbb400640 1 -- 172.21.15.3:0/1362789283 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f4fb008afe0 con 0x7f4fc40d19d0 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.695+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 0x7f4fa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.695+0000 7f4fcceb2740 1 Processor -- start 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.696+0000 7f4fcceb2740 1 -- start start 2024-09-10T07:52:17.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.696+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 0x7f4fa406e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4034000 tx=0x7f4fb402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 0x56337bb9f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bb99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56337bafd3b0 con 0x56337bb9ecf0 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56337bb02410 con 0x56337bba0480 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56337baaa620 con 0x56337bb9c4b0 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54306/0 (socket says 172.21.15.3:54306) 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 -- 172.21.15.3:0/1357403037 learned_addr learned my addr 172.21.15.3:0/1357403037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bb99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 0x56337bb9f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 -- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 msgr2=0x56337bb9f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 0x56337bb9f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 -- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 msgr2=0x56337bb99570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bb99570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc3400640 1 -- 172.21.15.3:0/1357403037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337b8152d0 con 0x56337bb9ecf0 2024-09-10T07:52:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.697+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 0x56337bb9f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402a730 tx=0x7f4fb402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547565997cee481a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4faf400640 1 -- 172.21.15.3:0/1357403037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb404e070 con 0x56337bb9ecf0 2024-09-10T07:52:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4faf400640 1 -- 172.21.15.3:0/1357403037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb4032b80 con 0x56337bb9ecf0 2024-09-10T07:52:17.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4faf400640 1 -- 172.21.15.3:0/1357403037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4037040 con 0x56337bb9ecf0 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1357403037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 msgr2=0x56337bbe4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.698+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402a730 tx=0x7f4fb402c000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1357403037 shutdown_connections 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bb99570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9c4b0 0x56337bb9f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1357403037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9ecf0 0x56337bbe4490 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1357403037 >> 172.21.15.3:0/1357403037 conn(0x56337bb9b740 msgr2=0x56337bbe3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1357403037 shutdown_connections 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1357403037 wait complete. 2024-09-10T07:52:17.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.699+0000 7f4fcceb2740 1 -- start start 2024-09-10T07:52:17.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.700+0000 7f4fcceb2740 1 Processor -- start 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bbbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bbbcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56337babd780 con 0x56337bb9d2c0 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56337bafe3d0 con 0x56337bba0480 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56337bb9abe0 con 0x56337bb9ecf0 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fcceb2740 10 client.0 _ll_register_callbacks cb 0x56337bafb5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54312/0 (socket says 172.21.15.3:54312) 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 -- 172.21.15.3:0/784467803 learned_addr learned my addr 172.21.15.3:0/784467803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bbbcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3400640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bbbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 -- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 msgr2=0x56337bbbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bbbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 -- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 msgr2=0x56337bbbcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bbbcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc3e00640 1 -- 172.21.15.3:0/784467803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337b8df6b0 con 0x56337bb9d2c0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.701+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bbbcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc007d60 tx=0x7f4fbc012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64de109e4ab35761 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc01c020 con 0x56337bb9d2c0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb4032b80 con 0x56337bb9d2c0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f4fbc018670 con 0x56337bb9d2c0 2024-09-10T07:52:17.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc026050 con 0x56337bb9d2c0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fcceb2740 10 client.7440 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56337bbbef40 con 0x56337bb9d2c0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.702+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb4032b80 con 0x56337bb9d2c0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f4fbc029020 con 0x56337bb9d2c0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 10 client.7440 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 10 client.7440 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 20 client.7440 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbf7620 0x56337bbf7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 4 client.7440 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337bb0fb40 con 0x56337bbf7620 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fc3400640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbf7620 0x56337bbf7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fc3400640 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbf7620 0x56337bbf7a00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69c158e58262d790 server_cookie=c47929a6cb5997a6 in_seq=0 out_seq=0 2024-09-10T07:52:17.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.703+0000 7f4fa3e00640 10 client.7440 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:17.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.704+0000 7f4fa3e00640 1 -- 172.21.15.3:0/784467803 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56337bb0fb40 con 0x56337bbf7620 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.704+0000 7f4fa3e00640 10 client.7440 handle_command_reply: tid=0 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.704+0000 7f4fcceb2740 1 client.7440 shutdown 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 msgr2=0x56337bb9d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc007d60 tx=0x7f4fbc012610 comp rx=0 tx=0).stop 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 shutdown_connections 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bb9d2c0 0x56337bb9d720 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bba0480 0x56337bbbcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbf7620 0x56337bbf7a00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/784467803 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bbbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 >> 172.21.15.3:0/784467803 conn(0x56337bb9c4b0 msgr2=0x56337bbe2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fa3e00640 0 client.7440 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 shutdown_connections 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 1 -- 172.21.15.3:0/784467803 wait complete. 2024-09-10T07:52:17.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.705+0000 7f4fcceb2740 20 client.7440 trim_cache size 0 max 16384 2024-09-10T07:52:17.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.724+0000 7f4fcceb2740 1 Processor -- start 2024-09-10T07:52:17.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.724+0000 7f4fcceb2740 1 -- start start 2024-09-10T07:52:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 0x56337bbf7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bc07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4200c60 con 0x56337bba0480 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ffec0 con 0x56337bb9d2c0 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc4200590 con 0x56337bb9ecf0 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54318/0 (socket says 172.21.15.3:54318) 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 -- 172.21.15.3:0/1525436153 learned_addr learned my addr 172.21.15.3:0/1525436153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bc07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 0x56337bbf7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 -- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 msgr2=0x56337bc07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bc07460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 -- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 msgr2=0x56337bbf7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 0x56337bbf7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3e00640 1 -- 172.21.15.3:0/1525436153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337bbe19e0 con 0x56337bba0480 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.725+0000 7f4fc3400640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 0x56337bbf7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc002230 tx=0x7f4fbc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6540f7e3afeb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4faea00640 1 -- 172.21.15.3:0/1525436153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc02f070 con 0x56337bba0480 2024-09-10T07:52:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4faea00640 1 -- 172.21.15.3:0/1525436153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fbc018cb0 con 0x56337bba0480 2024-09-10T07:52:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4faea00640 1 -- 172.21.15.3:0/1525436153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc01c040 con 0x56337bba0480 2024-09-10T07:52:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1525436153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 msgr2=0x56337bc079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc002230 tx=0x7f4fbc018000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1525436153 shutdown_connections 2024-09-10T07:52:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc079a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9ecf0 0x56337bc07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/1525436153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9d2c0 0x56337bbf7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.726+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1525436153 >> 172.21.15.3:0/1525436153 conn(0x56337bb9c4b0 msgr2=0x56337bb94e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.727+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1525436153 shutdown_connections 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.727+0000 7f4fcceb2740 1 -- 172.21.15.3:0/1525436153 wait complete. 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.727+0000 7f4fcceb2740 1 -- start start 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.727+0000 7f4fcceb2740 1 Processor -- start 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9d2c0 0x56337bc61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 0x56337bc06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56337bbbf220 con 0x56337bba0480 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41fef60 con 0x56337bb9ecf0 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56337bb9a0e0 con 0x56337bb9d2c0 2024-09-10T07:52:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fcceb2740 10 client.0 _ll_register_callbacks cb 0x7f4fbc009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.728+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54334/0 (socket says 172.21.15.3:54334) 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 -- 172.21.15.3:0/3486903066 learned_addr learned my addr 172.21.15.3:0/3486903066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3400640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9d2c0 0x56337bc61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 0x56337bc06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 -- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9d2c0 msgr2=0x56337bc61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9d2c0 0x56337bc61230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 -- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 msgr2=0x56337bc06450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 0x56337bc06450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 -- 172.21.15.3:0/3486903066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337bb99ad0 con 0x56337bba0480 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc2a00640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 0x56337bc06450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fc3e00640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00b010 tx=0x7f4fbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76b455a29a3ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.729+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc039020 con 0x56337bba0480 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f4fbc036b00 con 0x56337bba0480 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc018cb0 con 0x56337bba0480 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc02f040 con 0x56337bba0480 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 10 client.7446 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56337bc62720 con 0x56337bba0480 2024-09-10T07:52:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fbc018cb0 con 0x56337bba0480 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f4fbc029040 con 0x56337bba0480 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 10 client.7446 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 10 client.7446 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 20 client.7446 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbc4c00 0x56337bbc4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 4 client.7446 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "status"}) -- 0x56337bb09e90 con 0x56337bbc4c00 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.730+0000 7f4fc3400640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbc4c00 0x56337bbc4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fc3400640 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbc4c00 0x56337bbc4fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77cc8ad04c2d257 server_cookie=60c0da27a762f168 in_seq=0 out_seq=0 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fa3e00640 10 client.7446 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fa3e00640 1 -- 172.21.15.3:0/3486903066 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x7f4fb405a160 con 0x56337bbc4c00 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fa3e00640 10 client.7446 handle_command_reply: tid=0 2024-09-10T07:52:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 client.7446 shutdown 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 msgr2=0x56337bc06990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00b010 tx=0x7f4fbc004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 shutdown_connections 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56337bba0480 0x56337bc06990 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56337bb9ecf0 0x56337bc06450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56337bbc4c00 0x56337bbc4fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 --2- 172.21.15.3:0/3486903066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56337bb9d2c0 0x56337bc61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 >> 172.21.15.3:0/3486903066 conn(0x56337bb9c4b0 msgr2=0x56337bb9cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.731+0000 7f4fa3e00640 0 client.7446 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 shutdown_connections 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fcceb2740 1 -- 172.21.15.3:0/3486903066 wait complete. 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fcceb2740 20 client.7446 trim_cache size 0 max 16384 2024-09-10T07:52:17.735 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7236, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.1016526830000002, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 90, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.222720171999999 2024-09-10T07:52:17.736 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 msgr2=0x7f4fa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 0x7f4fa406e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4034000 tx=0x7f4fb402e000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 msgr2=0x7f4fc40d3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000d800 tx=0x7f4fb000dcc0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 shutdown_connections 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fa406be30 0x7f4fa406e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc415e8e0 0x7f4fc40d41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fc40d19d0 0x7f4fc40d3c60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 --2- 172.21.15.3:0/1362789283 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc40cf1f0 0x7f4fc415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.732+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 >> 172.21.15.3:0/1362789283 conn(0x7f4fc406bd00 msgr2=0x7f4fc415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.733+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 shutdown_connections 2024-09-10T07:52:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.733+0000 7f4fca000640 1 -- 172.21.15.3:0/1362789283 wait complete. 2024-09-10T07:52:17.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:52:17.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.905+0000 7f1eb0000640 1 Processor -- start 2024-09-10T07:52:17.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.905+0000 7f1eb0000640 1 -- start start 2024-09-10T07:52:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.906+0000 7f1eb0000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 0x7f1ea80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.906+0000 7f1eb0000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ea80b40d0 0x7f1ea80b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.906+0000 7f1eb0000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea80cf930 con 0x7f1ea80cd7d0 2024-09-10T07:52:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.906+0000 7f1eb0000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea80cfc10 con 0x7f1ea80b40d0 2024-09-10T07:52:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.906+0000 7f1eb0000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea80b38d0 con 0x7f1ea8162ac0 2024-09-10T07:52:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ea80b40d0 0x7f1ea80b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1ead600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 0x7f1ea80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ea80b40d0 0x7f1ea80b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54340/0 (socket says 172.21.15.3:54340) 2024-09-10T07:52:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 -- 172.21.15.3:0/3130862782 learned_addr learned my addr 172.21.15.3:0/3130862782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 -- 172.21.15.3:0/3130862782 >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 msgr2=0x7f1ea80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 --2- 172.21.15.3:0/3130862782 >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 0x7f1ea80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 -- 172.21.15.3:0/3130862782 >> v1:172.21.15.70:6789/0 conn(0x7f1ea80cd7d0 legacy=0x7f1ea8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1eae000640 1 -- 172.21.15.3:0/3130862782 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea80b3bb0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.907+0000 7f1ead600640 1 --2- 172.21.15.3:0/3130862782 >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 0x7f1ea80cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.908+0000 7f1eae000640 1 --2- 172.21.15.3:0/3130862782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea80b44e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1ea806e300 tx=0x7f1ea002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fccc938fc2bbe57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.908+0000 7f1ea7e00640 1 -- 172.21.15.3:0/3130862782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0036070 con 0x7f1ea80b40d0 2024-09-10T07:52:17.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.908+0000 7f1ea7e00640 1 -- 172.21.15.3:0/3130862782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea0031600 con 0x7f1ea80b40d0 2024-09-10T07:52:17.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.908+0000 7f1ea7e00640 1 -- 172.21.15.3:0/3130862782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00319d0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 -- 172.21.15.3:0/3130862782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 msgr2=0x7f1ea80b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3130862782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea80b44e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1ea806e300 tx=0x7f1ea002a740 comp rx=0 tx=0).stop 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 -- 172.21.15.3:0/3130862782 shutdown_connections 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3130862782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea80b44e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3130862782 >> v2:172.21.15.70:3300/0 conn(0x7f1ea8162ac0 0x7f1ea80cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 -- 172.21.15.3:0/3130862782 >> 172.21.15.3:0/3130862782 conn(0x7f1ea806d000 msgr2=0x7f1ea806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 -- 172.21.15.3:0/3130862782 shutdown_connections 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.909+0000 7f1eb0000640 1 -- 172.21.15.3:0/3130862782 wait complete. 2024-09-10T07:52:17.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.911+0000 7f1eb0000640 1 Processor -- start 2024-09-10T07:52:17.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.911+0000 7f1eb0000640 1 -- start start 2024-09-10T07:52:17.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 0x7f1ea80b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea8162ac0 0x7f1ea8164420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8162ea0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8071590 con 0x7f1ea8162ac0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1eb0000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea8163020 con 0x7f1ea80cd7d0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1ead600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1ead600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54344/0 (socket says 172.21.15.3:54344) 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.912+0000 7f1ead600640 1 -- 172.21.15.3:0/3929657849 learned_addr learned my addr 172.21.15.3:0/3929657849 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1eae000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea8162ac0 0x7f1ea8164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 0x7f1ea80b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 -- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 msgr2=0x7f1ea80b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 0x7f1ea80b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 -- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea8162ac0 msgr2=0x7f1ea8164420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea8162ac0 0x7f1ea8164420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 -- 172.21.15.3:0/3929657849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea80b1c00 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 0x7f1ea80b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ead600640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00e960 tx=0x7f1e9c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ea5600640 1 -- 172.21.15.3:0/3929657849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c017070 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ea5600640 1 -- 172.21.15.3:0/3929657849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e9c00ca00 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.913+0000 7f1ea5600640 1 -- 172.21.15.3:0/3929657849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00cdd0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.914+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea80b1ee0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.914+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea80b24a0 con 0x7f1ea80b40d0 2024-09-10T07:52:17.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.916+0000 7f1ea5600640 1 -- 172.21.15.3:0/3929657849 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e9c01a070 con 0x7f1ea80b40d0 2024-09-10T07:52:17.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.917+0000 7f1ea5600640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 0x7f1e8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.917+0000 7f1ea5600640 1 -- 172.21.15.3:0/3929657849 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1e9c08b170 con 0x7f1ea80b40d0 2024-09-10T07:52:17.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.917+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 0x7f1e8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.917+0000 7f1eb2ea4740 1 Processor -- start 2024-09-10T07:52:17.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.917+0000 7f1eb2ea4740 1 -- start start 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 0x7f1e8806e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b1030 tx=0x7f1e94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4ccf0 0x55efffa92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4a4b0 0x55efffa4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55efff9ab3b0 con 0x55efffa4e480 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55efff9b0410 con 0x55efffa4ccf0 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55efff958620 con 0x55efffa4a4b0 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.918+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54348/0 (socket says 172.21.15.3:54348) 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 -- 172.21.15.3:0/4011732909 learned_addr learned my addr 172.21.15.3:0/4011732909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eacc00640 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4a4b0 0x55efffa4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1ead600640 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4ccf0 0x55efffa92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 -- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4a4b0 msgr2=0x55efffa4d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4a4b0 0x55efffa4d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 -- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4ccf0 msgr2=0x55efffa92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4ccf0 0x55efffa92490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 -- 172.21.15.3:0/4011732909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efff6c32d0 con 0x55efffa4e480 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1eae000640 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009f10 tx=0x7f1ea002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb5616d134d067b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.919+0000 7f1e99c00640 1 -- 172.21.15.3:0/4011732909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0041070 con 0x55efffa4e480 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1e99c00640 1 -- 172.21.15.3:0/4011732909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea0031680 con 0x55efffa4e480 2024-09-10T07:52:17.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1e99c00640 1 -- 172.21.15.3:0/4011732909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0036040 con 0x55efffa4e480 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4011732909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 msgr2=0x55efffa47570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009f10 tx=0x7f1ea002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4011732909 shutdown_connections 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa47570 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4a4b0 0x55efffa4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4011732909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4ccf0 0x55efffa92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4011732909 >> 172.21.15.3:0/4011732909 conn(0x55efffa49740 msgr2=0x55efffa91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4011732909 shutdown_connections 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.920+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4011732909 wait complete. 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.921+0000 7f1eb2ea4740 1 -- start start 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.921+0000 7f1eb2ea4740 1 Processor -- start 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffa6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa6b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55efff96b780 con 0x55efffa75ed0 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55efff9ac3d0 con 0x55efffa4e480 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55efffa48be0 con 0x55efffa4ccf0 2024-09-10T07:52:17.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eb2ea4740 10 client.0 _ll_register_callbacks cb 0x55efff9a95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54360/0 (socket says 172.21.15.3:54360) 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.922+0000 7f1eae000640 1 -- 172.21.15.3:0/2144363250 learned_addr learned my addr 172.21.15.3:0/2144363250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1ead600640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffa6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eacc00640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa6b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 -- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 msgr2=0x55efffa6cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffa6cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 -- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 msgr2=0x55efffa6b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa6b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 -- 172.21.15.3:0/2144363250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efff78d6b0 con 0x55efffa75ed0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eae000640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1ea002a780 tx=0x7f1ea0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722feb698776ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1e8e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0036070 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea0031680 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.923+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1ea0031a40 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1ea0052050 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 10 client.7458 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55efffa467e0 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea0031680 con 0x55efffa75ed0 2024-09-10T07:52:17.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f1ea0055020 con 0x55efffa75ed0 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 10 client.7458 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 10 client.7458 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 20 client.7458 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa5870 0x55efffaa5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 4 client.7458 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:17.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efff9bdb40 con 0x55efffaa5870 2024-09-10T07:52:17.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.924+0000 7f1ead600640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa5870 0x55efffaa5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.925+0000 7f1ead600640 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa5870 0x55efffaa5c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3096b5812e3873 server_cookie=2563c0a65a6b4ba2 in_seq=0 out_seq=0 2024-09-10T07:52:17.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.925+0000 7f1e8e000640 10 client.7458 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1e8e000640 1 -- 172.21.15.3:0/2144363250 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efff9bdb40 con 0x55efffaa5870 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1e8e000640 10 client.7458 handle_command_reply: tid=0 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 client.7458 shutdown 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 msgr2=0x55efffa72d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1ea002a780 tx=0x7f1ea0004ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 shutdown_connections 2024-09-10T07:52:17.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa5870 0x55efffaa5c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa72d20 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa6b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/2144363250 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffa6cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1e8e000640 0 client.7458 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 >> 172.21.15.3:0/2144363250 conn(0x55efffa4a4b0 msgr2=0x55efffa8f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 shutdown_connections 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/2144363250 wait complete. 2024-09-10T07:52:17.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.926+0000 7f1eb2ea4740 20 client.7458 trim_cache size 0 max 16384 2024-09-10T07:52:17.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.945+0000 7f1eb2ea4740 1 Processor -- start 2024-09-10T07:52:17.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.945+0000 7f1eb2ea4740 1 -- start start 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffab5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa8ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81f9d00 con 0x55efffa75ed0 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8204a10 con 0x55efffa4e480 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea80b0ce0 con 0x55efffa4ccf0 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54362/0 (socket says 172.21.15.3:54362) 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 -- 172.21.15.3:0/4001589611 learned_addr learned my addr 172.21.15.3:0/4001589611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eacc00640 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa8ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1ead600640 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffab5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 -- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 msgr2=0x55efffab5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffab5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 -- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 msgr2=0x55efffa8ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa8ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.946+0000 7f1eae000640 1 -- 172.21.15.3:0/4001589611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efffa73090 con 0x55efffa75ed0 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1eae000640 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009680 tx=0x7f1ea0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea93a1895a607be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1e99200640 1 -- 172.21.15.3:0/4001589611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0041050 con 0x55efffa75ed0 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1e99200640 1 -- 172.21.15.3:0/4001589611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea005a020 con 0x55efffa75ed0 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1e99200640 1 -- 172.21.15.3:0/4001589611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0031c40 con 0x55efffa75ed0 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4001589611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 msgr2=0x55efffa8f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.947+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009680 tx=0x7f1ea0044000 comp rx=0 tx=0).stop 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4001589611 shutdown_connections 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa75ed0 0x55efffa8f9e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa4e480 0x55efffa8ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/4001589611 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffab5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4001589611 >> 172.21.15.3:0/4001589611 conn(0x55efffa4a4b0 msgr2=0x55efffa94e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4001589611 shutdown_connections 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/4001589611 wait complete. 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.948+0000 7f1eb2ea4740 1 -- start start 2024-09-10T07:52:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.949+0000 7f1eb2ea4740 1 Processor -- start 2024-09-10T07:52:17.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.949+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffb0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.949+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.949+0000 7f1eb2ea4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa75ed0 0x55efffa4b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.949+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55efffa6d170 con 0x55efffa4e480 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55efffa46a60 con 0x55efffa75ed0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eb2ea4740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55efffa47450 con 0x55efffa4ccf0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eb2ea4740 10 client.0 _ll_register_callbacks cb 0x7f1e9c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54364/0 (socket says 172.21.15.3:54364) 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 -- 172.21.15.3:0/3834184058 learned_addr learned my addr 172.21.15.3:0/3834184058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1ead600640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffb0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eae000640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa75ed0 0x55efffa4b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 -- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 msgr2=0x55efffb0fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffb0fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 -- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa75ed0 msgr2=0x55efffa4b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa75ed0 0x55efffa4b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 -- 172.21.15.3:0/3834184058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efffa6c180 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1eacc00640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007fd0 tx=0x7f1e94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c5f4f30d91e0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.950+0000 7f1e8e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94027070 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1e94023450 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea005a020 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1e94032050 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 10 client.7464 fetch_fsmap learned FSMap version 90 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55efffaa9c00 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea005a020 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 90) ==== 3542+0+0 (secure 0 0 0) 0x7f1e94035020 con 0x55efffa4e480 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 10 client.7464 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T07:52:17.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 10 client.7464 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 20 client.7464 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa8b80 0x55efffaa8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 4 client.7464 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.951+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "status"}) -- 0x55efff9b7e90 con 0x55efffaa8b80 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1ead600640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa8b80 0x55efffaa8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1ead600640 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa8b80 0x55efffaa8f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9129c565d618a1ee server_cookie=6a9ff9242ebd2605 in_seq=0 out_seq=0 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1e8e000640 10 client.7464 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1e8e000640 1 -- 172.21.15.3:0/3834184058 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55efff9b7e90 con 0x55efffaa8b80 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1e8e000640 10 client.7464 handle_command_reply: tid=0 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.952+0000 7f1eb2ea4740 1 client.7464 shutdown 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 msgr2=0x55efffa4af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007fd0 tx=0x7f1e94000bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 shutdown_connections 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55efffaa8b80 0x55efffaa8f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55efffa75ed0 0x55efffa4b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55efffa4e480 0x55efffa4af50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 --2- 172.21.15.3:0/3834184058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55efffa4ccf0 0x55efffb0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1e8e000640 0 client.7464 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 >> 172.21.15.3:0/3834184058 conn(0x55efffa4a4b0 msgr2=0x55efffa6dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 shutdown_connections 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 1 -- 172.21.15.3:0/3834184058 wait complete. 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.953+0000 7f1eb2ea4740 20 client.7464 trim_cache size 0 max 16384 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:17.956 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7239, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 16.376300454999999, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 90, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.441765175 2024-09-10T07:52:17.957 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 msgr2=0x7f1e8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 0x7f1e8806e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b1030 tx=0x7f1e94009040 comp rx=0 tx=0).stop 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 msgr2=0x7f1ea8151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00e960 tx=0x7f1e9c00ee20 comp rx=0 tx=0).stop 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 shutdown_connections 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1e8806be30 0x7f1e8806e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea8162ac0 0x7f1ea8164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea80cd7d0 0x7f1ea80b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 --2- 172.21.15.3:0/3929657849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ea80b40d0 0x7f1ea8151d60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 >> 172.21.15.3:0/3929657849 conn(0x7f1ea806d000 msgr2=0x7f1ea8161a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 shutdown_connections 2024-09-10T07:52:17.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:17.954+0000 7f1eb0000640 1 -- 172.21.15.3:0/3929657849 wait complete. 2024-09-10T07:52:17.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.125+0000 7f0343600640 1 Processor -- start 2024-09-10T07:52:18.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.125+0000 7f0343600640 1 -- start start 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0343600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f033c15abc0 0x7f033c0cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0343600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c0ccf60 0x7f033c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0343600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033c15bfb0 con 0x7f033c15e700 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0343600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f033c0b4140 con 0x7f033c15abc0 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0343600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f033c15ee70 con 0x7f033c0ccf60 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f033be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f033c15e700 0x7f033c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41774/0 (socket says 172.21.15.3:41774) 2024-09-10T07:52:18.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0340c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f033c15abc0 0x7f033c0cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f033be00640 1 -- 172.21.15.3:0/2707182369 learned_addr learned my addr 172.21.15.3:0/2707182369 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0341600640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c0ccf60 0x7f033c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f033b400640 1 -- 172.21.15.3:0/2707182369 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124854260 0 0) 0x7f033c15bfb0 con 0x7f033c15e700 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f033b400640 1 -- 172.21.15.3:0/2707182369 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0324003520 con 0x7f033c15e700 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0340c00640 1 -- 172.21.15.3:0/2707182369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c0ccf60 msgr2=0x7f033c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.127+0000 7f0340c00640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c0ccf60 0x7f033c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.128+0000 7f0340c00640 1 -- 172.21.15.3:0/2707182369 >> v1:172.21.15.3:6789/0 conn(0x7f033c15e700 legacy=0x7f033c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.128+0000 7f0340c00640 1 -- 172.21.15.3:0/2707182369 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c15f150 con 0x7f033c15abc0 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0340c00640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c0cfba0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f03300097a0 tx=0x7f033002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277f88b8add49aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f033b400640 1 -- 172.21.15.3:0/2707182369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330037070 con 0x7f033c15abc0 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f033b400640 1 -- 172.21.15.3:0/2707182369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033002ae50 con 0x7f033c15abc0 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f033b400640 1 -- 172.21.15.3:0/2707182369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330032660 con 0x7f033c15abc0 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 -- 172.21.15.3:0/2707182369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 msgr2=0x7f033c0cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c0cfba0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f03300097a0 tx=0x7f033002a100 comp rx=0 tx=0).stop 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 -- 172.21.15.3:0/2707182369 shutdown_connections 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c0ccf60 0x7f033c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 --2- 172.21.15.3:0/2707182369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c0cfba0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 -- 172.21.15.3:0/2707182369 >> 172.21.15.3:0/2707182369 conn(0x7f033c06d000 msgr2=0x7f033c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.129+0000 7f0343600640 1 -- 172.21.15.3:0/2707182369 shutdown_connections 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.130+0000 7f0343600640 1 -- 172.21.15.3:0/2707182369 wait complete. 2024-09-10T07:52:18.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.131+0000 7f0343600640 1 Processor -- start 2024-09-10T07:52:18.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.131+0000 7f0343600640 1 -- start start 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f033c0ccf60 0x7f033c1e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 0x7f033c14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f033c0d0250 con 0x7f033c15abc0 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f033c15eb50 con 0x7f033c0ccf60 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f0343600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f033c0d03d0 con 0x7f033c15e700 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f033be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f033be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54386/0 (socket says 172.21.15.3:54386) 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.132+0000 7f033be00640 1 -- 172.21.15.3:0/1412502409 learned_addr learned my addr 172.21.15.3:0/1412502409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0341600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 0x7f033c14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0340c00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f033c0ccf60 0x7f033c1e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 -- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 msgr2=0x7f033c14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 0x7f033c14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 -- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f033c0ccf60 msgr2=0x7f033c1e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f033c0ccf60 0x7f033c1e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 -- 172.21.15.3:0/1412502409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c1e5e10 con 0x7f033c15abc0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0341600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 0x7f033c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f033be00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3c0 tx=0x7f032800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328017070 con 0x7f033c15abc0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03280064e0 con 0x7f033c15abc0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328004a00 con 0x7f033c15abc0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.133+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c1e60f0 con 0x7f033c15abc0 2024-09-10T07:52:18.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.134+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c1e66b0 con 0x7f033c15abc0 2024-09-10T07:52:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.134+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033c0d0800 con 0x7f033c15abc0 2024-09-10T07:52:18.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.135+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f032801a070 con 0x7f033c15abc0 2024-09-10T07:52:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.136+0000 7f0338c00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.136+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f032808a1e0 con 0x7f033c15abc0 2024-09-10T07:52:18.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.137+0000 7f0340c00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:18.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.137+0000 7f0340c00640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 0x7f031c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0330034000 tx=0x7f033002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:18.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.141+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03280584a0 con 0x7f033c15abc0 2024-09-10T07:52:18.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.366+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f033c0b4b30 con 0x7f033c15abc0 2024-09-10T07:52:18.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.367+0000 7f0338c00640 1 -- 172.21.15.3:0/1412502409 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5254 (secure 0 0 0) 0x7f032805c7a0 con 0x7f033c15abc0 2024-09-10T07:52:18.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:18.367 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"btime":"2024-09-10T07:52:16:643731+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:16.643724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:18.367 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 90 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 msgr2=0x7f031c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 0x7f031c06e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0330034000 tx=0x7f033002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 msgr2=0x7f033c1e4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3c0 tx=0x7f032800c880 comp rx=0 tx=0).stop 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 shutdown_connections 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f033c15e700 0x7f033c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f033c15abc0 0x7f033c1e4f80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 --2- 172.21.15.3:0/1412502409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f033c0ccf60 0x7f033c1e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:18.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 >> 172.21.15.3:0/1412502409 conn(0x7f033c06d000 msgr2=0x7f033c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:18.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 shutdown_connections 2024-09-10T07:52:18.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:18.369+0000 7f0343600640 1 -- 172.21.15.3:0/1412502409 wait complete. 2024-09-10T07:52:18.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:18.385 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:52:18.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:18.432 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:52:18.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:18.474 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:52:18.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:18.550 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:52:18.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:18.626 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:52:18.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:52:18.919 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:52:18.919 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:18.919 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:52:20.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:52:21.107 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:52:21.107 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:21.107 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:52:22.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:52:22.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:52:23.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:52:23.294 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:52:23.295 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:23.295 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:52:25.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:52:25.481 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:52:25.482 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:25.482 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:52:27.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:52:27.672 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:52:27.673 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:52:27.673 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:52:27.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:52:27.848 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:52:27.848 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:27.848 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:52:29.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:52:30.035 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:52:30.036 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:30.036 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:52:32.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:52:32.225 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-10T07:52:32.226 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:52:32.226 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:52:32.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-10T07:52:32.406 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:52:32.407 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:52:32.407 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:52:32.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-10T07:52:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.567+0000 7fbfab200640 1 Processor -- start 2024-09-10T07:52:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.567+0000 7fbfab200640 1 -- start start 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.568+0000 7fbfab200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfac0cfcf0 0x7fbfac0d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.568+0000 7fbfab200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfac0ce850 con 0x7fbfac15feb0 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.568+0000 7fbfab200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfac0ceb30 con 0x7fbfac15f940 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.568+0000 7fbfab200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbfac0d0670 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfa9400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbfac15feb0 0x7fbfac15d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58042/0 (socket says 172.21.15.3:58042) 2024-09-10T07:52:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfa9400640 1 -- 172.21.15.3:0/3444494110 learned_addr learned my addr 172.21.15.3:0/3444494110 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:32.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfaa800640 1 --2- 172.21.15.3:0/3444494110 >> v2:172.21.15.70:3300/0 conn(0x7fbfac0cfcf0 0x7fbfac0d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfa8a00640 1 -- 172.21.15.3:0/3444494110 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678095804 0 0) 0x7fbfac0ce850 con 0x7fbfac15feb0 2024-09-10T07:52:32.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfa8a00640 1 -- 172.21.15.3:0/3444494110 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf90003520 con 0x7fbfac15feb0 2024-09-10T07:52:32.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfaa800640 1 -- 172.21.15.3:0/3444494110 >> v1:172.21.15.70:6789/0 conn(0x7fbfac15f940 legacy=0x7fbfac0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfaa800640 1 -- 172.21.15.3:0/3444494110 >> v1:172.21.15.3:6789/0 conn(0x7fbfac15feb0 legacy=0x7fbfac15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.569+0000 7fbfaa800640 1 -- 172.21.15.3:0/3444494110 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac0cee10 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.570+0000 7fbfaa800640 1 --2- 172.21.15.3:0/3444494110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac0d0130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009d90 tx=0x7fbfa402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d74fa6fc0235a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfa8a00640 1 -- 172.21.15.3:0/3444494110 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031600 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfa8a00640 1 -- 172.21.15.3:0/3444494110 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa40317a0 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfa8a00640 1 -- 172.21.15.3:0/3444494110 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031b70 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 -- 172.21.15.3:0/3444494110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 msgr2=0x7fbfac0d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 --2- 172.21.15.3:0/3444494110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac0d0130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009d90 tx=0x7fbfa402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 -- 172.21.15.3:0/3444494110 shutdown_connections 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 --2- 172.21.15.3:0/3444494110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac0d0130 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 -- 172.21.15.3:0/3444494110 >> 172.21.15.3:0/3444494110 conn(0x7fbfac0b1e90 msgr2=0x7fbfac0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 -- 172.21.15.3:0/3444494110 shutdown_connections 2024-09-10T07:52:32.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.571+0000 7fbfab200640 1 -- 172.21.15.3:0/3444494110 wait complete. 2024-09-10T07:52:32.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.573+0000 7fbfab200640 1 Processor -- start 2024-09-10T07:52:32.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.573+0000 7fbfab200640 1 -- start start 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfac15feb0 0x7fbfac151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac0726f0 con 0x7fbfac15f940 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac071ea0 con 0x7fbfac0cfcf0 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfab200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac072870 con 0x7fbfac15feb0 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36672/0 (socket says 172.21.15.3:36672) 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfa9400640 1 -- 172.21.15.3:0/3164930658 learned_addr learned my addr 172.21.15.3:0/3164930658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfaa800640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfac15feb0 0x7fbfac151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.574+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 -- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfac15feb0 msgr2=0x7fbfac151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfac15feb0 0x7fbfac151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 -- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 msgr2=0x7fbfac157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 -- 172.21.15.3:0/3164930658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac152370 con 0x7fbfac15f940 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa9400640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c430 tx=0x7fbf9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa2000640 1 -- 172.21.15.3:0/3164930658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98017070 con 0x7fbfac15f940 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa2000640 1 -- 172.21.15.3:0/3164930658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98005ce0 con 0x7fbfac15f940 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfa2000640 1 -- 172.21.15.3:0/3164930658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980072d0 con 0x7fbfac15f940 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.575+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac152650 con 0x7fbfac15f940 2024-09-10T07:52:32.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.576+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac152c10 con 0x7fbfac15f940 2024-09-10T07:52:32.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.578+0000 7fbfa2000640 1 -- 172.21.15.3:0/3164930658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf9801a070 con 0x7fbfac15f940 2024-09-10T07:52:32.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.579+0000 7fbfa2000640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 0x7fbf8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.579+0000 7fbfa2000640 1 -- 172.21.15.3:0/3164930658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fbf9808a220 con 0x7fbfac15f940 2024-09-10T07:52:32.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.579+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 0x7fbf8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.579+0000 7fbfb3e17740 1 Processor -- start 2024-09-10T07:52:32.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 -- start start 2024-09-10T07:52:32.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 0x7fbf8c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c001780 tx=0x7fbf9c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5639183ea660 con 0x5639184829e0 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563918392680 con 0x5639184ccea0 2024-09-10T07:52:32.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.580+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5639183e6620 con 0x56391849c4f0 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:53016/0 (socket says 172.21.15.3:53016) 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36684/0 (socket says 172.21.15.3:36684) 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9e00640 1 -- 172.21.15.3:0/483758531 learned_addr learned my addr 172.21.15.3:0/483758531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfaa800640 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 -- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 msgr2=0x56391849c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 -- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 msgr2=0x5639184cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 -- 172.21.15.3:0/483758531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639181c77d0 con 0x5639184829e0 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.581+0000 7fbfa9400640 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbf98088680 tx=0x7fbf980584e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc0f24d89c6745f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbf96a00640 1 -- 172.21.15.3:0/483758531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98061010 con 0x5639184829e0 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbf96a00640 1 -- 172.21.15.3:0/483758531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf980567e0 con 0x5639184829e0 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbf96a00640 1 -- 172.21.15.3:0/483758531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007470 con 0x5639184829e0 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 -- 172.21.15.3:0/483758531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 msgr2=0x563918481240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbf98088680 tx=0x7fbf980584e0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 -- 172.21.15.3:0/483758531 shutdown_connections 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184829e0 0x563918481240 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/483758531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x56391849c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.582+0000 7fbfb3e17740 1 -- 172.21.15.3:0/483758531 >> 172.21.15.3:0/483758531 conn(0x563918483410 msgr2=0x5639184cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.583+0000 7fbfb3e17740 1 -- 172.21.15.3:0/483758531 shutdown_connections 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.583+0000 7fbfb3e17740 1 -- 172.21.15.3:0/483758531 wait complete. 2024-09-10T07:52:32.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.583+0000 7fbfb3e17740 1 -- start start 2024-09-10T07:52:32.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 Processor -- start 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x5639184c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563918481940 con 0x5639184b1ee0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5639183734c0 con 0x5639184ccea0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.584+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563918481c20 con 0x56391849c4f0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfb3e17740 10 client.0 _ll_register_callbacks cb 0x5639183e2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfa9e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x5639184c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36696/0 (socket says 172.21.15.3:36696) 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 -- 172.21.15.3:0/4086284200 learned_addr learned my addr 172.21.15.3:0/4086284200 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfa9400640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 -- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 msgr2=0x5639184c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x5639184c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 -- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 msgr2=0x5639184c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfaa800640 1 -- 172.21.15.3:0/4086284200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639180fd3f0 con 0x5639184b1ee0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.585+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x5639184c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbfaa800640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbfa402abc0 tx=0x7fbfa4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab06cff5ec9a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbf8b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403de60 con 0x5639184b1ee0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fbfa4031400 con 0x5639184b1ee0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf980567e0 con 0x5639184b1ee0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbfa4050050 con 0x5639184b1ee0 2024-09-10T07:52:32.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbfb3e17740 10 client.7485 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5639180bd0f0 con 0x5639184b1ee0 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.586+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf980567e0 con 0x5639184b1ee0 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fbfa4053020 con 0x5639184b1ee0 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 10 client.7485 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 10 client.7485 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 20 client.7485 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184dfc30 0x5639184e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 4 client.7485 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639183f7d90 con 0x5639184dfc30 2024-09-10T07:52:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.587+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184dfc30 0x5639184e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.588+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184dfc30 0x5639184e0080 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec207688de0e22f2 server_cookie=b8168b2c3fbeca62 in_seq=0 out_seq=0 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.588+0000 7fbf8b400640 10 client.7485 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.588+0000 7fbf8b400640 1 -- 172.21.15.3:0/4086284200 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639183f7d90 con 0x5639184dfc30 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.588+0000 7fbf8b400640 10 client.7485 handle_command_reply: tid=0 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.588+0000 7fbfb3e17740 1 client.7485 shutdown 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 msgr2=0x5639184db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbfa402abc0 tx=0x7fbfa4004ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 shutdown_connections 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184dfc30 0x5639184e0080 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639184b1ee0 0x5639184db1b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184ccea0 0x5639184c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/4086284200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56391849c4f0 0x5639184c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbf8b400640 0 client.7485 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 >> 172.21.15.3:0/4086284200 conn(0x5639184829e0 msgr2=0x5639184ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 shutdown_connections 2024-09-10T07:52:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 1 -- 172.21.15.3:0/4086284200 wait complete. 2024-09-10T07:52:32.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.589+0000 7fbfb3e17740 20 client.7485 trim_cache size 0 max 16384 2024-09-10T07:52:32.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.605+0000 7fbfb3e17740 1 Processor -- start 2024-09-10T07:52:32.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.605+0000 7fbfb3e17740 1 -- start start 2024-09-10T07:52:32.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 0x5639184e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184b1ee0 0x56391854bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1fe8a0 con 0x56391849c4f0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1fda70 con 0x5639184829e0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac1fe170 con 0x5639184b1ee0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36708/0 (socket says 172.21.15.3:36708) 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfa9400640 1 -- 172.21.15.3:0/2646207394 learned_addr learned my addr 172.21.15.3:0/2646207394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.606+0000 7fbfaa800640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184b1ee0 0x56391854bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 0x5639184e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 -- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184b1ee0 msgr2=0x56391854bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184b1ee0 0x56391854bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 -- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 msgr2=0x5639184e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 0x5639184e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 -- 172.21.15.3:0/2646207394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639184c21e0 con 0x56391849c4f0 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 0x5639184e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbfa9400640 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbf98088680 tx=0x7fbf9801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc6dd4a8abc683 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbf96000640 1 -- 172.21.15.3:0/2646207394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980567e0 con 0x56391849c4f0 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbf96000640 1 -- 172.21.15.3:0/2646207394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf980882a0 con 0x56391849c4f0 2024-09-10T07:52:32.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.607+0000 7fbf96000640 1 -- 172.21.15.3:0/2646207394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007470 con 0x56391849c4f0 2024-09-10T07:52:32.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 -- 172.21.15.3:0/2646207394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 msgr2=0x563918480e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbf98088680 tx=0x7fbf9801d000 comp rx=0 tx=0).stop 2024-09-10T07:52:32.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 -- 172.21.15.3:0/2646207394 shutdown_connections 2024-09-10T07:52:32.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184b1ee0 0x56391854bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918480e00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/2646207394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184829e0 0x5639184e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 -- 172.21.15.3:0/2646207394 >> 172.21.15.3:0/2646207394 conn(0x5639184db310 msgr2=0x5639184a5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 -- 172.21.15.3:0/2646207394 shutdown_connections 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.608+0000 7fbfb3e17740 1 -- 172.21.15.3:0/2646207394 wait complete. 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.609+0000 7fbfb3e17740 1 -- start start 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.609+0000 7fbfb3e17740 1 Processor -- start 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184829e0 0x5639184cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184b1ee0 0x563918534dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563918481940 con 0x56391849c4f0 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac154490 con 0x5639184b1ee0 2024-09-10T07:52:32.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5639184f0020 con 0x5639184829e0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfb3e17740 10 client.0 _ll_register_callbacks cb 0x7fbfa403d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36712/0 (socket says 172.21.15.3:36712) 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 -- 172.21.15.3:0/1555706290 learned_addr learned my addr 172.21.15.3:0/1555706290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184829e0 0x5639184cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfaa800640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184b1ee0 0x563918534dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 -- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184829e0 msgr2=0x5639184cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184829e0 0x5639184cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 -- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184b1ee0 msgr2=0x563918534dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184b1ee0 0x563918534dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.610+0000 7fbfa9400640 1 -- 172.21.15.3:0/1555706290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391850dc30 con 0x56391849c4f0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfa9400640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800cdf0 tx=0x7fbf98014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9ae7d1676aace9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007670 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fbf98071d40 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf980882a0 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbf98077300 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 10 client.7491 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5639184ee9e0 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf980882a0 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fbf98070170 con 0x56391849c4f0 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 10 client.7491 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 10 client.7491 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 20 client.7491 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184ccea0 0x56391849f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 4 client.7491 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.611+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639183f20e0 con 0x5639184ccea0 2024-09-10T07:52:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.612+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184ccea0 0x56391849f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.612+0000 7fbfa9e00640 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184ccea0 0x56391849f460 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5cb75c22a3d5a9 server_cookie=ab65234e4c4fdb8b in_seq=0 out_seq=0 2024-09-10T07:52:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.612+0000 7fbf8b400640 10 client.7491 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbf8b400640 1 -- 172.21.15.3:0/1555706290 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639183f20e0 con 0x5639184ccea0 2024-09-10T07:52:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbf8b400640 10 client.7491 handle_command_reply: tid=0 2024-09-10T07:52:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 client.7491 shutdown 2024-09-10T07:52:32.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 msgr2=0x563918534890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800cdf0 tx=0x7fbf98014070 comp rx=0 tx=0).stop 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 shutdown_connections 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5639184ccea0 0x56391849f460 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639184b1ee0 0x563918534dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56391849c4f0 0x563918534890 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 --2- 172.21.15.3:0/1555706290 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5639184829e0 0x5639184cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbf8b400640 0 client.7491 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.625+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 >> 172.21.15.3:0/1555706290 conn(0x5639184db310 msgr2=0x5639184c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 shutdown_connections 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfb3e17740 1 -- 172.21.15.3:0/1555706290 wait complete. 2024-09-10T07:52:32.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfb3e17740 20 client.7491 trim_cache size 0 max 16384 2024-09-10T07:52:32.627 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:32.627 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:52:32.627 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:52:32.627 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 msgr2=0x7fbf8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 0x7fbf8c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c001780 tx=0x7fbf9c009040 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 msgr2=0x7fbfac14a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.626+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c430 tx=0x7fbf9800c8f0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 shutdown_connections 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbf8c06be30 0x7fbf8c06e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbfac15feb0 0x7fbfac151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfac15f940 0x7fbfac14a1c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 --2- 172.21.15.3:0/3164930658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfac0cfcf0 0x7fbfac157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 >> 172.21.15.3:0/3164930658 conn(0x7fbfac0b1e90 msgr2=0x7fbfac0afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 shutdown_connections 2024-09-10T07:52:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.627+0000 7fbfab200640 1 -- 172.21.15.3:0/3164930658 wait complete. 2024-09-10T07:52:32.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2024-09-10T07:52:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.802+0000 7fcb6f000640 1 Processor -- start 2024-09-10T07:52:32.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.802+0000 7fcb6f000640 1 -- start start 2024-09-10T07:52:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 0x7fcb680cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb680b7780 0x7fcb680b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 0x7fcb680b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcb680b8120 con 0x7fcb680b6170 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcb680cfc90 con 0x7fcb680b7780 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.803+0000 7fcb6f000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcb680d2a90 con 0x7fcb6815e7e0 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb6d000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 0x7fcb680b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb680b7780 0x7fcb680b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb67e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 0x7fcb680cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcb680b7780 0x7fcb680b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:43738/0 (socket says 172.21.15.3:43738) 2024-09-10T07:52:32.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 -- 172.21.15.3:0/300973105 learned_addr learned my addr 172.21.15.3:0/300973105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 -- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 msgr2=0x7fcb680cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 0x7fcb680cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 -- 172.21.15.3:0/300973105 >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 msgr2=0x7fcb680b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/300973105 >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 0x7fcb680b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb5fe00640 1 -- 172.21.15.3:0/300973105 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb680d2d70 con 0x7fcb680b7780 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb67e00640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 0x7fcb680cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.804+0000 7fcb6d000640 1 --2- 172.21.15.3:0/300973105 >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 0x7fcb680b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.805+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb680b7be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcb680d53b0 tx=0x7fcb5002a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f633a36c17914fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb67400640 1 -- 172.21.15.3:0/300973105 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50037070 con 0x7fcb680b7780 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb67400640 1 -- 172.21.15.3:0/300973105 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50032440 con 0x7fcb680b7780 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb67400640 1 -- 172.21.15.3:0/300973105 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50032810 con 0x7fcb680b7780 2024-09-10T07:52:32.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 -- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 msgr2=0x7fcb680b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb680b7be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcb680d53b0 tx=0x7fcb5002a1d0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 -- 172.21.15.3:0/300973105 shutdown_connections 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 --2- 172.21.15.3:0/300973105 >> v2:172.21.15.3:3300/0 conn(0x7fcb680b6170 0x7fcb680b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb680b7be0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 --2- 172.21.15.3:0/300973105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb6815e7e0 0x7fcb680cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 -- 172.21.15.3:0/300973105 >> 172.21.15.3:0/300973105 conn(0x7fcb68072560 msgr2=0x7fcb68072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 -- 172.21.15.3:0/300973105 shutdown_connections 2024-09-10T07:52:32.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.806+0000 7fcb6f000640 1 -- 172.21.15.3:0/300973105 wait complete. 2024-09-10T07:52:32.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.808+0000 7fcb6f000640 1 Processor -- start 2024-09-10T07:52:32.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.808+0000 7fcb6f000640 1 -- start start 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 0x7fcb68150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb6814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680d3610 con 0x7fcb6815e7e0 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680d3330 con 0x7fcb680b7780 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6f000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcb680b3940 con 0x7fcb680b6170 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36742/0 (socket says 172.21.15.3:36742) 2024-09-10T07:52:32.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.809+0000 7fcb6d000640 1 -- 172.21.15.3:0/900893760 learned_addr learned my addr 172.21.15.3:0/900893760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 -- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 msgr2=0x7fcb68150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb6814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb67e00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 0x7fcb68150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 0x7fcb68150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 -- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 msgr2=0x7fcb6814d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb6814d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 -- 172.21.15.3:0/900893760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb680ab6a0 con 0x7fcb6815e7e0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb6814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb67e00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 0x7fcb68150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6d000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000eed0 tx=0x7fcb6000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb64c00640 1 -- 172.21.15.3:0/900893760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018030 con 0x7fcb6815e7e0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb64c00640 1 -- 172.21.15.3:0/900893760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6000bd20 con 0x7fcb6815e7e0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6814df90 con 0x7fcb6815e7e0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.810+0000 7fcb64c00640 1 -- 172.21.15.3:0/900893760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60003bd0 con 0x7fcb6815e7e0 2024-09-10T07:52:32.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.811+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6814e550 con 0x7fcb6815e7e0 2024-09-10T07:52:32.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.813+0000 7fcb64c00640 1 -- 172.21.15.3:0/900893760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb60005d10 con 0x7fcb6815e7e0 2024-09-10T07:52:32.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.813+0000 7fcb64c00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 0x7fcb4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.814+0000 7fcb64c00640 1 -- 172.21.15.3:0/900893760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fcb6008b580 con 0x7fcb6815e7e0 2024-09-10T07:52:32.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.814+0000 7fcb67e00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 0x7fcb4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.814+0000 7fcb71e44740 1 Processor -- start 2024-09-10T07:52:32.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.814+0000 7fcb71e44740 1 -- start start 2024-09-10T07:52:32.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.814+0000 7fcb67e00640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 0x7fcb4406e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ac0 tx=0x7fcb58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x560456256200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 0x560456256740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560456173660 con 0x560456225420 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56045611b680 con 0x56045620b480 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56045616f620 con 0x560456225800 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb67e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36752/0 (socket says 172.21.15.3:36752) 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.815+0000 7fcb67e00640 1 -- 172.21.15.3:0/1376318786 learned_addr learned my addr 172.21.15.3:0/1376318786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb6d000640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 0x560456256740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x560456256200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 -- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 msgr2=0x560456256200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x560456256200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 -- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 msgr2=0x560456256740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 0x560456256740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 -- 172.21.15.3:0/1376318786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560455f507d0 con 0x560456225420 2024-09-10T07:52:32.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb6d000640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 0x560456256740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:32.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x560456256200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:32.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcb580094b0 tx=0x7fcb58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e125c739e9e1bc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb5cc00640 1 -- 172.21.15.3:0/1376318786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5801c050 con 0x560456225420 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb5cc00640 1 -- 172.21.15.3:0/1376318786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb58026070 con 0x560456225420 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.816+0000 7fcb5cc00640 1 -- 172.21.15.3:0/1376318786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800f040 con 0x560456225420 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 -- 172.21.15.3:0/1376318786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 msgr2=0x560456255c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcb580094b0 tx=0x7fcb58013000 comp rx=0 tx=0).stop 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 -- 172.21.15.3:0/1376318786 shutdown_connections 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56045620b480 0x560456256740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x560456256200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1376318786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456255c40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 -- 172.21.15.3:0/1376318786 >> 172.21.15.3:0/1376318786 conn(0x56045620c220 msgr2=0x560456254840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 -- 172.21.15.3:0/1376318786 shutdown_connections 2024-09-10T07:52:32.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.817+0000 7fcb71e44740 1 -- 172.21.15.3:0/1376318786 wait complete. 2024-09-10T07:52:32.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.818+0000 7fcb71e44740 1 -- start start 2024-09-10T07:52:32.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.818+0000 7fcb71e44740 1 Processor -- start 2024-09-10T07:52:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456225420 0x560456239750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56045620a3e0 con 0x560456212dd0 2024-09-10T07:52:32.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5604560fc4c0 con 0x560456225420 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56045620a6c0 con 0x560456225800 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb71e44740 10 client.0 _ll_register_callbacks cb 0x56045616be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36764/0 (socket says 172.21.15.3:36764) 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb6d000640 1 -- 172.21.15.3:0/965014160 learned_addr learned my addr 172.21.15.3:0/965014160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb67e00640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456225420 0x560456239750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.819+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 -- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 msgr2=0x56045624df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 -- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456225420 msgr2=0x560456239750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456225420 0x560456239750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 -- 172.21.15.3:0/965014160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560455e863f0 con 0x560456212dd0 2024-09-10T07:52:32.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb6d000640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcb600596a0 tx=0x7fcb60007470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efcecdb02513bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb55600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6006dda0 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fcb6008b0c0 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb58026070 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcb60077170 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb71e44740 10 client.7503 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x560455e460f0 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.820+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb58026070 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fcb60058b60 con 0x560456212dd0 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 10 client.7503 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 10 client.7503 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 20 client.7503 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56045624f1f0 0x56045624f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 4 client.7503 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560456180d90 con 0x56045624f1f0 2024-09-10T07:52:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.821+0000 7fcb67e00640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56045624f1f0 0x56045624f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb67e00640 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56045624f1f0 0x56045624f5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b30ca1dd387de761 server_cookie=84cb8846099389fe in_seq=0 out_seq=0 2024-09-10T07:52:32.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb55600640 10 client.7503 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb55600640 1 -- 172.21.15.3:0/965014160 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560456180d90 con 0x56045624f1f0 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb55600640 10 client.7503 handle_command_reply: tid=0 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb71e44740 1 client.7503 shutdown 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 msgr2=0x56045624e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.822+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcb600596a0 tx=0x7fcb60007470 comp rx=0 tx=0).stop 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 shutdown_connections 2024-09-10T07:52:32.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456212dd0 0x56045624e470 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x56045624f1f0 0x56045624f5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 --2- 172.21.15.3:0/965014160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456225420 0x560456239750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 >> 172.21.15.3:0/965014160 conn(0x56045620b480 msgr2=0x560456208f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb55600640 0 client.7503 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 shutdown_connections 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 1 -- 172.21.15.3:0/965014160 wait complete. 2024-09-10T07:52:32.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.823+0000 7fcb71e44740 20 client.7503 trim_cache size 0 max 16384 2024-09-10T07:52:32.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.838+0000 7fcb71e44740 1 Processor -- start 2024-09-10T07:52:32.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.838+0000 7fcb71e44740 1 -- start start 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x560456250120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb68200e00 con 0x560456225420 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681fffd0 con 0x560456212dd0 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcb682006d0 con 0x560456225800 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36776/0 (socket says 172.21.15.3:36776) 2024-09-10T07:52:32.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.839+0000 7fcb5fe00640 1 -- 172.21.15.3:0/2100047457 learned_addr learned my addr 172.21.15.3:0/2100047457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb6d000640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb67e00640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x560456250120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 -- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 msgr2=0x56045624da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 -- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 msgr2=0x560456250120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x560456250120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 -- 172.21.15.3:0/2100047457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560456257030 con 0x560456225420 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb6d000640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb67e00640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x560456250120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002a210 tx=0x7fcb5002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8617d2a4f3afa7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb57e00640 1 -- 172.21.15.3:0/2100047457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50037040 con 0x560456225420 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb57e00640 1 -- 172.21.15.3:0/2100047457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50032780 con 0x560456225420 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.840+0000 7fcb57e00640 1 -- 172.21.15.3:0/2100047457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50032b30 con 0x560456225420 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 -- 172.21.15.3:0/2100047457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 msgr2=0x560456250660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb5002a210 tx=0x7fcb5002c000 comp rx=0 tx=0).stop 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 -- 172.21.15.3:0/2100047457 shutdown_connections 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225800 0x56045624da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225420 0x560456250660 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 --2- 172.21.15.3:0/2100047457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x560456250120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 -- 172.21.15.3:0/2100047457 >> 172.21.15.3:0/2100047457 conn(0x56045620b480 msgr2=0x560456214720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 -- 172.21.15.3:0/2100047457 shutdown_connections 2024-09-10T07:52:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.841+0000 7fcb71e44740 1 -- 172.21.15.3:0/2100047457 wait complete. 2024-09-10T07:52:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.842+0000 7fcb71e44740 1 -- start start 2024-09-10T07:52:32.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.842+0000 7fcb71e44740 1 Processor -- start 2024-09-10T07:52:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x5604562093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225420 0x5604562d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6814fb10 con 0x560456225800 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56045626ec70 con 0x560456212dd0 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560456267680 con 0x560456225420 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb71e44740 10 client.0 _ll_register_callbacks cb 0x7fcb60055a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36780/0 (socket says 172.21.15.3:36780) 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 -- 172.21.15.3:0/1186878400 learned_addr learned my addr 172.21.15.3:0/1186878400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb5fe00640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225420 0x5604562d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x5604562093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 -- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225420 msgr2=0x5604562d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225420 0x5604562d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 -- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 msgr2=0x5604562093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x5604562093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.843+0000 7fcb6d000640 1 -- 172.21.15.3:0/1186878400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045626e1e0 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb6d000640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcb6006f040 tx=0x7fcb6005c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebe3bd9fb9e9eac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb600594e0 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fcb6008b2d0 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb50032780 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcb600622f0 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 10 client.7509 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x560456267420 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50032780 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fcb60078b30 con 0x560456225800 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 10 client.7509 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:32.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 10 client.7509 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 20 client.7509 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x560456266f80 0x560456278980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 4 client.7509 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.844+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56045617b0e0 con 0x560456266f80 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.845+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x560456266f80 0x560456278980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.845+0000 7fcb55600640 10 client.7509 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:32.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.845+0000 7fcb67e00640 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x560456266f80 0x560456278980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=617dea921d703073 server_cookie=5098f4841157936d in_seq=0 out_seq=0 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb55600640 1 -- 172.21.15.3:0/1186878400 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56045617b0e0 con 0x560456266f80 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb55600640 10 client.7509 handle_command_reply: tid=0 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 client.7509 shutdown 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 msgr2=0x560456278440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcb6006f040 tx=0x7fcb6005c070 comp rx=0 tx=0).stop 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 shutdown_connections 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560456225800 0x560456278440 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x560456266f80 0x560456278980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560456225420 0x5604562d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 --2- 172.21.15.3:0/1186878400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560456212dd0 0x5604562093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb55600640 0 client.7509 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 >> 172.21.15.3:0/1186878400 conn(0x56045620b480 msgr2=0x560456250090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 shutdown_connections 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 1 -- 172.21.15.3:0/1186878400 wait complete. 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.854+0000 7fcb71e44740 20 client.7509 trim_cache size 0 max 16384 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:52:32.855 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:52:32.856 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:32.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 msgr2=0x7fcb4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 0x7fcb4406e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ac0 tx=0x7fcb58008040 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 msgr2=0x7fcb680ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000eed0 tx=0x7fcb6000b480 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 shutdown_connections 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcb4406be30 0x7fcb4406e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6815e7e0 0x7fcb680ab160 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb680b7780 0x7fcb6814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 --2- 172.21.15.3:0/900893760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcb680b6170 0x7fcb68150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 >> 172.21.15.3:0/900893760 conn(0x7fcb68072560 msgr2=0x7fcb680d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:32.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 shutdown_connections 2024-09-10T07:52:32.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:32.855+0000 7fcb6f000640 1 -- 172.21.15.3:0/900893760 wait complete. 2024-09-10T07:52:32.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2024-09-10T07:52:33.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.028+0000 7fc923e00640 1 Processor -- start 2024-09-10T07:52:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.028+0000 7fc923e00640 1 -- start start 2024-09-10T07:52:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.029+0000 7fc923e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc92406fb60 0x7fc9240d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.029+0000 7fc923e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc9240d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.029+0000 7fc923e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9240d0960 con 0x7fc924162c10 2024-09-10T07:52:33.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.029+0000 7fc923e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc9240ced60 con 0x7fc92406fb60 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.029+0000 7fc923e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9240cf000 con 0x7fc924162240 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc923400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc9240d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc922a00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc924162c10 0x7fc9240d0250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:34132/0 (socket says 172.21.15.3:34132) 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc922a00640 1 -- 172.21.15.3:0/143460625 learned_addr learned my addr 172.21.15.3:0/143460625 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc922000640 1 --2- 172.21.15.3:0/143460625 >> v2:172.21.15.70:3300/0 conn(0x7fc92406fb60 0x7fc9240d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc923400640 1 -- 172.21.15.3:0/143460625 >> v2:172.21.15.70:3300/0 conn(0x7fc92406fb60 msgr2=0x7fc9240d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc923400640 1 --2- 172.21.15.3:0/143460625 >> v2:172.21.15.70:3300/0 conn(0x7fc92406fb60 0x7fc9240d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc923400640 1 -- 172.21.15.3:0/143460625 >> v1:172.21.15.70:6789/0 conn(0x7fc924162c10 legacy=0x7fc9240d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc923400640 1 -- 172.21.15.3:0/143460625 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240cf2e0 con 0x7fc924162240 2024-09-10T07:52:33.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.030+0000 7fc921600640 1 -- 172.21.15.3:0/143460625 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508511925 0 0) 0x7fc9240d0960 con 0x7fc924162c10 2024-09-10T07:52:33.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.031+0000 7fc923400640 1 --2- 172.21.15.3:0/143460625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc9240d2b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009d90 tx=0x7fc91c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a57409003441f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.031+0000 7fc921600640 1 -- 172.21.15.3:0/143460625 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007740 con 0x7fc924162240 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.031+0000 7fc921600640 1 -- 172.21.15.3:0/143460625 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c0078e0 con 0x7fc924162240 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.031+0000 7fc921600640 1 -- 172.21.15.3:0/143460625 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007cb0 con 0x7fc924162240 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 -- 172.21.15.3:0/143460625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 msgr2=0x7fc9240d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 --2- 172.21.15.3:0/143460625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc9240d2b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009d90 tx=0x7fc91c02a660 comp rx=0 tx=0).stop 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 -- 172.21.15.3:0/143460625 shutdown_connections 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 --2- 172.21.15.3:0/143460625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc9240d2b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 --2- 172.21.15.3:0/143460625 >> v2:172.21.15.70:3300/0 conn(0x7fc92406fb60 0x7fc9240d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 -- 172.21.15.3:0/143460625 >> 172.21.15.3:0/143460625 conn(0x7fc92406df10 msgr2=0x7fc92406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 -- 172.21.15.3:0/143460625 shutdown_connections 2024-09-10T07:52:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.032+0000 7fc923e00640 1 -- 172.21.15.3:0/143460625 wait complete. 2024-09-10T07:52:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.034+0000 7fc923e00640 1 Processor -- start 2024-09-10T07:52:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.034+0000 7fc923e00640 1 -- start start 2024-09-10T07:52:33.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92406fb60 0x7fc9240aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc924160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc924162ff0 con 0x7fc924162c10 2024-09-10T07:52:33.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240cf8a0 con 0x7fc92406fb60 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc924163170 con 0x7fc924162240 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36796/0 (socket says 172.21.15.3:36796) 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc923400640 1 -- 172.21.15.3:0/3785471688 learned_addr learned my addr 172.21.15.3:0/3785471688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc922000640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc924160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.035+0000 7fc922a00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92406fb60 0x7fc9240aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 -- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 msgr2=0x7fc924160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc924160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 -- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92406fb60 msgr2=0x7fc9240aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92406fb60 0x7fc9240aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 -- 172.21.15.3:0/3785471688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240ac520 con 0x7fc924162c10 2024-09-10T07:52:33.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923400640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009eb0 tx=0x7fc91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc91aa00640 1 -- 172.21.15.3:0/3785471688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c040070 con 0x7fc924162c10 2024-09-10T07:52:33.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc91aa00640 1 -- 172.21.15.3:0/3785471688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c031bc0 con 0x7fc924162c10 2024-09-10T07:52:33.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc91aa00640 1 -- 172.21.15.3:0/3785471688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c009450 con 0x7fc924162c10 2024-09-10T07:52:33.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9240ab480 con 0x7fc924162c10 2024-09-10T07:52:33.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.036+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9240aba40 con 0x7fc924162c10 2024-09-10T07:52:33.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.038+0000 7fc91aa00640 1 -- 172.21.15.3:0/3785471688 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc91c039040 con 0x7fc924162c10 2024-09-10T07:52:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.039+0000 7fc91aa00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 0x7fc90406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.039+0000 7fc91aa00640 1 -- 172.21.15.3:0/3785471688 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc91c0ac4d0 con 0x7fc924162c10 2024-09-10T07:52:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.039+0000 7fc922a00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 0x7fc90406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.040+0000 7fc92c9d1740 1 Processor -- start 2024-09-10T07:52:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.040+0000 7fc92c9d1740 1 -- start start 2024-09-10T07:52:33.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.040+0000 7fc922a00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 0x7fc90406e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc914007fd0 tx=0x7fc914007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 0x5593971a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5593970ef660 con 0x5593971879e0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559397097680 con 0x5593971a14f0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5593970eb620 con 0x5593971d1ea0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36800/0 (socket says 172.21.15.3:36800) 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 -- 172.21.15.3:0/302156417 learned_addr learned my addr 172.21.15.3:0/302156417 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922a00640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 0x5593971a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc923400640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 -- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 msgr2=0x5593971d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 -- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 msgr2=0x5593971a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.041+0000 7fc922000640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 0x5593971a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc922000640 1 -- 172.21.15.3:0/302156417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559396ecc7d0 con 0x5593971879e0 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc922a00640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 0x5593971a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc922000640 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc910009c30 tx=0x7fc910011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5d89e7709daef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc90f400640 1 -- 172.21.15.3:0/302156417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910019070 con 0x5593971879e0 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc90f400640 1 -- 172.21.15.3:0/302156417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910004030 con 0x5593971879e0 2024-09-10T07:52:33.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.042+0000 7fc90f400640 1 -- 172.21.15.3:0/302156417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100043e0 con 0x5593971879e0 2024-09-10T07:52:33.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 -- 172.21.15.3:0/302156417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 msgr2=0x559397186240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc910009c30 tx=0x7fc910011690 comp rx=0 tx=0).stop 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 -- 172.21.15.3:0/302156417 shutdown_connections 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971879e0 0x559397186240 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/302156417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971a14f0 0x5593971a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 -- 172.21.15.3:0/302156417 >> 172.21.15.3:0/302156417 conn(0x559397188410 msgr2=0x5593971d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 -- 172.21.15.3:0/302156417 shutdown_connections 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.043+0000 7fc92c9d1740 1 -- 172.21.15.3:0/302156417 wait complete. 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.044+0000 7fc92c9d1740 1 -- start start 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.044+0000 7fc92c9d1740 1 Processor -- start 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559397186940 con 0x5593971a14f0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5593970784c0 con 0x5593971b5a60 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559397186c20 con 0x5593971d1ea0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc92c9d1740 10 client.0 _ll_register_callbacks cb 0x5593970e7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36802/0 (socket says 172.21.15.3:36802) 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 -- 172.21.15.3:0/2763703833 learned_addr learned my addr 172.21.15.3:0/2763703833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922000640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc923400640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 -- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 msgr2=0x5593971e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.045+0000 7fc922a00640 1 -- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 msgr2=0x5593971c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc922a00640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc922a00640 1 -- 172.21.15.3:0/2763703833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559396e023f0 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc922a00640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc914013750 tx=0x7fc914000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70e219465481f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc903e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914027060 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc910004030 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc914022ab0 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc914022cb0 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc92c9d1740 10 client.7518 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.046+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x559396dc20f0 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910004030 con 0x5593971a14f0 2024-09-10T07:52:33.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fc914012560 con 0x5593971a14f0 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 10 client.7518 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 10 client.7518 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 20 client.7518 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971b5ee0 0x5593971cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 4 client.7518 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593970fcd90 con 0x5593971b5ee0 2024-09-10T07:52:33.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.047+0000 7fc922000640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971b5ee0 0x5593971cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.048+0000 7fc922000640 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971b5ee0 0x5593971cd290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ec3d80f9c2d05d6 server_cookie=6f15e783d1cf3259 in_seq=0 out_seq=0 2024-09-10T07:52:33.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.048+0000 7fc903e00640 10 client.7518 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.048+0000 7fc903e00640 1 -- 172.21.15.3:0/2763703833 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593970fcd90 con 0x5593971b5ee0 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.048+0000 7fc903e00640 10 client.7518 handle_command_reply: tid=0 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.048+0000 7fc92c9d1740 1 client.7518 shutdown 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 msgr2=0x5593971cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc914013750 tx=0x7fc914000ed0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 shutdown_connections 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971b5ee0 0x5593971cd290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2763703833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971cbcd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 >> 172.21.15.3:0/2763703833 conn(0x5593971879e0 msgr2=0x5593971d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc903e00640 0 client.7518 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:33.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 shutdown_connections 2024-09-10T07:52:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2763703833 wait complete. 2024-09-10T07:52:33.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.049+0000 7fc92c9d1740 20 client.7518 trim_cache size 0 max 16384 2024-09-10T07:52:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.064+0000 7fc92c9d1740 1 Processor -- start 2024-09-10T07:52:33.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.064+0000 7fc92c9d1740 1 -- start start 2024-09-10T07:52:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc924200950 con 0x5593971a14f0 2024-09-10T07:52:33.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241ffb20 con 0x5593971b5a60 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc924200220 con 0x5593971d1ea0 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc922a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc922a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36814/0 (socket says 172.21.15.3:36814) 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.065+0000 7fc922a00640 1 -- 172.21.15.3:0/4072436559 learned_addr learned my addr 172.21.15.3:0/4072436559 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc923400640 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922000640 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 -- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 msgr2=0x5593971c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 -- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 msgr2=0x5593971c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 -- 172.21.15.3:0/4072436559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593971cce70 con 0x5593971a14f0 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc922a00640 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc9140016b0 tx=0x7fc914020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50818a708f4aef32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc90ea00640 1 -- 172.21.15.3:0/4072436559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914004d30 con 0x5593971a14f0 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc90ea00640 1 -- 172.21.15.3:0/4072436559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc914012560 con 0x5593971a14f0 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.066+0000 7fc90ea00640 1 -- 172.21.15.3:0/4072436559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914012ab0 con 0x5593971a14f0 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 -- 172.21.15.3:0/4072436559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 msgr2=0x5593971f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc9140016b0 tx=0x7fc914020000 comp rx=0 tx=0).stop 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 -- 172.21.15.3:0/4072436559 shutdown_connections 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971d1ea0 0x5593971c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x5593971c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/4072436559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971a14f0 0x5593971f5f30 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 -- 172.21.15.3:0/4072436559 >> 172.21.15.3:0/4072436559 conn(0x5593971879e0 msgr2=0x559397181830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 -- 172.21.15.3:0/4072436559 shutdown_connections 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.067+0000 7fc92c9d1740 1 -- 172.21.15.3:0/4072436559 wait complete. 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.068+0000 7fc92c9d1740 1 -- start start 2024-09-10T07:52:33.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.068+0000 7fc92c9d1740 1 Processor -- start 2024-09-10T07:52:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 0x559397245190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x559397251e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559397188180 con 0x5593971d1ea0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241feb60 con 0x5593971b5a60 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559397125b80 con 0x5593971a14f0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc92c9d1740 10 client.0 _ll_register_callbacks cb 0x7fc914007340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc923400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc922a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 0x559397245190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc923400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36830/0 (socket says 172.21.15.3:36830) 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc923400640 1 -- 172.21.15.3:0/2132000707 learned_addr learned my addr 172.21.15.3:0/2132000707 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc922000640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x559397251e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc923400640 1 -- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 msgr2=0x559397245190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.069+0000 7fc923400640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 0x559397245190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc923400640 1 -- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 msgr2=0x559397251e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc923400640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x559397251e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc923400640 1 -- 172.21.15.3:0/2132000707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593972456d0 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc922a00640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 0x559397245190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc923400640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc91c07a8d0 tx=0x7fc91c07edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ed1a65f30f9592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0ac0c0 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc91c0ac260 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc914012560 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc91c089050 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 10 client.7524 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5593971ce560 con 0x5593971d1ea0 2024-09-10T07:52:33.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc914012560 con 0x5593971d1ea0 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fc91c08c020 con 0x5593971d1ea0 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 10 client.7524 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 10 client.7524 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 20 client.7524 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.070+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971ca160 0x5593971df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.071+0000 7fc92c9d1740 4 client.7524 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.071+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593970f70e0 con 0x5593971ca160 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.071+0000 7fc922a00640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971ca160 0x5593971df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.071+0000 7fc922a00640 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971ca160 0x5593971df630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b695067b097885e server_cookie=2634f3f9a427dd1c in_seq=0 out_seq=0 2024-09-10T07:52:33.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.071+0000 7fc903e00640 10 client.7524 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:33.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc903e00640 1 -- 172.21.15.3:0/2132000707 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593970f70e0 con 0x5593971ca160 2024-09-10T07:52:33.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc903e00640 10 client.7524 handle_command_reply: tid=0 2024-09-10T07:52:33.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 client.7524 shutdown 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 msgr2=0x5593971c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc91c07a8d0 tx=0x7fc91c07edc0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 shutdown_connections 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x5593971ca160 0x5593971df630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593971d1ea0 0x5593971c9b80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5593971b5a60 0x559397251e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc903e00640 0 client.7524 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.079+0000 7fc92c9d1740 1 --2- 172.21.15.3:0/2132000707 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593971a14f0 0x559397245190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 >> 172.21.15.3:0/2132000707 conn(0x5593971879e0 msgr2=0x5593971cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 shutdown_connections 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc92c9d1740 1 -- 172.21.15.3:0/2132000707 wait complete. 2024-09-10T07:52:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc92c9d1740 20 client.7524 trim_cache size 0 max 16384 2024-09-10T07:52:33.081 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:33.081 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:52:33.081 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:52:33.081 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 msgr2=0x7fc90406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 0x7fc90406e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc914007fd0 tx=0x7fc914007de0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 msgr2=0x7fc9241455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009eb0 tx=0x7fc91c004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.080+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 shutdown_connections 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc90406be30 0x7fc90406e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc924162c10 0x7fc9241455e0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc924162240 0x7fc924160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 --2- 172.21.15.3:0/3785471688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc92406fb60 0x7fc9240aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 >> 172.21.15.3:0/3785471688 conn(0x7fc92406df10 msgr2=0x7fc924146b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 shutdown_connections 2024-09-10T07:52:33.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33.081+0000 7fc923e00640 1 -- 172.21.15.3:0/3785471688 wait complete. 2024-09-10T07:52:33.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:33.095 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-10T07:52:33.138 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/ 2024-09-10T07:52:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4 2024-09-10T07:52:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5 2024-09-10T07:52:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6 2024-09-10T07:52:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-10T07:52:33.139 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-10T07:52:33.140 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:33.280 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627778 2024-09-10T07:52:33.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-10T07:52:33.292 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:33.432 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627779 2024-09-10T07:52:33.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-10T07:52:33.443 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:33.585 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627780 2024-09-10T07:52:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:33.598 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.598 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.598 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.598 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.599 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-10T07:52:33.599 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:33.739 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627781 2024-09-10T07:52:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-10T07:52:33.751 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:33.891 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627782 2024-09-10T07:52:33.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-10T07:52:33.903 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:52:34.043 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627783 2024-09-10T07:52:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:52:34.054 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-10T07:52:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.172+0000 7fd2bf12ff40 1 Processor -- start 2024-09-10T07:52:34.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.172+0000 7fd2bf12ff40 1 -- start start 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b012ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c6b00a1cb0 con 0x55c6b0057be0 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c6afffdb30 con 0x55c6b00593f0 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c6b0058910 con 0x55c6b0112850 2024-09-10T07:52:34.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bb800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bb800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44336/0 (socket says 172.21.15.3:44336) 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.173+0000 7fd2bb800640 1 -- 172.21.15.3:0/2751913914 learned_addr learned my addr 172.21.15.3:0/2751913914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bc200640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b012ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bae00640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 -- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 msgr2=0x55c6b012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b012ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 -- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 msgr2=0x55c6b012e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 -- 172.21.15.3:0/2751913914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b012d220 con 0x55c6b0057be0 2024-09-10T07:52:34.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bc200640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b012ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bae00640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2bb800640 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac000c00 tx=0x7fd2ac051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa45f2230322a2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2b9a00640 1 -- 172.21.15.3:0/2751913914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac063070 con 0x55c6b0057be0 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2b9a00640 1 -- 172.21.15.3:0/2751913914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0559d0 con 0x55c6b0057be0 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.174+0000 7fd2b9a00640 1 -- 172.21.15.3:0/2751913914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac055da0 con 0x55c6b0057be0 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/2751913914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 msgr2=0x55c6b0112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac000c00 tx=0x7fd2ac051000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/2751913914 shutdown_connections 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b012ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/2751913914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b0112ce0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/2751913914 >> 172.21.15.3:0/2751913914 conn(0x55c6b0083e30 msgr2=0x55c6b012bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/2751913914 shutdown_connections 2024-09-10T07:52:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.175+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/2751913914 wait complete. 2024-09-10T07:52:34.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.176+0000 7fd2bf12ff40 1 Processor -- start 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.176+0000 7fd2bf12ff40 1 -- start start 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b009ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b010db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c6b0086af0 con 0x55c6b0057be0 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c6b00868a0 con 0x55c6b00593f0 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bf12ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c6b012d4e0 con 0x55c6b0112850 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b009ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b009ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44340/0 (socket says 172.21.15.3:44340) 2024-09-10T07:52:34.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 -- 172.21.15.3:0/755329507 learned_addr learned my addr 172.21.15.3:0/755329507 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bae00640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 -- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 msgr2=0x55c6b010db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bc200640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b010db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b010db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 -- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 msgr2=0x55c6b012e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bb800640 1 -- 172.21.15.3:0/755329507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b009d610 con 0x55c6b0057be0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bae00640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c6b00593f0 0x55c6b012e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.177+0000 7fd2bc200640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c6b0112850 0x55c6b010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2bb800640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b009ddf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac045ef0 tx=0x7fd2ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2b3e00640 1 -- 172.21.15.3:0/755329507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac076070 con 0x55c6b0057be0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2b3e00640 1 -- 172.21.15.3:0/755329507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac068460 con 0x55c6b0057be0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6b012c2a0 con 0x55c6b0057be0 2024-09-10T07:52:34.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2b3e00640 1 -- 172.21.15.3:0/755329507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac06d070 con 0x55c6b0057be0 2024-09-10T07:52:34.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.178+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b0044ee0 con 0x55c6b0057be0 2024-09-10T07:52:34.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.180+0000 7fd2b3e00640 1 -- 172.21.15.3:0/755329507 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2ac079070 con 0x55c6b0057be0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.181+0000 7fd2b3e00640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd29c06a840 0x7fd29c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.181+0000 7fd2b3e00640 1 -- 172.21.15.3:0/755329507 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fd2ac0cf350 con 0x55c6b0057be0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.181+0000 7fd2bae00640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd29c06a840 0x7fd29c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.181+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c6b012e6d0 0x55c6b010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.181+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 12.12 12:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55c6b010a330 con 0x55c6b012e6d0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.182+0000 7fd2bae00640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd29c06a840 0x7fd29c06cd00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55c6b01c7790 tx=0x7fd2a800c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.182+0000 7fd2bc200640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c6b012e6d0 0x55c6b010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.182+0000 7fd2bc200640 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c6b012e6d0 0x55c6b010e0b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bc200640 1 -- 172.21.15.3:0/755329507 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v90'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd2b4006cf0 con 0x55c6b012e6d0 2024-09-10T07:52:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c6b012e6d0 msgr2=0x55c6b010e0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55c6b012e6d0 0x55c6b010e0b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd29c06a840 msgr2=0x7fd29c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd29c06a840 0x7fd29c06cd00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55c6b01c7790 tx=0x7fd2a800c040 comp rx=0 tx=0).stop 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 msgr2=0x55c6b009ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bf12ff40 1 --2- 172.21.15.3:0/755329507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c6b0057be0 0x55c6b009ddf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac045ef0 tx=0x7fd2ac002fb0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.184+0000 7fd2bc200640 1 -- 172.21.15.3:0/755329507 reap_dead start 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.185+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 shutdown_connections 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.185+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 >> 172.21.15.3:0/755329507 conn(0x55c6b0083e30 msgr2=0x55c6b012a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.185+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 shutdown_connections 2024-09-10T07:52:34.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.185+0000 7fd2bf12ff40 1 -- 172.21.15.3:0/755329507 wait complete. 2024-09-10T07:52:34.189 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-10T07:52:34.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.266+0000 7fe21191ef40 1 Processor -- start 2024-09-10T07:52:34.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.266+0000 7fe21191ef40 1 -- start start 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bcd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bcd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560092a9ab30 con 0x560092b247a0 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560092a9eb70 con 0x560092b26220 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe21191ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560092bcb900 con 0x560092b25e40 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe20e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe20e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44350/0 (socket says 172.21.15.3:44350) 2024-09-10T07:52:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.267+0000 7fe20e000640 1 -- 172.21.15.3:0/42767775 learned_addr learned my addr 172.21.15.3:0/42767775 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20d600640 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bcd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20ea00640 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bcd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 -- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 msgr2=0x560092bcd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bcd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 -- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 msgr2=0x560092bcd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bcd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 -- 172.21.15.3:0/42767775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560092af36e0 con 0x560092b247a0 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe20e000640 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe200001e50 tx=0x7fe20000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa899f46d84d32a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe207e00640 1 -- 172.21.15.3:0/42767775 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200012020 con 0x560092b247a0 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe207e00640 1 -- 172.21.15.3:0/42767775 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20000e690 con 0x560092b247a0 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.268+0000 7fe207e00640 1 -- 172.21.15.3:0/42767775 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20000eac0 con 0x560092b247a0 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 -- 172.21.15.3:0/42767775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 msgr2=0x560092bbd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe200001e50 tx=0x7fe20000b780 comp rx=0 tx=0).stop 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 -- 172.21.15.3:0/42767775 shutdown_connections 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bcd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bcd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 --2- 172.21.15.3:0/42767775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bbd550 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 -- 172.21.15.3:0/42767775 >> 172.21.15.3:0/42767775 conn(0x560092b22d40 msgr2=0x560092af2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 -- 172.21.15.3:0/42767775 shutdown_connections 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.269+0000 7fe21191ef40 1 -- 172.21.15.3:0/42767775 wait complete. 2024-09-10T07:52:34.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.270+0000 7fe21191ef40 1 Processor -- start 2024-09-10T07:52:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.270+0000 7fe21191ef40 1 -- start start 2024-09-10T07:52:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bc65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bbac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560092af39c0 con 0x560092b247a0 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560092bcdbe0 con 0x560092b26220 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe21191ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560092b3f240 con 0x560092b25e40 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bc65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bc65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44366/0 (socket says 172.21.15.3:44366) 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 -- 172.21.15.3:0/3531769838 learned_addr learned my addr 172.21.15.3:0/3531769838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20d600640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20ea00640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 -- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 msgr2=0x560092bbcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560092b25e40 0x560092bbcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 -- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 msgr2=0x560092bbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560092b26220 0x560092bbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.271+0000 7fe20e000640 1 -- 172.21.15.3:0/3531769838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560092bc1e90 con 0x560092b247a0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe20e000640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bc65a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe200000f60 tx=0x7fe200000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe206a00640 1 -- 172.21.15.3:0/3531769838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200027070 con 0x560092b247a0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe206a00640 1 -- 172.21.15.3:0/3531769838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe200017630 con 0x560092b247a0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe206a00640 1 -- 172.21.15.3:0/3531769838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200017a20 con 0x560092b247a0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560092bc15f0 con 0x560092b247a0 2024-09-10T07:52:34.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.272+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560092bc1bb0 con 0x560092b247a0 2024-09-10T07:52:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.274+0000 7fe206a00640 1 -- 172.21.15.3:0/3531769838 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe20002a070 con 0x560092b247a0 2024-09-10T07:52:34.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.275+0000 7fe206a00640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe1f006a810 0x7fe1f006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.275+0000 7fe206a00640 1 -- 172.21.15.3:0/3531769838 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fe20008b240 con 0x560092b247a0 2024-09-10T07:52:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.275+0000 7fe20d600640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe1f006a810 0x7fe1f006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.275+0000 7fe21191ef40 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x560092bbb140 0x560092bc3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.275+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 12.12 12:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x560092bb8940 con 0x560092bbb140 2024-09-10T07:52:34.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.276+0000 7fe20ea00640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x560092bbb140 0x560092bc3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.276+0000 7fe20d600640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe1f006a810 0x7fe1f006ccd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc056000 tx=0x7fe1fc050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.276+0000 7fe20ea00640 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x560092bbb140 0x560092bc3580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe20ea00640 1 -- 172.21.15.3:0/3531769838 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v90'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe208002600 con 0x560092bbb140 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x560092bbb140 msgr2=0x560092bc3580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x560092bbb140 0x560092bc3580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe1f006a810 msgr2=0x7fe1f006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe1f006a810 0x7fe1f006ccd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc056000 tx=0x7fe1fc050000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 msgr2=0x560092bc65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 --2- 172.21.15.3:0/3531769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560092b247a0 0x560092bc65a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe200000f60 tx=0x7fe200000bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe20ea00640 1 -- 172.21.15.3:0/3531769838 reap_dead start 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 shutdown_connections 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 >> 172.21.15.3:0/3531769838 conn(0x560092b22d40 msgr2=0x560092af2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 shutdown_connections 2024-09-10T07:52:34.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.278+0000 7fe21191ef40 1 -- 172.21.15.3:0/3531769838 wait complete. 2024-09-10T07:52:34.283 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-10T07:52:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.361+0000 7fc8b5bdff40 1 Processor -- start 2024-09-10T07:52:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.361+0000 7fc8b5bdff40 1 -- start start 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 0x563a0444c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 0x563a04479fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563a043f3b30 con 0x563a0447ee40 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563a043f7b70 con 0x563a0447d7a0 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563a04524900 con 0x563a0447f220 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b2400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b2400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44382/0 (socket says 172.21.15.3:44382) 2024-09-10T07:52:34.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.362+0000 7fc8b2400640 1 -- 172.21.15.3:0/98845413 learned_addr learned my addr 172.21.15.3:0/98845413 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b3800640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 0x563a04479fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2e00640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 0x563a0444c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 -- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 msgr2=0x563a04479fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 0x563a04479fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 -- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 msgr2=0x563a0444c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 0x563a0444c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 -- 172.21.15.3:0/98845413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a045267e0 con 0x563a0447ee40 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b3800640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 0x563a04479fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2e00640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 0x563a0444c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b2400640 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b1f0 tx=0x7fc8a000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7037f44f12685fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b1000640 1 -- 172.21.15.3:0/98845413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0016070 con 0x563a0447ee40 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b1000640 1 -- 172.21.15.3:0/98845413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a000d6d0 con 0x563a0447ee40 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.363+0000 7fc8b1000640 1 -- 172.21.15.3:0/98845413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000dac0 con 0x563a0447ee40 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/98845413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 msgr2=0x563a04479a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b1f0 tx=0x7fc8a000b6b0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/98845413 shutdown_connections 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447f220 0x563a04479fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447ee40 0x563a04479a90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/98845413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447d7a0 0x563a0444c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/98845413 >> 172.21.15.3:0/98845413 conn(0x563a0447bd40 msgr2=0x563a0444bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/98845413 shutdown_connections 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.364+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/98845413 wait complete. 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.365+0000 7fc8b5bdff40 1 Processor -- start 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.365+0000 7fc8b5bdff40 1 -- start start 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447d7a0 0x563a04515060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447ee40 0x563a0451c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 0x563a0444bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563a0451cff0 con 0x563a0447f220 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563a04519210 con 0x563a0447ee40 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b5bdff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563a0444cb80 con 0x563a0447d7a0 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 0x563a0444bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 0x563a0444bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44396/0 (socket says 172.21.15.3:44396) 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 -- 172.21.15.3:0/771675734 learned_addr learned my addr 172.21.15.3:0/771675734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b2400640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447ee40 0x563a0451c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b2e00640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447d7a0 0x563a04515060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 -- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447d7a0 msgr2=0x563a04515060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a0447d7a0 0x563a04515060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 -- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447ee40 msgr2=0x563a0451c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447ee40 0x563a0451c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b3800640 1 -- 172.21.15.3:0/771675734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a04518820 con 0x563a0447f220 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.366+0000 7fc8b2400640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a0447ee40 0x563a0451c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8b3800640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 0x563a0444bf30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac006fc0 tx=0x7fc8ac00c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8ab400640 1 -- 172.21.15.3:0/771675734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac017070 con 0x563a0447f220 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8ab400640 1 -- 172.21.15.3:0/771675734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ac0062f0 con 0x563a0447f220 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a0451e930 con 0x563a0447f220 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8ab400640 1 -- 172.21.15.3:0/771675734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004700 con 0x563a0447f220 2024-09-10T07:52:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.367+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a04517fc0 con 0x563a0447f220 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.369+0000 7fc8ab400640 1 -- 172.21.15.3:0/771675734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8ac01a070 con 0x563a0447f220 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.370+0000 7fc8ab400640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc89406a810 0x7fc89406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.370+0000 7fc8ab400640 1 -- 172.21.15.3:0/771675734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc8ac0893a0 con 0x563a0447f220 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.370+0000 7fc8b2e00640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc89406a810 0x7fc89406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.370+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x563a0451b560 0x563a0451b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.370+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 12.5 12:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x563a0451be80 con 0x563a0451b560 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.371+0000 7fc8b2400640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x563a0451b560 0x563a0451b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.371+0000 7fc8b2e00640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc89406a810 0x7fc89406ccd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4057000 tx=0x7fc8a4052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.371+0000 7fc8b2400640 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x563a0451b560 0x563a0451b940 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b2400640 1 -- 172.21.15.3:0/771675734 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v90'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8a000d980 con 0x563a0451b560 2024-09-10T07:52:34.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x563a0451b560 msgr2=0x563a0451b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x563a0451b560 0x563a0451b940 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc89406a810 msgr2=0x7fc89406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc89406a810 0x7fc89406ccd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4057000 tx=0x7fc8a4052000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 msgr2=0x563a0444bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 --2- 172.21.15.3:0/771675734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a0447f220 0x563a0444bf30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac006fc0 tx=0x7fc8ac00c5e0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b3800640 1 -- 172.21.15.3:0/771675734 reap_dead start 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 shutdown_connections 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 >> 172.21.15.3:0/771675734 conn(0x563a0447bd40 msgr2=0x563a0444d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 shutdown_connections 2024-09-10T07:52:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.373+0000 7fc8b5bdff40 1 -- 172.21.15.3:0/771675734 wait complete. 2024-09-10T07:52:34.377 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-10T07:52:34.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.454+0000 7fb39eb1ef40 1 Processor -- start 2024-09-10T07:52:34.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.454+0000 7fb39eb1ef40 1 -- start start 2024-09-10T07:52:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6b7a0 0x55614ce85130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 0x55614cf03ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55614ce6c120 con 0x55614ce6d220 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55614cde1b30 con 0x55614ce6ce40 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55614cf12620 con 0x55614ce6b7a0 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39bc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39bc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44406/0 (socket says 172.21.15.3:44406) 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.455+0000 7fb39bc00640 1 -- 172.21.15.3:0/2588579268 learned_addr learned my addr 172.21.15.3:0/2588579268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 -- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6b7a0 msgr2=0x55614ce85130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39a800640 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 0x55614cf03ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6b7a0 0x55614ce85130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 -- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 msgr2=0x55614cf03ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 0x55614cf03ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 -- 172.21.15.3:0/2588579268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614ce87110 con 0x55614ce6d220 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39a800640 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 0x55614cf03ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb39bc00640 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb394002880 tx=0x7fb394002d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dc3b2f97f4d18b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb399400640 1 -- 172.21.15.3:0/2588579268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394016020 con 0x55614ce6d220 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb399400640 1 -- 172.21.15.3:0/2588579268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39400d9c0 con 0x55614ce6d220 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.456+0000 7fb399400640 1 -- 172.21.15.3:0/2588579268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39400ddb0 con 0x55614ce6d220 2024-09-10T07:52:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/2588579268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 msgr2=0x55614cf043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb394002880 tx=0x7fb394002d40 comp rx=0 tx=0).stop 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/2588579268 shutdown_connections 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf043e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6ce40 0x55614cf03ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/2588579268 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6b7a0 0x55614ce85130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/2588579268 >> 172.21.15.3:0/2588579268 conn(0x55614ce69d40 msgr2=0x55614ce68db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/2588579268 shutdown_connections 2024-09-10T07:52:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.457+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/2588579268 wait complete. 2024-09-10T07:52:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.458+0000 7fb39eb1ef40 1 Processor -- start 2024-09-10T07:52:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.458+0000 7fb39eb1ef40 1 -- start start 2024-09-10T07:52:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6b7a0 0x55614cf002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6ce40 0x55614cf08d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55614cf13310 con 0x55614ce6d220 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55614cf07680 con 0x55614ce6b7a0 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39eb1ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55614cf078d0 con 0x55614ce6ce40 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44422/0 (socket says 172.21.15.3:44422) 2024-09-10T07:52:34.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 -- 172.21.15.3:0/3821835180 learned_addr learned my addr 172.21.15.3:0/3821835180 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39a800640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6ce40 0x55614cf08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39b200640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6b7a0 0x55614cf002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 -- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6ce40 msgr2=0x55614cf08d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55614ce6ce40 0x55614cf08d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 -- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6b7a0 msgr2=0x55614cf002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55614ce6b7a0 0x55614cf002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 -- 172.21.15.3:0/3821835180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614cf00ae0 con 0x55614ce6d220 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.459+0000 7fb39bc00640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf049b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb394003010 tx=0x7fb3940033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.460+0000 7fb393e00640 1 -- 172.21.15.3:0/3821835180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394012d40 con 0x55614ce6d220 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.460+0000 7fb393e00640 1 -- 172.21.15.3:0/3821835180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb394025430 con 0x55614ce6d220 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.460+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55614cf0d1f0 con 0x55614ce6d220 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.460+0000 7fb393e00640 1 -- 172.21.15.3:0/3821835180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394025820 con 0x55614ce6d220 2024-09-10T07:52:34.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.460+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614cf09ba0 con 0x55614ce6d220 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.462+0000 7fb393e00640 1 -- 172.21.15.3:0/3821835180 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb394019020 con 0x55614ce6d220 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.462+0000 7fb393e00640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb37c06a810 0x7fb37c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb393e00640 1 -- 172.21.15.3:0/3821835180 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fb39408bd80 con 0x55614ce6d220 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb39b200640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb37c06a810 0x7fb37c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55614cf09290 0x55614cf05400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 12.9 12:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55614ceec1d0 con 0x55614cf09290 2024-09-10T07:52:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb39b200640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb37c06a810 0x7fb37c06ccd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb38c004170 tx=0x7fb38c000bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.463+0000 7fb39a800640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55614cf09290 0x55614cf05400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.464+0000 7fb39a800640 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55614cf09290 0x55614cf05400 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39a800640 1 -- 172.21.15.3:0/3821835180 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v90'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38805c040 con 0x55614cf09290 2024-09-10T07:52:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55614cf09290 msgr2=0x55614cf05400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55614cf09290 0x55614cf05400 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb37c06a810 msgr2=0x7fb37c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb37c06a810 0x7fb37c06ccd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb38c004170 tx=0x7fb38c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 msgr2=0x55614cf049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39eb1ef40 1 --2- 172.21.15.3:0/3821835180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55614ce6d220 0x55614cf049b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb394003010 tx=0x7fb3940033a0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.466+0000 7fb39bc00640 1 -- 172.21.15.3:0/3821835180 reap_dead start 2024-09-10T07:52:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.467+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 shutdown_connections 2024-09-10T07:52:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.467+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 >> 172.21.15.3:0/3821835180 conn(0x55614ce69d40 msgr2=0x55614ce85210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.467+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 shutdown_connections 2024-09-10T07:52:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.467+0000 7fb39eb1ef40 1 -- 172.21.15.3:0/3821835180 wait complete. 2024-09-10T07:52:34.476 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-10T07:52:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.553+0000 7fa19ce0ef40 1 Processor -- start 2024-09-10T07:52:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.553+0000 7fa19ce0ef40 1 -- start start 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec41220 0x55600ecca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec59130 0x55600ece7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55600ece6620 con 0x55600ec40e40 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55600ebb5b30 con 0x55600ec59130 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55600ece6900 con 0x55600ec41220 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa199200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa199200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44442/0 (socket says 172.21.15.3:44442) 2024-09-10T07:52:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.554+0000 7fa199200640 1 -- 172.21.15.3:0/2790379952 learned_addr learned my addr 172.21.15.3:0/2790379952 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 -- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec41220 msgr2=0x55600ecca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec41220 0x55600ecca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 -- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec59130 msgr2=0x55600ece7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec59130 0x55600ece7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 -- 172.21.15.3:0/2790379952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600ec403e0 con 0x55600ec40e40 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa199200640 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00a440 tx=0x7fa18c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c59648b504bc7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa192a00640 1 -- 172.21.15.3:0/2790379952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c063070 con 0x55600ec40e40 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa192a00640 1 -- 172.21.15.3:0/2790379952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18c054c10 con 0x55600ec40e40 2024-09-10T07:52:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.555+0000 7fa192a00640 1 -- 172.21.15.3:0/2790379952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c068460 con 0x55600ec40e40 2024-09-10T07:52:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/2790379952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 msgr2=0x55600ecd8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00a440 tx=0x7fa18c051000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/2790379952 shutdown_connections 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec59130 0x55600ece7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec41220 0x55600ecca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/2790379952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec40e40 0x55600ecd8a20 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/2790379952 >> 172.21.15.3:0/2790379952 conn(0x55600ec3dd40 msgr2=0x55600ec0e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/2790379952 shutdown_connections 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.556+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/2790379952 wait complete. 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.557+0000 7fa19ce0ef40 1 Processor -- start 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.557+0000 7fa19ce0ef40 1 -- start start 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.557+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec40e40 0x55600ece1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 0x55600ecc9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa19ce0ef40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec59130 0x55600ece3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55600ecde1e0 con 0x55600ec41220 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55600ec40680 con 0x55600ec40e40 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa19ce0ef40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55600ece7290 con 0x55600ec59130 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 0x55600ecc9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 0x55600ecc9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44448/0 (socket says 172.21.15.3:44448) 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 -- 172.21.15.3:0/212701471 learned_addr learned my addr 172.21.15.3:0/212701471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa199c00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec59130 0x55600ece3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa199200640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec40e40 0x55600ece1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 -- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec59130 msgr2=0x55600ece3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55600ec59130 0x55600ece3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 -- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec40e40 msgr2=0x55600ece1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55600ec40e40 0x55600ece1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 -- 172.21.15.3:0/212701471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600ece2380 con 0x55600ec41220 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.558+0000 7fa193e00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 0x55600ecc9380 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d70 tx=0x7fa184006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.559+0000 7fa191600640 1 -- 172.21.15.3:0/212701471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184017070 con 0x55600ec41220 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.559+0000 7fa191600640 1 -- 172.21.15.3:0/212701471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1840090d0 con 0x55600ec41220 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.559+0000 7fa191600640 1 -- 172.21.15.3:0/212701471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184012630 con 0x55600ec41220 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.559+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55600ece2660 con 0x55600ec41220 2024-09-10T07:52:34.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.559+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55600ece0b30 con 0x55600ec41220 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.561+0000 7fa191600640 1 -- 172.21.15.3:0/212701471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa18401a070 con 0x55600ec41220 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.561+0000 7fa191600640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa17806a810 0x7fa17806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa191600640 1 -- 172.21.15.3:0/212701471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fa184088b30 con 0x55600ec41220 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa199200640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa17806a810 0x7fa17806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55600ec3cd20 0x55600ece1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 12.5 12:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55600ece4330 con 0x55600ec3cd20 2024-09-10T07:52:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa199c00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55600ec3cd20 0x55600ece1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.562+0000 7fa199200640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa17806a810 0x7fa17806ccd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa18c064000 tx=0x7fa18c053000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.563+0000 7fa199c00640 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55600ec3cd20 0x55600ece1560 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa199c00640 1 -- 172.21.15.3:0/212701471 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v90'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1940014d0 con 0x55600ec3cd20 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55600ec3cd20 msgr2=0x55600ece1560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x55600ec3cd20 0x55600ece1560 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa17806a810 msgr2=0x7fa17806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa17806a810 0x7fa17806ccd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa18c064000 tx=0x7fa18c053000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 msgr2=0x55600ecc9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 --2- 172.21.15.3:0/212701471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55600ec41220 0x55600ecc9380 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d70 tx=0x7fa184006750 comp rx=0 tx=0).stop 2024-09-10T07:52:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa199c00640 1 -- 172.21.15.3:0/212701471 reap_dead start 2024-09-10T07:52:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 shutdown_connections 2024-09-10T07:52:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 >> 172.21.15.3:0/212701471 conn(0x55600ec3dd40 msgr2=0x55600ec0df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 shutdown_connections 2024-09-10T07:52:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.565+0000 7fa19ce0ef40 1 -- 172.21.15.3:0/212701471 wait complete. 2024-09-10T07:52:34.569 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-10T07:52:34.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.646+0000 7f0f1ee68f40 1 Processor -- start 2024-09-10T07:52:34.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.646+0000 7f0f1ee68f40 1 -- start start 2024-09-10T07:52:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf8f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 0x56356d024460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56356cf96620 con 0x56356cef1220 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56356ce65b30 con 0x56356cf09130 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56356cf96900 con 0x56356cef0e40 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44474/0 (socket says 172.21.15.3:44474) 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 -- 172.21.15.3:0/3680789891 learned_addr learned my addr 172.21.15.3:0/3680789891 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1c000640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf8f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ca00640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 0x56356d024460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 -- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 msgr2=0x56356cf8f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf8f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 -- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 msgr2=0x56356d024460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 0x56356d024460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1b600640 1 -- 172.21.15.3:0/3680789891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356cef02e0 con 0x56356cef1220 2024-09-10T07:52:34.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.647+0000 7f0f1ca00640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 0x56356d024460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:34.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1b600640 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f0f08025c00 tx=0x7f0f08051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f088953f6bde6bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1a200640 1 -- 172.21.15.3:0/3680789891 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08053af0 con 0x56356cef1220 2024-09-10T07:52:34.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1a200640 1 -- 172.21.15.3:0/3680789891 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f08063070 con 0x56356cef1220 2024-09-10T07:52:34.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1a200640 1 -- 172.21.15.3:0/3680789891 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08068470 con 0x56356cef1220 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/3680789891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 msgr2=0x56356ceef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.648+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f0f08025c00 tx=0x7f0f08051000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/3680789891 shutdown_connections 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cf09130 0x56356d024460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cef1220 0x56356ceef890 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/3680789891 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf8f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/3680789891 >> 172.21.15.3:0/3680789891 conn(0x56356ceedd40 msgr2=0x56356cebe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/3680789891 shutdown_connections 2024-09-10T07:52:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/3680789891 wait complete. 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.649+0000 7f0f1ee68f40 1 Processor -- start 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 -- start start 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf84d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cef1220 0x56356cf8c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 0x56356cf8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56356cf97720 con 0x56356cf09130 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56356cf0b520 con 0x56356cef1220 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.650+0000 7f0f1ee68f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56356cf0b6a0 con 0x56356cef0e40 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 0x56356cf8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1c000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf84d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 0x56356cf8c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44484/0 (socket says 172.21.15.3:44484) 2024-09-10T07:52:34.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1c000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf84d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:42042/0 (socket says 172.21.15.3:42042) 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1b600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cef1220 0x56356cf8c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 -- 172.21.15.3:0/402001209 learned_addr learned my addr 172.21.15.3:0/402001209 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 -- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 msgr2=0x56356cf84d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56356cef0e40 0x56356cf84d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 -- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cef1220 msgr2=0x56356cf8c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56356cef1220 0x56356cf8c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 -- 172.21.15.3:0/402001209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356cf89a40 con 0x56356cf09130 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.651+0000 7f0f1ca00640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 0x56356cf8c7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400dfd0 tx=0x7f0f140037f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.654+0000 7f0f18e00640 1 -- 172.21.15.3:0/402001209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14018070 con 0x56356cf09130 2024-09-10T07:52:34.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.654+0000 7f0f18e00640 1 -- 172.21.15.3:0/402001209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f140030d0 con 0x56356cf09130 2024-09-10T07:52:34.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.654+0000 7f0f18e00640 1 -- 172.21.15.3:0/402001209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14003280 con 0x56356cf09130 2024-09-10T07:52:34.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.654+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56356cf89d20 con 0x56356cf09130 2024-09-10T07:52:34.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.654+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356cf8a2e0 con 0x56356cf09130 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.656+0000 7f0f18e00640 1 -- 172.21.15.3:0/402001209 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f140123e0 con 0x56356cf09130 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f18e00640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0efc06a810 0x7f0efc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f18e00640 1 -- 172.21.15.3:0/402001209 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f0f14088f00 con 0x56356cf09130 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f1c000640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0efc06a810 0x7f0efc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x56356ceecf00 0x56356cf8ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 12.1d 12:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x56356cf8b250 con 0x56356ceecf00 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.657+0000 7f0f1c000640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0efc06a810 0x7f0efc06ccd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005fd0 tx=0x7f0f0c00e040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.658+0000 7f0f1b600640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x56356ceecf00 0x56356cf8ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.658+0000 7f0f1b600640 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x56356ceecf00 0x56356cf8ad10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1b600640 1 -- 172.21.15.3:0/402001209 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v90'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0f080538f0 con 0x56356ceecf00 2024-09-10T07:52:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x56356ceecf00 msgr2=0x56356cf8ad10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x56356ceecf00 0x56356cf8ad10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0efc06a810 msgr2=0x7f0efc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0efc06a810 0x7f0efc06ccd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005fd0 tx=0x7f0f0c00e040 comp rx=0 tx=0).stop 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 msgr2=0x56356cf8c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 --2- 172.21.15.3:0/402001209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56356cf09130 0x56356cf8c7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400dfd0 tx=0x7f0f140037f0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ca00640 1 -- 172.21.15.3:0/402001209 reap_dead start 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 shutdown_connections 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 >> 172.21.15.3:0/402001209 conn(0x56356ceedd40 msgr2=0x56356cebdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.660+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 shutdown_connections 2024-09-10T07:52:34.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.661+0000 7f0f1ee68f40 1 -- 172.21.15.3:0/402001209 wait complete. 2024-09-10T07:52:34.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:34.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2024-09-10T07:52:34.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.825+0000 7fd5e5800640 1 Processor -- start 2024-09-10T07:52:34.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.826+0000 7fd5e5800640 1 -- start start 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5e5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5e5800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5e00b3e10 0x7fd5e00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5e5800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e00cd8d0 con 0x7fd5e00ccc50 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5e5800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e015b900 con 0x7fd5e00b3e10 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5e5800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e00b47b0 con 0x7fd5e01591b0 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:42048/0 (socket says 172.21.15.3:42048) 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5dea00640 1 -- 172.21.15.3:0/3158184154 learned_addr learned my addr 172.21.15.3:0/3158184154 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.827+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3158184154 >> v2:172.21.15.70:3300/0 conn(0x7fd5e00b3e10 0x7fd5e00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.828+0000 7fd5dea00640 1 -- 172.21.15.3:0/3158184154 >> v2:172.21.15.70:3300/0 conn(0x7fd5e00b3e10 msgr2=0x7fd5e00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.828+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3158184154 >> v2:172.21.15.70:3300/0 conn(0x7fd5e00b3e10 0x7fd5e00b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.828+0000 7fd5dea00640 1 -- 172.21.15.3:0/3158184154 >> v1:172.21.15.70:6789/0 conn(0x7fd5e00ccc50 legacy=0x7fd5e00cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.828+0000 7fd5dea00640 1 -- 172.21.15.3:0/3158184154 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e015bb20 con 0x7fd5e01591b0 2024-09-10T07:52:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.829+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3158184154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e2ef3ba0055238 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.829+0000 7fd5de000640 1 -- 172.21.15.3:0/3158184154 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0037070 con 0x7fd5e01591b0 2024-09-10T07:52:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.829+0000 7fd5de000640 1 -- 172.21.15.3:0/3158184154 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0032440 con 0x7fd5e01591b0 2024-09-10T07:52:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.829+0000 7fd5de000640 1 -- 172.21.15.3:0/3158184154 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0032810 con 0x7fd5e01591b0 2024-09-10T07:52:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 -- 172.21.15.3:0/3158184154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 msgr2=0x7fd5e015c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3158184154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002a270 comp rx=0 tx=0).stop 2024-09-10T07:52:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 -- 172.21.15.3:0/3158184154 shutdown_connections 2024-09-10T07:52:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3158184154 >> v2:172.21.15.70:3300/0 conn(0x7fd5e00b3e10 0x7fd5e00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3158184154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e01591b0 0x7fd5e015c600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 -- 172.21.15.3:0/3158184154 >> 172.21.15.3:0/3158184154 conn(0x7fd5e0062cc0 msgr2=0x7fd5e006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 -- 172.21.15.3:0/3158184154 shutdown_connections 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.830+0000 7fd5e5800640 1 -- 172.21.15.3:0/3158184154 wait complete. 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.831+0000 7fd5e5800640 1 Processor -- start 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.831+0000 7fd5e5800640 1 -- start start 2024-09-10T07:52:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e00ccc50 0x7fd5e015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e014dee0 0x7fd5e00728d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e00ce290 con 0x7fd5e00b3e10 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e00cdbb0 con 0x7fd5e014dee0 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5e5800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e00ce410 con 0x7fd5e00ccc50 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44500/0 (socket says 172.21.15.3:44500) 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 -- 172.21.15.3:0/3556112514 learned_addr learned my addr 172.21.15.3:0/3556112514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e00ccc50 0x7fd5e015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5df400640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e014dee0 0x7fd5e00728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 -- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e00ccc50 msgr2=0x7fd5e015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e00ccc50 0x7fd5e015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 -- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e014dee0 msgr2=0x7fd5e00728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e014dee0 0x7fd5e00728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.833+0000 7fd5dea00640 1 -- 172.21.15.3:0/3556112514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e0071830 con 0x7fd5e00b3e10 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0000c00 tx=0x7fd5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5d7400640 1 -- 172.21.15.3:0/3556112514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0037040 con 0x7fd5e00b3e10 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5d7400640 1 -- 172.21.15.3:0/3556112514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0042070 con 0x7fd5e00b3e10 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5d7400640 1 -- 172.21.15.3:0/3556112514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0030880 con 0x7fd5e00b3e10 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e0071b10 con 0x7fd5e00b3e10 2024-09-10T07:52:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.834+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e00720d0 con 0x7fd5e00b3e10 2024-09-10T07:52:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.836+0000 7fd5d7400640 1 -- 172.21.15.3:0/3556112514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5d0030a20 con 0x7fd5e00b3e10 2024-09-10T07:52:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.837+0000 7fd5d7400640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 0x7fd5bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.837+0000 7fd5d7400640 1 -- 172.21.15.3:0/3556112514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fd5d00ab550 con 0x7fd5e00b3e10 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.837+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 0x7fd5bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.838+0000 7fd5e8820740 1 Processor -- start 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.838+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 0x7fd5bc06e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd5e01f7cd0 tx=0x7fd5c8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.838+0000 7fd5e8820740 1 -- start start 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 0x55f8af36ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 0x55f8af345e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8af2a89c0 con 0x55f8af38a5a0 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f8af2508f0 con 0x55f8af343540 2024-09-10T07:52:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f8af2a4980 con 0x55f8af36fbe0 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44504/0 (socket says 172.21.15.3:44504) 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 -- 172.21.15.3:0/2104910566 learned_addr learned my addr 172.21.15.3:0/2104910566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 -- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 msgr2=0x55f8af36ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5df400640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 0x55f8af345e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5dea00640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 0x55f8af36ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 0x55f8af36ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 -- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 msgr2=0x55f8af345e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 0x55f8af345e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5d6000640 1 -- 172.21.15.3:0/2104910566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8af083a50 con 0x55f8af38a5a0 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5dea00640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 0x55f8af36ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.839+0000 7fd5df400640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 0x55f8af345e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.840+0000 7fd5d6000640 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8011df0 tx=0x7fd5c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c2cfc0c8a399e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.840+0000 7fd5cf400640 1 -- 172.21.15.3:0/2104910566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8025070 con 0x55f8af38a5a0 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.840+0000 7fd5cf400640 1 -- 172.21.15.3:0/2104910566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c801b070 con 0x55f8af38a5a0 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.840+0000 7fd5cf400640 1 -- 172.21.15.3:0/2104910566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8014dd0 con 0x55f8af38a5a0 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- 172.21.15.3:0/2104910566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 msgr2=0x55f8af3889e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8011df0 tx=0x7fd5c8014000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- 172.21.15.3:0/2104910566 shutdown_connections 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af343540 0x55f8af345e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af3889e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 --2- 172.21.15.3:0/2104910566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af36fbe0 0x55f8af36ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- 172.21.15.3:0/2104910566 >> 172.21.15.3:0/2104910566 conn(0x55f8af3427d0 msgr2=0x55f8af38a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- 172.21.15.3:0/2104910566 shutdown_connections 2024-09-10T07:52:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- 172.21.15.3:0/2104910566 wait complete. 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.841+0000 7fd5e8820740 1 -- start start 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.842+0000 7fd5e8820740 1 Processor -- start 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.842+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af385b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af344960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8af370500 con 0x55f8af36fbe0 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f8af263b40 con 0x55f8af38a5a0 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f8af341c00 con 0x55f8af3444e0 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5e8820740 10 client.0 _ll_register_callbacks cb 0x55f8af265bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44508/0 (socket says 172.21.15.3:44508) 2024-09-10T07:52:34.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 -- 172.21.15.3:0/3942104046 learned_addr learned my addr 172.21.15.3:0/3942104046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af385b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5df400640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af344960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 -- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 msgr2=0x55f8af344960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af344960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 -- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 msgr2=0x55f8af385b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af385b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5dea00640 1 -- 172.21.15.3:0/3942104046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8aef7b3e0 con 0x55f8af36fbe0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.843+0000 7fd5df400640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af344960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00a9a10 tx=0x7fd5d0032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafd4506c78ba815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00823d0 con 0x55f8af36fbe0 2024-09-10T07:52:34.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fd5d0081f40 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c801b070 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd5d00800b0 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 10 client.7587 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55f8af0e41e0 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c801b070 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fd5d003a070 con 0x55f8af36fbe0 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 10 client.7587 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 10 client.7587 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 20 client.7587 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:34.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af39e780 0x55f8af39eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 4 client.7587 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.844+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8af2b60f0 con 0x55f8af39e780 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.845+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af39e780 0x55f8af39eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.845+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af39e780 0x55f8af39eb90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fde2d4fa78853 server_cookie=3d95d509f954755c in_seq=0 out_seq=0 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.845+0000 7fd5c3e00640 10 client.7587 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5c3e00640 1 -- 172.21.15.3:0/3942104046 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8af2b60f0 con 0x55f8af39e780 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5c3e00640 10 client.7587 handle_command_reply: tid=0 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 client.7587 shutdown 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 msgr2=0x55f8af35e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00a9a10 tx=0x7fd5d0032440 comp rx=0 tx=0).stop 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 shutdown_connections 2024-09-10T07:52:34.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af39e780 0x55f8af39eb90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af344960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af385b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3942104046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af35e540 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5c3e00640 0 client.7587 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 >> 172.21.15.3:0/3942104046 conn(0x55f8af343540 msgr2=0x55f8af36d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 shutdown_connections 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.846+0000 7fd5e8820740 1 -- 172.21.15.3:0/3942104046 wait complete. 2024-09-10T07:52:34.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.847+0000 7fd5e8820740 20 client.7587 trim_cache size 0 max 16384 2024-09-10T07:52:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.865+0000 7fd5e8820740 1 Processor -- start 2024-09-10T07:52:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.865+0000 7fd5e8820740 1 -- start start 2024-09-10T07:52:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af343b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af36fbe0 0x55f8af38cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e00b2240 con 0x55f8af38a5a0 2024-09-10T07:52:34.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0143250 con 0x55f8af36fbe0 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e00b1b10 con 0x55f8af3444e0 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44516/0 (socket says 172.21.15.3:44516) 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 -- 172.21.15.3:0/3778135656 learned_addr learned my addr 172.21.15.3:0/3778135656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5dea00640 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af343b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af36fbe0 0x55f8af38cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 -- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 msgr2=0x55f8af343b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af343b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 -- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af36fbe0 msgr2=0x55f8af38cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af36fbe0 0x55f8af38cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.866+0000 7fd5df400640 1 -- 172.21.15.3:0/3778135656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8af382c10 con 0x55f8af38a5a0 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5df400640 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8003ea0 tx=0x7fd5d8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06ff71ecbd7e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5cea00640 1 -- 172.21.15.3:0/3778135656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8005190 con 0x55f8af38a5a0 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5cea00640 1 -- 172.21.15.3:0/3778135656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d8005330 con 0x55f8af38a5a0 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5cea00640 1 -- 172.21.15.3:0/3778135656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8005520 con 0x55f8af38a5a0 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 -- 172.21.15.3:0/3778135656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 msgr2=0x55f8af36c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8003ea0 tx=0x7fd5d8001670 comp rx=0 tx=0).stop 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 -- 172.21.15.3:0/3778135656 shutdown_connections 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af38a5a0 0x55f8af36c5f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af36fbe0 0x55f8af38cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 --2- 172.21.15.3:0/3778135656 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af343b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 -- 172.21.15.3:0/3778135656 >> 172.21.15.3:0/3778135656 conn(0x55f8af343540 msgr2=0x55f8af3859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.867+0000 7fd5e8820740 1 -- 172.21.15.3:0/3778135656 shutdown_connections 2024-09-10T07:52:34.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.868+0000 7fd5e8820740 1 -- 172.21.15.3:0/3778135656 wait complete. 2024-09-10T07:52:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.868+0000 7fd5e8820740 1 -- start start 2024-09-10T07:52:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.868+0000 7fd5e8820740 1 Processor -- start 2024-09-10T07:52:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af408b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af38bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8af386660 con 0x55f8af36fbe0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0151760 con 0x55f8af38a5a0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f8af346780 con 0x55f8af3444e0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.869+0000 7fd5e8820740 10 client.0 _ll_register_callbacks cb 0x7fd5d00094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44520/0 (socket says 172.21.15.3:44520) 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 -- 172.21.15.3:0/1248030218 learned_addr learned my addr 172.21.15.3:0/1248030218 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5df400640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af38bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5dea00640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af408b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 -- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 msgr2=0x55f8af408b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af408b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 -- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 msgr2=0x55f8af38bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af38bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 -- 172.21.15.3:0/1248030218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8aef544a0 con 0x55f8af36fbe0 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5df400640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af38bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5dea00640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af408b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5d6000640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x55f8af36d190 tx=0x7fd5c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f57bcbd5508ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8025040 con 0x55f8af36fbe0 2024-09-10T07:52:34.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.870+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fd5c8004220 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d8005330 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd5c803d050 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 10 client.7593 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55f8af4091e0 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d8005330 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fd5c8040020 con 0x55f8af36fbe0 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 10 client.7593 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 10 client.7593 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 20 client.7593 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af38c060 0x55f8af399f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:34.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 4 client.7593 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.871+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55f8af2b0440 con 0x55f8af38c060 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.872+0000 7fd5dea00640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af38c060 0x55f8af399f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.872+0000 7fd5dea00640 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af38c060 0x55f8af399f20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6101e816059d15 server_cookie=b54b916513417d59 in_seq=0 out_seq=0 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.872+0000 7fd5c3e00640 10 client.7593 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5c3e00640 1 -- 172.21.15.3:0/1248030218 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f8af2b0440 con 0x55f8af38c060 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5c3e00640 10 client.7593 handle_command_reply: tid=0 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 client.7593 shutdown 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 msgr2=0x55f8af39cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x55f8af36d190 tx=0x7fd5c8019000 comp rx=0 tx=0).stop 2024-09-10T07:52:34.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 shutdown_connections 2024-09-10T07:52:34.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f8af38c060 0x55f8af399f20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8af38a5a0 0x55f8af38bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8af36fbe0 0x55f8af39cd40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 --2- 172.21.15.3:0/1248030218 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f8af3444e0 0x55f8af408b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 >> 172.21.15.3:0/1248030218 conn(0x55f8af343540 msgr2=0x55f8af344f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.878 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:52:34.873+0000 7f8ca5000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1) n(v3 rc2024-09-10T07:52:18.703983+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561e6517b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:52:18.703983+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5c3e00640 0 client.7593 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:34.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 shutdown_connections 2024-09-10T07:52:34.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 1 -- 172.21.15.3:0/1248030218 wait complete. 2024-09-10T07:52:34.879 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:52:34.873+0000 7f9536000640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1) n(v2 rc2024-09-10T07:52:18.703983+0000 4=0+4) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5564a90e9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:52:18.703983+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.873+0000 7fd5e8820740 20 client.7593 trim_cache size 0 max 16384 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-10T07:52:34.880 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:34.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 msgr2=0x7fd5bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 0x7fd5bc06e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd5e01f7cd0 tx=0x7fd5c8009190 comp rx=0 tx=0).stop 2024-09-10T07:52:34.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 msgr2=0x7fd5e00711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:34.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0000c00 tx=0x7fd5d00046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 shutdown_connections 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd5bc06be30 0x7fd5bc06e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e014dee0 0x7fd5e00728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5e00ccc50 0x7fd5e015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 --2- 172.21.15.3:0/3556112514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e00b3e10 0x7fd5e00711a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 >> 172.21.15.3:0/3556112514 conn(0x7fd5e0062cc0 msgr2=0x7fd5e006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.874+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 shutdown_connections 2024-09-10T07:52:34.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:34.875+0000 7fd5e5800640 1 -- 172.21.15.3:0/3556112514 wait complete. 2024-09-10T07:52:34.882 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:52:34.875+0000 7f8cad200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1) n(v4 rc2024-09-10T07:52:18.703983+0000 5=0+5) (inest sync->mix g=1,2) caps={7392=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561e6517bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:52:18.703983+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.883 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:52:34.875+0000 7f953e200640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1) n(v1 rc2024-09-10T07:52:18.703983+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7392=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x5564a90e8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:52:18.703983+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.883 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:52:34.874+0000 7f3877800640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1) n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e641253b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.883 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:52:34.874+0000 7f3877800640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/AsLsXsFsx@2},l=7392 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e641254100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:52:34.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:34.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub pause 2024-09-10T07:52:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.049+0000 7fae22e00640 1 Processor -- start 2024-09-10T07:52:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.049+0000 7fae22e00640 1 -- start start 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae22e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae1c1451d0 0x7fae1c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae22e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae1c0cf270 0x7fae1c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae22e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1c0cd900 con 0x7fae1c145730 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae22e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c06d7c0 con 0x7fae1c1451d0 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae22e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c0cdbe0 con 0x7fae1c0cf270 2024-09-10T07:52:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae1be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fae1c1451d0 0x7fae1c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae13e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fae1c145730 0x7fae1c0b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51760/0 (socket says 172.21.15.3:51760) 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae13e00640 1 -- 172.21.15.3:0/2655054922 learned_addr learned my addr 172.21.15.3:0/2655054922 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.050+0000 7fae20e00640 1 --2- 172.21.15.3:0/2655054922 >> v2:172.21.15.70:3300/0 conn(0x7fae1c0cf270 0x7fae1c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.051+0000 7fae1be00640 1 -- 172.21.15.3:0/2655054922 >> v2:172.21.15.70:3300/0 conn(0x7fae1c0cf270 msgr2=0x7fae1c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.051+0000 7fae1be00640 1 --2- 172.21.15.3:0/2655054922 >> v2:172.21.15.70:3300/0 conn(0x7fae1c0cf270 0x7fae1c0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.051+0000 7fae1be00640 1 -- 172.21.15.3:0/2655054922 >> v1:172.21.15.3:6789/0 conn(0x7fae1c145730 legacy=0x7fae1c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.051+0000 7fae1be00640 1 -- 172.21.15.3:0/2655054922 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c06da10 con 0x7fae1c1451d0 2024-09-10T07:52:35.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.051+0000 7fae1b400640 1 -- 172.21.15.3:0/2655054922 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518317426 0 0) 0x7fae1c0cd900 con 0x7fae1c145730 2024-09-10T07:52:35.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.052+0000 7fae1be00640 1 --2- 172.21.15.3:0/2655054922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c1451d0 0x7fae1c06cfc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0b5370 tx=0x7fae0c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bc9bce4c5c842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.052+0000 7fae1b400640 1 -- 172.21.15.3:0/2655054922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c037070 con 0x7fae1c1451d0 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.052+0000 7fae1b400640 1 -- 172.21.15.3:0/2655054922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c032440 con 0x7fae1c1451d0 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.052+0000 7fae1b400640 1 -- 172.21.15.3:0/2655054922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c02ae60 con 0x7fae1c1451d0 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 -- 172.21.15.3:0/2655054922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c1451d0 msgr2=0x7fae1c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 --2- 172.21.15.3:0/2655054922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c1451d0 0x7fae1c06cfc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0b5370 tx=0x7fae0c02a110 comp rx=0 tx=0).stop 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 -- 172.21.15.3:0/2655054922 shutdown_connections 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 --2- 172.21.15.3:0/2655054922 >> v2:172.21.15.70:3300/0 conn(0x7fae1c0cf270 0x7fae1c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 --2- 172.21.15.3:0/2655054922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c1451d0 0x7fae1c06cfc0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 -- 172.21.15.3:0/2655054922 >> 172.21.15.3:0/2655054922 conn(0x7fae1c0b5cb0 msgr2=0x7fae1c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 -- 172.21.15.3:0/2655054922 shutdown_connections 2024-09-10T07:52:35.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.053+0000 7fae22e00640 1 -- 172.21.15.3:0/2655054922 wait complete. 2024-09-10T07:52:35.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.054+0000 7fae22e00640 1 Processor -- start 2024-09-10T07:52:35.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.054+0000 7fae22e00640 1 -- start start 2024-09-10T07:52:35.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 0x7fae1c160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 0x7fae1c1f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c06df30 con 0x7fae1c0cf270 2024-09-10T07:52:35.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c0cfc10 con 0x7fae1c145730 2024-09-10T07:52:35.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.055+0000 7fae22e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c06e0b0 con 0x7fae1c1451d0 2024-09-10T07:52:35.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44538/0 (socket says 172.21.15.3:44538) 2024-09-10T07:52:35.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 -- 172.21.15.3:0/3112217026 learned_addr learned my addr 172.21.15.3:0/3112217026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae20e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 0x7fae1c1f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae13e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 0x7fae1c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 -- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 msgr2=0x7fae1c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 0x7fae1c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 -- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 msgr2=0x7fae1c1f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 0x7fae1c1f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 -- 172.21.15.3:0/3112217026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c13e130 con 0x7fae1c0cf270 2024-09-10T07:52:35.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae13e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 0x7fae1c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae20e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 0x7fae1c1f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.056+0000 7fae1be00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fae0c035040 tx=0x7fae0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.057+0000 7fae18c00640 1 -- 172.21.15.3:0/3112217026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c037050 con 0x7fae1c0cf270 2024-09-10T07:52:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.057+0000 7fae18c00640 1 -- 172.21.15.3:0/3112217026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c038040 con 0x7fae1c0cf270 2024-09-10T07:52:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.057+0000 7fae18c00640 1 -- 172.21.15.3:0/3112217026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c004450 con 0x7fae1c0cf270 2024-09-10T07:52:35.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.057+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c13e410 con 0x7fae1c0cf270 2024-09-10T07:52:35.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.057+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c15ce10 con 0x7fae1c0cf270 2024-09-10T07:52:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.059+0000 7fae18c00640 1 -- 172.21.15.3:0/3112217026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae0c03c6d0 con 0x7fae1c0cf270 2024-09-10T07:52:35.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.060+0000 7fae18c00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 0x7fadf806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.060+0000 7fae18c00640 1 -- 172.21.15.3:0/3112217026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fae0c0ab8b0 con 0x7fae1c0cf270 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.060+0000 7fae13e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 0x7fadf806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.060+0000 7fae25c38740 1 Processor -- start 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.060+0000 7fae25c38740 1 -- start start 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae13e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 0x7fadf806e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0cca90 tx=0x7fae04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef870e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e5210 0x55ef870de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef870aacf0 con 0x55ef870e3a80 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef870467f0 con 0x55ef870e5210 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef86fee800 con 0x55ef870e1240 2024-09-10T07:52:35.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae1be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44550/0 (socket says 172.21.15.3:44550) 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.061+0000 7fae1be00640 1 -- 172.21.15.3:0/1039761478 learned_addr learned my addr 172.21.15.3:0/1039761478 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae13e00640 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef870e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae20e00640 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e5210 0x55ef870de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 -- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 msgr2=0x55ef870e3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef870e3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 -- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e5210 msgr2=0x55ef870de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e5210 0x55ef870de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 -- 172.21.15.3:0/1039761478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef86e237d0 con 0x55ef870e3a80 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae1be00640 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0797c0 tx=0x7fae0c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9405897b9f2d346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae10c00640 1 -- 172.21.15.3:0/1039761478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c079ac0 con 0x55ef870e3a80 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae10c00640 1 -- 172.21.15.3:0/1039761478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c07de70 con 0x55ef870e3a80 2024-09-10T07:52:35.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.062+0000 7fae10c00640 1 -- 172.21.15.3:0/1039761478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0826a0 con 0x55ef870e3a80 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 -- 172.21.15.3:0/1039761478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 msgr2=0x55ef87129220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0797c0 tx=0x7fae0c0093c0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 -- 172.21.15.3:0/1039761478 shutdown_connections 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e5210 0x55ef870de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef870e3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 --2- 172.21.15.3:0/1039761478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef87129220 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 -- 172.21.15.3:0/1039761478 >> 172.21.15.3:0/1039761478 conn(0x55ef870e0240 msgr2=0x55ef871286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 -- 172.21.15.3:0/1039761478 shutdown_connections 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.063+0000 7fae25c38740 1 -- 172.21.15.3:0/1039761478 wait complete. 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.064+0000 7fae25c38740 1 -- start start 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.064+0000 7fae25c38740 1 Processor -- start 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e3a80 0x55ef871035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef871059b0 0x55ef870de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ef86fcf750 con 0x55ef870e5210 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef870427b0 con 0x55ef871059b0 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef870aacf0 con 0x55ef870e3a80 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae25c38740 10 client.0 _ll_register_callbacks cb 0x55ef86d58be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44564/0 (socket says 172.21.15.3:44564) 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae13e00640 1 -- 172.21.15.3:0/25124361 learned_addr learned my addr 172.21.15.3:0/25124361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae1be00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e3a80 0x55ef871035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.065+0000 7fae20e00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef871059b0 0x55ef870de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 -- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e3a80 msgr2=0x55ef871035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e3a80 0x55ef871035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 -- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef871059b0 msgr2=0x55ef870de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef871059b0 0x55ef870de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 -- 172.21.15.3:0/25124361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef86d190f0 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae13e00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fae04005ad0 tx=0x7fae04000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d586a4b79f41e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae09600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04026060 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fae04021a90 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae0c07de70 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.066+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fae04021c90 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 10 client.7608 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55ef86e83f60 con 0x55ef870e5210 2024-09-10T07:52:35.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c07de70 con 0x55ef870e5210 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fae04012410 con 0x55ef870e5210 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 10 client.7608 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 10 client.7608 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 20 client.7608 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef87103ce0 0x55ef871040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 4 client.7608 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef87053f20 con 0x55ef87103ce0 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.067+0000 7fae1be00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef87103ce0 0x55ef871040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.068+0000 7fae1be00640 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef87103ce0 0x55ef871040f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4faa25c918c11a9 server_cookie=dcca1eb34025ebc in_seq=0 out_seq=0 2024-09-10T07:52:35.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.068+0000 7fae09600640 10 client.7608 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.068+0000 7fae09600640 1 -- 172.21.15.3:0/25124361 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef87053f20 con 0x55ef87103ce0 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.068+0000 7fae09600640 10 client.7608 handle_command_reply: tid=0 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.068+0000 7fae25c38740 1 client.7608 shutdown 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 msgr2=0x55ef87105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fae04005ad0 tx=0x7fae04000ed0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 shutdown_connections 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef87103ce0 0x55ef871040f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef871059b0 0x55ef870de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef87105470 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 --2- 172.21.15.3:0/25124361 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e3a80 0x55ef871035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae09600640 0 client.7608 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 >> 172.21.15.3:0/25124361 conn(0x55ef870e1240 msgr2=0x55ef87126770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 shutdown_connections 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 1 -- 172.21.15.3:0/25124361 wait complete. 2024-09-10T07:52:35.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.069+0000 7fae25c38740 20 client.7608 trim_cache size 0 max 16384 2024-09-10T07:52:35.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.087+0000 7fae25c38740 1 Processor -- start 2024-09-10T07:52:35.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.087+0000 7fae25c38740 1 -- start start 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef8710b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1fe820 con 0x55ef870e5210 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1fda10 con 0x55ef870e3a80 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c1fe0f0 con 0x55ef870e1240 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44580/0 (socket says 172.21.15.3:44580) 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 -- 172.21.15.3:0/2432549238 learned_addr learned my addr 172.21.15.3:0/2432549238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae1be00640 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef8710b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae13e00640 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 -- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 msgr2=0x55ef8710b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef8710b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 -- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e3a80 msgr2=0x55ef8710db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.088+0000 7fae20e00640 1 -- 172.21.15.3:0/2432549238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef86cf21b0 con 0x55ef870e5210 2024-09-10T07:52:35.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae20e00640 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fae14016730 tx=0x7fae14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a458ce7cf3585ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae0be00640 1 -- 172.21.15.3:0/2432549238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14016e90 con 0x55ef870e5210 2024-09-10T07:52:35.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae0be00640 1 -- 172.21.15.3:0/2432549238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae14007aa0 con 0x55ef870e5210 2024-09-10T07:52:35.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae0be00640 1 -- 172.21.15.3:0/2432549238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14007e50 con 0x55ef870e5210 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 -- 172.21.15.3:0/2432549238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 msgr2=0x55ef8710e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fae14016730 tx=0x7fae14009750 comp rx=0 tx=0).stop 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 -- 172.21.15.3:0/2432549238 shutdown_connections 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 --2- 172.21.15.3:0/2432549238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e1240 0x55ef8710b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.089+0000 7fae25c38740 1 -- 172.21.15.3:0/2432549238 >> 172.21.15.3:0/2432549238 conn(0x55ef87128030 msgr2=0x55ef87142620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.090+0000 7fae25c38740 1 -- 172.21.15.3:0/2432549238 shutdown_connections 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.090+0000 7fae25c38740 1 -- 172.21.15.3:0/2432549238 wait complete. 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.090+0000 7fae25c38740 1 -- start start 2024-09-10T07:52:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.090+0000 7fae25c38740 1 Processor -- start 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e1240 0x55ef871a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1f5770 con 0x55ef870e3a80 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55ef871054b0 con 0x55ef870e1240 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55ef8710bbc0 con 0x55ef870e5210 2024-09-10T07:52:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae25c38740 10 client.0 _ll_register_callbacks cb 0x7fae040076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.091+0000 7fae13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44588/0 (socket says 172.21.15.3:44588) 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 -- 172.21.15.3:0/907556700 learned_addr learned my addr 172.21.15.3:0/907556700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae1be00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e1240 0x55ef871a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae20e00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 -- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e5210 msgr2=0x55ef8710e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 -- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e1240 msgr2=0x55ef871a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e1240 0x55ef871a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 -- 172.21.15.3:0/907556700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef87126dd0 con 0x55ef870e3a80 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae13e00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fae04012ce0 tx=0x7fae04007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec89d4c367bb4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae09600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04012280 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fae0401d050 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.092+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae14007aa0 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fae04012420 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 10 client.7614 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55ef87105120 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae14007aa0 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7fae04042020 con 0x55ef870e3a80 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 10 client.7614 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 10 client.7614 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 20 client.7614 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef871059b0 0x55ef871aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 4 client.7614 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55ef8704e270 con 0x55ef871059b0 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.093+0000 7fae1be00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef871059b0 0x55ef871aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae1be00640 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef871059b0 0x55ef871aacc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce064288c1de78c6 server_cookie=c905413bcbe9ac62 in_seq=0 out_seq=0 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae09600640 10 client.7614 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae09600640 1 -- 172.21.15.3:0/907556700 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef8704e270 con 0x55ef871059b0 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae09600640 10 client.7614 handle_command_reply: tid=0 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 client.7614 shutdown 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 msgr2=0x55ef8710db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fae04012ce0 tx=0x7fae04007000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 shutdown_connections 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55ef871059b0 0x55ef871aacc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55ef870e5210 0x55ef8710e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ef870e3a80 0x55ef8710db00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 --2- 172.21.15.3:0/907556700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ef870e1240 0x55ef871a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae09600640 0 client.7614 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.094+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 >> 172.21.15.3:0/907556700 conn(0x55ef87128030 msgr2=0x55ef87142620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 shutdown_connections 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae25c38740 1 -- 172.21.15.3:0/907556700 wait complete. 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae25c38740 20 client.7614 trim_cache size 0 max 16384 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:52:35.099 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:52:35.100 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 msgr2=0x7fadf806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 0x7fadf806e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0cca90 tx=0x7fae04009040 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 msgr2=0x7fae1c15be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fae0c035040 tx=0x7fae0c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 shutdown_connections 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fadf806be60 0x7fadf806e320 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c145730 0x7fae1c1f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae1c1451d0 0x7fae1c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 --2- 172.21.15.3:0/3112217026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae1c0cf270 0x7fae1c15be80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.095+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 >> 172.21.15.3:0/3112217026 conn(0x7fae1c0b5cb0 msgr2=0x7fae1c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.096+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 shutdown_connections 2024-09-10T07:52:35.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.096+0000 7fae22e00640 1 -- 172.21.15.3:0/3112217026 wait complete. 2024-09-10T07:52:35.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:35.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-10T07:52:35.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.269+0000 7f79e8c00640 1 Processor -- start 2024-09-10T07:52:35.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.269+0000 7f79e8c00640 1 -- start start 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.270+0000 7f79e8c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79e40cf980 0x7f79e406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.270+0000 7f79e8c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e40cd900 con 0x7f79e4145730 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.270+0000 7f79e8c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e40cdbe0 con 0x7f79e41451d0 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.270+0000 7f79e8c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f79e406d7c0 con 0x7f79e40cf980 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e1400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79e4145730 0x7f79e40cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51768/0 (socket says 172.21.15.3:51768) 2024-09-10T07:52:35.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e1400640 1 -- 172.21.15.3:0/4104717658 learned_addr learned my addr 172.21.15.3:0/4104717658 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:35.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e2800640 1 --2- 172.21.15.3:0/4104717658 >> v2:172.21.15.3:3300/0 conn(0x7f79e40cf980 0x7f79e406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e0a00640 1 -- 172.21.15.3:0/4104717658 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222416776 0 0) 0x7f79e40cd900 con 0x7f79e4145730 2024-09-10T07:52:35.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e0a00640 1 -- 172.21.15.3:0/4104717658 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79c8003520 con 0x7f79e4145730 2024-09-10T07:52:35.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e2800640 1 -- 172.21.15.3:0/4104717658 >> v1:172.21.15.70:6789/0 conn(0x7f79e41451d0 legacy=0x7f79e40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e2800640 1 -- 172.21.15.3:0/4104717658 >> v1:172.21.15.3:6789/0 conn(0x7f79e4145730 legacy=0x7f79e40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.271+0000 7f79e2800640 1 -- 172.21.15.3:0/4104717658 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e406da10 con 0x7f79e40cf980 2024-09-10T07:52:35.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.272+0000 7f79e2800640 1 --2- 172.21.15.3:0/4104717658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e406cfc0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f79e406d940 tx=0x7f79dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb06828c44b8865 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.272+0000 7f79e0a00640 1 -- 172.21.15.3:0/4104717658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007740 con 0x7f79e40cf980 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.272+0000 7f79e0a00640 1 -- 172.21.15.3:0/4104717658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc0078e0 con 0x7f79e40cf980 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.272+0000 7f79e0a00640 1 -- 172.21.15.3:0/4104717658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007cb0 con 0x7f79e40cf980 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 -- 172.21.15.3:0/4104717658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 msgr2=0x7f79e406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 --2- 172.21.15.3:0/4104717658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e406cfc0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f79e406d940 tx=0x7f79dc02a660 comp rx=0 tx=0).stop 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 -- 172.21.15.3:0/4104717658 shutdown_connections 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 --2- 172.21.15.3:0/4104717658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e406cfc0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 -- 172.21.15.3:0/4104717658 >> 172.21.15.3:0/4104717658 conn(0x7f79e40b5cb0 msgr2=0x7f79e406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 -- 172.21.15.3:0/4104717658 shutdown_connections 2024-09-10T07:52:35.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.273+0000 7f79e8c00640 1 -- 172.21.15.3:0/4104717658 wait complete. 2024-09-10T07:52:35.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.275+0000 7f79e8c00640 1 Processor -- start 2024-09-10T07:52:35.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.275+0000 7f79e8c00640 1 -- start start 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e41451d0 0x7f79e41f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e4145730 0x7f79e415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79e406e0b0 con 0x7f79e40cf980 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79e406df30 con 0x7f79e4145730 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f79e406e230 con 0x7f79e41451d0 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44606/0 (socket says 172.21.15.3:44606) 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 -- 172.21.15.3:0/2467822952 learned_addr learned my addr 172.21.15.3:0/2467822952 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1400640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e41451d0 0x7f79e41f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e2800640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e4145730 0x7f79e415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 -- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e41451d0 msgr2=0x7f79e41f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e41451d0 0x7f79e41f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 -- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e4145730 msgr2=0x7f79e415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e4145730 0x7f79e415ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.276+0000 7f79e1e00640 1 -- 172.21.15.3:0/2467822952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e40abd10 con 0x7f79e40cf980 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79e1e00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c480 tx=0x7f79d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79da000640 1 -- 172.21.15.3:0/2467822952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d40052d0 con 0x7f79e40cf980 2024-09-10T07:52:35.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79da000640 1 -- 172.21.15.3:0/2467822952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d4005470 con 0x7f79e40cf980 2024-09-10T07:52:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79da000640 1 -- 172.21.15.3:0/2467822952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4012870 con 0x7f79e40cf980 2024-09-10T07:52:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e40abfb0 con 0x7f79e40cf980 2024-09-10T07:52:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.277+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e40ac570 con 0x7f79e40cf980 2024-09-10T07:52:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.280+0000 7f79da000640 1 -- 172.21.15.3:0/2467822952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79d4012a10 con 0x7f79e40cf980 2024-09-10T07:52:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.281+0000 7f79da000640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 0x7f79c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.281+0000 7f79da000640 1 -- 172.21.15.3:0/2467822952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f79d408a9b0 con 0x7f79e40cf980 2024-09-10T07:52:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.281+0000 7f79e1400640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 0x7f79c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.281+0000 7f79eba37740 1 Processor -- start 2024-09-10T07:52:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.282+0000 7f79eba37740 1 -- start start 2024-09-10T07:52:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.282+0000 7f79e1400640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 0x7f79c406e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cdf70 tx=0x7f79d0007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.282+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 0x5636a17eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.282+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.282+0000 7f79eba37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79eba37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636a170a660 con 0x5636a17ecc20 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5636a16b2680 con 0x5636a17bcbc0 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636a1706620 con 0x5636a17bc490 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44610/0 (socket says 172.21.15.3:44610) 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 -- 172.21.15.3:0/93058243 learned_addr learned my addr 172.21.15.3:0/93058243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e1400640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e1e00640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 0x5636a17eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 -- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 msgr2=0x5636a17eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 0x5636a17eaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 -- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 msgr2=0x5636a17ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e2800640 1 -- 172.21.15.3:0/93058243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a14e77d0 con 0x5636a17ecc20 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e1e00640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 0x5636a17eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.283+0000 7f79e1400640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79e2800640 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0095c0 tx=0x7f79dc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b669b7b4d392fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79ce600640 1 -- 172.21.15.3:0/93058243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0079a0 con 0x5636a17ecc20 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79ce600640 1 -- 172.21.15.3:0/93058243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc007b40 con 0x5636a17ecc20 2024-09-10T07:52:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79ce600640 1 -- 172.21.15.3:0/93058243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc03c420 con 0x5636a17ecc20 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 -- 172.21.15.3:0/93058243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 msgr2=0x5636a17ed740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0095c0 tx=0x7f79dc02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 -- 172.21.15.3:0/93058243 shutdown_connections 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17ed740 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.284+0000 7f79eba37740 1 --2- 172.21.15.3:0/93058243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bc490 0x5636a17eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.285+0000 7f79eba37740 1 -- 172.21.15.3:0/93058243 >> 172.21.15.3:0/93058243 conn(0x5636a17a3220 msgr2=0x5636a17eb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.285+0000 7f79eba37740 1 -- 172.21.15.3:0/93058243 shutdown_connections 2024-09-10T07:52:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.285+0000 7f79eba37740 1 -- 172.21.15.3:0/93058243 wait complete. 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.285+0000 7f79eba37740 1 -- start start 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.285+0000 7f79eba37740 1 Processor -- start 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636a17e9ef0 con 0x5636a17ecc20 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5636a16934c0 con 0x5636a17bcbc0 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636a17ea1d0 con 0x5636a17d0aa0 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.286+0000 7f79eba37740 10 client.0 _ll_register_callbacks cb 0x5636a1702e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44618/0 (socket says 172.21.15.3:44618) 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 -- 172.21.15.3:0/324885854 learned_addr learned my addr 172.21.15.3:0/324885854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 -- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 msgr2=0x5636a17e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 -- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 msgr2=0x5636a17d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 -- 172.21.15.3:0/324885854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a141d3f0 con 0x5636a17ecc20 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79e1400640 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f79d00065b0 tx=0x7f79d0001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b40c1da040e99d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0005e40 con 0x5636a17ecc20 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.287+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f79d0024780 con 0x5636a17ecc20 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc007b40 con 0x5636a17ecc20 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f79d00249c0 con 0x5636a17ecc20 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 10 client.7629 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5636a13dd0f0 con 0x5636a17ecc20 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc007b40 con 0x5636a17ecc20 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f79d0024b60 con 0x5636a17ecc20 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 10 client.7629 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 10 client.7629 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 20 client.7629 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a1803710 0x5636a1803b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 4 client.7629 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636a1717d90 con 0x5636a1803710 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.288+0000 7f79e1e00640 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a1803710 0x5636a1803b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.289+0000 7f79e1e00640 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a1803710 0x5636a1803b60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6e09247a7c79788 server_cookie=9ea1ac799600b787 in_seq=0 out_seq=0 2024-09-10T07:52:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.289+0000 7f79c2a00640 10 client.7629 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79c2a00640 1 -- 172.21.15.3:0/324885854 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636a1717d90 con 0x5636a1803710 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79c2a00640 10 client.7629 handle_command_reply: tid=0 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 client.7629 shutdown 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 msgr2=0x5636a17e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f79d00065b0 tx=0x7f79d0001150 comp rx=0 tx=0).stop 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 shutdown_connections 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a1803710 0x5636a1803b60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17e8200 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 --2- 172.21.15.3:0/324885854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a17d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79c2a00640 0 client.7629 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 >> 172.21.15.3:0/324885854 conn(0x5636a17bc490 msgr2=0x5636a179ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 shutdown_connections 2024-09-10T07:52:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 1 -- 172.21.15.3:0/324885854 wait complete. 2024-09-10T07:52:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.290+0000 7f79eba37740 20 client.7629 trim_cache size 0 max 16384 2024-09-10T07:52:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.308+0000 7f79eba37740 1 Processor -- start 2024-09-10T07:52:35.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.308+0000 7f79eba37740 1 -- start start 2024-09-10T07:52:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 0x5636a17e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 0x5636a17cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41fc810 con 0x5636a17ecc20 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4072a80 con 0x5636a17d0aa0 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f79e40ab130 con 0x5636a17bcbc0 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44624/0 (socket says 172.21.15.3:44624) 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.309+0000 7f79e2800640 1 -- 172.21.15.3:0/3982823241 learned_addr learned my addr 172.21.15.3:0/3982823241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e1e00640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 0x5636a17e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e1400640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 0x5636a17cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 -- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 msgr2=0x5636a17e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 0x5636a17e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 -- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 msgr2=0x5636a17cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 0x5636a17cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 -- 172.21.15.3:0/3982823241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a17e7770 con 0x5636a17ecc20 2024-09-10T07:52:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e1e00640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 0x5636a17e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e1400640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 0x5636a17cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79e2800640 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0075f0 tx=0x7f79dc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d690b28511311606 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79cdc00640 1 -- 172.21.15.3:0/3982823241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc003990 con 0x5636a17ecc20 2024-09-10T07:52:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79cdc00640 1 -- 172.21.15.3:0/3982823241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc003b30 con 0x5636a17ecc20 2024-09-10T07:52:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.310+0000 7f79cdc00640 1 -- 172.21.15.3:0/3982823241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0077a0 con 0x5636a17ecc20 2024-09-10T07:52:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 -- 172.21.15.3:0/3982823241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 msgr2=0x5636a17cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0075f0 tx=0x7f79dc02c000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 -- 172.21.15.3:0/3982823241 shutdown_connections 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a17cf5c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17d0aa0 0x5636a17cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 --2- 172.21.15.3:0/3982823241 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17bcbc0 0x5636a17e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 -- 172.21.15.3:0/3982823241 >> 172.21.15.3:0/3982823241 conn(0x5636a17bc490 msgr2=0x5636a179ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 -- 172.21.15.3:0/3982823241 shutdown_connections 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.311+0000 7f79eba37740 1 -- 172.21.15.3:0/3982823241 wait complete. 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.312+0000 7f79eba37740 1 -- start start 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.312+0000 7f79eba37740 1 Processor -- start 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a1868810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636a17a2860 con 0x5636a17ecc20 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79e40ad980 con 0x5636a17bcbc0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5636a17a1540 con 0x5636a17d0aa0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79eba37740 10 client.0 _ll_register_callbacks cb 0x7f79d0005930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44640/0 (socket says 172.21.15.3:44640) 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 -- 172.21.15.3:0/1991987882 learned_addr learned my addr 172.21.15.3:0/1991987882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e1e00640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a1868810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e1400640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 -- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 msgr2=0x5636a17e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 -- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 msgr2=0x5636a1868810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a1868810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 -- 172.21.15.3:0/1991987882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a186f8d0 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e1e00640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a1868810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.313+0000 7f79e2800640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0071e0 tx=0x7f79dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4c245de2a2024c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007930 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f79dc04a070 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc003b30 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f79dc05d050 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 10 client.7635 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5636a17a0fc0 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc003b30 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f79dc060020 con 0x5636a17ecc20 2024-09-10T07:52:35.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 10 client.7635 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 10 client.7635 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 20 client.7635 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a17e3b50 0x5636a1802a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 4 client.7635 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.314+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636a17120e0 con 0x5636a17e3b50 2024-09-10T07:52:35.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79e1e00640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a17e3b50 0x5636a1802a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79e1e00640 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a17e3b50 0x5636a1802a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78b7201f9a612bac server_cookie=6c82eb17dc26f47d in_seq=0 out_seq=0 2024-09-10T07:52:35.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79c2a00640 10 client.7635 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:35.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79c2a00640 1 -- 172.21.15.3:0/1991987882 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5636a17120e0 con 0x5636a17e3b50 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79c2a00640 10 client.7635 handle_command_reply: tid=0 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.315+0000 7f79eba37740 1 client.7635 shutdown 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 msgr2=0x5636a1802510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0071e0 tx=0x7f79dc004ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 shutdown_connections 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636a17ecc20 0x5636a1802510 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5636a17e3b50 0x5636a1802a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5636a17d0aa0 0x5636a17e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 --2- 172.21.15.3:0/1991987882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5636a17bcbc0 0x5636a1868810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79c2a00640 0 client.7635 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 >> 172.21.15.3:0/1991987882 conn(0x5636a17bc490 msgr2=0x5636a17e5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 shutdown_connections 2024-09-10T07:52:35.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 1 -- 172.21.15.3:0/1991987882 wait complete. 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79eba37740 20 client.7635 trim_cache size 0 max 16384 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:35.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.322 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.322 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 msgr2=0x7f79c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 0x7f79c406e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cdf70 tx=0x7f79d0007d30 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 msgr2=0x7f79e4140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.316+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c480 tx=0x7f79d400c940 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 shutdown_connections 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c406be30 0x7f79c406e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e4145730 0x7f79e415ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e41451d0 0x7f79e41f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 --2- 172.21.15.3:0/2467822952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e40cf980 0x7f79e4140f20 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 >> 172.21.15.3:0/2467822952 conn(0x7f79e40b5cb0 msgr2=0x7f79e4160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 shutdown_connections 2024-09-10T07:52:35.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.317+0000 7f79e8c00640 1 -- 172.21.15.3:0/2467822952 wait complete. 2024-09-10T07:52:35.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:35.332 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:35.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-10T07:52:35.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.490+0000 7f221e200640 1 Processor -- start 2024-09-10T07:52:35.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.490+0000 7f221e200640 1 -- start start 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.491+0000 7f221e200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2218145730 0x7f221806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.491+0000 7f221e200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22180cd900 con 0x7f22181451d0 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.491+0000 7f221e200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22180cdbe0 con 0x7f22180cf270 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.491+0000 7f221e200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f221806d7c0 con 0x7f2218145730 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2217400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f22181451d0 0x7f22180b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51784/0 (socket says 172.21.15.3:51784) 2024-09-10T07:52:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2217400640 1 -- 172.21.15.3:0/3356259919 learned_addr learned my addr 172.21.15.3:0/3356259919 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f220fe00640 1 --2- 172.21.15.3:0/3356259919 >> v2:172.21.15.70:3300/0 conn(0x7f2218145730 0x7f221806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2122700276 0 0) 0x7f22180cd900 con 0x7f22181451d0 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21fc003520 con 0x7f22181451d0 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123966019 0 0) 0x7f22180cdbe0 con 0x7f22180cf270 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22180cd900 con 0x7f22180cf270 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f220fe00640 1 -- 172.21.15.3:0/3356259919 >> v1:172.21.15.70:6789/0 conn(0x7f22180cf270 legacy=0x7f22180cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f220fe00640 1 -- 172.21.15.3:0/3356259919 >> v1:172.21.15.3:6789/0 conn(0x7f22181451d0 legacy=0x7f22180b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.492+0000 7f220fe00640 1 -- 172.21.15.3:0/3356259919 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221806da10 con 0x7f2218145730 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.493+0000 7f220fe00640 1 --2- 172.21.15.3:0/3356259919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f221806cfc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f221806d940 tx=0x7f220002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46f514e7ecb620d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037070 con 0x7f2218145730 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2200032440 con 0x7f2218145730 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f2216a00640 1 -- 172.21.15.3:0/3356259919 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200032810 con 0x7f2218145730 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 -- 172.21.15.3:0/3356259919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 msgr2=0x7f221806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 --2- 172.21.15.3:0/3356259919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f221806cfc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f221806d940 tx=0x7f220002a180 comp rx=0 tx=0).stop 2024-09-10T07:52:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 -- 172.21.15.3:0/3356259919 shutdown_connections 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 --2- 172.21.15.3:0/3356259919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f221806cfc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 -- 172.21.15.3:0/3356259919 >> 172.21.15.3:0/3356259919 conn(0x7f22180b5cb0 msgr2=0x7f221806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 -- 172.21.15.3:0/3356259919 shutdown_connections 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.494+0000 7f221e200640 1 -- 172.21.15.3:0/3356259919 wait complete. 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.496+0000 7f221e200640 1 Processor -- start 2024-09-10T07:52:35.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.496+0000 7f221e200640 1 -- start start 2024-09-10T07:52:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22181451d0 0x7f22180addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f22180c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f221806e0b0 con 0x7f22180cf270 2024-09-10T07:52:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f221806df30 con 0x7f2218145730 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f221e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f221806e230 con 0x7f22181451d0 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f2217400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f2217400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44644/0 (socket says 172.21.15.3:44644) 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f2217400640 1 -- 172.21.15.3:0/953160262 learned_addr learned my addr 172.21.15.3:0/953160262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f2217e00640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f22180c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.497+0000 7f220fe00640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22181451d0 0x7f22180addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 -- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22181451d0 msgr2=0x7f22180addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22181451d0 0x7f22180addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 -- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 msgr2=0x7f22180c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f22180c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 -- 172.21.15.3:0/953160262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22180abec0 con 0x7f22180cf270 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f2217400640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f220800c4d0 tx=0x7f220800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f220f400640 1 -- 172.21.15.3:0/953160262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f22180cf270 2024-09-10T07:52:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f220f400640 1 -- 172.21.15.3:0/953160262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208005ce0 con 0x7f22180cf270 2024-09-10T07:52:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f220f400640 1 -- 172.21.15.3:0/953160262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208007290 con 0x7f22180cf270 2024-09-10T07:52:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22180ac190 con 0x7f22180cf270 2024-09-10T07:52:35.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.498+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22180ac750 con 0x7f22180cf270 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.501+0000 7f220f400640 1 -- 172.21.15.3:0/953160262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f220801a070 con 0x7f22180cf270 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.502+0000 7f220f400640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 0x7f21f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.502+0000 7f220f400640 1 -- 172.21.15.3:0/953160262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f220808a2e0 con 0x7f22180cf270 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.502+0000 7f220fe00640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 0x7f21f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.502+0000 7f2221209740 1 Processor -- start 2024-09-10T07:52:35.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.503+0000 7f2221209740 1 -- start start 2024-09-10T07:52:35.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.503+0000 7f220fe00640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 0x7f21f406e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2200034000 tx=0x7f220002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af57200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 0x56258af57740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56258ae74660 con 0x56258af26420 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56258ae1c680 con 0x56258af0c480 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56258ae70620 con 0x56258af26800 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44652/0 (socket says 172.21.15.3:44652) 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 -- 172.21.15.3:0/661426193 learned_addr learned my addr 172.21.15.3:0/661426193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f220fe00640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af57200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217e00640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 0x56258af57740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 -- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 msgr2=0x56258af57200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af57200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 -- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 msgr2=0x56258af57740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 0x56258af57740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 -- 172.21.15.3:0/661426193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258ac517d0 con 0x56258af26420 2024-09-10T07:52:35.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217e00640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 0x56258af57740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.504+0000 7f2217400640 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f220800ce90 tx=0x7f22080585b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dea4b152a262577 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2207400640 1 -- 172.21.15.3:0/661426193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220808a0c0 con 0x56258af26420 2024-09-10T07:52:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2207400640 1 -- 172.21.15.3:0/661426193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22080567f0 con 0x56258af26420 2024-09-10T07:52:35.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2207400640 1 -- 172.21.15.3:0/661426193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208088bd0 con 0x56258af26420 2024-09-10T07:52:35.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 -- 172.21.15.3:0/661426193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 msgr2=0x56258af56c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f220800ce90 tx=0x7f22080585b0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 -- 172.21.15.3:0/661426193 shutdown_connections 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af0c480 0x56258af57740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af57200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 --2- 172.21.15.3:0/661426193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26420 0x56258af56c40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 -- 172.21.15.3:0/661426193 >> 172.21.15.3:0/661426193 conn(0x56258af0d220 msgr2=0x56258af55840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 -- 172.21.15.3:0/661426193 shutdown_connections 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.505+0000 7f2221209740 1 -- 172.21.15.3:0/661426193 wait complete. 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.506+0000 7f2221209740 1 -- start start 2024-09-10T07:52:35.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.506+0000 7f2221209740 1 Processor -- start 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26420 0x56258af505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af694b0 0x56258af3e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56258af0b3e0 con 0x56258af26800 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56258adfd4c0 con 0x56258af694b0 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56258af0b6c0 con 0x56258af26420 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f2221209740 10 client.0 _ll_register_callbacks cb 0x56258ae6ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.507+0000 7f220fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44664/0 (socket says 172.21.15.3:44664) 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 -- 172.21.15.3:0/244697197 learned_addr learned my addr 172.21.15.3:0/244697197 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f2217e00640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af694b0 0x56258af3e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f2217400640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26420 0x56258af505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 -- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26420 msgr2=0x56258af505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26420 0x56258af505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 -- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af694b0 msgr2=0x56258af3e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af694b0 0x56258af3e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 -- 172.21.15.3:0/244697197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258ab873f0 con 0x56258af26800 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f220fe00640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2200035040 tx=0x7f2200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc1cfb4b35cd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.508+0000 7f21fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037040 con 0x56258af26800 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f220003ea30 con 0x56258af26800 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22080567f0 con 0x56258af26800 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2200056050 con 0x56258af26800 2024-09-10T07:52:35.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 10 client.7647 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x56258ab470f0 con 0x56258af26800 2024-09-10T07:52:35.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22080567f0 con 0x56258af26800 2024-09-10T07:52:35.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f2200059020 con 0x56258af26800 2024-09-10T07:52:35.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 10 client.7647 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 10 client.7647 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 20 client.7647 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af716e0 0x56258af73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 4 client.7647 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.509+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258ae81d90 con 0x56258af716e0 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.510+0000 7f2217400640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af716e0 0x56258af73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.510+0000 7f2217400640 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af716e0 0x56258af73ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc53288d3dcd93e4 server_cookie=6bbf279bf687fddc in_seq=0 out_seq=0 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.510+0000 7f21fbe00640 10 client.7647 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.511+0000 7f21fbe00640 1 -- 172.21.15.3:0/244697197 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56258ae81d90 con 0x56258af716e0 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.511+0000 7f21fbe00640 10 client.7647 handle_command_reply: tid=0 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.511+0000 7f2221209740 1 client.7647 shutdown 2024-09-10T07:52:35.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 msgr2=0x56258af3e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2200035040 tx=0x7f2200004730 comp rx=0 tx=0).stop 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 shutdown_connections 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af716e0 0x56258af73ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af694b0 0x56258af3e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af3e0b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 --2- 172.21.15.3:0/244697197 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26420 0x56258af505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f21fbe00640 0 client.7647 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 >> 172.21.15.3:0/244697197 conn(0x56258af0c480 msgr2=0x56258af107d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 shutdown_connections 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 1 -- 172.21.15.3:0/244697197 wait complete. 2024-09-10T07:52:35.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.512+0000 7f2221209740 20 client.7647 trim_cache size 0 max 16384 2024-09-10T07:52:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.530+0000 7f2221209740 1 Processor -- start 2024-09-10T07:52:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 -- start start 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258af50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af694b0 0x56258af64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22182009d0 con 0x56258af26800 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f22181ffc50 con 0x56258af26420 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.531+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f22182002a0 con 0x56258af694b0 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44672/0 (socket says 172.21.15.3:44672) 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 -- 172.21.15.3:0/3963770745 learned_addr learned my addr 172.21.15.3:0/3963770745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f2217400640 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258af50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 -- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af694b0 msgr2=0x56258af64410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af694b0 0x56258af64410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 -- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 msgr2=0x56258af50060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258af50060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 -- 172.21.15.3:0/3963770745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258af3b270 con 0x56258af26800 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f2217400640 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258af50060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f220fe00640 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2200032fd0 tx=0x7f2200041070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d302ad37dd6d231f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f2206a00640 1 -- 172.21.15.3:0/3963770745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037070 con 0x56258af26800 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f2206a00640 1 -- 172.21.15.3:0/3963770745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220004e070 con 0x56258af26800 2024-09-10T07:52:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.532+0000 7f2206a00640 1 -- 172.21.15.3:0/3963770745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220003c4b0 con 0x56258af26800 2024-09-10T07:52:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 -- 172.21.15.3:0/3963770745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 msgr2=0x56258af63ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2200032fd0 tx=0x7f2200041070 comp rx=0 tx=0).stop 2024-09-10T07:52:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 -- 172.21.15.3:0/3963770745 shutdown_connections 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af694b0 0x56258af64410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af26800 0x56258af63ed0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 --2- 172.21.15.3:0/3963770745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258af50060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 -- 172.21.15.3:0/3963770745 >> 172.21.15.3:0/3963770745 conn(0x56258af0c480 msgr2=0x56258af068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 -- 172.21.15.3:0/3963770745 shutdown_connections 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.533+0000 7f2221209740 1 -- 172.21.15.3:0/3963770745 wait complete. 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.534+0000 7f2221209740 1 -- start start 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.534+0000 7f2221209740 1 Processor -- start 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258afd3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af53e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22180af2f0 con 0x56258af694b0 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56258af6f690 con 0x56258af26420 2024-09-10T07:52:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56258af3da30 con 0x56258af26800 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2221209740 10 client.0 _ll_register_callbacks cb 0x7f2200035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44684/0 (socket says 172.21.15.3:44684) 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 -- 172.21.15.3:0/1799869004 learned_addr learned my addr 172.21.15.3:0/1799869004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217400640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258afd3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f220fe00640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af53e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 -- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 msgr2=0x56258af53e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af53e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.535+0000 7f2217e00640 1 -- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 msgr2=0x56258afd3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2217e00640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258afd3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2217e00640 1 -- 172.21.15.3:0/1799869004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258af3e800 con 0x56258af694b0 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f220fe00640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af53e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2217e00640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f22100058a0 tx=0x7f2210001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd6e9d8d9cd7a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f21fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221001a020 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f2210016660 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220004e070 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2210024050 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2221209740 10 client.7653 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.536+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x56258af6f3d0 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220004e070 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f2210027020 con 0x56258af694b0 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 10 client.7653 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 10 client.7653 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:35.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 20 client.7653 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af50640 0x56258af4ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 4 client.7653 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:35.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56258ae7c0e0 con 0x56258af50640 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.537+0000 7f2217400640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af50640 0x56258af4ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2217400640 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af50640 0x56258af4ed40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e8e051781a8e049 server_cookie=575e8dd923ecad97 in_seq=0 out_seq=0 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f21fbe00640 10 client.7653 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f21fbe00640 1 -- 172.21.15.3:0/1799869004 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56258ae7c0e0 con 0x56258af50640 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f21fbe00640 10 client.7653 handle_command_reply: tid=0 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 client.7653 shutdown 2024-09-10T07:52:35.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 msgr2=0x56258af50060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f22100058a0 tx=0x7f2210001a20 comp rx=0 tx=0).stop 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 shutdown_connections 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56258af50640 0x56258af4ed40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258af694b0 0x56258af50060 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56258af26800 0x56258af53e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 --2- 172.21.15.3:0/1799869004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56258af26420 0x56258afd3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f21fbe00640 0 client.7653 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 >> 172.21.15.3:0/1799869004 conn(0x56258af0c480 msgr2=0x56258af54820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.538+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 shutdown_connections 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f2221209740 1 -- 172.21.15.3:0/1799869004 wait complete. 2024-09-10T07:52:35.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f2221209740 20 client.7653 trim_cache size 0 max 16384 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.543 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:35.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 msgr2=0x7f21f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 0x7f21f406e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2200034000 tx=0x7f220002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 msgr2=0x7f22181f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f220800c4d0 tx=0x7f220800c990 comp rx=0 tx=0).stop 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 shutdown_connections 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f21f406be60 0x7f21f406e320 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2218145730 0x7f22180c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f22181451d0 0x7f22180addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 --2- 172.21.15.3:0/953160262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22180cf270 0x7f22181f71c0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 >> 172.21.15.3:0/953160262 conn(0x7f22180b5cb0 msgr2=0x7f22180cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.539+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 shutdown_connections 2024-09-10T07:52:35.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.540+0000 7f221e200640 1 -- 172.21.15.3:0/953160262 wait complete. 2024-09-10T07:52:35.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:35.554 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:35.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T07:52:35.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.715+0000 7f5504800640 1 Processor -- start 2024-09-10T07:52:35.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.715+0000 7f5504800640 1 -- start start 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5504800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54fc06ccf0 0x7f54fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5504800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 0x7f54fc06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5504800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc06bc50 con 0x7f54fc15e630 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5504800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f54fc06bf30 con 0x7f54fc06ccf0 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5504800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc06c210 con 0x7f54fc070a90 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.716+0000 7f5501400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54fc06ccf0 0x7f54fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f54fc06ccf0 0x7f54fc06d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44694/0 (socket says 172.21.15.3:44694) 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 -- 172.21.15.3:0/1022407964 learned_addr learned my addr 172.21.15.3:0/1022407964 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5502800640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 0x7f54fc06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 -- 172.21.15.3:0/1022407964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 msgr2=0x7f54fc06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 0x7f54fc06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 -- 172.21.15.3:0/1022407964 >> v1:172.21.15.70:6789/0 conn(0x7f54fc15e630 legacy=0x7f54fc071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5501400640 1 -- 172.21.15.3:0/1022407964 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc06c4f0 con 0x7f54fc06ccf0 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.717+0000 7f5502800640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 0x7f54fc06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.718+0000 7f5501400640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc06ccf0 0x7f54fc06d190 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009870 tx=0x7f54ec02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8856ac47c36245a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.718+0000 7f5500a00640 1 -- 172.21.15.3:0/1022407964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec037070 con 0x7f54fc06ccf0 2024-09-10T07:52:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.718+0000 7f5500a00640 1 -- 172.21.15.3:0/1022407964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec032440 con 0x7f54fc06ccf0 2024-09-10T07:52:35.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.718+0000 7f5500a00640 1 -- 172.21.15.3:0/1022407964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec032810 con 0x7f54fc06ccf0 2024-09-10T07:52:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 -- 172.21.15.3:0/1022407964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc06ccf0 msgr2=0x7f54fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc06ccf0 0x7f54fc06d190 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009870 tx=0x7f54ec02a130 comp rx=0 tx=0).stop 2024-09-10T07:52:35.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 -- 172.21.15.3:0/1022407964 shutdown_connections 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc070a90 0x7f54fc06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 --2- 172.21.15.3:0/1022407964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc06ccf0 0x7f54fc06d190 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 -- 172.21.15.3:0/1022407964 >> 172.21.15.3:0/1022407964 conn(0x7f54fc06f180 msgr2=0x7f54fc06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 -- 172.21.15.3:0/1022407964 shutdown_connections 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.719+0000 7f5504800640 1 -- 172.21.15.3:0/1022407964 wait complete. 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.720+0000 7f5504800640 1 Processor -- start 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.721+0000 7f5504800640 1 -- start start 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.721+0000 7f5504800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 0x7f54fc0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.721+0000 7f5504800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.721+0000 7f5504800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54fc15e630 0x7f54fc15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5504800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc15eb90 con 0x7f54fc070a90 2024-09-10T07:52:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5504800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc15ea10 con 0x7f54fc15e630 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5504800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc15ed10 con 0x7f54fc06ccf0 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44702/0 (socket says 172.21.15.3:44702) 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 -- 172.21.15.3:0/590474944 learned_addr learned my addr 172.21.15.3:0/590474944 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501e00640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 0x7f54fc0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5502800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54fc15e630 0x7f54fc15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 -- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 msgr2=0x7f54fc0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 0x7f54fc0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 -- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54fc15e630 msgr2=0x7f54fc15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54fc15e630 0x7f54fc15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501400640 1 -- 172.21.15.3:0/590474944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc158060 con 0x7f54fc070a90 2024-09-10T07:52:35.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.722+0000 7f5501e00640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 0x7f54fc0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f5501400640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f54ec02a170 tx=0x7f54ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f54f6000640 1 -- 172.21.15.3:0/590474944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec037040 con 0x7f54fc070a90 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f54f6000640 1 -- 172.21.15.3:0/590474944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec042070 con 0x7f54fc070a90 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc158340 con 0x7f54fc070a90 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f54f6000640 1 -- 172.21.15.3:0/590474944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec030980 con 0x7f54fc070a90 2024-09-10T07:52:35.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.723+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc158900 con 0x7f54fc070a90 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.725+0000 7f54f6000640 1 -- 172.21.15.3:0/590474944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54ec030b20 con 0x7f54fc070a90 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.726+0000 7f54f6000640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 0x7f54e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.726+0000 7f54f6000640 1 -- 172.21.15.3:0/590474944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f54ec0ab7d0 con 0x7f54fc070a90 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.726+0000 7f5501e00640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 0x7f54e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.727+0000 7f55075d2740 1 Processor -- start 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.727+0000 7f5501e00640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 0x7f54e006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f54f0001660 tx=0x7f54f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.727+0000 7f55075d2740 1 -- start start 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 0x55d7ea8dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea7f9660 con 0x55d7ea8ab420 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea7a1680 con 0x55d7ea891480 2024-09-10T07:52:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d7ea7f5620 con 0x55d7ea8ab800 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44716/0 (socket says 172.21.15.3:44716) 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 -- 172.21.15.3:0/222435002 learned_addr learned my addr 172.21.15.3:0/222435002 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 -- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 msgr2=0x55d7ea8dc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5502800640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 0x55d7ea8dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501400640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8dc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 -- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 msgr2=0x55d7ea8dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 0x55d7ea8dc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.728+0000 7f5501e00640 1 -- 172.21.15.3:0/222435002 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ea5d67d0 con 0x55d7ea8ab420 2024-09-10T07:52:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f5502800640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 0x55d7ea8dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f5501400640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8dc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f5501e00640 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f54f0007fa0 tx=0x7f54f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c6774d886bc9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f54ea600640 1 -- 172.21.15.3:0/222435002 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00035f0 con 0x55d7ea8ab420 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f54ea600640 1 -- 172.21.15.3:0/222435002 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0003790 con 0x55d7ea8ab420 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.729+0000 7f54ea600640 1 -- 172.21.15.3:0/222435002 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0003b80 con 0x55d7ea8ab420 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- 172.21.15.3:0/222435002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 msgr2=0x55d7ea8dbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f54f0007fa0 tx=0x7f54f0011000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- 172.21.15.3:0/222435002 shutdown_connections 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea891480 0x55d7ea8dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8dc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 --2- 172.21.15.3:0/222435002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8dbc40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- 172.21.15.3:0/222435002 >> 172.21.15.3:0/222435002 conn(0x55d7ea892220 msgr2=0x55d7ea8da840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- 172.21.15.3:0/222435002 shutdown_connections 2024-09-10T07:52:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- 172.21.15.3:0/222435002 wait complete. 2024-09-10T07:52:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.730+0000 7f55075d2740 1 -- start start 2024-09-10T07:52:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.731+0000 7f55075d2740 1 Processor -- start 2024-09-10T07:52:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab800 0x55d7ea8d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea8987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea8903e0 con 0x55d7ea8d5370 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea7824c0 con 0x55d7ea8ab800 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d7ea8906c0 con 0x55d7ea8ab420 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f55075d2740 10 client.0 _ll_register_callbacks cb 0x55d7ea7f1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5502800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea8987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:42260/0 (socket says 172.21.15.3:42260) 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 -- 172.21.15.3:0/1761003012 learned_addr learned my addr 172.21.15.3:0/1761003012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501400640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab800 0x55d7ea8d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 -- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab800 msgr2=0x55d7ea8d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab800 0x55d7ea8d6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 -- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 msgr2=0x55d7ea8987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea8987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5501e00640 1 -- 172.21.15.3:0/1761003012 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ea50c3f0 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.732+0000 7f5502800640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea8987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f5501e00640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e530 tx=0x7f54f0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed94a0253aecff87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f54dea00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0021650 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f0003790 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f54f00217f0 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f54f0032050 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f55075d2740 10 client.6602 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.733+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55d7ea4cc0f0 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0003790 con 0x55d7ea8ab420 2024-09-10T07:52:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f54f0035020 con 0x55d7ea8ab420 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 10 client.6602 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 10 client.6602 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 20 client.6602 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea899bd0 0x55d7ea89a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 4 client.6602 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ea806d90 con 0x55d7ea899bd0 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.734+0000 7f5501400640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea899bd0 0x55d7ea89a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.735+0000 7f5501400640 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea899bd0 0x55d7ea89a000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1d9d4f8811f5a1 server_cookie=9a4b8b904243b315 in_seq=0 out_seq=0 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.735+0000 7f54dea00640 10 client.6602 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.735+0000 7f54dea00640 1 -- 172.21.15.3:0/1761003012 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7ea806d90 con 0x55d7ea899bd0 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.735+0000 7f54dea00640 10 client.6602 handle_command_reply: tid=0 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.735+0000 7f55075d2740 1 client.6602 shutdown 2024-09-10T07:52:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 msgr2=0x55d7ea8aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e530 tx=0x7f54f0013000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 shutdown_connections 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea899bd0 0x55d7ea89a000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea8987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab800 0x55d7ea8d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f54dea00640 0 client.6602 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 --2- 172.21.15.3:0/1761003012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab420 0x55d7ea8aa6c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 >> 172.21.15.3:0/1761003012 conn(0x55d7ea891480 msgr2=0x55d7ea88ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 shutdown_connections 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 1 -- 172.21.15.3:0/1761003012 wait complete. 2024-09-10T07:52:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.736+0000 7f55075d2740 20 client.6602 trim_cache size 0 max 16384 2024-09-10T07:52:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.754+0000 7f55075d2740 1 Processor -- start 2024-09-10T07:52:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.754+0000 7f55075d2740 1 -- start start 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8d5370 0x55d7ea8dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc200d30 con 0x55d7ea8ab420 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1fff20 con 0x55d7ea8d5370 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc200600 con 0x55d7ea8ab800 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f5501e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f5501e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44732/0 (socket says 172.21.15.3:44732) 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.755+0000 7f5501e00640 1 -- 172.21.15.3:0/3725995875 learned_addr learned my addr 172.21.15.3:0/3725995875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 -- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 msgr2=0x55d7ea8d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 -- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8d5370 msgr2=0x55d7ea8dc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8d5370 0x55d7ea8dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 -- 172.21.15.3:0/3725995875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ea8dcf00 con 0x55d7ea8ab420 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f5501e00640 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f54f000ed20 tx=0x7f54f001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ff7214153a8e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f54e9c00640 1 -- 172.21.15.3:0/3725995875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0021450 con 0x55d7ea8ab420 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f54e9c00640 1 -- 172.21.15.3:0/3725995875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f00215f0 con 0x55d7ea8ab420 2024-09-10T07:52:35.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.756+0000 7f54e9c00640 1 -- 172.21.15.3:0/3725995875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f002b430 con 0x55d7ea8ab420 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- 172.21.15.3:0/3725995875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 msgr2=0x55d7ea8fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f54f000ed20 tx=0x7f54f001d000 comp rx=0 tx=0).stop 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- 172.21.15.3:0/3725995875 shutdown_connections 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8d5370 0x55d7ea8dc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 --2- 172.21.15.3:0/3725995875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8ab420 0x55d7ea8fd620 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- 172.21.15.3:0/3725995875 >> 172.21.15.3:0/3725995875 conn(0x55d7ea891480 msgr2=0x55d7ea8d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- 172.21.15.3:0/3725995875 shutdown_connections 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- 172.21.15.3:0/3725995875 wait complete. 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.757+0000 7f55075d2740 1 -- start start 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.758+0000 7f55075d2740 1 Processor -- start 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 0x55d7ea8f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc160350 con 0x55d7ea8d5370 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea8a9d30 con 0x55d7ea8ab420 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d7ea8d8ec0 con 0x55d7ea8ab800 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f55075d2740 10 client.0 _ll_register_callbacks cb 0x7f54f0001d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44736/0 (socket says 172.21.15.3:44736) 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 -- 172.21.15.3:0/2824115388 learned_addr learned my addr 172.21.15.3:0/2824115388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5501e00640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 0x55d7ea8f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5501400640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 -- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 msgr2=0x55d7ea8fd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8fd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 -- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 msgr2=0x55d7ea8f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 0x55d7ea8f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5502800640 1 -- 172.21.15.3:0/2824115388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ea9588d0 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.759+0000 7f5501e00640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 0x55d7ea8f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f5502800640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f54f8004770 tx=0x7f54f8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22002f47ec74fca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f801b070 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f54f8017440 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f00215f0 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f54f8026050 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 10 client.7671 fetch_fsmap learned FSMap version 92 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55d7ea8d8cc0 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f00215f0 con 0x55d7ea8d5370 2024-09-10T07:52:35.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 92) ==== 3554+0+0 (secure 0 0 0) 0x7f54f8029020 con 0x55d7ea8d5370 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 10 client.7671 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 10 client.7671 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 20 client.7671 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.760+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea88f690 0x55d7ea898560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f55075d2740 4 client.7671 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7ea8010e0 con 0x55d7ea88f690 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f5501e00640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea88f690 0x55d7ea898560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f5501e00640 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea88f690 0x55d7ea898560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f441a4d83478eb server_cookie=8732d17c15e55811 in_seq=0 out_seq=0 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f54dea00640 10 client.7671 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f54dea00640 1 -- 172.21.15.3:0/2824115388 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d7ea8010e0 con 0x55d7ea88f690 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f54dea00640 10 client.7671 handle_command_reply: tid=0 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.761+0000 7f55075d2740 1 client.7671 shutdown 2024-09-10T07:52:35.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 msgr2=0x55d7ea88f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f54f8004770 tx=0x7f54f8007700 comp rx=0 tx=0).stop 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 shutdown_connections 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55d7ea88f690 0x55d7ea898560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d7ea8d5370 0x55d7ea88f150 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d7ea8ab800 0x55d7ea8fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 --2- 172.21.15.3:0/2824115388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d7ea8ab420 0x55d7ea8f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f54dea00640 0 client.7671 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 >> 172.21.15.3:0/2824115388 conn(0x55d7ea891480 msgr2=0x55d7ea89b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 shutdown_connections 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 1 -- 172.21.15.3:0/2824115388 wait complete. 2024-09-10T07:52:35.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.762+0000 7f55075d2740 20 client.7671 trim_cache size 0 max 16384 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:35.767 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 msgr2=0x7f54e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 0x7f54e006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f54f0001660 tx=0x7f54f0009040 comp rx=0 tx=0).stop 2024-09-10T07:52:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 msgr2=0x7f54fc1576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f54ec02a170 tx=0x7f54ec0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 shutdown_connections 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f54e006be30 0x7f54e006e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54fc15e630 0x7f54fc15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc070a90 0x7f54fc1576f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 --2- 172.21.15.3:0/590474944 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54fc06ccf0 0x7f54fc0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 >> 172.21.15.3:0/590474944 conn(0x7f54fc06f180 msgr2=0x7f54fc0b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 shutdown_connections 2024-09-10T07:52:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.763+0000 7f5504800640 1 -- 172.21.15.3:0/590474944 wait complete. 2024-09-10T07:52:35.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:35.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.935+0000 7fc51be00640 1 Processor -- start 2024-09-10T07:52:35.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.935+0000 7fc51be00640 1 -- start start 2024-09-10T07:52:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.936+0000 7fc51be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc51415b110 0x7fc5140b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.936+0000 7fc51be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc51415b4f0 0x7fc5140d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.936+0000 7fc51be00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5140b4250 con 0x7fc514073d10 2024-09-10T07:52:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.936+0000 7fc51be00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc5140b44f0 con 0x7fc51415b110 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.936+0000 7fc51be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc5140ccd80 con 0x7fc51415b4f0 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc51415b110 0x7fc5140b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc51415b110 0x7fc5140b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44742/0 (socket says 172.21.15.3:44742) 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 -- 172.21.15.3:0/1943852769 learned_addr learned my addr 172.21.15.3:0/1943852769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc518a00640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc51415b4f0 0x7fc5140d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 -- 172.21.15.3:0/1943852769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc51415b4f0 msgr2=0x7fc5140d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc51415b4f0 0x7fc5140d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 -- 172.21.15.3:0/1943852769 >> v1:172.21.15.70:6789/0 conn(0x7fc514073d10 legacy=0x7fc51415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.937+0000 7fc519400640 1 -- 172.21.15.3:0/1943852769 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5140cd060 con 0x7fc51415b110 2024-09-10T07:52:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.938+0000 7fc519400640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b110 0x7fc5140b3d10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc508009870 tx=0x7fc50802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7697665dc15e958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc513e00640 1 -- 172.21.15.3:0/1943852769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508037070 con 0x7fc51415b110 2024-09-10T07:52:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc513e00640 1 -- 172.21.15.3:0/1943852769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc508032440 con 0x7fc51415b110 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc513e00640 1 -- 172.21.15.3:0/1943852769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508032810 con 0x7fc51415b110 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 -- 172.21.15.3:0/1943852769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b110 msgr2=0x7fc5140b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b110 0x7fc5140b3d10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc508009870 tx=0x7fc50802a270 comp rx=0 tx=0).stop 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 -- 172.21.15.3:0/1943852769 shutdown_connections 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc51415b4f0 0x7fc5140d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 --2- 172.21.15.3:0/1943852769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b110 0x7fc5140b3d10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 -- 172.21.15.3:0/1943852769 >> 172.21.15.3:0/1943852769 conn(0x7fc51406d000 msgr2=0x7fc51406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 -- 172.21.15.3:0/1943852769 shutdown_connections 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.939+0000 7fc51be00640 1 -- 172.21.15.3:0/1943852769 wait complete. 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.941+0000 7fc51be00640 1 Processor -- start 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.941+0000 7fc51be00640 1 -- start start 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc514073d10 0x7fc5141f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 0x7fc51415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc5140ceb50 con 0x7fc51415b4f0 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc5140cd620 con 0x7fc51415b110 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc51be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc5140cecd0 con 0x7fc514073d10 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc519e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc519e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44744/0 (socket says 172.21.15.3:44744) 2024-09-10T07:52:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc519e00640 1 -- 172.21.15.3:0/2666053680 learned_addr learned my addr 172.21.15.3:0/2666053680 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc519400640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc514073d10 0x7fc5141f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.942+0000 7fc518a00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 0x7fc51415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 -- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc514073d10 msgr2=0x7fc5141f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc514073d10 0x7fc5141f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 -- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 msgr2=0x7fc51415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 0x7fc51415ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 -- 172.21.15.3:0/2666053680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51415d190 con 0x7fc51415b4f0 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc518a00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 0x7fc51415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc519e00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c730 tx=0x7fc50c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003b90 con 0x7fc51415b4f0 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c003d30 con 0x7fc51415b4f0 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00aa30 con 0x7fc51415b4f0 2024-09-10T07:52:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51415d470 con 0x7fc51415b4f0 2024-09-10T07:52:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.943+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141f3ec0 con 0x7fc51415b4f0 2024-09-10T07:52:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.945+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5140b3d10 con 0x7fc51415b4f0 2024-09-10T07:52:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.945+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc50c00abd0 con 0x7fc51415b4f0 2024-09-10T07:52:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.946+0000 7fc511600640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 0x7fc4f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.946+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc50c08ad50 con 0x7fc51415b4f0 2024-09-10T07:52:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.947+0000 7fc519400640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 0x7fc4f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.947+0000 7fc519400640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 0x7fc4f406e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc508034000 tx=0x7fc50802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:35.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:35.951+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc50c059660 con 0x7fc51415b4f0 2024-09-10T07:52:36.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.171+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc514152500 con 0x7fc51415b4f0 2024-09-10T07:52:36.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.172+0000 7fc511600640 1 -- 172.21.15.3:0/2666053680 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5258 (secure 0 0 0) 0x7fc50c058f60 con 0x7fc51415b4f0 2024-09-10T07:52:36.172 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:36.172 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"btime":"2024-09-10T07:52:31:640554+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:31.513935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:36.172 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 92 2024-09-10T07:52:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.174+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 msgr2=0x7fc4f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.174+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 0x7fc4f406e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc508034000 tx=0x7fc50802e000 comp rx=0 tx=0).stop 2024-09-10T07:52:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.174+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 msgr2=0x7fc514151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.174+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c730 tx=0x7fc50c00cbf0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 shutdown_connections 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4f406be30 0x7fc4f406e2f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc51415b4f0 0x7fc514151b30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc51415b110 0x7fc51415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 --2- 172.21.15.3:0/2666053680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc514073d10 0x7fc5141f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 >> 172.21.15.3:0/2666053680 conn(0x7fc51406d000 msgr2=0x7fc5140af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 shutdown_connections 2024-09-10T07:52:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.175+0000 7fc51be00640 1 -- 172.21.15.3:0/2666053680 wait complete. 2024-09-10T07:52:36.190 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:36.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:36.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.345+0000 7f4a24600640 1 Processor -- start 2024-09-10T07:52:36.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.345+0000 7f4a24600640 1 -- start start 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.346+0000 7f4a24600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a1c15b410 0x7f4a1c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a24600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a1c0cfac0 0x7f4a1c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a24600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1c0ccd40 con 0x7f4a1c0b4aa0 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a24600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c06f250 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a24600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c0cd020 con 0x7f4a1c15b410 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a22600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a1c0cfac0 0x7f4a1c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a21200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4a1c0b4aa0 0x7f4a1c0cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51786/0 (socket says 172.21.15.3:51786) 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a21200640 1 -- 172.21.15.3:0/4280059455 learned_addr learned my addr 172.21.15.3:0/4280059455 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a21c00640 1 --2- 172.21.15.3:0/4280059455 >> v2:172.21.15.70:3300/0 conn(0x7f4a1c15b410 0x7f4a1c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a22600640 1 -- 172.21.15.3:0/4280059455 >> v2:172.21.15.70:3300/0 conn(0x7f4a1c15b410 msgr2=0x7f4a1c06ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:36.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a22600640 1 --2- 172.21.15.3:0/4280059455 >> v2:172.21.15.70:3300/0 conn(0x7f4a1c15b410 0x7f4a1c06ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a22600640 1 -- 172.21.15.3:0/4280059455 >> v1:172.21.15.3:6789/0 conn(0x7f4a1c0b4aa0 legacy=0x7f4a1c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a1be00640 1 -- 172.21.15.3:0/4280059455 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696964622 0 0) 0x7f4a1c0ccd40 con 0x7f4a1c0b4aa0 2024-09-10T07:52:36.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.347+0000 7f4a22600640 1 -- 172.21.15.3:0/4280059455 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0cd300 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.348+0000 7f4a22600640 1 --2- 172.21.15.3:0/4280059455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c0b3690 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06f440 tx=0x7f4a1402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66bdf78ee8d52e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a1be00640 1 -- 172.21.15.3:0/4280059455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14007740 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a1be00640 1 -- 172.21.15.3:0/4280059455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a140078e0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a1be00640 1 -- 172.21.15.3:0/4280059455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14007cb0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 -- 172.21.15.3:0/4280059455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 msgr2=0x7f4a1c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 --2- 172.21.15.3:0/4280059455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c0b3690 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06f440 tx=0x7f4a1402a680 comp rx=0 tx=0).stop 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 -- 172.21.15.3:0/4280059455 shutdown_connections 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 --2- 172.21.15.3:0/4280059455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c0b3690 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 --2- 172.21.15.3:0/4280059455 >> v2:172.21.15.70:3300/0 conn(0x7f4a1c15b410 0x7f4a1c06ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 -- 172.21.15.3:0/4280059455 >> 172.21.15.3:0/4280059455 conn(0x7f4a1c06cfc0 msgr2=0x7f4a1c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:36.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 -- 172.21.15.3:0/4280059455 shutdown_connections 2024-09-10T07:52:36.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.349+0000 7f4a24600640 1 -- 172.21.15.3:0/4280059455 wait complete. 2024-09-10T07:52:36.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.351+0000 7f4a24600640 1 Processor -- start 2024-09-10T07:52:36.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.351+0000 7f4a24600640 1 -- start start 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a1c0b4aa0 0x7f4a1c0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a1c15b410 0x7f4a1c1e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c0cdba0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c0cd8c0 con 0x7f4a1c15b410 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a24600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c072540 con 0x7f4a1c0b4aa0 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a21200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a21200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44762/0 (socket says 172.21.15.3:44762) 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a21200640 1 -- 172.21.15.3:0/1045292032 learned_addr learned my addr 172.21.15.3:0/1045292032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:36.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a21c00640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a1c0b4aa0 0x7f4a1c0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.352+0000 7f4a22600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a1c15b410 0x7f4a1c1e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 -- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a1c0b4aa0 msgr2=0x7f4a1c0b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a1c0b4aa0 0x7f4a1c0b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 -- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a1c15b410 msgr2=0x7f4a1c1e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a1c15b410 0x7f4a1c1e60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 -- 172.21.15.3:0/1045292032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0b1450 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a21200640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c410 tx=0x7f4a0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080074e0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a080047e0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.353+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08004bd0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.354+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c0afe70 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.354+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c0b0430 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.355+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c15b7f0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.356+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a08011070 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.356+0000 7f4a19600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 0x7f49fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.357+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f4a0808a990 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.357+0000 7f4a21c00640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 0x7f49fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:36.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.358+0000 7f4a21c00640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 0x7f49fc06e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009b40 tx=0x7f4a10009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:36.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.361+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a08058ce0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.616+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4a1c0b2a90 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.617+0000 7f4a19600640 1 -- 172.21.15.3:0/1045292032 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1361 (secure 0 0 0) 0x7f4a0805cfe0 con 0x7f4a1c0cfac0 2024-09-10T07:52:36.616 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:36.616 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":390,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":668443,"bytes_used":269938688,"bytes_avail":772824174592,"bytes_total":773094113280,"write_bytes_sec":5033,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":92,"btime":"2024-09-10T07:52:31:640554+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-09-10T07:51:49.561868+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 msgr2=0x7f49fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 0x7f49fc06e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009b40 tx=0x7f4a10009190 comp rx=0 tx=0).stop 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 msgr2=0x7f4a1c1e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c410 tx=0x7f4a0800c8d0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 shutdown_connections 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f49fc06be30 0x7f49fc06e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a1c15b410 0x7f4a1c1e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a1c0cfac0 0x7f4a1c1e5ba0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 --2- 172.21.15.3:0/1045292032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4a1c0b4aa0 0x7f4a1c0b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 >> 172.21.15.3:0/1045292032 conn(0x7f4a1c06cfc0 msgr2=0x7f4a1c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 shutdown_connections 2024-09-10T07:52:36.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:36.619+0000 7f4a24600640 1 -- 172.21.15.3:0/1045292032 wait complete. 2024-09-10T07:52:36.634 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-10T07:52:37.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:37.666 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:52:37.666+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:52:37.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.805+0000 7fde2a200640 1 Processor -- start 2024-09-10T07:52:37.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.805+0000 7fde2a200640 1 -- start start 2024-09-10T07:52:37.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde2a200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde240d3c10 0x7fde24073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde2a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 0x7fde24073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde2a200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde240cfdc0 con 0x7fde2415e860 2024-09-10T07:52:37.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde2a200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fde240d3780 con 0x7fde240d3c10 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde2a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fde240cee40 con 0x7fde240d4050 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde23e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde240d3c10 0x7fde24073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde23e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fde240d3c10 0x7fde24073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44766/0 (socket says 172.21.15.3:44766) 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.806+0000 7fde23e00640 1 -- 172.21.15.3:0/1221257563 learned_addr learned my addr 172.21.15.3:0/1221257563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde29800640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 0x7fde24073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde23e00640 1 -- 172.21.15.3:0/1221257563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 msgr2=0x7fde24073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde23e00640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 0x7fde24073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde23e00640 1 -- 172.21.15.3:0/1221257563 >> v1:172.21.15.70:6789/0 conn(0x7fde2415e860 legacy=0x7fde240cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde23e00640 1 -- 172.21.15.3:0/1221257563 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde240cf120 con 0x7fde240d3c10 2024-09-10T07:52:37.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.807+0000 7fde29800640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 0x7fde24073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:37.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.808+0000 7fde23e00640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d3c10 0x7fde24073150 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fde10009870 tx=0x7fde1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104c6c86d1a0c665 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:37.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.808+0000 7fde23400640 1 -- 172.21.15.3:0/1221257563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10037070 con 0x7fde240d3c10 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.808+0000 7fde23400640 1 -- 172.21.15.3:0/1221257563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde10032440 con 0x7fde240d3c10 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.808+0000 7fde23400640 1 -- 172.21.15.3:0/1221257563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10032810 con 0x7fde240d3c10 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 -- 172.21.15.3:0/1221257563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d3c10 msgr2=0x7fde24073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d3c10 0x7fde24073150 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fde10009870 tx=0x7fde1002a270 comp rx=0 tx=0).stop 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 -- 172.21.15.3:0/1221257563 shutdown_connections 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde240d4050 0x7fde24073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 --2- 172.21.15.3:0/1221257563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d3c10 0x7fde24073150 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 -- 172.21.15.3:0/1221257563 >> 172.21.15.3:0/1221257563 conn(0x7fde240d4650 msgr2=0x7fde240d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 -- 172.21.15.3:0/1221257563 shutdown_connections 2024-09-10T07:52:37.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.809+0000 7fde2a200640 1 -- 172.21.15.3:0/1221257563 wait complete. 2024-09-10T07:52:37.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.810+0000 7fde2a200640 1 Processor -- start 2024-09-10T07:52:37.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 -- start start 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 0x7fde2415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2415e860 0x7fde241f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde240d2460 con 0x7fde240d4050 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde240cf400 con 0x7fde240d3c10 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.811+0000 7fde2a200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fde240b7880 con 0x7fde2415e860 2024-09-10T07:52:37.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44768/0 (socket says 172.21.15.3:44768) 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 -- 172.21.15.3:0/1707811242 learned_addr learned my addr 172.21.15.3:0/1707811242 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde28e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 0x7fde2415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde29800640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2415e860 0x7fde241f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 -- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2415e860 msgr2=0x7fde241f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2415e860 0x7fde241f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 -- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 msgr2=0x7fde2415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 0x7fde2415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 -- 172.21.15.3:0/1707811242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde240ac440 con 0x7fde240d4050 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde28e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 0x7fde2415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:37.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.812+0000 7fde23e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fde10000c00 tx=0x7fde100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:37.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.813+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10037040 con 0x7fde240d4050 2024-09-10T07:52:37.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.813+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde100304c0 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.813+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100308b0 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.813+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde240ac720 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.813+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde240acce0 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.814+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde240cf6b0 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.815+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde10044020 con 0x7fde240d4050 2024-09-10T07:52:37.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.816+0000 7fde20c00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 0x7fde0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.816+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fde100aaec0 con 0x7fde240d4050 2024-09-10T07:52:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.816+0000 7fde28e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 0x7fde0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:37.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.817+0000 7fde28e00640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 0x7fde0406e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde18001660 tx=0x7fde18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:37.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:37.821+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fde1004b020 con 0x7fde240d4050 2024-09-10T07:52:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.044+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde240d4a30 con 0x7fde240d4050 2024-09-10T07:52:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.045+0000 7fde20c00640 1 -- 172.21.15.3:0/1707811242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5261 (secure 0 0 0) 0x7fde10079870 con 0x7fde240d4050 2024-09-10T07:52:38.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:38.045 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":94,"btime":"2024-09-10T07:52:37:667453+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:37.618905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 94 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 msgr2=0x7fde0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 0x7fde0406e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde18001660 tx=0x7fde18009040 comp rx=0 tx=0).stop 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 msgr2=0x7fde241f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fde10000c00 tx=0x7fde100046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 shutdown_connections 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fde0406be60 0x7fde0406e320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fde2415e860 0x7fde241f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde240d4050 0x7fde241f7d40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 --2- 172.21.15.3:0/1707811242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde240d3c10 0x7fde2415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 >> 172.21.15.3:0/1707811242 conn(0x7fde240d4650 msgr2=0x7fde240b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 shutdown_connections 2024-09-10T07:52:38.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.047+0000 7fde2a200640 1 -- 172.21.15.3:0/1707811242 wait complete. 2024-09-10T07:52:38.063 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:38.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.258+0000 7fdc76200640 1 Processor -- start 2024-09-10T07:52:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.258+0000 7fdc76200640 1 -- start start 2024-09-10T07:52:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.259+0000 7fdc76200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc7015f8e0 0x7fdc700b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc76200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7006e8d0 con 0x7fdc70148080 2024-09-10T07:52:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc76200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc700af210 con 0x7fdc700b4e20 2024-09-10T07:52:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc76200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdc700aefb0 con 0x7fdc7015f8e0 2024-09-10T07:52:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc70148080 0x7fdc700ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51794/0 (socket says 172.21.15.3:51794) 2024-09-10T07:52:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6f400640 1 -- 172.21.15.3:0/2353148927 learned_addr learned my addr 172.21.15.3:0/2353148927 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321512568 0 0) 0x7fdc7006e8d0 con 0x7fdc70148080 2024-09-10T07:52:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc67e00640 1 --2- 172.21.15.3:0/2353148927 >> v2:172.21.15.70:3300/0 conn(0x7fdc7015f8e0 0x7fdc700b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc54003520 con 0x7fdc70148080 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1836697139 0 0) 0x7fdc54003520 con 0x7fdc70148080 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 >> v2:172.21.15.70:3300/0 conn(0x7fdc7015f8e0 msgr2=0x7fdc700b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.260+0000 7fdc6ea00640 1 --2- 172.21.15.3:0/2353148927 >> v2:172.21.15.70:3300/0 conn(0x7fdc7015f8e0 0x7fdc700b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 >> v1:172.21.15.70:6789/0 conn(0x7fdc700b4e20 legacy=0x7fdc7006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc700af4f0 con 0x7fdc70148080 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fdc60001670 con 0x7fdc70148080 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 >> v1:172.21.15.3:6789/0 conn(0x7fdc70148080 legacy=0x7fdc700ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc54003a00 0x7fdc54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc700aefb0 con 0x7fdc54004800 2024-09-10T07:52:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5400c0c0 con 0x7fdc54003a00 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5400c2c0 con 0x7fdc540051c0 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6f400640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc54003a00 0x7fdc54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6fe00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 -- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc540051c0 msgr2=0x7fdc5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 -- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc54003a00 msgr2=0x7fdc54003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc54003a00 0x7fdc54003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc67e00640 1 -- 172.21.15.3:0/2353148927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc54003520 con 0x7fdc54004800 2024-09-10T07:52:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.261+0000 7fdc6fe00640 1 -- 172.21.15.3:0/2353148927 reap_dead start 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.262+0000 7fdc67e00640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009a90 tx=0x7fdc54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e63a1fd84b24f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58036070 con 0x7fdc54004800 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc58031400 con 0x7fdc54004800 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc6ea00640 1 -- 172.21.15.3:0/2353148927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58009690 con 0x7fdc54004800 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 -- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 msgr2=0x7fdc54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009a90 tx=0x7fdc54005830 comp rx=0 tx=0).stop 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 -- 172.21.15.3:0/2353148927 shutdown_connections 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 --2- 172.21.15.3:0/2353148927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 -- 172.21.15.3:0/2353148927 >> 172.21.15.3:0/2353148927 conn(0x7fdc700b24d0 msgr2=0x7fdc700b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 -- 172.21.15.3:0/2353148927 shutdown_connections 2024-09-10T07:52:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.263+0000 7fdc76200640 1 -- 172.21.15.3:0/2353148927 wait complete. 2024-09-10T07:52:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.265+0000 7fdc76200640 1 Processor -- start 2024-09-10T07:52:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.265+0000 7fdc76200640 1 -- start start 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc54004800 0x7fdc7013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 0x7fdc701f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7015fd70 con 0x7fdc70143110 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdc700b5250 con 0x7fdc7013ffb0 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc76200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdc700afab0 con 0x7fdc54004800 2024-09-10T07:52:38.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44796/0 (socket says 172.21.15.3:44796) 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.266+0000 7fdc6fe00640 1 -- 172.21.15.3:0/3432664438 learned_addr learned my addr 172.21.15.3:0/3432664438 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6f400640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc54004800 0x7fdc7013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc67e00640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 0x7fdc701f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 -- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc54004800 msgr2=0x7fdc7013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc54004800 0x7fdc7013f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 -- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 msgr2=0x7fdc701f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 0x7fdc701f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 -- 172.21.15.3:0/3432664438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc701f53f0 con 0x7fdc70143110 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc67e00640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 0x7fdc701f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc6fe00640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdc70163240 tx=0x7fdc6800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68003b80 con 0x7fdc70143110 2024-09-10T07:52:38.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc68018070 con 0x7fdc70143110 2024-09-10T07:52:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc70140c80 con 0x7fdc70143110 2024-09-10T07:52:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.267+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680050d0 con 0x7fdc70143110 2024-09-10T07:52:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.268+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc70141690 con 0x7fdc70143110 2024-09-10T07:52:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.269+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc701411a0 con 0x7fdc70143110 2024-09-10T07:52:38.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.270+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc68005270 con 0x7fdc70143110 2024-09-10T07:52:38.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.270+0000 7fdc67400640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 0x7fdc4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.271+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fdc6808ab80 con 0x7fdc70143110 2024-09-10T07:52:38.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.271+0000 7fdc6f400640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 0x7fdc4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.271+0000 7fdc6f400640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 0x7fdc4c06e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc70140440 tx=0x7fdc60000f80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.275+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc701f53f0 con 0x7fdc70143110 2024-09-10T07:52:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.527+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdc70148460 con 0x7fdc70143110 2024-09-10T07:52:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.528+0000 7fdc67400640 1 -- 172.21.15.3:0/3432664438 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7fdc680595c0 con 0x7fdc70143110 2024-09-10T07:52:38.528 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:38.528 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":392,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":270024704,"bytes_avail":772824088576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9896,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":94,"btime":"2024-09-10T07:52:37:667453+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-10T07:52:37.572776+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 msgr2=0x7fdc4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 0x7fdc4c06e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc70140440 tx=0x7fdc60000f80 comp rx=0 tx=0).stop 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 msgr2=0x7fdc701435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdc70163240 tx=0x7fdc6800fd40 comp rx=0 tx=0).stop 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 shutdown_connections 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdc4c06be30 0x7fdc4c06e2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc70143110 0x7fdc701435a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc7013ffb0 0x7fdc701f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 --2- 172.21.15.3:0/3432664438 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdc54004800 0x7fdc7013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 >> 172.21.15.3:0/3432664438 conn(0x7fdc700b24d0 msgr2=0x7fdc700b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 shutdown_connections 2024-09-10T07:52:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.530+0000 7fdc76200640 1 -- 172.21.15.3:0/3432664438 wait complete. 2024-09-10T07:52:38.546 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:52:38.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:38.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub resume 2024-09-10T07:52:38.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.706+0000 7f3e5a600640 1 Processor -- start 2024-09-10T07:52:38.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.706+0000 7f3e5a600640 1 -- start start 2024-09-10T07:52:38.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 0x7f3e540ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e540aede0 0x7f3e54072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e540728d0 0x7f3e54070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3e540af250 con 0x7f3e540aede0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5415eaf0 con 0x7f3e54147ce0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e5a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e540730c0 con 0x7f3e540728d0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.707+0000 7f3e52a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e540aede0 0x7f3e54072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e540aede0 0x7f3e54072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44802/0 (socket says 172.21.15.3:44802) 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 -- 172.21.15.3:0/1091468224 learned_addr learned my addr 172.21.15.3:0/1091468224 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e53400640 1 --2- 172.21.15.3:0/1091468224 >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 0x7f3e540ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e53e00640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e540728d0 0x7f3e54070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 -- 172.21.15.3:0/1091468224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e540728d0 msgr2=0x7f3e54070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e540728d0 0x7f3e54070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 -- 172.21.15.3:0/1091468224 >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 msgr2=0x7f3e540ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 --2- 172.21.15.3:0/1091468224 >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 0x7f3e540ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e52a00640 1 -- 172.21.15.3:0/1091468224 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5415faf0 con 0x7f3e540aede0 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.708+0000 7f3e53400640 1 --2- 172.21.15.3:0/1091468224 >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 0x7f3e540ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.709+0000 7f3e52a00640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540aede0 0x7f3e54072390 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3e5406c370 tx=0x7f3e4002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd495d0341f1919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.709+0000 7f3e52000640 1 -- 172.21.15.3:0/1091468224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40037070 con 0x7f3e540aede0 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.709+0000 7f3e52000640 1 -- 172.21.15.3:0/1091468224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e40032440 con 0x7f3e540aede0 2024-09-10T07:52:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.709+0000 7f3e52000640 1 -- 172.21.15.3:0/1091468224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40032810 con 0x7f3e540aede0 2024-09-10T07:52:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 -- 172.21.15.3:0/1091468224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540aede0 msgr2=0x7f3e54072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540aede0 0x7f3e54072390 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3e5406c370 tx=0x7f3e4002a200 comp rx=0 tx=0).stop 2024-09-10T07:52:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 -- 172.21.15.3:0/1091468224 shutdown_connections 2024-09-10T07:52:38.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e540728d0 0x7f3e54070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 --2- 172.21.15.3:0/1091468224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540aede0 0x7f3e54072390 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 --2- 172.21.15.3:0/1091468224 >> v2:172.21.15.70:3300/0 conn(0x7f3e54147ce0 0x7f3e540ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 -- 172.21.15.3:0/1091468224 >> 172.21.15.3:0/1091468224 conn(0x7f3e540b5cb0 msgr2=0x7f3e5406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 -- 172.21.15.3:0/1091468224 shutdown_connections 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.710+0000 7f3e5a600640 1 -- 172.21.15.3:0/1091468224 wait complete. 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.711+0000 7f3e5a600640 1 Processor -- start 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.712+0000 7f3e5a600640 1 -- start start 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.712+0000 7f3e5a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.712+0000 7f3e5a600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e540aede0 0x7f3e540b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e5a600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e54147ce0 0x7f3e54144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e5a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5415f010 con 0x7f3e540728d0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e5a600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5415f660 con 0x7f3e540aede0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e5a600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5415f2f0 con 0x7f3e54147ce0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44816/0 (socket says 172.21.15.3:44816) 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 -- 172.21.15.3:0/3328492409 learned_addr learned my addr 172.21.15.3:0/3328492409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e540aede0 0x7f3e540b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53e00640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e54147ce0 0x7f3e54144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 -- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e54147ce0 msgr2=0x7f3e54144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e54147ce0 0x7f3e54144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 -- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e540aede0 msgr2=0x7f3e540b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e540aede0 0x7f3e540b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.713+0000 7f3e53400640 1 -- 172.21.15.3:0/3328492409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e54133880 con 0x7f3e540728d0 2024-09-10T07:52:38.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e53400640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400d7d0 tx=0x7f3e4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e4b400640 1 -- 172.21.15.3:0/3328492409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44004030 con 0x7f3e540728d0 2024-09-10T07:52:38.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e4b400640 1 -- 172.21.15.3:0/3328492409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e440041d0 con 0x7f3e540728d0 2024-09-10T07:52:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e4b400640 1 -- 172.21.15.3:0/3328492409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440096a0 con 0x7f3e540728d0 2024-09-10T07:52:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e54130c40 con 0x7f3e540728d0 2024-09-10T07:52:38.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.714+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e541314e0 con 0x7f3e540728d0 2024-09-10T07:52:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.716+0000 7f3e4b400640 1 -- 172.21.15.3:0/3328492409 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e44009840 con 0x7f3e540728d0 2024-09-10T07:52:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.717+0000 7f3e4b400640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 0x7f3e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.718+0000 7f3e4b400640 1 -- 172.21.15.3:0/3328492409 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f3e4408b0b0 con 0x7f3e540728d0 2024-09-10T07:52:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.718+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 0x7f3e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.718+0000 7f3e5d5c3740 1 Processor -- start 2024-09-10T07:52:38.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.718+0000 7f3e5d5c3740 1 -- start start 2024-09-10T07:52:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.718+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 0x7f3e3406e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3e40034000 tx=0x7f3e4002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x560898233200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981e8480 0x560898233740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560898150660 con 0x560898202420 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5608980f8680 con 0x560898202800 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56089814c620 con 0x5608981e8480 2024-09-10T07:52:38.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44832/0 (socket says 172.21.15.3:44832) 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.719+0000 7f3e53400640 1 -- 172.21.15.3:0/3904989629 learned_addr learned my addr 172.21.15.3:0/3904989629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x560898233200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53e00640 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981e8480 0x560898233740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 -- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981e8480 msgr2=0x560898233740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981e8480 0x560898233740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 -- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 msgr2=0x560898233200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x560898233200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 -- 172.21.15.3:0/3904989629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560897f2d7d0 con 0x560898202420 2024-09-10T07:52:38.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e53400640 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3e44061970 tx=0x7f3e440594b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2810be5aa640020e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e3fe00640 1 -- 172.21.15.3:0/3904989629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44087070 con 0x560898202420 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e3fe00640 1 -- 172.21.15.3:0/3904989629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e44057670 con 0x560898202420 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.720+0000 7f3e3fe00640 1 -- 172.21.15.3:0/3904989629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44071030 con 0x560898202420 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3904989629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 msgr2=0x560898232c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3e44061970 tx=0x7f3e440594b0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3904989629 shutdown_connections 2024-09-10T07:52:38.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981e8480 0x560898233740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x560898233200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3904989629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x560898232c40 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3904989629 >> 172.21.15.3:0/3904989629 conn(0x5608981e9220 msgr2=0x560898231840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3904989629 shutdown_connections 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.721+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3904989629 wait complete. 2024-09-10T07:52:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.722+0000 7f3e5d5c3740 1 -- start start 2024-09-10T07:52:38.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.722+0000 7f3e5d5c3740 1 Processor -- start 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x560898244920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x56089822eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5608981e73e0 con 0x5608981f06f0 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5608980d94c0 con 0x560898202420 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5608981e76c0 con 0x560898202800 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e5d5c3740 10 client.0 _ll_register_callbacks cb 0x560898148e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44836/0 (socket says 172.21.15.3:44836) 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e53e00640 1 -- 172.21.15.3:0/2946980557 learned_addr learned my addr 172.21.15.3:0/2946980557 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e52a00640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x56089822eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.723+0000 7f3e53400640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x560898244920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 -- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 msgr2=0x56089822eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x56089822eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 -- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 msgr2=0x560898244920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x560898244920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 -- 172.21.15.3:0/2946980557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560897e633f0 con 0x5608981f06f0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e53e00640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c004270 tx=0x7f3e4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e0ecbb0c2bdcfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c01a020 con 0x5608981f06f0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f3e4c007890 con 0x5608981f06f0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e44057670 con 0x5608981f06f0 2024-09-10T07:52:38.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c024050 con 0x5608981f06f0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e5d5c3740 10 client.7710 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.724+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x560897e230f0 con 0x5608981f06f0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e44057670 con 0x5608981f06f0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f3e4c027020 con 0x5608981f06f0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 10 client.7710 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 10 client.7710 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 20 client.7710 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5608981eedc0 0x5608981ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 4 client.7710 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089815dd90 con 0x5608981eedc0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.725+0000 7f3e53400640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5608981eedc0 0x5608981ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e53400640 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5608981eedc0 0x5608981ef1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9019715fac610375 server_cookie=7aff89cfd1650c89 in_seq=0 out_seq=0 2024-09-10T07:52:38.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e3cc00640 10 client.7710 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e3cc00640 1 -- 172.21.15.3:0/2946980557 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56089815dd90 con 0x5608981eedc0 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e3cc00640 10 client.7710 handle_command_reply: tid=0 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 client.7710 shutdown 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 msgr2=0x5608981f0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c004270 tx=0x7f3e4c004730 comp rx=0 tx=0).stop 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 shutdown_connections 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x5608981eedc0 0x5608981ef1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608981f0b70 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x56089822eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.726+0000 7f3e3cc00640 0 client.7710 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.727+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/2946980557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x560898244920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.727+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 >> 172.21.15.3:0/2946980557 conn(0x5608981e8480 msgr2=0x56089822fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.727+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 shutdown_connections 2024-09-10T07:52:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.727+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/2946980557 wait complete. 2024-09-10T07:52:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.727+0000 7f3e5d5c3740 20 client.7710 trim_cache size 0 max 16384 2024-09-10T07:52:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.745+0000 7f3e5d5c3740 1 Processor -- start 2024-09-10T07:52:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.745+0000 7f3e5d5c3740 1 -- start start 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 0x560898254460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54200d30 con 0x560898202420 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541ffed0 con 0x560898202800 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3e54200600 con 0x5608981f06f0 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44852/0 (socket says 172.21.15.3:44852) 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 -- 172.21.15.3:0/3690300885 learned_addr learned my addr 172.21.15.3:0/3690300885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e53400640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 0x560898254460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 -- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 msgr2=0x560898254460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e53e00640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 0x560898254460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 -- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 msgr2=0x5608981f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e52a00640 1 -- 172.21.15.3:0/3690300885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560898231640 con 0x560898202420 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e53400640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 0x560898254460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.746+0000 7f3e53e00640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.747+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3e4003e040 tx=0x7f3e4002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c6a651b489759f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.747+0000 7f3e3f400640 1 -- 172.21.15.3:0/3690300885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40037040 con 0x560898202420 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.747+0000 7f3e3f400640 1 -- 172.21.15.3:0/3690300885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e40032db0 con 0x560898202420 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.747+0000 7f3e3f400640 1 -- 172.21.15.3:0/3690300885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40048660 con 0x560898202420 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3690300885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 msgr2=0x5608982549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3e4003e040 tx=0x7f3e4002c000 comp rx=0 tx=0).stop 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3690300885 shutdown_connections 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560898202420 0x5608982549a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3690300885 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5608981f06f0 0x560898254460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3690300885 >> 172.21.15.3:0/3690300885 conn(0x5608981e8480 msgr2=0x560898234fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3690300885 shutdown_connections 2024-09-10T07:52:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3690300885 wait complete. 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 -- start start 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.748+0000 7f3e5d5c3740 1 Processor -- start 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x56089822d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560898244360 con 0x5608981f06f0 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54143670 con 0x560898202420 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560898229f00 con 0x560898202800 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.749+0000 7f3e5d5c3740 10 client.0 _ll_register_callbacks cb 0x7f3e4c009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44868/0 (socket says 172.21.15.3:44868) 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 -- 172.21.15.3:0/3641768531 learned_addr learned my addr 172.21.15.3:0/3641768531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x56089822d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53e00640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 -- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 msgr2=0x5608981f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 -- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 msgr2=0x56089822d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x56089822d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 -- 172.21.15.3:0/3641768531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560898253230 con 0x5608981f06f0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e53400640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3e44059300 tx=0x7f3e44006d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9cec2c9037be80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.750+0000 7f3e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44071040 con 0x5608981f06f0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f3e440711e0 con 0x5608981f06f0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e40032db0 con 0x5608981f06f0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f3e44087040 con 0x5608981f06f0 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 10 client.7716 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x560898230090 con 0x5608981f06f0 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e40032db0 con 0x5608981f06f0 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f3e44013070 con 0x5608981f06f0 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 10 client.7716 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 10 client.7716 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 20 client.7716 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x560898229790 0x5608981f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 4 client.7716 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x5608981580e0 con 0x560898229790 2024-09-10T07:52:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.751+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x560898229790 0x5608981f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e52a00640 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x560898229790 0x5608981f1200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3826869830b9 server_cookie=859ee83770c5db26 in_seq=0 out_seq=0 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e3cc00640 10 client.7716 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e3cc00640 1 -- 172.21.15.3:0/3641768531 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5608981580e0 con 0x560898229790 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e3cc00640 10 client.7716 handle_command_reply: tid=0 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e5d5c3740 1 client.7716 shutdown 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 msgr2=0x5608982afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.752+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3e44059300 tx=0x7f3e44006d40 comp rx=0 tx=0).stop 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 shutdown_connections 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x560898229790 0x5608981f1200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560898202800 0x5608981f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560898202420 0x56089822d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 --2- 172.21.15.3:0/3641768531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5608981f06f0 0x5608982afac0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e3cc00640 0 client.7716 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 >> 172.21.15.3:0/3641768531 conn(0x5608981e8480 msgr2=0x560898200830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 shutdown_connections 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 1 -- 172.21.15.3:0/3641768531 wait complete. 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.753+0000 7f3e5d5c3740 20 client.7716 trim_cache size 0 max 16384 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:52:38.757 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 msgr2=0x7f3e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 0x7f3e3406e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3e40034000 tx=0x7f3e4002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 msgr2=0x7f3e54133340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400d7d0 tx=0x7f3e4400dc90 comp rx=0 tx=0).stop 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 shutdown_connections 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3e3406be30 0x7f3e3406e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3e54147ce0 0x7f3e54144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e540aede0 0x7f3e540b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 --2- 172.21.15.3:0/3328492409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e540728d0 0x7f3e54133340 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 >> 172.21.15.3:0/3328492409 conn(0x7f3e540b5cb0 msgr2=0x7f3e54160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 shutdown_connections 2024-09-10T07:52:38.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.754+0000 7f3e5a600640 1 -- 172.21.15.3:0/3328492409 wait complete. 2024-09-10T07:52:38.768 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:38.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-10T07:52:38.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.926+0000 7f704ba00640 1 Processor -- start 2024-09-10T07:52:38.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.927+0000 7f704ba00640 1 -- start start 2024-09-10T07:52:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7044145730 0x7f70440b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70440cf270 0x7f70440cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f704406d7c0 con 0x7f70440cf270 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f70440cd900 con 0x7f7044145730 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f704ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f70440cdbe0 con 0x7f70441451d0 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f7049a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70440cf270 0x7f70440cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f7049000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f7049a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70440cf270 0x7f70440cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44876/0 (socket says 172.21.15.3:44876) 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f7049a00640 1 -- 172.21.15.3:0/1662623323 learned_addr learned my addr 172.21.15.3:0/1662623323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.928+0000 7f703be00640 1 --2- 172.21.15.3:0/1662623323 >> v2:172.21.15.70:3300/0 conn(0x7f7044145730 0x7f70440b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049a00640 1 -- 172.21.15.3:0/1662623323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 msgr2=0x7f704406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049a00640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049a00640 1 -- 172.21.15.3:0/1662623323 >> v2:172.21.15.70:3300/0 conn(0x7f7044145730 msgr2=0x7f70440b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049a00640 1 --2- 172.21.15.3:0/1662623323 >> v2:172.21.15.70:3300/0 conn(0x7f7044145730 0x7f70440b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049a00640 1 -- 172.21.15.3:0/1662623323 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704406da10 con 0x7f70440cf270 2024-09-10T07:52:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.929+0000 7f7049000640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f7049a00640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70440cf270 0x7f70440cf720 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f704406c460 tx=0x7f703c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5282e2e8d563ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f7043e00640 1 -- 172.21.15.3:0/1662623323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c036070 con 0x7f70440cf270 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f7043e00640 1 -- 172.21.15.3:0/1662623323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703c031600 con 0x7f70440cf270 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f7043e00640 1 -- 172.21.15.3:0/1662623323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0319d0 con 0x7f70440cf270 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 -- 172.21.15.3:0/1662623323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70440cf270 msgr2=0x7f70440cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70440cf270 0x7f70440cf720 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f704406c460 tx=0x7f703c02a610 comp rx=0 tx=0).stop 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 -- 172.21.15.3:0/1662623323 shutdown_connections 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70440cf270 0x7f70440cf720 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 --2- 172.21.15.3:0/1662623323 >> v2:172.21.15.70:3300/0 conn(0x7f7044145730 0x7f70440b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 --2- 172.21.15.3:0/1662623323 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.930+0000 7f704ba00640 1 -- 172.21.15.3:0/1662623323 >> 172.21.15.3:0/1662623323 conn(0x7f70440b5cb0 msgr2=0x7f704406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.931+0000 7f704ba00640 1 -- 172.21.15.3:0/1662623323 shutdown_connections 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.931+0000 7f704ba00640 1 -- 172.21.15.3:0/1662623323 wait complete. 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.932+0000 7f704ba00640 1 Processor -- start 2024-09-10T07:52:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.932+0000 7f704ba00640 1 -- start start 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 0x7f7044140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7044145730 0x7f70440b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70440b38b0 con 0x7f7044145730 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7044145f10 con 0x7f70440cf270 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.933+0000 7f704ba00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f704406df30 con 0x7f70441451d0 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f7049a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7044145730 0x7f70440b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f7049000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 0x7f7044140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f7049000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 0x7f7044140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:35734/0 (socket says 172.21.15.3:35734) 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f7049000640 1 -- 172.21.15.3:0/232655330 learned_addr learned my addr 172.21.15.3:0/232655330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 -- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 msgr2=0x7f7044140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 0x7f7044140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 -- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7044145730 msgr2=0x7f70440b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7044145730 0x7f70440b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 -- 172.21.15.3:0/232655330 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704415f910 con 0x7f70441451d0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.934+0000 7f703be00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704415f3d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e870 tx=0x7f702c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.935+0000 7f7041600640 1 -- 172.21.15.3:0/232655330 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c017070 con 0x7f70441451d0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.935+0000 7f7041600640 1 -- 172.21.15.3:0/232655330 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c00ee60 con 0x7f70441451d0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.935+0000 7f7041600640 1 -- 172.21.15.3:0/232655330 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c00cb80 con 0x7f70441451d0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.935+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704415cba0 con 0x7f70441451d0 2024-09-10T07:52:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.935+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f704415e610 con 0x7f70441451d0 2024-09-10T07:52:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.937+0000 7f7041600640 1 -- 172.21.15.3:0/232655330 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f702c01a070 con 0x7f70441451d0 2024-09-10T07:52:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.938+0000 7f7041600640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 0x7f702006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.938+0000 7f7041600640 1 -- 172.21.15.3:0/232655330 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f702c08ad00 con 0x7f70441451d0 2024-09-10T07:52:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.938+0000 7f7049000640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 0x7f702006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.938+0000 7f704e88f740 1 Processor -- start 2024-09-10T07:52:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 -- start start 2024-09-10T07:52:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f7049000640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 0x7f702006e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7034009560 tx=0x7f7034009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a673c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 0x5652a673c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652a6659660 con 0x5652a670b420 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652a6601680 con 0x5652a670b800 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.939+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652a6655620 con 0x5652a66f1480 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44900/0 (socket says 172.21.15.3:44900) 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 -- 172.21.15.3:0/1807962330 learned_addr learned my addr 172.21.15.3:0/1807962330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049a00640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 0x5652a673c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f703be00640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a673c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 -- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 msgr2=0x5652a673c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 0x5652a673c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 -- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 msgr2=0x5652a673c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a673c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 -- 172.21.15.3:0/1807962330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a64367d0 con 0x5652a670b420 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049a00640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 0x5652a673c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f703be00640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a673c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.940+0000 7f7049000640 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7034009410 tx=0x7f7034014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58304740b26c3cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f7039600640 1 -- 172.21.15.3:0/1807962330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034025070 con 0x5652a670b420 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f7039600640 1 -- 172.21.15.3:0/1807962330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703401b070 con 0x5652a670b420 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f7039600640 1 -- 172.21.15.3:0/1807962330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340213f0 con 0x5652a670b420 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 -- 172.21.15.3:0/1807962330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 msgr2=0x5652a673bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7034009410 tx=0x7f7034014000 comp rx=0 tx=0).stop 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 -- 172.21.15.3:0/1807962330 shutdown_connections 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a66f1480 0x5652a673c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a673c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 --2- 172.21.15.3:0/1807962330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a673bc40 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 -- 172.21.15.3:0/1807962330 >> 172.21.15.3:0/1807962330 conn(0x5652a66f2220 msgr2=0x5652a673a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 -- 172.21.15.3:0/1807962330 shutdown_connections 2024-09-10T07:52:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.941+0000 7f704e88f740 1 -- 172.21.15.3:0/1807962330 wait complete. 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.942+0000 7f704e88f740 1 -- start start 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.942+0000 7f704e88f740 1 Processor -- start 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a671f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a670b800 0x5652a66f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652a66f03e0 con 0x5652a6737fc0 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652a65e24c0 con 0x5652a670b420 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652a66f06c0 con 0x5652a670b800 2024-09-10T07:52:38.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f704e88f740 10 client.0 _ll_register_callbacks cb 0x5652a6651e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.943+0000 7f7049a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44912/0 (socket says 172.21.15.3:44912) 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 -- 172.21.15.3:0/2786700367 learned_addr learned my addr 172.21.15.3:0/2786700367 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049000640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a671f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f703be00640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a670b800 0x5652a66f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 -- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a670b800 msgr2=0x5652a66f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a670b800 0x5652a66f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 -- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 msgr2=0x5652a671f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a671f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 -- 172.21.15.3:0/2786700367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a636c3f0 con 0x5652a6737fc0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7049a00640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f703c02ac00 tx=0x7f703c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6239af6df5c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7032000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:38.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c042070 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.944+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f703c03c3f0 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703401b070 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f703c04c050 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 10 client.7728 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x5652a632c0f0 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703401b070 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f703c008040 con 0x5652a6737fc0 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 10 client.7728 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 10 client.7728 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 20 client.7728 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a6735eb0 0x5652a67362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 4 client.7728 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.945+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652a6666d90 con 0x5652a6735eb0 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.946+0000 7f7049000640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a6735eb0 0x5652a67362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.946+0000 7f7049000640 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a6735eb0 0x5652a67362c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d54802e78050b8ae server_cookie=42de7d7cae8846ad in_seq=0 out_seq=0 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.946+0000 7f7032000640 10 client.7728 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f7032000640 1 -- 172.21.15.3:0/2786700367 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652a6666d90 con 0x5652a6735eb0 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f7032000640 10 client.7728 handle_command_reply: tid=0 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 client.7728 shutdown 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 msgr2=0x5652a66f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f703c02ac00 tx=0x7f703c004ab0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 shutdown_connections 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a6737fc0 0x5652a66f7e30 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a6735eb0 0x5652a67362c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a670b800 0x5652a66f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 --2- 172.21.15.3:0/2786700367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a671f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 >> 172.21.15.3:0/2786700367 conn(0x5652a66f1480 msgr2=0x5652a673a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f7032000640 0 client.7728 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 shutdown_connections 2024-09-10T07:52:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.947+0000 7f704e88f740 1 -- 172.21.15.3:0/2786700367 wait complete. 2024-09-10T07:52:38.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.948+0000 7f704e88f740 20 client.7728 trim_cache size 0 max 16384 2024-09-10T07:52:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.966+0000 7f704e88f740 1 Processor -- start 2024-09-10T07:52:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.966+0000 7f704e88f740 1 -- start start 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.966+0000 7f704e88f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.966+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a6739010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a66f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f704e88f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7044205200 con 0x5652a670b420 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f70442043a0 con 0x5652a670b800 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7044204ad0 con 0x5652a6737fc0 2024-09-10T07:52:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44916/0 (socket says 172.21.15.3:44916) 2024-09-10T07:52:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 -- 172.21.15.3:0/3779421336 learned_addr learned my addr 172.21.15.3:0/3779421336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f703be00640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a6739010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049a00640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a66f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 -- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 msgr2=0x5652a66f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a66f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 -- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 msgr2=0x5652a6739010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a6739010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 -- 172.21.15.3:0/3779421336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a66f1990 con 0x5652a670b420 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049a00640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a66f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.967+0000 7f7049000640 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7034024000 tx=0x7f7034016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fdda4f501a29c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f7038c00640 1 -- 172.21.15.3:0/3779421336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703403a070 con 0x5652a670b420 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f7038c00640 1 -- 172.21.15.3:0/3779421336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703401b070 con 0x5652a670b420 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f7038c00640 1 -- 172.21.15.3:0/3779421336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034025040 con 0x5652a670b420 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 -- 172.21.15.3:0/3779421336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 msgr2=0x5652a6738ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7034024000 tx=0x7f7034016000 comp rx=0 tx=0).stop 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 -- 172.21.15.3:0/3779421336 shutdown_connections 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a66f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b800 0x5652a6739010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 --2- 172.21.15.3:0/3779421336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b420 0x5652a6738ad0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 -- 172.21.15.3:0/3779421336 >> 172.21.15.3:0/3779421336 conn(0x5652a66f1480 msgr2=0x5652a6739d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 -- 172.21.15.3:0/3779421336 shutdown_connections 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.968+0000 7f704e88f740 1 -- 172.21.15.3:0/3779421336 wait complete. 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.969+0000 7f704e88f740 1 -- start start 2024-09-10T07:52:38.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.969+0000 7f704e88f740 1 Processor -- start 2024-09-10T07:52:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a66effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a67b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652a66f05f0 con 0x5652a670b800 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7044071cc0 con 0x5652a670b420 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652a673bff0 con 0x5652a6737fc0 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f704e88f740 10 client.0 _ll_register_callbacks cb 0x7f702c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f703be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f703be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44918/0 (socket says 172.21.15.3:44918) 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.970+0000 7f703be00640 1 -- 172.21.15.3:0/449635353 learned_addr learned my addr 172.21.15.3:0/449635353 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7049a00640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a67b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7049000640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a66effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 -- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 msgr2=0x5652a67b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a67b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 -- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 msgr2=0x5652a66effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a66effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 -- 172.21.15.3:0/449635353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a6737590 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7049000640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a66effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7049a00640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a67b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f703be00640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f702c055310 tx=0x7f702c058f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e359c634a863e404 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7032000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c021040 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f702c076b30 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703401b070 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f702c076d30 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f704e88f740 10 client.7734 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.971+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x5652a675df90 con 0x5652a670b800 2024-09-10T07:52:38.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703401b070 con 0x5652a670b800 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f702c024060 con 0x5652a670b800 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 10 client.7734 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 10 client.7734 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.a 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 20 client.7734 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a67b7f40 0x5652a673a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 4 client.7734 mds_command: new command op to 7236 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 --> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652a66610e0 con 0x5652a67b7f40 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.972+0000 7f7049000640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a67b7f40 0x5652a673a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f7049000640 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a67b7f40 0x5652a673a640 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14c146d2671c874a server_cookie=c387d36c0676ba46 in_seq=0 out_seq=0 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f7032000640 10 client.7734 ms_handle_connect on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f7032000640 1 -- 172.21.15.3:0/449635353 <== mds.1 v2:172.21.15.3:6834/3872292817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5652a66610e0 con 0x5652a67b7f40 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f7032000640 10 client.7734 handle_command_reply: tid=0 2024-09-10T07:52:38.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 client.7734 shutdown 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 msgr2=0x5652a673bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f702c055310 tx=0x7f702c058f70 comp rx=0 tx=0).stop 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 shutdown_connections 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:6834/3872292817,v1:172.21.15.3:6836/3872292817] conn(0x5652a67b7f40 0x5652a673a640 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652a6737fc0 0x5652a67b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652a670b800 0x5652a673bab0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f7032000640 0 client.7734 ms_handle_reset on v2:172.21.15.3:6834/3872292817 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 --2- 172.21.15.3:0/449635353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652a670b420 0x5652a66effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 >> 172.21.15.3:0/449635353 conn(0x5652a66f1480 msgr2=0x5652a673dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 shutdown_connections 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 1 -- 172.21.15.3:0/449635353 wait complete. 2024-09-10T07:52:38.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.973+0000 7f704e88f740 20 client.7734 trim_cache size 0 max 16384 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:38.978 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 msgr2=0x7f702006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 0x7f702006e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7034009560 tx=0x7f7034009190 comp rx=0 tx=0).stop 2024-09-10T07:52:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 msgr2=0x7f704415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704415f3d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e870 tx=0x7f702c00ed30 comp rx=0 tx=0).stop 2024-09-10T07:52:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 shutdown_connections 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f702006be60 0x7f702006e320 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7044145730 0x7f70440b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70441451d0 0x7f704415f3d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 --2- 172.21.15.3:0/232655330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70440cf270 0x7f7044140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 >> 172.21.15.3:0/232655330 conn(0x7f70440b5cb0 msgr2=0x7f704406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 shutdown_connections 2024-09-10T07:52:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:38.974+0000 7f704ba00640 1 -- 172.21.15.3:0/232655330 wait complete. 2024-09-10T07:52:38.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:38.989 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:38.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-10T07:52:39.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.147+0000 7f1b96600640 1 Processor -- start 2024-09-10T07:52:39.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.147+0000 7f1b96600640 1 -- start start 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b96600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b96600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b90072c50 0x7f1b900b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b96600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9015eff0 con 0x7f1b900b4ac0 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b96600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9015fbe0 con 0x7f1b90072c50 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b96600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9015c8d0 con 0x7f1b9015b230 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b8f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b87e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b90072c50 0x7f1b900b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b8f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:42454/0 (socket says 172.21.15.3:42454) 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b8f400640 1 -- 172.21.15.3:0/700353811 learned_addr learned my addr 172.21.15.3:0/700353811 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b8f400640 1 -- 172.21.15.3:0/700353811 >> v2:172.21.15.70:3300/0 conn(0x7f1b90072c50 msgr2=0x7f1b900b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.149+0000 7f1b8f400640 1 --2- 172.21.15.3:0/700353811 >> v2:172.21.15.70:3300/0 conn(0x7f1b90072c50 0x7f1b900b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.150+0000 7f1b8f400640 1 -- 172.21.15.3:0/700353811 >> v1:172.21.15.70:6789/0 conn(0x7f1b900b4ac0 legacy=0x7f1b9006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.150+0000 7f1b8f400640 1 -- 172.21.15.3:0/700353811 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9015f2b0 con 0x7f1b9015b230 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.150+0000 7f1b8f400640 1 --2- 172.21.15.3:0/700353811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009870 tx=0x7f1b8002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4222422b25c46bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b8ea00640 1 -- 172.21.15.3:0/700353811 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80037070 con 0x7f1b9015b230 2024-09-10T07:52:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b8ea00640 1 -- 172.21.15.3:0/700353811 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80032440 con 0x7f1b9015b230 2024-09-10T07:52:39.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b8ea00640 1 -- 172.21.15.3:0/700353811 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80032810 con 0x7f1b9015b230 2024-09-10T07:52:39.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 -- 172.21.15.3:0/700353811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 msgr2=0x7f1b9015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 --2- 172.21.15.3:0/700353811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009870 tx=0x7f1b8002a250 comp rx=0 tx=0).stop 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 -- 172.21.15.3:0/700353811 shutdown_connections 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 --2- 172.21.15.3:0/700353811 >> v2:172.21.15.70:3300/0 conn(0x7f1b90072c50 0x7f1b900b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 --2- 172.21.15.3:0/700353811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9015b230 0x7f1b9015c390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.151+0000 7f1b96600640 1 -- 172.21.15.3:0/700353811 >> 172.21.15.3:0/700353811 conn(0x7f1b900b5cb0 msgr2=0x7f1b9006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.152+0000 7f1b96600640 1 -- 172.21.15.3:0/700353811 shutdown_connections 2024-09-10T07:52:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.152+0000 7f1b96600640 1 -- 172.21.15.3:0/700353811 wait complete. 2024-09-10T07:52:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.153+0000 7f1b96600640 1 Processor -- start 2024-09-10T07:52:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.153+0000 7f1b96600640 1 -- start start 2024-09-10T07:52:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b90072c50 0x7f1b9014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9015b230 0x7f1b900affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9015b610 con 0x7f1b900b4ac0 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9015f870 con 0x7f1b9015b230 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b96600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9015b790 con 0x7f1b90072c50 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.154+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44928/0 (socket says 172.21.15.3:44928) 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 -- 172.21.15.3:0/929215951 learned_addr learned my addr 172.21.15.3:0/929215951 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b8f400640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b90072c50 0x7f1b9014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9015b230 0x7f1b900affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 -- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b90072c50 msgr2=0x7f1b9014a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b90072c50 0x7f1b9014a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 -- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9015b230 msgr2=0x7f1b900affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9015b230 0x7f1b900affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 -- 172.21.15.3:0/929215951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9015d400 con 0x7f1b900b4ac0 2024-09-10T07:52:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87e00640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e980 tx=0x7f1b7800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.155+0000 7f1b87400640 1 -- 172.21.15.3:0/929215951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017070 con 0x7f1b900b4ac0 2024-09-10T07:52:39.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.156+0000 7f1b87400640 1 -- 172.21.15.3:0/929215951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7800ca00 con 0x7f1b900b4ac0 2024-09-10T07:52:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.156+0000 7f1b87400640 1 -- 172.21.15.3:0/929215951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800cdd0 con 0x7f1b900b4ac0 2024-09-10T07:52:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.156+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9015d6e0 con 0x7f1b900b4ac0 2024-09-10T07:52:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.156+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9015dca0 con 0x7f1b900b4ac0 2024-09-10T07:52:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.158+0000 7f1b87400640 1 -- 172.21.15.3:0/929215951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b7801a070 con 0x7f1b900b4ac0 2024-09-10T07:52:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.159+0000 7f1b87400640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.159+0000 7f1b87400640 1 -- 172.21.15.3:0/929215951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1b7808ae90 con 0x7f1b900b4ac0 2024-09-10T07:52:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.159+0000 7f1b8f400640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.160+0000 7f1b99472740 1 Processor -- start 2024-09-10T07:52:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.160+0000 7f1b99472740 1 -- start start 2024-09-10T07:52:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.160+0000 7f1b8f400640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b80034000 tx=0x7f1b8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd8dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd8e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56350dcab660 con 0x56350dd43480 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56350dc53680 con 0x56350dd5d420 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56350dca7620 con 0x56350dd5d800 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44932/0 (socket says 172.21.15.3:44932) 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b8fe00640 1 -- 172.21.15.3:0/2821630238 learned_addr learned my addr 172.21.15.3:0/2821630238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b8f400640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd8dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.161+0000 7f1b87e00640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd8e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 -- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 msgr2=0x56350dd8e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd8e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 -- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 msgr2=0x56350dd8dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd8dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 -- 172.21.15.3:0/2821630238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350da887d0 con 0x56350dd43480 2024-09-10T07:52:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b87e00640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd8e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1b88001f30 tx=0x7f1b88001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71859cbe3012890 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b7fe00640 1 -- 172.21.15.3:0/2821630238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800fdf0 con 0x56350dd43480 2024-09-10T07:52:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b7fe00640 1 -- 172.21.15.3:0/2821630238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88018070 con 0x56350dd43480 2024-09-10T07:52:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.162+0000 7f1b7fe00640 1 -- 172.21.15.3:0/2821630238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88003b70 con 0x56350dd43480 2024-09-10T07:52:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- 172.21.15.3:0/2821630238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 msgr2=0x56350dd8e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1b88001f30 tx=0x7f1b88001670 comp rx=0 tx=0).stop 2024-09-10T07:52:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- 172.21.15.3:0/2821630238 shutdown_connections 2024-09-10T07:52:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd43480 0x56350dd8e740 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd8e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 --2- 172.21.15.3:0/2821630238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd8dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- 172.21.15.3:0/2821630238 >> 172.21.15.3:0/2821630238 conn(0x56350dd44220 msgr2=0x56350dd8c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- 172.21.15.3:0/2821630238 shutdown_connections 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- 172.21.15.3:0/2821630238 wait complete. 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.163+0000 7f1b99472740 1 -- start start 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.164+0000 7f1b99472740 1 Processor -- start 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56350dd423e0 con 0x56350dd89f80 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56350dc344c0 con 0x56350dd5d420 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56350dd426c0 con 0x56350dd5d800 2024-09-10T07:52:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b99472740 10 client.0 _ll_register_callbacks cb 0x56350dca3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44938/0 (socket says 172.21.15.3:44938) 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 -- 172.21.15.3:0/643385745 learned_addr learned my addr 172.21.15.3:0/643385745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b87e00640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8f400640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 -- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 msgr2=0x56350dd89a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd89a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 -- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 msgr2=0x56350dd466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.165+0000 7f1b8fe00640 1 -- 172.21.15.3:0/643385745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350d9be3f0 con 0x56350dd89f80 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1b88004da0 tx=0x7f1b88001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7658d001a851f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:39.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88006d50 con 0x56350dd89f80 2024-09-10T07:52:39.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1b880050d0 con 0x56350dd89f80 2024-09-10T07:52:39.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b88018070 con 0x56350dd89f80 2024-09-10T07:52:39.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f1b8802e050 con 0x56350dd89f80 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b99472740 10 client.7743 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x56350d97e0f0 con 0x56350dd89f80 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.166+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88018070 con 0x56350dd89f80 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f1b88031020 con 0x56350dd89f80 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 10 client.7743 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 10 client.7743 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 20 client.7743 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350dd9c490 0x56350dd63680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 4 client.7743 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350dcb8d90 con 0x56350dd9c490 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.167+0000 7f1b8f400640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350dd9c490 0x56350dd63680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.168+0000 7f1b8f400640 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350dd9c490 0x56350dd63680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7085f6c51402215 server_cookie=1062829e2550ee21 in_seq=0 out_seq=0 2024-09-10T07:52:39.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.168+0000 7f1b7cc00640 10 client.7743 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b7cc00640 1 -- 172.21.15.3:0/643385745 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56350dcb8d90 con 0x56350dd9c490 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b7cc00640 10 client.7743 handle_command_reply: tid=0 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 client.7743 shutdown 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 msgr2=0x56350dd820f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1b88004da0 tx=0x7f1b88001250 comp rx=0 tx=0).stop 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 shutdown_connections 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350dd9c490 0x56350dd63680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd89f80 0x56350dd820f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d800 0x56350dd89a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 --2- 172.21.15.3:0/643385745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350dd466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b7cc00640 0 client.7743 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 >> 172.21.15.3:0/643385745 conn(0x56350dd43480 msgr2=0x56350dd40f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 shutdown_connections 2024-09-10T07:52:39.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 1 -- 172.21.15.3:0/643385745 wait complete. 2024-09-10T07:52:39.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.169+0000 7f1b99472740 20 client.7743 trim_cache size 0 max 16384 2024-09-10T07:52:39.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 Processor -- start 2024-09-10T07:52:39.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 -- start start 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d420 0x56350dd67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd89f80 0x56350dd41150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901fc900 con 0x56350dd5d800 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901fbaa0 con 0x56350dd89f80 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.188+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b901fc1d0 con 0x56350dd5d420 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44944/0 (socket says 172.21.15.3:44944) 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 -- 172.21.15.3:0/3415279123 learned_addr learned my addr 172.21.15.3:0/3415279123 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b8f400640 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d420 0x56350dd67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd89f80 0x56350dd41150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 -- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d420 msgr2=0x56350dd67480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d420 0x56350dd67480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 -- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd89f80 msgr2=0x56350dd41150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd89f80 0x56350dd41150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 -- 172.21.15.3:0/3415279123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350dd48770 con 0x56350dd5d800 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.189+0000 7f1b87e00640 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1b78055560 tx=0x7f1b780096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238893f4a93189ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b7f400640 1 -- 172.21.15.3:0/3415279123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78021040 con 0x56350dd5d800 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b7f400640 1 -- 172.21.15.3:0/3415279123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78061c80 con 0x56350dd5d800 2024-09-10T07:52:39.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b7f400640 1 -- 172.21.15.3:0/3415279123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780557c0 con 0x56350dd5d800 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 -- 172.21.15.3:0/3415279123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 msgr2=0x56350dd8aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1b78055560 tx=0x7f1b780096c0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 -- 172.21.15.3:0/3415279123 shutdown_connections 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd89f80 0x56350dd41150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350dd8aad0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 --2- 172.21.15.3:0/3415279123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd5d420 0x56350dd67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 -- 172.21.15.3:0/3415279123 >> 172.21.15.3:0/3415279123 conn(0x56350dd43480 msgr2=0x56350dd457c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 -- 172.21.15.3:0/3415279123 shutdown_connections 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.190+0000 7f1b99472740 1 -- 172.21.15.3:0/3415279123 wait complete. 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.191+0000 7f1b99472740 1 -- start start 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.191+0000 7f1b99472740 1 Processor -- start 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350de010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd89f80 0x56350de05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56350dd82080 con 0x56350dd5d800 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90139f20 con 0x56350dd5d420 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56350de05d10 con 0x56350dd89f80 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b99472740 10 client.0 _ll_register_callbacks cb 0x7f1b78009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44954/0 (socket says 172.21.15.3:44954) 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b87e00640 1 -- 172.21.15.3:0/65928523 learned_addr learned my addr 172.21.15.3:0/65928523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b8f400640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350de010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.192+0000 7f1b8fe00640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd89f80 0x56350de05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 -- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd89f80 msgr2=0x56350de05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd89f80 0x56350de05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 -- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 msgr2=0x56350de010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350de010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 -- 172.21.15.3:0/65928523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350dd419d0 con 0x56350dd5d800 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b87e00640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000c00 tx=0x7f1b7800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3b6e4b9e9f32f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78077070 con 0x56350dd5d800 2024-09-10T07:52:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1b78054ea0 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b78061c80 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f1b78021070 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b99472740 10 client.7749 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.193+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x56350dd413f0 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78061c80 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f1b78024060 con 0x56350dd5d800 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 10 client.7749 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 10 client.7749 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.d 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 20 client.7749 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350de01920 0x56350dd5b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 4 client.7749 mds_command: new command op to 6799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 --> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56350dcb30e0 con 0x56350de01920 2024-09-10T07:52:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.194+0000 7f1b8f400640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350de01920 0x56350dd5b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b8f400640 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350de01920 0x56350dd5b980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbe22706095b2e3d server_cookie=6ef21b44f5f1eafe in_seq=0 out_seq=0 2024-09-10T07:52:39.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b7cc00640 10 client.7749 ms_handle_connect on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b7cc00640 1 -- 172.21.15.3:0/65928523 <== mds.2 v2:172.21.15.70:6833/3096818521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56350dcb30e0 con 0x56350de01920 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b7cc00640 10 client.7749 handle_command_reply: tid=0 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 client.7749 shutdown 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 msgr2=0x56350de03560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000c00 tx=0x7f1b7800cf70 comp rx=0 tx=0).stop 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 shutdown_connections 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:6833/3096818521,v1:172.21.15.70:6835/3096818521] conn(0x56350de01920 0x56350dd5b980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56350dd89f80 0x56350de05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56350dd5d800 0x56350de03560 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b7cc00640 0 client.7749 ms_handle_reset on v2:172.21.15.70:6833/3096818521 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 --2- 172.21.15.3:0/65928523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56350dd5d420 0x56350de010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 >> 172.21.15.3:0/65928523 conn(0x56350dd43480 msgr2=0x56350dd67480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.195+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 shutdown_connections 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b99472740 1 -- 172.21.15.3:0/65928523 wait complete. 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b99472740 20 client.7749 trim_cache size 0 max 16384 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:39.200 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:39.201 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:39.201 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:39.201 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:39.201 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:39.201 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 msgr2=0x7f1b6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b80034000 tx=0x7f1b8002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 msgr2=0x7f1b9015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e980 tx=0x7f1b7800ee40 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 shutdown_connections 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1b6c06be30 0x7f1b6c06e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9015b230 0x7f1b900affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b900b4ac0 0x7f1b9015ca90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 --2- 172.21.15.3:0/929215951 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b90072c50 0x7f1b9014a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 >> 172.21.15.3:0/929215951 conn(0x7f1b900b5cb0 msgr2=0x7f1b90155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 shutdown_connections 2024-09-10T07:52:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.196+0000 7f1b96600640 1 -- 172.21.15.3:0/929215951 wait complete. 2024-09-10T07:52:39.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:39.212 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:52:39.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T07:52:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.370+0000 7f1a3f200640 1 Processor -- start 2024-09-10T07:52:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.371+0000 7f1a3f200640 1 -- start start 2024-09-10T07:52:39.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a3815b230 0x7f1a3815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 0x7f1a38071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3815eff0 con 0x7f1a38072c50 2024-09-10T07:52:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3815fbe0 con 0x7f1a3815b230 2024-09-10T07:52:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3815c8d0 con 0x7f1a380b4c90 2024-09-10T07:52:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a2f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1a38072c50 0x7f1a380b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51808/0 (socket says 172.21.15.3:51808) 2024-09-10T07:52:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a37e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a3815b230 0x7f1a3815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a2f400640 1 -- 172.21.15.3:0/2948198856 learned_addr learned my addr 172.21.15.3:0/2948198856 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a3d200640 1 --2- 172.21.15.3:0/2948198856 >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 0x7f1a38071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.372+0000 7f1a37e00640 1 -- 172.21.15.3:0/2948198856 >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 msgr2=0x7f1a38071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.373+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2948198856 >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 0x7f1a38071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.373+0000 7f1a37e00640 1 -- 172.21.15.3:0/2948198856 >> v1:172.21.15.3:6789/0 conn(0x7f1a38072c50 legacy=0x7f1a380b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.373+0000 7f1a37e00640 1 -- 172.21.15.3:0/2948198856 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3815f2b0 con 0x7f1a3815b230 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.373+0000 7f1a3d200640 1 --2- 172.21.15.3:0/2948198856 >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 0x7f1a38071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:39.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2948198856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3815b230 0x7f1a3815c390 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1a380ccca0 tx=0x7f1a2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d69932986d32651 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a37400640 1 -- 172.21.15.3:0/2948198856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28037070 con 0x7f1a3815b230 2024-09-10T07:52:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a37400640 1 -- 172.21.15.3:0/2948198856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a28032440 con 0x7f1a3815b230 2024-09-10T07:52:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a37400640 1 -- 172.21.15.3:0/2948198856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28032810 con 0x7f1a3815b230 2024-09-10T07:52:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 -- 172.21.15.3:0/2948198856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3815b230 msgr2=0x7f1a3815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2948198856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3815b230 0x7f1a3815c390 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1a380ccca0 tx=0x7f1a2802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 -- 172.21.15.3:0/2948198856 shutdown_connections 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2948198856 >> v2:172.21.15.70:3300/0 conn(0x7f1a380b4c90 0x7f1a38071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2948198856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a3815b230 0x7f1a3815c390 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 -- 172.21.15.3:0/2948198856 >> 172.21.15.3:0/2948198856 conn(0x7f1a380b5cb0 msgr2=0x7f1a3806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.374+0000 7f1a3f200640 1 -- 172.21.15.3:0/2948198856 shutdown_connections 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.375+0000 7f1a3f200640 1 -- 172.21.15.3:0/2948198856 wait complete. 2024-09-10T07:52:39.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.376+0000 7f1a3f200640 1 Processor -- start 2024-09-10T07:52:39.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.376+0000 7f1a3f200640 1 -- start start 2024-09-10T07:52:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a380b4c90 0x7f1a38142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a3815b230 0x7f1a380ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3815b610 con 0x7f1a38072c50 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3815f870 con 0x7f1a3815b230 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3815b790 con 0x7f1a380b4c90 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44978/0 (socket says 172.21.15.3:44978) 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a37e00640 1 -- 172.21.15.3:0/2038555625 learned_addr learned my addr 172.21.15.3:0/2038555625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a380b4c90 0x7f1a38142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.377+0000 7f1a3d200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a3815b230 0x7f1a380ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 -- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a380b4c90 msgr2=0x7f1a38142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a380b4c90 0x7f1a38142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 -- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a3815b230 msgr2=0x7f1a380ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a3815b230 0x7f1a380ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 -- 172.21.15.3:0/2038555625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380b26b0 con 0x7f1a38072c50 2024-09-10T07:52:39.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a34c00640 1 -- 172.21.15.3:0/2038555625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28037040 con 0x7f1a38072c50 2024-09-10T07:52:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a34c00640 1 -- 172.21.15.3:0/2038555625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a28042070 con 0x7f1a38072c50 2024-09-10T07:52:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a34c00640 1 -- 172.21.15.3:0/2038555625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28030a50 con 0x7f1a38072c50 2024-09-10T07:52:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a380b0160 con 0x7f1a38072c50 2024-09-10T07:52:39.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.378+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a380b2920 con 0x7f1a38072c50 2024-09-10T07:52:39.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.380+0000 7f1a34c00640 1 -- 172.21.15.3:0/2038555625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a28030bf0 con 0x7f1a38072c50 2024-09-10T07:52:39.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.381+0000 7f1a34c00640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.381+0000 7f1a34c00640 1 -- 172.21.15.3:0/2038555625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1a280ab960 con 0x7f1a38072c50 2024-09-10T07:52:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.381+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.382+0000 7f1a421f0740 1 Processor -- start 2024-09-10T07:52:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.382+0000 7f1a421f0740 1 -- start start 2024-09-10T07:52:39.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.382+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 0x7f1a1406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a3815ab20 tx=0x7f1a20007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f641102c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f6410b8480 0x55f641103740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f641020660 con 0x55f6410d2800 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f640fc8680 con 0x55f6410d2420 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f64101c620 con 0x55f6410b8480 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44988/0 (socket says 172.21.15.3:44988) 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 -- 172.21.15.3:0/2401510769 learned_addr learned my addr 172.21.15.3:0/2401510769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a37e00640 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f641102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a3d200640 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f6410b8480 0x55f641103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 -- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f6410b8480 msgr2=0x55f641103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f6410b8480 0x55f641103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 -- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 msgr2=0x55f641102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f641102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.383+0000 7f1a2f400640 1 -- 172.21.15.3:0/2401510769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f640dfd7d0 con 0x55f6410d2800 2024-09-10T07:52:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a2f400640 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1a200016c0 tx=0x7f1a20013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0618db3e7b9dabd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a2cc00640 1 -- 172.21.15.3:0/2401510769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20026070 con 0x55f6410d2800 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a2cc00640 1 -- 172.21.15.3:0/2401510769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a20013840 con 0x55f6410d2800 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a2cc00640 1 -- 172.21.15.3:0/2401510769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20013bf0 con 0x55f6410d2800 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a421f0740 1 -- 172.21.15.3:0/2401510769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 msgr2=0x55f641103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.384+0000 7f1a421f0740 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1a200016c0 tx=0x7f1a20013000 comp rx=0 tx=0).stop 2024-09-10T07:52:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 -- 172.21.15.3:0/2401510769 shutdown_connections 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f6410b8480 0x55f641103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641103200 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 --2- 172.21.15.3:0/2401510769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f641102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 -- 172.21.15.3:0/2401510769 >> 172.21.15.3:0/2401510769 conn(0x55f6410b9220 msgr2=0x55f641101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 -- 172.21.15.3:0/2401510769 shutdown_connections 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 -- 172.21.15.3:0/2401510769 wait complete. 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.385+0000 7f1a421f0740 1 -- start start 2024-09-10T07:52:39.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.386+0000 7f1a421f0740 1 Processor -- start 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.386+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.386+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.386+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6411126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a421f0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f6410b73e0 con 0x55f6410d2800 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f640fa94c0 con 0x55f6410d2420 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f6410b76c0 con 0x55f641112230 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a421f0740 10 client.0 _ll_register_callbacks cb 0x55f641018e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44990/0 (socket says 172.21.15.3:44990) 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 -- 172.21.15.3:0/66587418 learned_addr learned my addr 172.21.15.3:0/66587418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a3d200640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6411126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a37e00640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 -- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 msgr2=0x55f6411126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6411126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 -- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 msgr2=0x55f6410e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 -- 172.21.15.3:0/66587418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f640d333f0 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a2f400640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000f60 tx=0x7f1a20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e5500ef4428634 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.387+0000 7f1a25600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20026070 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1a20021740 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a20013840 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f1a20038050 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a421f0740 10 client.7764 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55f640cf30f0 con 0x55f6410d2800 2024-09-10T07:52:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a20013840 con 0x55f6410d2800 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.388+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f1a2003b020 con 0x55f6410d2800 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 10 client.7764 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 10 client.7764 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 20 client.7764 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f64111dae0 0x55f64111fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 4 client.7764 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64102dd90 con 0x55f64111dae0 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a37e00640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f64111dae0 0x55f64111fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a37e00640 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f64111dae0 0x55f64111fed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33396838aeedc1ac server_cookie=cede69c99c70156e in_seq=0 out_seq=0 2024-09-10T07:52:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.389+0000 7f1a25600640 10 client.7764 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a25600640 1 -- 172.21.15.3:0/66587418 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f64102dd90 con 0x55f64111dae0 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a25600640 10 client.7764 handle_command_reply: tid=0 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 client.7764 shutdown 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 msgr2=0x55f641111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000f60 tx=0x7f1a20000bc0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 shutdown_connections 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f64111dae0 0x55f64111fed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6411126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f641111cf0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 --2- 172.21.15.3:0/66587418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a25600640 0 client.7764 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 >> 172.21.15.3:0/66587418 conn(0x55f6410b8480 msgr2=0x55f6410b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.390+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 shutdown_connections 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.391+0000 7f1a421f0740 1 -- 172.21.15.3:0/66587418 wait complete. 2024-09-10T07:52:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.391+0000 7f1a421f0740 20 client.7764 trim_cache size 0 max 16384 2024-09-10T07:52:39.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.409+0000 7f1a421f0740 1 Processor -- start 2024-09-10T07:52:39.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.409+0000 7f1a421f0740 1 -- start start 2024-09-10T07:52:39.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a38200c60 con 0x55f6410d2800 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381ffe20 con 0x55f6410d2420 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1a38200530 con 0x55f641112230 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44992/0 (socket says 172.21.15.3:44992) 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 -- 172.21.15.3:0/3551935257 learned_addr learned my addr 172.21.15.3:0/3551935257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a37e00640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a3d200640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 -- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 msgr2=0x55f6410bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410bad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 -- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 msgr2=0x55f6410e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a2f400640 1 -- 172.21.15.3:0/3551935257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6410b3050 con 0x55f6410d2800 2024-09-10T07:52:39.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.410+0000 7f1a37e00640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004b80 tx=0x7f1a20029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e11d2dcd0165e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a27e00640 1 -- 172.21.15.3:0/3551935257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20021740 con 0x55f6410d2800 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a27e00640 1 -- 172.21.15.3:0/3551935257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a20030070 con 0x55f6410d2800 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a27e00640 1 -- 172.21.15.3:0/3551935257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20021270 con 0x55f6410d2800 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 -- 172.21.15.3:0/3551935257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 msgr2=0x55f6410ba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004b80 tx=0x7f1a20029000 comp rx=0 tx=0).stop 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 -- 172.21.15.3:0/3551935257 shutdown_connections 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2800 0x55f6410ba7c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3551935257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2420 0x55f6410e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 -- 172.21.15.3:0/3551935257 >> 172.21.15.3:0/3551935257 conn(0x55f6410b8480 msgr2=0x55f6410fb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.411+0000 7f1a421f0740 1 -- 172.21.15.3:0/3551935257 shutdown_connections 2024-09-10T07:52:39.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.412+0000 7f1a421f0740 1 -- 172.21.15.3:0/3551935257 wait complete. 2024-09-10T07:52:39.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.412+0000 7f1a421f0740 1 -- start start 2024-09-10T07:52:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.412+0000 7f1a421f0740 1 Processor -- start 2024-09-10T07:52:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 0x55f6410d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3815d930 con 0x55f6410d2420 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f6410fd7f0 con 0x55f6410d2800 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f6410d1660 con 0x55f641112230 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.413+0000 7f1a421f0740 10 client.0 _ll_register_callbacks cb 0x7f1a20024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 -- 172.21.15.3:0/3801416523 learned_addr learned my addr 172.21.15.3:0/3801416523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 0x55f6410d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a3d200640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 -- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 msgr2=0x55f6410d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 -- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 msgr2=0x55f6410d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 0x55f6410d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 -- 172.21.15.3:0/3801416523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6410fb030 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 0x55f6410d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a37e00640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1a2802a1e0 tx=0x7f1a28049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1c41739fab101c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a25600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28095070 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f1a280761e0 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.414+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a20030070 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f1a280a0050 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 10 client.7770 fetch_fsmap learned FSMap version 95 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55f64110e5d0 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a20030070 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 95) ==== 3558+0+0 (secure 0 0 0) 0x7f1a280a3020 con 0x55f6410d2420 2024-09-10T07:52:39.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 10 client.7770 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 10 client.7770 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 20 client.7770 populate_metadata read hostname 'smithi003' 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f641101f80 0x55f6410e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 4 client.7770 mds_command: new command op to 7239 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 --> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6410280e0 con 0x55f641101f80 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.415+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f641101f80 0x55f6410e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a2f400640 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f641101f80 0x55f6410e7990 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6636ace1868186a server_cookie=788b96b7c67b7d15 in_seq=0 out_seq=0 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a25600640 10 client.7770 ms_handle_connect on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a25600640 1 -- 172.21.15.3:0/3801416523 <== mds.0 v2:172.21.15.3:6835/2433616379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f6410280e0 con 0x55f641101f80 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a25600640 10 client.7770 handle_command_reply: tid=0 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 client.7770 shutdown 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 msgr2=0x55f641100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1a2802a1e0 tx=0x7f1a28049040 comp rx=0 tx=0).stop 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 shutdown_connections 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:6835/2433616379,v1:172.21.15.3:6837/2433616379] conn(0x55f641101f80 0x55f6410e7990 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f641112230 0x55f6410d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f6410d2800 0x55f6410d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a25600640 0 client.7770 ms_handle_reset on v2:172.21.15.3:6835/2433616379 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 --2- 172.21.15.3:0/3801416523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6410d2420 0x55f641100780 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 >> 172.21.15.3:0/3801416523 conn(0x55f6410b8480 msgr2=0x55f6410e6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 shutdown_connections 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.416+0000 7f1a421f0740 1 -- 172.21.15.3:0/3801416523 wait complete. 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a421f0740 20 client.7770 trim_cache size 0 max 16384 2024-09-10T07:52:39.421 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "9d2337a3-568d-475d-b05c-cfb86e4116fd": { 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "9d2337a3-568d-475d-b05c-cfb86e4116fd", 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:52:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 msgr2=0x7f1a1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 0x7f1a1406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a3815ab20 tx=0x7f1a20007e80 comp rx=0 tx=0).stop 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 msgr2=0x7f1a38147ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a280046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 shutdown_connections 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a1406be30 0x7f1a1406e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a3815b230 0x7f1a380ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a380b4c90 0x7f1a38142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 --2- 172.21.15.3:0/2038555625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a38072c50 0x7f1a38147ef0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 >> 172.21.15.3:0/2038555625 conn(0x7f1a380b5cb0 msgr2=0x7f1a38151280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 shutdown_connections 2024-09-10T07:52:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.417+0000 7f1a3f200640 1 -- 172.21.15.3:0/2038555625 wait complete. 2024-09-10T07:52:39.431 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:52:39.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:39.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.590+0000 7f2b4d200640 1 Processor -- start 2024-09-10T07:52:39.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.590+0000 7f2b4d200640 1 -- start start 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b4815b330 0x7f2b480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b480b2540 0x7f2b480cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 0x7f2b4815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2b480b29a0 con 0x7f2b4815b330 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4815bac0 con 0x7f2b480cfb00 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.591+0000 7f2b4d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4815bd30 con 0x7f2b480b2540 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b4815b330 0x7f2b480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2b4815b330 0x7f2b480b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45018/0 (socket says 172.21.15.3:45018) 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 -- 172.21.15.3:0/2754184420 learned_addr learned my addr 172.21.15.3:0/2754184420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47e00640 1 --2- 172.21.15.3:0/2754184420 >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 0x7f2b4815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b46a00640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b480b2540 0x7f2b480cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 -- 172.21.15.3:0/2754184420 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b480b2540 msgr2=0x7f2b480cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b480b2540 0x7f2b480cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 -- 172.21.15.3:0/2754184420 >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 msgr2=0x7f2b4815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:39.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 --2- 172.21.15.3:0/2754184420 >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 0x7f2b4815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47400640 1 -- 172.21.15.3:0/2754184420 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4815c010 con 0x7f2b4815b330 2024-09-10T07:52:39.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.592+0000 7f2b47e00640 1 --2- 172.21.15.3:0/2754184420 >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 0x7f2b4815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.593+0000 7f2b47400640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b4815b330 0x7f2b480b2000 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2b480b2c30 tx=0x7f2b3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d952f1acc3a8d26b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.593+0000 7f2b46000640 1 -- 172.21.15.3:0/2754184420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38037070 con 0x7f2b4815b330 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.593+0000 7f2b46000640 1 -- 172.21.15.3:0/2754184420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38032440 con 0x7f2b4815b330 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.593+0000 7f2b46000640 1 -- 172.21.15.3:0/2754184420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38032810 con 0x7f2b4815b330 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 -- 172.21.15.3:0/2754184420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b4815b330 msgr2=0x7f2b480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b4815b330 0x7f2b480b2000 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2b480b2c30 tx=0x7f2b3802a1e0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 -- 172.21.15.3:0/2754184420 shutdown_connections 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 --2- 172.21.15.3:0/2754184420 >> v2:172.21.15.70:3300/0 conn(0x7f2b480cfb00 0x7f2b4815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b480b2540 0x7f2b480cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 --2- 172.21.15.3:0/2754184420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b4815b330 0x7f2b480b2000 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 -- 172.21.15.3:0/2754184420 >> 172.21.15.3:0/2754184420 conn(0x7f2b4806bb20 msgr2=0x7f2b4806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 -- 172.21.15.3:0/2754184420 shutdown_connections 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.594+0000 7f2b4d200640 1 -- 172.21.15.3:0/2754184420 wait complete. 2024-09-10T07:52:39.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.596+0000 7f2b4d200640 1 Processor -- start 2024-09-10T07:52:39.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.596+0000 7f2b4d200640 1 -- start start 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b480b2540 0x7f2b4814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b4815b330 0x7f2b480c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2b480ce1c0 con 0x7f2b480cfb00 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4815c5d0 con 0x7f2b480b2540 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b4d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4815b710 con 0x7f2b4815b330 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45034/0 (socket says 172.21.15.3:45034) 2024-09-10T07:52:39.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 -- 172.21.15.3:0/80889105 learned_addr learned my addr 172.21.15.3:0/80889105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b47400640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b480b2540 0x7f2b4814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b47e00640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b4815b330 0x7f2b480c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 -- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b4815b330 msgr2=0x7f2b480c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b4815b330 0x7f2b480c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 -- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b480b2540 msgr2=0x7f2b4814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b480b2540 0x7f2b4814f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.597+0000 7f2b46a00640 1 -- 172.21.15.3:0/80889105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4814fb60 con 0x7f2b480cfb00 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b46a00640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400d830 tx=0x7f2b3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34018030 con 0x7f2b480cfb00 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3400be10 con 0x7f2b480cfb00 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340098b0 con 0x7f2b480cfb00 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b480c5e00 con 0x7f2b480cfb00 2024-09-10T07:52:39.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.598+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b480c63c0 con 0x7f2b480cfb00 2024-09-10T07:52:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.599+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b480cff90 con 0x7f2b480cfb00 2024-09-10T07:52:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.600+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b3401b070 con 0x7f2b480cfb00 2024-09-10T07:52:39.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.601+0000 7f2b3f400640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.601+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f2b3408aef0 con 0x7f2b480cfb00 2024-09-10T07:52:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.601+0000 7f2b47400640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:39.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.602+0000 7f2b47400640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 0x7f2b2806e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2b38034000 tx=0x7f2b3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:39.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.605+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2b34059240 con 0x7f2b480cfb00 2024-09-10T07:52:39.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.831+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b48072a50 con 0x7f2b480cfb00 2024-09-10T07:52:39.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.831+0000 7f2b3f400640 1 -- 172.21.15.3:0/80889105 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f2b3405d540 con 0x7f2b480cfb00 2024-09-10T07:52:39.831 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:39.831 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:39.831 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 msgr2=0x7f2b2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 0x7f2b2806e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2b38034000 tx=0x7f2b3802e000 comp rx=0 tx=0).stop 2024-09-10T07:52:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 msgr2=0x7f2b48159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400d830 tx=0x7f2b3400dcf0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 shutdown_connections 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2b2806be30 0x7f2b2806e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2b4815b330 0x7f2b480c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b480cfb00 0x7f2b48159c20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 --2- 172.21.15.3:0/80889105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b480b2540 0x7f2b4814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.833+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 >> 172.21.15.3:0/80889105 conn(0x7f2b4806bb20 msgr2=0x7f2b480b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.834+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 shutdown_connections 2024-09-10T07:52:39.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:39.834+0000 7f2b4d200640 1 -- 172.21.15.3:0/80889105 wait complete. 2024-09-10T07:52:39.849 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:39.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.005+0000 7f8b81c00640 1 Processor -- start 2024-09-10T07:52:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.005+0000 7f8b81c00640 1 -- start start 2024-09-10T07:52:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b7c15b410 0x7f8b7c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b7c0cf8f0 0x7f8b7c15f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c06f250 con 0x7f8b7c15b410 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c0ccd40 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.006+0000 7f8b81c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c0cd020 con 0x7f8b7c0b4aa0 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b7c15b410 0x7f8b7c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b7c15b410 0x7f8b7c06ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45048/0 (socket says 172.21.15.3:45048) 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 -- 172.21.15.3:0/1864716887 learned_addr learned my addr 172.21.15.3:0/1864716887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7a000640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7b400640 1 --2- 172.21.15.3:0/1864716887 >> v2:172.21.15.70:3300/0 conn(0x7f8b7c0cf8f0 0x7f8b7c15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 -- 172.21.15.3:0/1864716887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c0b4aa0 msgr2=0x7f8b7c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 -- 172.21.15.3:0/1864716887 >> v2:172.21.15.70:3300/0 conn(0x7f8b7c0cf8f0 msgr2=0x7f8b7c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/1864716887 >> v2:172.21.15.70:3300/0 conn(0x7f8b7c0cf8f0 0x7f8b7c15f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.007+0000 7f8b7aa00640 1 -- 172.21.15.3:0/1864716887 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0cd300 con 0x7f8b7c15b410 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.008+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c15b410 0x7f8b7c06ed10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c070350 tx=0x7f8b6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176c5b74b81fa880 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.008+0000 7f8b79600640 1 -- 172.21.15.3:0/1864716887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c037070 con 0x7f8b7c15b410 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.008+0000 7f8b79600640 1 -- 172.21.15.3:0/1864716887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6c032440 con 0x7f8b7c15b410 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.008+0000 7f8b79600640 1 -- 172.21.15.3:0/1864716887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c032810 con 0x7f8b7c15b410 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 -- 172.21.15.3:0/1864716887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c15b410 msgr2=0x7f8b7c06ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c15b410 0x7f8b7c06ed10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c070350 tx=0x7f8b6c02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 -- 172.21.15.3:0/1864716887 shutdown_connections 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 --2- 172.21.15.3:0/1864716887 >> v2:172.21.15.70:3300/0 conn(0x7f8b7c0cf8f0 0x7f8b7c15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 --2- 172.21.15.3:0/1864716887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c15b410 0x7f8b7c06ed10 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 -- 172.21.15.3:0/1864716887 >> 172.21.15.3:0/1864716887 conn(0x7f8b7c06cfc0 msgr2=0x7f8b7c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 -- 172.21.15.3:0/1864716887 shutdown_connections 2024-09-10T07:52:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.009+0000 7f8b81c00640 1 -- 172.21.15.3:0/1864716887 wait complete. 2024-09-10T07:52:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.011+0000 7f8b81c00640 1 Processor -- start 2024-09-10T07:52:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.011+0000 7f8b81c00640 1 -- start start 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 0x7f8b7c0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0cdba0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0cd8c0 con 0x7f8b7c0b4aa0 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b81c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c072540 con 0x7f8b7c15b410 2024-09-10T07:52:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45062/0 (socket says 172.21.15.3:45062) 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 -- 172.21.15.3:0/3110735398 learned_addr learned my addr 172.21.15.3:0/3110735398 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7b400640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 0x7f8b7c0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 -- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 msgr2=0x7f8b7c0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 0x7f8b7c0c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 -- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 msgr2=0x7f8b7c0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7a000640 1 -- 172.21.15.3:0/3110735398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c15d6d0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.012+0000 7f8b7b400640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 0x7f8b7c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:40.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b7a000640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d7d0 tx=0x7f8b6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800b6c0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b68019070 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800ba80 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c15d9b0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.013+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c15df70 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.014+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7c15b7f0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.015+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b6800bc20 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.016+0000 7f8b72a00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 0x7f8b5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:40.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.017+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f8b6808b390 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.017+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 0x7f8b5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.017+0000 7f8b7aa00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 0x7f8b5c06e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c034000 tx=0x7f8b6c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.021+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b6801c360 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.279+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8b7c0b4630 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.280+0000 7f8b72a00640 1 -- 172.21.15.3:0/3110735398 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7f8b68059ca0 con 0x7f8b7c0cf8f0 2024-09-10T07:52:40.280 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:40.280 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":393,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":270024704,"bytes_avail":772824088576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6995,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-10T07:52:37.572776+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 msgr2=0x7f8b5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 0x7f8b5c06e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c034000 tx=0x7f8b6c02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 msgr2=0x7f8b7c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:40.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d7d0 tx=0x7f8b6800dc90 comp rx=0 tx=0).stop 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 shutdown_connections 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b5c06be30 0x7f8b5c06e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b7c15b410 0x7f8b7c0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b7c0cf8f0 0x7f8b7c0c86b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 --2- 172.21.15.3:0/3110735398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b7c0b4aa0 0x7f8b7c0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.282+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 >> 172.21.15.3:0/3110735398 conn(0x7f8b7c06cfc0 msgr2=0x7f8b7c0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.283+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 shutdown_connections 2024-09-10T07:52:40.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:40.283+0000 7f8b81c00640 1 -- 172.21.15.3:0/3110735398 wait complete. 2024-09-10T07:52:40.298 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:52:41.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.467+0000 7f2880c00640 1 Processor -- start 2024-09-10T07:52:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.467+0000 7f2880c00640 1 -- start start 2024-09-10T07:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.468+0000 7f2880c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 0x7f287c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.468+0000 7f2880c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.468+0000 7f2880c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287c0cfdd0 con 0x7f287c160af0 2024-09-10T07:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.468+0000 7f2880c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f287c15cc90 con 0x7f287c160710 2024-09-10T07:52:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.468+0000 7f2880c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f287c15cf70 con 0x7f287c0d07c0 2024-09-10T07:52:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2879400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f287c160af0 0x7f287c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51814/0 (socket says 172.21.15.3:51814) 2024-09-10T07:52:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2879400640 1 -- 172.21.15.3:0/3725736653 learned_addr learned my addr 172.21.15.3:0/3725736653 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f287a800640 1 --2- 172.21.15.3:0/3725736653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2879e00640 1 --2- 172.21.15.3:0/3725736653 >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 0x7f287c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2878a00640 1 -- 172.21.15.3:0/3725736653 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4221290984 0 0) 0x7f287c0cfdd0 con 0x7f287c160af0 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2878a00640 1 -- 172.21.15.3:0/3725736653 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2860003520 con 0x7f287c160af0 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f287a800640 1 -- 172.21.15.3:0/3725736653 >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 msgr2=0x7f287c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f287a800640 1 --2- 172.21.15.3:0/3725736653 >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 0x7f287c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f287a800640 1 -- 172.21.15.3:0/3725736653 >> v1:172.21.15.3:6789/0 conn(0x7f287c160af0 legacy=0x7f287c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f287a800640 1 -- 172.21.15.3:0/3725736653 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c15d250 con 0x7f287c0d07c0 2024-09-10T07:52:41.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.469+0000 7f2879e00640 1 --2- 172.21.15.3:0/3725736653 >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 0x7f287c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f287a800640 1 --2- 172.21.15.3:0/3725736653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c072070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9c85b1fdab79b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f2878a00640 1 -- 172.21.15.3:0/3725736653 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874007740 con 0x7f287c0d07c0 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f2878a00640 1 -- 172.21.15.3:0/3725736653 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28740078e0 con 0x7f287c0d07c0 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f2878a00640 1 -- 172.21.15.3:0/3725736653 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874007cb0 con 0x7f287c0d07c0 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f2880c00640 1 -- 172.21.15.3:0/3725736653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 msgr2=0x7f287c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.471+0000 7f2880c00640 1 --2- 172.21.15.3:0/3725736653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c072070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287402a660 comp rx=0 tx=0).stop 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 -- 172.21.15.3:0/3725736653 shutdown_connections 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 --2- 172.21.15.3:0/3725736653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c072070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 --2- 172.21.15.3:0/3725736653 >> v2:172.21.15.70:3300/0 conn(0x7f287c160710 0x7f287c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 -- 172.21.15.3:0/3725736653 >> 172.21.15.3:0/3725736653 conn(0x7f287c06d000 msgr2=0x7f287c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 -- 172.21.15.3:0/3725736653 shutdown_connections 2024-09-10T07:52:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.472+0000 7f2880c00640 1 -- 172.21.15.3:0/3725736653 wait complete. 2024-09-10T07:52:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.473+0000 7f2880c00640 1 Processor -- start 2024-09-10T07:52:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.473+0000 7f2880c00640 1 -- start start 2024-09-10T07:52:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f287c160710 0x7f287c0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f287c15d770 con 0x7f287c160af0 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0cf800 con 0x7f287c160710 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.474+0000 7f2880c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f287c1f6900 con 0x7f287c0d07c0 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45072/0 (socket says 172.21.15.3:45072) 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 -- 172.21.15.3:0/2226179997 learned_addr learned my addr 172.21.15.3:0/2226179997 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f2879400640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f287c160710 0x7f287c0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f2879e00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 -- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 msgr2=0x7f287c151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 -- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f287c160710 msgr2=0x7f287c0ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f287c160710 0x7f287c0ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 -- 172.21.15.3:0/2226179997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1f5d80 con 0x7f287c160af0 2024-09-10T07:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f2879e00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.475+0000 7f287a800640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2874009eb0 tx=0x7f2874004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.476+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874040070 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.476+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2874031bc0 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.476+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874009450 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.476+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c1f6060 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.476+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1f6620 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.477+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c0cc910 con 0x7f287c160af0 2024-09-10T07:52:41.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.478+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2874039040 con 0x7f287c160af0 2024-09-10T07:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.479+0000 7f2872000640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 0x7f285c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.479+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f28740ac580 con 0x7f287c160af0 2024-09-10T07:52:41.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.479+0000 7f2879e00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 0x7f285c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.480+0000 7f2879e00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 0x7f285c06e320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ba0 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.483+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f287407b040 con 0x7f287c160af0 2024-09-10T07:52:41.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.712+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f287c0afd50 con 0x7f287c160af0 2024-09-10T07:52:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.712+0000 7f2872000640 1 -- 172.21.15.3:0/2226179997 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f287407a940 con 0x7f287c160af0 2024-09-10T07:52:41.712 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:41.712 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:41.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.714+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 msgr2=0x7f285c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.714+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 0x7f285c06e320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ba0 tx=0x7f286c008040 comp rx=0 tx=0).stop 2024-09-10T07:52:41.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 msgr2=0x7f287c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2874009eb0 tx=0x7f2874004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 shutdown_connections 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f285c06be60 0x7f285c06e320 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f287c160af0 0x7f287c0aff70 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f287c160710 0x7f287c0ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 --2- 172.21.15.3:0/2226179997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f287c0d07c0 0x7f287c151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 >> 172.21.15.3:0/2226179997 conn(0x7f287c06d000 msgr2=0x7f287c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 shutdown_connections 2024-09-10T07:52:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.715+0000 7f2880c00640 1 -- 172.21.15.3:0/2226179997 wait complete. 2024-09-10T07:52:41.730 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:41.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.886+0000 7fcc59800640 1 Processor -- start 2024-09-10T07:52:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.886+0000 7fcc59800640 1 -- start start 2024-09-10T07:52:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc59800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc54070f30 0x7fcc54071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc59800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc540cfdc0 0x7fcc5415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc59800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc540cd7f0 con 0x7fcc5415c720 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc59800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcc54072210 con 0x7fcc54070f30 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc59800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc540cf880 con 0x7fcc540cfdc0 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc4a000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc54070f30 0x7fcc54071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc4a000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc54070f30 0x7fcc54071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc4a000640 1 -- 172.21.15.3:0/3203520713 learned_addr learned my addr 172.21.15.3:0/3203520713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc53400640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc540cfdc0 0x7fcc5415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc4a000640 1 -- 172.21.15.3:0/3203520713 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc540cfdc0 msgr2=0x7fcc5415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.888+0000 7fcc4a000640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc540cfdc0 0x7fcc5415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.889+0000 7fcc4a000640 1 -- 172.21.15.3:0/3203520713 >> v1:172.21.15.70:6789/0 conn(0x7fcc5415c720 legacy=0x7fcc54073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.889+0000 7fcc4a000640 1 -- 172.21.15.3:0/3203520713 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc540cda90 con 0x7fcc54070f30 2024-09-10T07:52:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc4a000640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc54070f30 0x7fcc54071cd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009870 tx=0x7fcc3c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db922325403dcb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc52000640 1 -- 172.21.15.3:0/3203520713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c037070 con 0x7fcc54070f30 2024-09-10T07:52:41.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc52000640 1 -- 172.21.15.3:0/3203520713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c032440 con 0x7fcc54070f30 2024-09-10T07:52:41.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc52000640 1 -- 172.21.15.3:0/3203520713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c032810 con 0x7fcc54070f30 2024-09-10T07:52:41.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc59800640 1 -- 172.21.15.3:0/3203520713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc54070f30 msgr2=0x7fcc54071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc59800640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc54070f30 0x7fcc54071cd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009870 tx=0x7fcc3c02a250 comp rx=0 tx=0).stop 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.890+0000 7fcc59800640 1 -- 172.21.15.3:0/3203520713 shutdown_connections 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.891+0000 7fcc59800640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc540cfdc0 0x7fcc5415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.891+0000 7fcc59800640 1 --2- 172.21.15.3:0/3203520713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc54070f30 0x7fcc54071cd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.891+0000 7fcc59800640 1 -- 172.21.15.3:0/3203520713 >> 172.21.15.3:0/3203520713 conn(0x7fcc5406cfc0 msgr2=0x7fcc5406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.891+0000 7fcc59800640 1 -- 172.21.15.3:0/3203520713 shutdown_connections 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.891+0000 7fcc59800640 1 -- 172.21.15.3:0/3203520713 wait complete. 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.892+0000 7fcc59800640 1 Processor -- start 2024-09-10T07:52:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.892+0000 7fcc59800640 1 -- start start 2024-09-10T07:52:41.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 0x7fcc540b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 0x7fcc54143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54073d20 con 0x7fcc540cfdc0 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5406ed10 con 0x7fcc5415c720 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc59800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc54073ea0 con 0x7fcc54070f30 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45088/0 (socket says 172.21.15.3:45088) 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.893+0000 7fcc4a000640 1 -- 172.21.15.3:0/2871417808 learned_addr learned my addr 172.21.15.3:0/2871417808 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc52a00640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 0x7fcc540b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc53400640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 0x7fcc54143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 -- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 msgr2=0x7fcc540b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 0x7fcc540b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 -- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 msgr2=0x7fcc54143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 0x7fcc54143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 -- 172.21.15.3:0/2871417808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5415d7c0 con 0x7fcc540cfdc0 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc53400640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 0x7fcc54143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc52a00640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 0x7fcc540b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:41.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4a000640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c000c00 tx=0x7fcc3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c041020 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c0306c0 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.894+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5415daa0 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.895+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c037040 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.895+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5415d3b0 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.896+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc54071cd0 con 0x7fcc540cfdc0 2024-09-10T07:52:41.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.897+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc3c044020 con 0x7fcc540cfdc0 2024-09-10T07:52:41.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.897+0000 7fcc4b400640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 0x7fcc3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:41.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.897+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fcc3c0ab780 con 0x7fcc540cfdc0 2024-09-10T07:52:41.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.898+0000 7fcc52a00640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 0x7fcc3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:41.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.898+0000 7fcc52a00640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 0x7fcc3006e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc44001700 tx=0x7fcc44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:41.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:41.902+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc3c04b030 con 0x7fcc540cfdc0 2024-09-10T07:52:42.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.162+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcc540cf2b0 con 0x7fcc540cfdc0 2024-09-10T07:52:42.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.162+0000 7fcc4b400640 1 -- 172.21.15.3:0/2871417808 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7fcc3c07a090 con 0x7fcc540cfdc0 2024-09-10T07:52:42.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:42.162 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":395,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":270024704,"bytes_avail":772824088576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6825,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-10T07:52:37.572776+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 msgr2=0x7fcc3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 0x7fcc3006e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc44001700 tx=0x7fcc44009040 comp rx=0 tx=0).stop 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 msgr2=0x7fcc5415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c000c00 tx=0x7fcc3c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 shutdown_connections 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcc3006be30 0x7fcc3006e2f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc5415c720 0x7fcc54143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc540cfdc0 0x7fcc5415e580 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 --2- 172.21.15.3:0/2871417808 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc54070f30 0x7fcc540b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.164+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 >> 172.21.15.3:0/2871417808 conn(0x7fcc5406cfc0 msgr2=0x7fcc540b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.165+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 shutdown_connections 2024-09-10T07:52:42.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:42.165+0000 7fcc59800640 1 -- 172.21.15.3:0/2871417808 wait complete. 2024-09-10T07:52:42.181 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:52:43.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:43.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.349+0000 7fab6aa00640 1 Processor -- start 2024-09-10T07:52:43.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.349+0000 7fab6aa00640 1 -- start start 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.350+0000 7fab6aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b110 0x7fab640b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.350+0000 7fab6aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab640d08f0 0x7fab640cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.350+0000 7fab6aa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab640b4250 con 0x7fab6415b4f0 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.350+0000 7fab6aa00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fab640b44f0 con 0x7fab640d08f0 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.350+0000 7fab6aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab640ccd80 con 0x7fab6415b110 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab640d08f0 0x7fab640cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab640d08f0 0x7fab640cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45098/0 (socket says 172.21.15.3:45098) 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 -- 172.21.15.3:0/730473040 learned_addr learned my addr 172.21.15.3:0/730473040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab63e00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b110 0x7fab640b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 -- 172.21.15.3:0/730473040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b110 msgr2=0x7fab640b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b110 0x7fab640b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 -- 172.21.15.3:0/730473040 >> v1:172.21.15.70:6789/0 conn(0x7fab6415b4f0 legacy=0x7fab640d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.351+0000 7fab68a00640 1 -- 172.21.15.3:0/730473040 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab640cd060 con 0x7fab640d08f0 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.352+0000 7fab68a00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab640cdb40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009e50 tx=0x7fab5c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e821df3696154 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.352+0000 7fab62a00640 1 -- 172.21.15.3:0/730473040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c036070 con 0x7fab640d08f0 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab62a00640 1 -- 172.21.15.3:0/730473040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c031600 con 0x7fab640d08f0 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab62a00640 1 -- 172.21.15.3:0/730473040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0319d0 con 0x7fab640d08f0 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 -- 172.21.15.3:0/730473040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 msgr2=0x7fab640cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab640cdb40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009e50 tx=0x7fab5c02a710 comp rx=0 tx=0).stop 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 -- 172.21.15.3:0/730473040 shutdown_connections 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab640cdb40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 --2- 172.21.15.3:0/730473040 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b110 0x7fab640b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 -- 172.21.15.3:0/730473040 >> 172.21.15.3:0/730473040 conn(0x7fab6406d000 msgr2=0x7fab6406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:43.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 -- 172.21.15.3:0/730473040 shutdown_connections 2024-09-10T07:52:43.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.353+0000 7fab6aa00640 1 -- 172.21.15.3:0/730473040 wait complete. 2024-09-10T07:52:43.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.355+0000 7fab6aa00640 1 Processor -- start 2024-09-10T07:52:43.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.355+0000 7fab6aa00640 1 -- start start 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6415b110 0x7fab64150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b4f0 0x7fab6415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab64073d10 con 0x7fab640d08f0 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fab640cd620 con 0x7fab6415b110 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab6aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fab64073e90 con 0x7fab6415b4f0 2024-09-10T07:52:43.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45100/0 (socket says 172.21.15.3:45100) 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 -- 172.21.15.3:0/2647363622 learned_addr learned my addr 172.21.15.3:0/2647363622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63400640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6415b110 0x7fab64150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab68a00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b4f0 0x7fab6415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 -- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b4f0 msgr2=0x7fab6415e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b4f0 0x7fab6415e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 -- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6415b110 msgr2=0x7fab64150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.356+0000 7fab63e00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6415b110 0x7fab64150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab63e00640 1 -- 172.21.15.3:0/2647363622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64152a80 con 0x7fab640d08f0 2024-09-10T07:52:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab63e00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fab5400d830 tx=0x7fab5400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400b6c0 con 0x7fab640d08f0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54019070 con 0x7fab640d08f0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab64151f00 con 0x7fab640d08f0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400ba80 con 0x7fab640d08f0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.357+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab641524c0 con 0x7fab640d08f0 2024-09-10T07:52:43.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.358+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6406e330 con 0x7fab640d08f0 2024-09-10T07:52:43.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.359+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab5401e460 con 0x7fab640d08f0 2024-09-10T07:52:43.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.360+0000 7fab5be00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 0x7fab4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.360+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fab5408b5b0 con 0x7fab640d08f0 2024-09-10T07:52:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.360+0000 7fab63400640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 0x7fab4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.361+0000 7fab63400640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 0x7fab4406e320 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab64073080 tx=0x7fab50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.364+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab6406e330 con 0x7fab640d08f0 2024-09-10T07:52:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.585+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab640d1380 con 0x7fab640d08f0 2024-09-10T07:52:43.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.586+0000 7fab5be00640 1 -- 172.21.15.3:0/2647363622 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7fab640d1380 con 0x7fab640d08f0 2024-09-10T07:52:43.585 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:43.586 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:43.586 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:43.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 msgr2=0x7fab4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 0x7fab4406e320 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab64073080 tx=0x7fab50009040 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 msgr2=0x7fab64151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fab5400d830 tx=0x7fab5400dcf0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 shutdown_connections 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fab4406be60 0x7fab4406e320 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fab6415b4f0 0x7fab6415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6415b110 0x7fab64150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 --2- 172.21.15.3:0/2647363622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab640d08f0 0x7fab64151840 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 >> 172.21.15.3:0/2647363622 conn(0x7fab6406d000 msgr2=0x7fab640aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 shutdown_connections 2024-09-10T07:52:43.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.588+0000 7fab6aa00640 1 -- 172.21.15.3:0/2647363622 wait complete. 2024-09-10T07:52:43.603 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:43.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:43.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.760+0000 7f0825a00640 1 Processor -- start 2024-09-10T07:52:43.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.760+0000 7f0825a00640 1 -- start start 2024-09-10T07:52:43.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.761+0000 7f0825a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08200af700 0x7f08200cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.761+0000 7f0825a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200ce9b0 0x7f0820162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.761+0000 7f0825a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08200cdf70 con 0x7f08200cfd40 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.761+0000 7f0825a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f08200ae940 con 0x7f08200af700 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.761+0000 7f0825a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08200aec20 con 0x7f08200ce9b0 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08200af700 0x7f08200cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08200af700 0x7f08200cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45114/0 (socket says 172.21.15.3:45114) 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 -- 172.21.15.3:0/3372773550 learned_addr learned my addr 172.21.15.3:0/3372773550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081f400640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200ce9b0 0x7f0820162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 -- 172.21.15.3:0/3372773550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200ce9b0 msgr2=0x7f0820162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200ce9b0 0x7f0820162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 -- 172.21.15.3:0/3372773550 >> v1:172.21.15.70:6789/0 conn(0x7f08200cfd40 legacy=0x7f08200ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081e000640 1 -- 172.21.15.3:0/3372773550 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08200aef00 con 0x7f08200af700 2024-09-10T07:52:43.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.762+0000 7f081d600640 1 -- 172.21.15.3:0/3372773550 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379597896 0 0) 0x7f08200cdf70 con 0x7f08200cfd40 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.763+0000 7f081e000640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200af700 0x7f08200cd840 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f080c009870 tx=0x7f080c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950ff9f8c38b116c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.763+0000 7f081d600640 1 -- 172.21.15.3:0/3372773550 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c037070 con 0x7f08200af700 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.763+0000 7f081d600640 1 -- 172.21.15.3:0/3372773550 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c032440 con 0x7f08200af700 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f081d600640 1 -- 172.21.15.3:0/3372773550 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c032810 con 0x7f08200af700 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 -- 172.21.15.3:0/3372773550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200af700 msgr2=0x7f08200cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200af700 0x7f08200cd840 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f080c009870 tx=0x7f080c02a1c0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 -- 172.21.15.3:0/3372773550 shutdown_connections 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200ce9b0 0x7f0820162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 --2- 172.21.15.3:0/3372773550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200af700 0x7f08200cd840 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 -- 172.21.15.3:0/3372773550 >> 172.21.15.3:0/3372773550 conn(0x7f082006cfc0 msgr2=0x7f082006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 -- 172.21.15.3:0/3372773550 shutdown_connections 2024-09-10T07:52:43.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.764+0000 7f0825a00640 1 -- 172.21.15.3:0/3372773550 wait complete. 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.766+0000 7f0825a00640 1 Processor -- start 2024-09-10T07:52:43.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.766+0000 7f0825a00640 1 -- start start 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08200af700 0x7f0820161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200cfd40 0x7f08200b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08200cefc0 con 0x7f08200ce9b0 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f08200cee40 con 0x7f08200af700 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.767+0000 7f0825a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08201628a0 con 0x7f08200cfd40 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57674/0 (socket says 172.21.15.3:57674) 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 -- 172.21.15.3:0/2969073931 learned_addr learned my addr 172.21.15.3:0/2969073931 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:43.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081f400640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200cfd40 0x7f08200b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081ea00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08200af700 0x7f0820161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 -- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200cfd40 msgr2=0x7f08200b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200cfd40 0x7f08200b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 -- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08200af700 msgr2=0x7f0820161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08200af700 0x7f0820161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 -- 172.21.15.3:0/2969073931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0820143a20 con 0x7f08200ce9b0 2024-09-10T07:52:43.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.768+0000 7f081e000640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f080c035040 tx=0x7f080c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.769+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c037040 con 0x7f08200ce9b0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.769+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c004230 con 0x7f08200ce9b0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.769+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c003610 con 0x7f08200ce9b0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.769+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0820142120 con 0x7f08200ce9b0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.769+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201426e0 con 0x7f08200ce9b0 2024-09-10T07:52:43.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.770+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0820162360 con 0x7f08200ce9b0 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.771+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f080c02ad10 con 0x7f08200ce9b0 2024-09-10T07:52:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.772+0000 7f0816a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 0x7f080006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.772+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f080c0ab960 con 0x7f08200ce9b0 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.772+0000 7f081ea00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 0x7f080006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.773+0000 7f081ea00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 0x7f080006e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0810000f60 tx=0x7f0810009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:43.776+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f080c07a390 con 0x7f08200ce9b0 2024-09-10T07:52:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.027+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f08200ccc10 con 0x7f08200ce9b0 2024-09-10T07:52:44.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.027+0000 7f0816a00640 1 -- 172.21.15.3:0/2969073931 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7f080c079c90 con 0x7f08200ce9b0 2024-09-10T07:52:44.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:44.027 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":397,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270024704,"bytes_avail":772824088576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7422,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-09-10T07:52:43.574059+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 msgr2=0x7f080006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 0x7f080006e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0810000f60 tx=0x7f0810009040 comp rx=0 tx=0).stop 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 msgr2=0x7f0820140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f080c035040 tx=0x7f080c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 shutdown_connections 2024-09-10T07:52:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f080006be30 0x7f080006e2f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08200cfd40 0x7f08200b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08200ce9b0 0x7f0820140fb0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 --2- 172.21.15.3:0/2969073931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08200af700 0x7f0820161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.029+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 >> 172.21.15.3:0/2969073931 conn(0x7f082006cfc0 msgr2=0x7f0820146ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.030+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 shutdown_connections 2024-09-10T07:52:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:44.030+0000 7f0825a00640 1 -- 172.21.15.3:0/2969073931 wait complete. 2024-09-10T07:52:44.044 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:45.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:45.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.213+0000 7f0203e00640 1 Processor -- start 2024-09-10T07:52:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.213+0000 7f0203e00640 1 -- start start 2024-09-10T07:52:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02040d3c10 0x7f0204073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 0x7f0204073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02040cfdc0 con 0x7f020415e860 2024-09-10T07:52:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f02040d3780 con 0x7f02040d3c10 2024-09-10T07:52:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f02040cee40 con 0x7f02040d4050 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0202a00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f020415e860 0x7f02040cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:43138/0 (socket says 172.21.15.3:43138) 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0203400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 0x7f0204073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.214+0000 7f0202a00640 1 -- 172.21.15.3:0/1765469745 learned_addr learned my addr 172.21.15.3:0/1765469745 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0202000640 1 --2- 172.21.15.3:0/1765469745 >> v2:172.21.15.70:3300/0 conn(0x7f02040d3c10 0x7f0204073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0201600640 1 -- 172.21.15.3:0/1765469745 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3543804298 0 0) 0x7f02040cfdc0 con 0x7f020415e860 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0203400640 1 -- 172.21.15.3:0/1765469745 >> v2:172.21.15.70:3300/0 conn(0x7f02040d3c10 msgr2=0x7f0204073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0203400640 1 --2- 172.21.15.3:0/1765469745 >> v2:172.21.15.70:3300/0 conn(0x7f02040d3c10 0x7f0204073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0203400640 1 -- 172.21.15.3:0/1765469745 >> v1:172.21.15.70:6789/0 conn(0x7f020415e860 legacy=0x7f02040cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.215+0000 7f0203400640 1 -- 172.21.15.3:0/1765469745 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02040cf120 con 0x7f02040d4050 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.216+0000 7f0203400640 1 --2- 172.21.15.3:0/1765469745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 0x7f0204073690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009d90 tx=0x7f01fc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828ff494e7c701e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.216+0000 7f0201600640 1 -- 172.21.15.3:0/1765469745 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc007740 con 0x7f02040d4050 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.216+0000 7f0201600640 1 -- 172.21.15.3:0/1765469745 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0078e0 con 0x7f02040d4050 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.216+0000 7f0201600640 1 -- 172.21.15.3:0/1765469745 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc007cb0 con 0x7f02040d4050 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 -- 172.21.15.3:0/1765469745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 msgr2=0x7f0204073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 --2- 172.21.15.3:0/1765469745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 0x7f0204073690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009d90 tx=0x7f01fc02a660 comp rx=0 tx=0).stop 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 -- 172.21.15.3:0/1765469745 shutdown_connections 2024-09-10T07:52:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 --2- 172.21.15.3:0/1765469745 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d4050 0x7f0204073690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 --2- 172.21.15.3:0/1765469745 >> v2:172.21.15.70:3300/0 conn(0x7f02040d3c10 0x7f0204073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 -- 172.21.15.3:0/1765469745 >> 172.21.15.3:0/1765469745 conn(0x7f02040d4650 msgr2=0x7f02040d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 -- 172.21.15.3:0/1765469745 shutdown_connections 2024-09-10T07:52:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.217+0000 7f0203e00640 1 -- 172.21.15.3:0/1765469745 wait complete. 2024-09-10T07:52:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.219+0000 7f0203e00640 1 Processor -- start 2024-09-10T07:52:45.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.219+0000 7f0203e00640 1 -- start start 2024-09-10T07:52:45.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 0x7f020414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02040d4050 0x7f0204160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 0x7f02040aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02040d2460 con 0x7f020415e860 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02040cf400 con 0x7f02040d4050 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f02040b7880 con 0x7f02040d3c10 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 0x7f02040aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0202a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 0x7f020414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 0x7f02040aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57680/0 (socket says 172.21.15.3:57680) 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0203400640 1 -- 172.21.15.3:0/4166665736 learned_addr learned my addr 172.21.15.3:0/4166665736 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.220+0000 7f0202000640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02040d4050 0x7f0204160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 -- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02040d4050 msgr2=0x7f0204160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02040d4050 0x7f0204160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 -- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 msgr2=0x7f02040aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 0x7f02040aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 -- 172.21.15.3:0/4166665736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020415a8f0 con 0x7f02040d3c10 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0202a00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 0x7f020414da10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c3e0 tx=0x7f01f400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f40074e0 con 0x7f02040d3c10 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40047e0 con 0x7f02040d3c10 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4004bd0 con 0x7f02040d3c10 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020415fd10 con 0x7f02040d3c10 2024-09-10T07:52:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.221+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041602d0 con 0x7f02040d3c10 2024-09-10T07:52:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.222+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02040b8dc0 con 0x7f02040d3c10 2024-09-10T07:52:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.223+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01f4011070 con 0x7f02040d3c10 2024-09-10T07:52:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.224+0000 7f01fb400640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 0x7f01e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.224+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f01f408a8e0 con 0x7f02040d3c10 2024-09-10T07:52:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.224+0000 7f0202000640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 0x7f01e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.225+0000 7f0202000640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 0x7f01e406e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f02041f77d0 tx=0x7f01f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.229+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01f4058ba0 con 0x7f02040d3c10 2024-09-10T07:52:45.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.449+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02040b55f0 con 0x7f02040d3c10 2024-09-10T07:52:45.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.450+0000 7f01fb400640 1 -- 172.21.15.3:0/4166665736 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f01f4059e20 con 0x7f02040d3c10 2024-09-10T07:52:45.450 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:45.450 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:45.450 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:45.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 msgr2=0x7f01e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 0x7f01e406e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f02041f77d0 tx=0x7f01f0009190 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 msgr2=0x7f020414da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 0x7f020414da10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c3e0 tx=0x7f01f400c8a0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 shutdown_connections 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01e406be30 0x7f01e406e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f020415e860 0x7f02040aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02040d4050 0x7f0204160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 --2- 172.21.15.3:0/4166665736 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02040d3c10 0x7f020414da10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 >> 172.21.15.3:0/4166665736 conn(0x7f02040d4650 msgr2=0x7f02040d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 shutdown_connections 2024-09-10T07:52:45.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.453+0000 7f0203e00640 1 -- 172.21.15.3:0/4166665736 wait complete. 2024-09-10T07:52:45.469 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:45.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.625+0000 7f37fe800640 1 Processor -- start 2024-09-10T07:52:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.626+0000 7f37fe800640 1 -- start start 2024-09-10T07:52:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37fe800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37f815e7c0 0x7f37f80d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37fe800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f80d2ba0 con 0x7f37f80cf030 2024-09-10T07:52:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37fe800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f80b8160 con 0x7f37f81622a0 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37fe800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f37f815eba0 con 0x7f37f815e7c0 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37f7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37f815e7c0 0x7f37f80d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37f7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f37f80cf030 0x7f37f80b9200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57626/0 (socket says 172.21.15.3:57626) 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37f7e00640 1 -- 172.21.15.3:0/1393329112 learned_addr learned my addr 172.21.15.3:0/1393329112 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37f7400640 1 -- 172.21.15.3:0/1393329112 >> v1:172.21.15.70:6789/0 conn(0x7f37f81622a0 legacy=0x7f37f80b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.627+0000 7f37f7400640 1 -- 172.21.15.3:0/1393329112 >> v1:172.21.15.3:6789/0 conn(0x7f37f80cf030 legacy=0x7f37f80b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.628+0000 7f37f7400640 1 -- 172.21.15.3:0/1393329112 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f80b8440 con 0x7f37f815e7c0 2024-09-10T07:52:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.628+0000 7f37f6000640 1 -- 172.21.15.3:0/1393329112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896637977 0 0) 0x7f37f80d2ba0 con 0x7f37f80cf030 2024-09-10T07:52:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37f7400640 1 --2- 172.21.15.3:0/1393329112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f815e7c0 0x7f37f80d2660 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009d90 tx=0x7f37e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6391d7446951b408 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37f6000640 1 -- 172.21.15.3:0/1393329112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8007740 con 0x7f37f815e7c0 2024-09-10T07:52:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37f6000640 1 -- 172.21.15.3:0/1393329112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e80078e0 con 0x7f37f815e7c0 2024-09-10T07:52:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37f6000640 1 -- 172.21.15.3:0/1393329112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8007cb0 con 0x7f37f815e7c0 2024-09-10T07:52:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 -- 172.21.15.3:0/1393329112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f815e7c0 msgr2=0x7f37f80d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 --2- 172.21.15.3:0/1393329112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f815e7c0 0x7f37f80d2660 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009d90 tx=0x7f37e802a6f0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 -- 172.21.15.3:0/1393329112 shutdown_connections 2024-09-10T07:52:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 --2- 172.21.15.3:0/1393329112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f815e7c0 0x7f37f80d2660 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 -- 172.21.15.3:0/1393329112 >> 172.21.15.3:0/1393329112 conn(0x7f37f8072560 msgr2=0x7f37f8072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.629+0000 7f37fe800640 1 -- 172.21.15.3:0/1393329112 shutdown_connections 2024-09-10T07:52:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.630+0000 7f37fe800640 1 -- 172.21.15.3:0/1393329112 wait complete. 2024-09-10T07:52:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.631+0000 7f37fe800640 1 Processor -- start 2024-09-10T07:52:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.631+0000 7f37fe800640 1 -- start start 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37f80cf030 0x7f37f8154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f815e7c0 0x7f37f81e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80b8a00 con 0x7f37f81622a0 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80b8d70 con 0x7f37f815e7c0 2024-09-10T07:52:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37fe800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f37f80b7e70 con 0x7f37f80cf030 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.632+0000 7f37f7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57690/0 (socket says 172.21.15.3:57690) 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 -- 172.21.15.3:0/563499089 learned_addr learned my addr 172.21.15.3:0/563499089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f6a00640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f815e7c0 0x7f37f81e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7400640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37f80cf030 0x7f37f8154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 -- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37f80cf030 msgr2=0x7f37f8154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37f80cf030 0x7f37f8154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 -- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f815e7c0 msgr2=0x7f37f81e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f815e7c0 0x7f37f81e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 -- 172.21.15.3:0/563499089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f81e9340 con 0x7f37f81622a0 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37f7e00640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c450 tx=0x7f37f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0018070 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f0007500 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.633+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f81e9620 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.634+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0004a20 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.634+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f81e9900 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.635+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f80cf4e0 con 0x7f37f81622a0 2024-09-10T07:52:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.635+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37f001b070 con 0x7f37f81622a0 2024-09-10T07:52:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.636+0000 7f37ef400640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 0x7f37d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.636+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f37f008a150 con 0x7f37f81622a0 2024-09-10T07:52:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.637+0000 7f37f7400640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 0x7f37d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.637+0000 7f37f7400640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 0x7f37d806e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f37e8034000 tx=0x7f37e802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.641+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37f0058af0 con 0x7f37f81622a0 2024-09-10T07:52:45.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.895+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f37f80b36a0 con 0x7f37f81622a0 2024-09-10T07:52:45.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.896+0000 7f37ef400640 1 -- 172.21.15.3:0/563499089 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7f37f00583f0 con 0x7f37f81622a0 2024-09-10T07:52:45.896 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270024704,"bytes_avail":772824088576,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5801,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-09-10T07:52:43.574059+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 msgr2=0x7f37d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 0x7f37d806e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f37e8034000 tx=0x7f37e802f000 comp rx=0 tx=0).stop 2024-09-10T07:52:45.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 msgr2=0x7f37f815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c450 tx=0x7f37f000c910 comp rx=0 tx=0).stop 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 shutdown_connections 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f37d806be30 0x7f37d806e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f81622a0 0x7f37f815c250 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f815e7c0 0x7f37f81e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 --2- 172.21.15.3:0/563499089 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37f80cf030 0x7f37f8154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 >> 172.21.15.3:0/563499089 conn(0x7f37f8072560 msgr2=0x7f37f8161890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:45.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 shutdown_connections 2024-09-10T07:52:45.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:45.898+0000 7f37fe800640 1 -- 172.21.15.3:0/563499089 wait complete. 2024-09-10T07:52:45.915 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:46.463 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:52:46.463+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:52:46.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:47.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.084+0000 7f19e2800640 1 Processor -- start 2024-09-10T07:52:47.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.084+0000 7f19e2800640 1 -- start start 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e2800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 0x7f19dc0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e2800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 0x7f19dc073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e2800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19dc0cee40 con 0x7f19dc0cfbf0 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e2800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc0d3780 con 0x7f19dc073150 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e2800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0cf120 con 0x7f19dc15e860 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.085+0000 7f19e0a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f19dc0cfbf0 0x7f19dc0d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57640/0 (socket says 172.21.15.3:57640) 2024-09-10T07:52:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e0a00640 1 -- 172.21.15.3:0/2912494497 learned_addr learned my addr 172.21.15.3:0/2912494497 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1e00640 1 --2- 172.21.15.3:0/2912494497 >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 0x7f19dc073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1400640 1 --2- 172.21.15.3:0/2912494497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 0x7f19dc0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285435102 0 0) 0x7f19dc0cee40 con 0x7f19dc0cfbf0 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19c4003520 con 0x7f19dc0cfbf0 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 228369508 0 0) 0x7f19c4003520 con 0x7f19dc0cfbf0 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1400640 1 -- 172.21.15.3:0/2912494497 >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 msgr2=0x7f19dc073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1400640 1 --2- 172.21.15.3:0/2912494497 >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 0x7f19dc073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1400640 1 -- 172.21.15.3:0/2912494497 >> v1:172.21.15.3:6789/0 conn(0x7f19dc0cfbf0 legacy=0x7f19dc0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1400640 1 -- 172.21.15.3:0/2912494497 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc0d3410 con 0x7f19dc15e860 2024-09-10T07:52:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.086+0000 7f19e1e00640 1 --2- 172.21.15.3:0/2912494497 >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 0x7f19dc073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.087+0000 7f19e1400640 1 --2- 172.21.15.3:0/2912494497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 0x7f19dc0cf6b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ce87003ccb0f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0037070 con 0x7f19dc15e860 2024-09-10T07:52:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d002aba0 con 0x7f19dc15e860 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19dbe00640 1 -- 172.21.15.3:0/2912494497 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8001670 con 0x7f19dc15e860 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 -- 172.21.15.3:0/2912494497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 msgr2=0x7f19dc0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 --2- 172.21.15.3:0/2912494497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 0x7f19dc0cf6b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002a110 comp rx=0 tx=0).stop 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 -- 172.21.15.3:0/2912494497 shutdown_connections 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 --2- 172.21.15.3:0/2912494497 >> v2:172.21.15.70:3300/0 conn(0x7f19dc073150 0x7f19dc073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 --2- 172.21.15.3:0/2912494497 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc15e860 0x7f19dc0cf6b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 -- 172.21.15.3:0/2912494497 >> 172.21.15.3:0/2912494497 conn(0x7f19dc0d4650 msgr2=0x7f19dc0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 -- 172.21.15.3:0/2912494497 shutdown_connections 2024-09-10T07:52:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.088+0000 7f19e2800640 1 -- 172.21.15.3:0/2912494497 wait complete. 2024-09-10T07:52:47.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.090+0000 7f19e2800640 1 Processor -- start 2024-09-10T07:52:47.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.090+0000 7f19e2800640 1 -- start start 2024-09-10T07:52:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc073150 0x7f19dc160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19dc0cfbf0 0x7f19dc0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0d2460 con 0x7f19dc15e860 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0cf400 con 0x7f19dc0cfbf0 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e2800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0b7880 con 0x7f19dc073150 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57692/0 (socket says 172.21.15.3:57692) 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.091+0000 7f19e1e00640 1 -- 172.21.15.3:0/3237780388 learned_addr learned my addr 172.21.15.3:0/3237780388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e0a00640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19dc0cfbf0 0x7f19dc0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1400640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc073150 0x7f19dc160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 -- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc073150 msgr2=0x7f19dc160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc073150 0x7f19dc160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 -- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19dc0cfbf0 msgr2=0x7f19dc0ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19dc0cfbf0 0x7f19dc0ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 -- 172.21.15.3:0/3237780388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc0ad360 con 0x7f19dc15e860 2024-09-10T07:52:47.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19e1e00640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f19d400efc0 tx=0x7f19d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4018030 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.092+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d400cde0 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.093+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003bb0 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.093+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc1f49e0 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.093+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc1f4fa0 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.094+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc0cf6b0 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.095+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19d4005d10 con 0x7f19dc15e860 2024-09-10T07:52:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.096+0000 7f19d9600640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 0x7f19bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.096+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f19d408b850 con 0x7f19dc15e860 2024-09-10T07:52:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.096+0000 7f19e1400640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 0x7f19bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.097+0000 7f19e1400640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 0x7f19bc06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f19d0034000 tx=0x7f19d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.101+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19d4022070 con 0x7f19dc15e860 2024-09-10T07:52:47.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.323+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19dc0d31d0 con 0x7f19dc15e860 2024-09-10T07:52:47.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.324+0000 7f19d9600640 1 -- 172.21.15.3:0/3237780388 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f19dc0ad360 con 0x7f19dc15e860 2024-09-10T07:52:47.323 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:47.324 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:47.324 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:47.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 msgr2=0x7f19bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 0x7f19bc06e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f19d0034000 tx=0x7f19d002e000 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 msgr2=0x7f19dc1f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f19d400efc0 tx=0x7f19d4009f90 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 shutdown_connections 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f19bc06be30 0x7f19bc06e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19dc15e860 0x7f19dc1f5300 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19dc0cfbf0 0x7f19dc0ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 --2- 172.21.15.3:0/3237780388 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19dc073150 0x7f19dc160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 >> 172.21.15.3:0/3237780388 conn(0x7f19dc0d4650 msgr2=0x7f19dc0cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 shutdown_connections 2024-09-10T07:52:47.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.326+0000 7f19e2800640 1 -- 172.21.15.3:0/3237780388 wait complete. 2024-09-10T07:52:47.342 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:47.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:47.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.498+0000 7f343e800640 1 Processor -- start 2024-09-10T07:52:47.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.498+0000 7f343e800640 1 -- start start 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f343e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f343806c230 0x7f343815f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f343e800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34380afa00 con 0x7f34380cfbc0 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f343e800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343815f6d0 con 0x7f343815ebc0 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f343e800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34380ae8a0 con 0x7f343806c230 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f3437400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34380cfbc0 0x7f34380af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57644/0 (socket says 172.21.15.3:57644) 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.499+0000 7f3437400640 1 -- 172.21.15.3:0/1287808066 learned_addr learned my addr 172.21.15.3:0/1287808066 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:47.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.500+0000 7f3437e00640 1 --2- 172.21.15.3:0/1287808066 >> v2:172.21.15.3:3300/0 conn(0x7f343806c230 0x7f343815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.500+0000 7f3437e00640 1 -- 172.21.15.3:0/1287808066 >> v1:172.21.15.70:6789/0 conn(0x7f343815ebc0 legacy=0x7f343806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.500+0000 7f3437e00640 1 -- 172.21.15.3:0/1287808066 >> v1:172.21.15.3:6789/0 conn(0x7f34380cfbc0 legacy=0x7f34380af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.500+0000 7f3437e00640 1 -- 172.21.15.3:0/1287808066 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343815f9b0 con 0x7f343806c230 2024-09-10T07:52:47.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.500+0000 7f3436000640 1 -- 172.21.15.3:0/1287808066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827646276 0 0) 0x7f34380afa00 con 0x7f34380cfbc0 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.501+0000 7f3437e00640 1 --2- 172.21.15.3:0/1287808066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343806c230 0x7f343815f000 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3430009d90 tx=0x7f343002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bf016edaf2f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.501+0000 7f3436000640 1 -- 172.21.15.3:0/1287808066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430007740 con 0x7f343806c230 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.501+0000 7f3436000640 1 -- 172.21.15.3:0/1287808066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34300078e0 con 0x7f343806c230 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.501+0000 7f3436000640 1 -- 172.21.15.3:0/1287808066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430007cb0 con 0x7f343806c230 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 -- 172.21.15.3:0/1287808066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343806c230 msgr2=0x7f343815f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 --2- 172.21.15.3:0/1287808066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343806c230 0x7f343815f000 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3430009d90 tx=0x7f343002a6f0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 -- 172.21.15.3:0/1287808066 shutdown_connections 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 --2- 172.21.15.3:0/1287808066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343806c230 0x7f343815f000 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 -- 172.21.15.3:0/1287808066 >> 172.21.15.3:0/1287808066 conn(0x7f34380b2870 msgr2=0x7f34380b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:47.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 -- 172.21.15.3:0/1287808066 shutdown_connections 2024-09-10T07:52:47.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.502+0000 7f343e800640 1 -- 172.21.15.3:0/1287808066 wait complete. 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.503+0000 7f343e800640 1 Processor -- start 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.504+0000 7f343e800640 1 -- start start 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.504+0000 7f343e800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 0x7f343815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.504+0000 7f343e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.504+0000 7f343e800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343815ebc0 0x7f34380727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f343e800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34380cffa0 con 0x7f34380cfbc0 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f343e800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f34380aee60 con 0x7f343815ebc0 2024-09-10T07:52:47.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f343e800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34380d0120 con 0x7f343806c230 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3436a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3436a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57718/0 (socket says 172.21.15.3:57718) 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3436a00640 1 -- 172.21.15.3:0/3892149682 learned_addr learned my addr 172.21.15.3:0/3892149682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 0x7f343815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437e00640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343815ebc0 0x7f34380727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 -- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343815ebc0 msgr2=0x7f34380727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343815ebc0 0x7f34380727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 -- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 msgr2=0x7f3438159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3437400640 1 -- 172.21.15.3:0/3892149682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34381e8020 con 0x7f343806c230 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.505+0000 7f3436a00640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:47.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f3437400640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 0x7f343815a020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f342800c410 tx=0x7f342800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280074e0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34280047e0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428004bd0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34380b50f0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.506+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34380b56b0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.507+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34381e81f0 con 0x7f343806c230 2024-09-10T07:52:47.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.508+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3428011070 con 0x7f343806c230 2024-09-10T07:52:47.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.509+0000 7f342f400640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:47.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.509+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f342808a970 con 0x7f343806c230 2024-09-10T07:52:47.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.509+0000 7f3436a00640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:47.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.510+0000 7f3436a00640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 0x7f341806e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f343815ad70 tx=0x7f3424009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:47.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.514+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3428058c30 con 0x7f343806c230 2024-09-10T07:52:47.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.767+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f34380b0310 con 0x7f343806c230 2024-09-10T07:52:47.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.768+0000 7f342f400640 1 -- 172.21.15.3:0/3892149682 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1979 (secure 0 0 0) 0x7f342805cf30 con 0x7f343806c230 2024-09-10T07:52:47.768 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:47.768 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":401,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270032896,"bytes_avail":772824080384,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5801,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-10T07:52:47.574928+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 msgr2=0x7f341806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 0x7f341806e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f343815ad70 tx=0x7f3424009040 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 msgr2=0x7f343815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 0x7f343815a020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f342800c410 tx=0x7f342800c8d0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 shutdown_connections 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343815ebc0 0x7f34380727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34380cfbc0 0x7f3438159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 --2- 172.21.15.3:0/3892149682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f343806c230 0x7f343815a020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 >> 172.21.15.3:0/3892149682 conn(0x7f34380b2870 msgr2=0x7f3438160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 shutdown_connections 2024-09-10T07:52:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:47.771+0000 7f343e800640 1 -- 172.21.15.3:0/3892149682 wait complete. 2024-09-10T07:52:47.787 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:48.789 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:48.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.958+0000 7f81c9200640 1 Processor -- start 2024-09-10T07:52:48.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.958+0000 7f81c9200640 1 -- start start 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81c9200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81c4160af0 0x7f81c40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81c9200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81c40d05f0 0x7f81c4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81c9200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c40cfdd0 con 0x7f81c4160710 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81c9200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f81c415cc90 con 0x7f81c4160af0 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81c9200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f81c415cf70 con 0x7f81c40d05f0 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81bbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81c4160af0 0x7f81c40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81bbe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f81c4160af0 0x7f81c40d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57724/0 (socket says 172.21.15.3:57724) 2024-09-10T07:52:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.959+0000 7f81bbe00640 1 -- 172.21.15.3:0/1414948487 learned_addr learned my addr 172.21.15.3:0/1414948487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.960+0000 7f81c2800640 1 --2- 172.21.15.3:0/1414948487 >> v2:172.21.15.70:3300/0 conn(0x7f81c40d05f0 0x7f81c4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.960+0000 7f81bbe00640 1 -- 172.21.15.3:0/1414948487 >> v2:172.21.15.70:3300/0 conn(0x7f81c40d05f0 msgr2=0x7f81c4072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.960+0000 7f81bbe00640 1 --2- 172.21.15.3:0/1414948487 >> v2:172.21.15.70:3300/0 conn(0x7f81c40d05f0 0x7f81c4072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.960+0000 7f81bbe00640 1 -- 172.21.15.3:0/1414948487 >> v1:172.21.15.70:6789/0 conn(0x7f81c4160710 legacy=0x7f81c40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.960+0000 7f81bbe00640 1 -- 172.21.15.3:0/1414948487 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c415d250 con 0x7f81c4160af0 2024-09-10T07:52:48.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.961+0000 7f81bbe00640 1 --2- 172.21.15.3:0/1414948487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c4160af0 0x7f81c40d00b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f81c40718d0 tx=0x7f81ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676beeba205fadb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.961+0000 7f81c1400640 1 -- 172.21.15.3:0/1414948487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81c4160af0 2024-09-10T07:52:48.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.961+0000 7f81c1400640 1 -- 172.21.15.3:0/1414948487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac032440 con 0x7f81c4160af0 2024-09-10T07:52:48.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.961+0000 7f81c1400640 1 -- 172.21.15.3:0/1414948487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac032810 con 0x7f81c4160af0 2024-09-10T07:52:48.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 -- 172.21.15.3:0/1414948487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c4160af0 msgr2=0x7f81c40d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 --2- 172.21.15.3:0/1414948487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c4160af0 0x7f81c40d00b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f81c40718d0 tx=0x7f81ac02a230 comp rx=0 tx=0).stop 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 -- 172.21.15.3:0/1414948487 shutdown_connections 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 --2- 172.21.15.3:0/1414948487 >> v2:172.21.15.70:3300/0 conn(0x7f81c40d05f0 0x7f81c4072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 --2- 172.21.15.3:0/1414948487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c4160af0 0x7f81c40d00b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 -- 172.21.15.3:0/1414948487 >> 172.21.15.3:0/1414948487 conn(0x7f81c406d000 msgr2=0x7f81c406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 -- 172.21.15.3:0/1414948487 shutdown_connections 2024-09-10T07:52:48.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.962+0000 7f81c9200640 1 -- 172.21.15.3:0/1414948487 wait complete. 2024-09-10T07:52:48.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.963+0000 7f81c9200640 1 Processor -- start 2024-09-10T07:52:48.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.964+0000 7f81c9200640 1 -- start start 2024-09-10T07:52:48.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.964+0000 7f81c9200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.964+0000 7f81c9200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81c4160710 0x7f81c415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.964+0000 7f81c9200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 0x7f81c415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c9200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415d770 con 0x7f81c40d05f0 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c9200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415c960 con 0x7f81c4160af0 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c9200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81c415e900 con 0x7f81c4160710 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57738/0 (socket says 172.21.15.3:57738) 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 -- 172.21.15.3:0/1928008977 learned_addr learned my addr 172.21.15.3:0/1928008977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c2800640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 0x7f81c415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81bbe00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81c4160710 0x7f81c415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 -- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81c4160710 msgr2=0x7f81c415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81c4160710 0x7f81c415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 -- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 msgr2=0x7f81c415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 0x7f81c415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 -- 172.21.15.3:0/1928008977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c415ebe0 con 0x7f81c40d05f0 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c2800640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 0x7f81c415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.965+0000 7f81c1e00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c460 tx=0x7f81b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.966+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4016020 con 0x7f81c40d05f0 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.966+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b4012870 con 0x7f81c40d05f0 2024-09-10T07:52:48.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.966+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4012c60 con 0x7f81c40d05f0 2024-09-10T07:52:48.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.966+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c415eec0 con 0x7f81c40d05f0 2024-09-10T07:52:48.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.966+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c415f480 con 0x7f81c40d05f0 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.967+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c40cfa60 con 0x7f81c40d05f0 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.968+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81b4019020 con 0x7f81c40d05f0 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.969+0000 7f81ba000640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.969+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f81b408ab70 con 0x7f81c40d05f0 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.969+0000 7f81bbe00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:48.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.970+0000 7f81bbe00640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 0x7f81a006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f81ac034000 tx=0x7f81ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:48.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:48.973+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81b4020030 con 0x7f81c40d05f0 2024-09-10T07:52:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.198+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c40afd50 con 0x7f81c40d05f0 2024-09-10T07:52:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.199+0000 7f81ba000640 1 -- 172.21.15.3:0/1928008977 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f81b4059510 con 0x7f81c40d05f0 2024-09-10T07:52:49.199 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:49.199 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 msgr2=0x7f81a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 0x7f81a006e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f81ac034000 tx=0x7f81ac02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 msgr2=0x7f81c4151050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c460 tx=0x7f81b400c920 comp rx=0 tx=0).stop 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 shutdown_connections 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f81a006be30 0x7f81a006e2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c4160af0 0x7f81c415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81c4160710 0x7f81c415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 --2- 172.21.15.3:0/1928008977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81c40d05f0 0x7f81c4151050 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 >> 172.21.15.3:0/1928008977 conn(0x7f81c406d000 msgr2=0x7f81c40b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 shutdown_connections 2024-09-10T07:52:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.202+0000 7f81c9200640 1 -- 172.21.15.3:0/1928008977 wait complete. 2024-09-10T07:52:49.217 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:49.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:49.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.372+0000 7f8ca8a00640 1 Processor -- start 2024-09-10T07:52:49.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.372+0000 7f8ca8a00640 1 -- start start 2024-09-10T07:52:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.373+0000 7f8ca8a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ca40b6c70 0x7f8ca406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.373+0000 7f8ca8a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ca40cd350 0x7f8ca406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.373+0000 7f8ca8a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca415fc00 con 0x7f8ca415f820 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.373+0000 7f8ca8a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca4160b50 con 0x7f8ca40b6c70 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.373+0000 7f8ca8a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca40d1280 con 0x7f8ca40cd350 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8ca3400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ca415f820 0x7f8ca40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57654/0 (socket says 172.21.15.3:57654) 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8ca3400640 1 -- 172.21.15.3:0/1778988031 learned_addr learned my addr 172.21.15.3:0/1778988031 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8c9be00640 1 --2- 172.21.15.3:0/1778988031 >> v2:172.21.15.3:3300/0 conn(0x7f8ca40b6c70 0x7f8ca406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8ca3e00640 1 --2- 172.21.15.3:0/1778988031 >> v2:172.21.15.70:3300/0 conn(0x7f8ca40cd350 0x7f8ca406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8ca2a00640 1 -- 172.21.15.3:0/1778988031 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163428758 0 0) 0x7f8ca415fc00 con 0x7f8ca415f820 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8c9be00640 1 -- 172.21.15.3:0/1778988031 >> v2:172.21.15.70:3300/0 conn(0x7f8ca40cd350 msgr2=0x7f8ca406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8c9be00640 1 --2- 172.21.15.3:0/1778988031 >> v2:172.21.15.70:3300/0 conn(0x7f8ca40cd350 0x7f8ca406c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8c9be00640 1 -- 172.21.15.3:0/1778988031 >> v1:172.21.15.3:6789/0 conn(0x7f8ca415f820 legacy=0x7f8ca40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.374+0000 7f8c9be00640 1 -- 172.21.15.3:0/1778988031 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40d1560 con 0x7f8ca40b6c70 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.375+0000 7f8c9be00640 1 --2- 172.21.15.3:0/1778988031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca40b6c70 0x7f8ca406bb20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40d0bd0 tx=0x7f8c8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa319f1e970292e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.375+0000 7f8ca2a00640 1 -- 172.21.15.3:0/1778988031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c037070 con 0x7f8ca40b6c70 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.375+0000 7f8ca2a00640 1 -- 172.21.15.3:0/1778988031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8c032440 con 0x7f8ca40b6c70 2024-09-10T07:52:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca2a00640 1 -- 172.21.15.3:0/1778988031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c032810 con 0x7f8ca40b6c70 2024-09-10T07:52:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 -- 172.21.15.3:0/1778988031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca40b6c70 msgr2=0x7f8ca406bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/1778988031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca40b6c70 0x7f8ca406bb20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40d0bd0 tx=0x7f8c8c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 -- 172.21.15.3:0/1778988031 shutdown_connections 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/1778988031 >> v2:172.21.15.70:3300/0 conn(0x7f8ca40cd350 0x7f8ca406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/1778988031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca40b6c70 0x7f8ca406bb20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 -- 172.21.15.3:0/1778988031 >> 172.21.15.3:0/1778988031 conn(0x7f8ca40b5ea0 msgr2=0x7f8ca40b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 -- 172.21.15.3:0/1778988031 shutdown_connections 2024-09-10T07:52:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.376+0000 7f8ca8a00640 1 -- 172.21.15.3:0/1778988031 wait complete. 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.378+0000 7f8ca8a00640 1 Processor -- start 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.378+0000 7f8ca8a00640 1 -- start start 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 0x7f8ca40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ca40cd350 0x7f8ca414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40b3c00 con 0x7f8ca415f820 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40b3a80 con 0x7f8ca40cd350 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca8a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca40b1090 con 0x7f8ca40b6c70 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57752/0 (socket says 172.21.15.3:57752) 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca3e00640 1 -- 172.21.15.3:0/3937737933 learned_addr learned my addr 172.21.15.3:0/3937737933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:49.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8ca3400640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 0x7f8ca40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.379+0000 7f8c9be00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ca40cd350 0x7f8ca414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 -- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 msgr2=0x7f8ca40ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 0x7f8ca40ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 -- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ca40cd350 msgr2=0x7f8ca414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ca40cd350 0x7f8ca414c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 -- 172.21.15.3:0/3937737933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca414c720 con 0x7f8ca415f820 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3400640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 0x7f8ca40ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca3e00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c018020 con 0x7f8ca415f820 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c014480 con 0x7f8ca415f820 2024-09-10T07:52:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c014870 con 0x7f8ca415f820 2024-09-10T07:52:49.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca40ae600 con 0x7f8ca415f820 2024-09-10T07:52:49.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.380+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca40aebc0 con 0x7f8ca415f820 2024-09-10T07:52:49.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.381+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca40d1a80 con 0x7f8ca415f820 2024-09-10T07:52:49.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.382+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c9c005d10 con 0x7f8ca415f820 2024-09-10T07:52:49.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.383+0000 7f8c9b400640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 0x7f8c8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:49.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.383+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f8c9c08b9d0 con 0x7f8ca415f820 2024-09-10T07:52:49.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.383+0000 7f8ca3400640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 0x7f8c8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:49.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.384+0000 7f8ca3400640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 0x7f8c8006e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c94000f60 tx=0x7f8c94007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:49.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.387+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c9c022070 con 0x7f8ca415f820 2024-09-10T07:52:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.644+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8ca415be70 con 0x7f8ca415f820 2024-09-10T07:52:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.644+0000 7f8c9b400640 1 -- 172.21.15.3:0/3937737933 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1979 (secure 0 0 0) 0x7f8c9c010040 con 0x7f8ca415f820 2024-09-10T07:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:49.644 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270032896,"bytes_avail":772824080384,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5801,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-10T07:52:47.574928+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 msgr2=0x7f8c8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 0x7f8c8006e320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c94000f60 tx=0x7f8c94007ca0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 msgr2=0x7f8ca4154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 shutdown_connections 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8c8006be60 0x7f8c8006e320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ca415f820 0x7f8ca4154ec0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ca40cd350 0x7f8ca414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 --2- 172.21.15.3:0/3937737933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ca40b6c70 0x7f8ca40ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 >> 172.21.15.3:0/3937737933 conn(0x7f8ca40b5ea0 msgr2=0x7f8ca40731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 shutdown_connections 2024-09-10T07:52:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:49.647+0000 7f8ca8a00640 1 -- 172.21.15.3:0/3937737933 wait complete. 2024-09-10T07:52:49.662 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:50.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:50.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.832+0000 7f5ae2000640 1 Processor -- start 2024-09-10T07:52:50.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.832+0000 7f5ae2000640 1 -- start start 2024-09-10T07:52:50.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5ae2000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5adc0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5ae2000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5adc0cee40 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5ae2000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5adc0cf120 con 0x7f5adc073150 2024-09-10T07:52:50.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5ae2000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc0d3780 con 0x7f5adc15e860 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5adbe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5adc0cfbf0 0x7f5adc0d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57662/0 (socket says 172.21.15.3:57662) 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.833+0000 7f5adbe00640 1 -- 172.21.15.3:0/3876534259 learned_addr learned my addr 172.21.15.3:0/3876534259 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5adc0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531943037 0 0) 0x7f5adc0cee40 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ac4003520 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387506999 0 0) 0x7f5adc0cf120 con 0x7f5adc073150 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5adc0cee40 con 0x7f5adc073150 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1982611732 0 0) 0x7f5ac4003520 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 msgr2=0x7f5adc0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5adc0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 >> v1:172.21.15.70:6789/0 conn(0x7f5adc073150 legacy=0x7f5adc0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc0d3410 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f5ac8001670 con 0x7f5adc0cfbf0 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 >> v1:172.21.15.3:6789/0 conn(0x7f5adc0cfbf0 legacy=0x7f5adc0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5ac4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.834+0000 7f5adb400640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5ac4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5adb400640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ac4004d40 0x7f5ac40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0d3780 con 0x7f5ac4003be0 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac400baa0 con 0x7f5ac4004d40 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac400bca0 con 0x7f5adc15e860 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5ac4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5adbe00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5ac4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae1600640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ac4004d40 0x7f5ac40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 -- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 msgr2=0x7f5ac4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc15e860 0x7f5ac4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 -- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ac4004d40 msgr2=0x7f5ac40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ac4004d40 0x7f5ac40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.835+0000 7f5ae0c00640 1 -- 172.21.15.3:0/3876534259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac4003520 con 0x7f5ac4003be0 2024-09-10T07:52:50.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.836+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5ac4003fe0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009a90 tx=0x7f5ac4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6385ad4dac546f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:50.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.836+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0036070 con 0x7f5ac4003be0 2024-09-10T07:52:50.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.836+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad0031400 con 0x7f5ac4003be0 2024-09-10T07:52:50.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.836+0000 7f5adb400640 1 -- 172.21.15.3:0/3876534259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0009690 con 0x7f5ac4003be0 2024-09-10T07:52:50.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 -- 172.21.15.3:0/3876534259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 msgr2=0x7f5ac4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 --2- 172.21.15.3:0/3876534259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5ac4003fe0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009a90 tx=0x7f5ac4005830 comp rx=0 tx=0).stop 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae1600640 1 -- 172.21.15.3:0/3876534259 reap_dead start 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 -- 172.21.15.3:0/3876534259 shutdown_connections 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 -- 172.21.15.3:0/3876534259 >> 172.21.15.3:0/3876534259 conn(0x7f5adc0d4650 msgr2=0x7f5adc0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 -- 172.21.15.3:0/3876534259 shutdown_connections 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.837+0000 7f5ae2000640 1 -- 172.21.15.3:0/3876534259 wait complete. 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.839+0000 7f5ae2000640 1 Processor -- start 2024-09-10T07:52:50.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.839+0000 7f5ae2000640 1 -- start start 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5adc0cc670 0x7f5adc1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc1f4220 0x7f5adc1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0d2460 con 0x7f5ac4003be0 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0cf400 con 0x7f5adc0cc670 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae2000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc0cf6b0 con 0x7f5adc1f4220 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57766/0 (socket says 172.21.15.3:57766) 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 -- 172.21.15.3:0/511417760 learned_addr learned my addr 172.21.15.3:0/511417760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:50.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5adbe00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5adc0cc670 0x7f5adc1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae1600640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc1f4220 0x7f5adc1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 -- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc1f4220 msgr2=0x7f5adc1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc1f4220 0x7f5adc1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 -- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5adc0cc670 msgr2=0x7f5adc1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.840+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5adc0cc670 0x7f5adc1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ae0c00640 1 -- 172.21.15.3:0/511417760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1f4c10 con 0x7f5ac4003be0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ae0c00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009bb0 tx=0x7f5ad0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0040020 con 0x7f5ac4003be0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad0003a40 con 0x7f5ac4003be0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0036040 con 0x7f5ac4003be0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc15c4f0 con 0x7f5ac4003be0 2024-09-10T07:52:50.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.841+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc0cceb0 con 0x7f5ac4003be0 2024-09-10T07:52:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.842+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc073600 con 0x7f5ac4003be0 2024-09-10T07:52:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.843+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ad0043020 con 0x7f5ac4003be0 2024-09-10T07:52:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.844+0000 7f5ad8c00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 0x7f5abc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:50.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.844+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad00ab390 con 0x7f5ac4003be0 2024-09-10T07:52:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.844+0000 7f5adbe00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 0x7f5abc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:50.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.845+0000 7f5adbe00640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 0x7f5abc06e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5adc161260 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:50.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:50.848+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ad004a030 con 0x7f5ac4003be0 2024-09-10T07:52:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.069+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5adc0cde10 con 0x7f5ac4003be0 2024-09-10T07:52:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.070+0000 7f5ad8c00640 1 -- 172.21.15.3:0/511417760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f5ad0079cb0 con 0x7f5ac4003be0 2024-09-10T07:52:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:51.070 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 msgr2=0x7f5abc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 0x7f5abc06e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5adc161260 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 msgr2=0x7f5adc0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009bb0 tx=0x7f5ad0004730 comp rx=0 tx=0).stop 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 shutdown_connections 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5abc06be60 0x7f5abc06e320 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5adc1f4220 0x7f5adc1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5adc0cc670 0x7f5adc1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 --2- 172.21.15.3:0/511417760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ac4003be0 0x7f5adc0cc130 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 >> 172.21.15.3:0/511417760 conn(0x7f5adc0d4650 msgr2=0x7f5adc0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 shutdown_connections 2024-09-10T07:52:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.072+0000 7f5ae2000640 1 -- 172.21.15.3:0/511417760 wait complete. 2024-09-10T07:52:51.087 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:51.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.244+0000 7f4ffd600640 1 Processor -- start 2024-09-10T07:52:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.244+0000 7f4ffd600640 1 -- start start 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.245+0000 7f4ffd600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ff8070f30 0x7f4ff8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.245+0000 7f4ffd600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff80cd7f0 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.245+0000 7f4ffd600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff80cf880 con 0x7f4ff815c720 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.245+0000 7f4ffd600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff8072210 con 0x7f4ff8070f30 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff7400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4ff80cfdc0 0x7f4ff815cb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57668/0 (socket says 172.21.15.3:57668) 2024-09-10T07:52:51.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff7400640 1 -- 172.21.15.3:0/3472613247 learned_addr learned my addr 172.21.15.3:0/3472613247 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff6000640 1 --2- 172.21.15.3:0/3472613247 >> v2:172.21.15.70:3300/0 conn(0x7f4ff8070f30 0x7f4ff8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265340965 0 0) 0x7f4ff80cd7f0 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fdc003520 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 328482082 0 0) 0x7f4fdc003520 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 >> v2:172.21.15.70:3300/0 conn(0x7f4ff8070f30 msgr2=0x7f4ff8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 --2- 172.21.15.3:0/3472613247 >> v2:172.21.15.70:3300/0 conn(0x7f4ff8070f30 0x7f4ff8071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 >> v1:172.21.15.70:6789/0 conn(0x7f4ff815c720 legacy=0x7f4ff8073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff80cda90 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f4ff00019f0 con 0x7f4ff80cfdc0 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.246+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 >> v1:172.21.15.3:6789/0 conn(0x7f4ff80cfdc0 legacy=0x7f4ff815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc0039b0 0x7f4fdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0043b0 0x7f4fdc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 0x7f4fdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8072210 con 0x7f4fdc005080 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc00bfe0 con 0x7f4fdc0043b0 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc00c1e0 con 0x7f4fdc0039b0 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 0x7f4fdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff6a00640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc0039b0 0x7f4fdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff6000640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0043b0 0x7f4fdc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 -- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc0039b0 msgr2=0x7f4fdc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc0039b0 0x7f4fdc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 -- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0043b0 msgr2=0x7f4fdc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0043b0 0x7f4fdc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.247+0000 7f4ff7400640 1 -- 172.21.15.3:0/3472613247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc003520 con 0x7f4fdc005080 2024-09-10T07:52:51.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.248+0000 7f4ff7400640 1 -- 172.21.15.3:0/3472613247 reap_dead start 2024-09-10T07:52:51.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.248+0000 7f4ff7400640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 0x7f4fdc00b9c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000a810 tx=0x7f4fdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721e060c5a04eac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000a430 con 0x7f4fdc005080 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff002f070 con 0x7f4fdc005080 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ff5600640 1 -- 172.21.15.3:0/3472613247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff002bc90 con 0x7f4fdc005080 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 -- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 msgr2=0x7f4fdc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 0x7f4fdc00b9c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000a810 tx=0x7f4fdc005830 comp rx=0 tx=0).stop 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 -- 172.21.15.3:0/3472613247 shutdown_connections 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 --2- 172.21.15.3:0/3472613247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fdc005080 0x7f4fdc00b9c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 -- 172.21.15.3:0/3472613247 >> 172.21.15.3:0/3472613247 conn(0x7f4ff806cfc0 msgr2=0x7f4ff806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 -- 172.21.15.3:0/3472613247 shutdown_connections 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.249+0000 7f4ffd600640 1 -- 172.21.15.3:0/3472613247 wait complete. 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.251+0000 7f4ffd600640 1 Processor -- start 2024-09-10T07:52:51.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.251+0000 7f4ffd600640 1 -- start start 2024-09-10T07:52:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 0x7f4ff80b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff814ffa0 0x7f4ff8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8071cd0 con 0x7f4ff80b21e0 2024-09-10T07:52:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff806ed10 con 0x7f4ff814ffa0 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ffd600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff8071e50 con 0x7f4fdc005080 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ff6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ff6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57782/0 (socket says 172.21.15.3:57782) 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.252+0000 7f4ff6000640 1 -- 172.21.15.3:0/1303074014 learned_addr learned my addr 172.21.15.3:0/1303074014 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6a00640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 0x7f4ff80b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff7400640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff814ffa0 0x7f4ff8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 -- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 msgr2=0x7f4ff80b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 0x7f4ff80b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 -- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff814ffa0 msgr2=0x7f4ff8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff814ffa0 0x7f4ff8152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 -- 172.21.15.3:0/1303074014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff8154650 con 0x7f4ff80b21e0 2024-09-10T07:52:51.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6a00640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 0x7f4ff80b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ff6000640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40118e0 tx=0x7f4fe4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400fa90 con 0x7f4ff80b21e0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe400fc30 con 0x7f4ff80b21e0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4014440 con 0x7f4ff80b21e0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.253+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff8154930 con 0x7f4ff80b21e0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.254+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff8154ef0 con 0x7f4ff80b21e0 2024-09-10T07:52:51.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.255+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff80cf880 con 0x7f4ff80b21e0 2024-09-10T07:52:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.256+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fe4012040 con 0x7f4ff80b21e0 2024-09-10T07:52:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.256+0000 7f4feea00640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 0x7f4fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.257+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe408b560 con 0x7f4ff80b21e0 2024-09-10T07:52:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.257+0000 7f4ff6a00640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 0x7f4fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:51.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.257+0000 7f4ff6a00640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 0x7f4fd806e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815d7c0 tx=0x7f4fe8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:51.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.261+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4fe4019030 con 0x7f4ff80b21e0 2024-09-10T07:52:51.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.519+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4ff80cf2b0 con 0x7f4ff80b21e0 2024-09-10T07:52:51.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.519+0000 7f4feea00640 1 -- 172.21.15.3:0/1303074014 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1959 (secure 0 0 0) 0x7f4fe4059f00 con 0x7f4ff80b21e0 2024-09-10T07:52:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:51.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270032896,"bytes_avail":772824080384,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-10T07:52:47.574928+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 msgr2=0x7f4fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 0x7f4fd806e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815d7c0 tx=0x7f4fe8009040 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 msgr2=0x7f4ff8152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40118e0 tx=0x7f4fe4011da0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 shutdown_connections 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4fd806be30 0x7f4fd806e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ff814ffa0 0x7f4ff8152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ff80b21e0 0x7f4ff8152890 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 --2- 172.21.15.3:0/1303074014 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fdc005080 0x7f4ff80b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:51.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 >> 172.21.15.3:0/1303074014 conn(0x7f4ff806cfc0 msgr2=0x7f4ff815d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:51.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 shutdown_connections 2024-09-10T07:52:51.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:51.522+0000 7f4ffd600640 1 -- 172.21.15.3:0/1303074014 wait complete. 2024-09-10T07:52:51.538 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:52.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:52:52.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:52:52.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:52.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.700+0000 7f38b2c00640 1 Processor -- start 2024-09-10T07:52:52.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.700+0000 7f38b2c00640 1 -- start start 2024-09-10T07:52:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac15e8e0 0x7f38ac162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38ac0cf1f0 0x7f38ac0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac0d19d0 0x7f38ac0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f38ac0b7920 con 0x7f38ac0d19d0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f38ac0cf940 con 0x7f38ac0cf1f0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac0cfc20 con 0x7f38ac15e8e0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b0c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac0d19d0 0x7f38ac0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b0c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38ac0d19d0 0x7f38ac0b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57790/0 (socket says 172.21.15.3:57790) 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.701+0000 7f38b0c00640 1 -- 172.21.15.3:0/2120073506 learned_addr learned my addr 172.21.15.3:0/2120073506 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38abe00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac15e8e0 0x7f38ac162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38ab400640 1 --2- 172.21.15.3:0/2120073506 >> v2:172.21.15.70:3300/0 conn(0x7f38ac0cf1f0 0x7f38ac0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38b0c00640 1 -- 172.21.15.3:0/2120073506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac15e8e0 msgr2=0x7f38ac162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac15e8e0 0x7f38ac162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38b0c00640 1 -- 172.21.15.3:0/2120073506 >> v2:172.21.15.70:3300/0 conn(0x7f38ac0cf1f0 msgr2=0x7f38ac0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2120073506 >> v2:172.21.15.70:3300/0 conn(0x7f38ac0cf1f0 0x7f38ac0d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.702+0000 7f38b0c00640 1 -- 172.21.15.3:0/2120073506 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac0d2ae0 con 0x7f38ac0d19d0 2024-09-10T07:52:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.703+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b6170 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009e50 tx=0x7f38a402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4724cfaae6e9d801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.703+0000 7f38aaa00640 1 -- 172.21.15.3:0/2120073506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4036070 con 0x7f38ac0d19d0 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.703+0000 7f38aaa00640 1 -- 172.21.15.3:0/2120073506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a4031600 con 0x7f38ac0d19d0 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.703+0000 7f38aaa00640 1 -- 172.21.15.3:0/2120073506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a40319d0 con 0x7f38ac0d19d0 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 -- 172.21.15.3:0/2120073506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac0d19d0 msgr2=0x7f38ac0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b6170 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009e50 tx=0x7f38a402a630 comp rx=0 tx=0).stop 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 -- 172.21.15.3:0/2120073506 shutdown_connections 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b6170 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2120073506 >> v2:172.21.15.70:3300/0 conn(0x7f38ac0cf1f0 0x7f38ac0d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2120073506 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac15e8e0 0x7f38ac162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 -- 172.21.15.3:0/2120073506 >> 172.21.15.3:0/2120073506 conn(0x7f38ac06bd00 msgr2=0x7f38ac06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 -- 172.21.15.3:0/2120073506 shutdown_connections 2024-09-10T07:52:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.704+0000 7f38b2c00640 1 -- 172.21.15.3:0/2120073506 wait complete. 2024-09-10T07:52:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.705+0000 7f38b2c00640 1 Processor -- start 2024-09-10T07:52:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 -- start start 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 0x7f38ac0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac0b66b0 con 0x7f38ac15e8e0 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac0cf630 con 0x7f38ac0d19d0 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.706+0000 7f38b2c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac0b6830 con 0x7f38ac0cf1f0 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57806/0 (socket says 172.21.15.3:57806) 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 -- 172.21.15.3:0/2243372621 learned_addr learned my addr 172.21.15.3:0/2243372621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38abe00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 0x7f38ac0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38ab400640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 -- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 msgr2=0x7f38ac0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 0x7f38ac0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 -- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38ac0d19d0 msgr2=0x7f38ac0b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 -- 172.21.15.3:0/2243372621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac1f4720 con 0x7f38ac15e8e0 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38abe00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 0x7f38ac0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.707+0000 7f38b0c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f38a4034040 tx=0x7f38a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.708+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4036050 con 0x7f38ac15e8e0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.708+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a4042070 con 0x7f38ac15e8e0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.708+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac1f4a00 con 0x7f38ac15e8e0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.708+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4009510 con 0x7f38ac15e8e0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.708+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1f4fc0 con 0x7f38ac15e8e0 2024-09-10T07:52:52.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.709+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac0b4a60 con 0x7f38ac15e8e0 2024-09-10T07:52:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.710+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38a4007b50 con 0x7f38ac15e8e0 2024-09-10T07:52:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.711+0000 7f38a3e00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 0x7f388c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.711+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f38a40abb80 con 0x7f38ac15e8e0 2024-09-10T07:52:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.711+0000 7f38abe00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 0x7f388c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.712+0000 7f38abe00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 0x7f388c06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f389c009af0 tx=0x7f389c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.715+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38ac0b4a60 con 0x7f38ac15e8e0 2024-09-10T07:52:52.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.939+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ac0d1490 con 0x7f38ac15e8e0 2024-09-10T07:52:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.939+0000 7f38a3e00640 1 -- 172.21.15.3:0/2243372621 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f38a407a520 con 0x7f38ac15e8e0 2024-09-10T07:52:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:52.939 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 msgr2=0x7f388c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 0x7f388c06e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f389c009af0 tx=0x7f389c008040 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 msgr2=0x7f38ac0b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f38a4034040 tx=0x7f38a4004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 shutdown_connections 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f388c06be30 0x7f388c06e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38ac15e8e0 0x7f38ac0b8ed0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38ac0d19d0 0x7f38ac0b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 --2- 172.21.15.3:0/2243372621 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38ac0cf1f0 0x7f38ac0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 >> 172.21.15.3:0/2243372621 conn(0x7f38ac06bd00 msgr2=0x7f38ac160d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:52.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 shutdown_connections 2024-09-10T07:52:52.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:52.942+0000 7f38b2c00640 1 -- 172.21.15.3:0/2243372621 wait complete. 2024-09-10T07:52:52.958 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:52.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.114+0000 7f2f8b400640 1 Processor -- start 2024-09-10T07:52:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.114+0000 7f2f8b400640 1 -- start start 2024-09-10T07:52:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.115+0000 7f2f8b400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2f840b4aa0 0x7f2f8406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.115+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f840b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f8b400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f840ccd40 con 0x7f2f8415b410 2024-09-10T07:52:53.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f8b400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8406f250 con 0x7f2f840b4aa0 2024-09-10T07:52:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f8b400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f840cd020 con 0x7f2f840cfac0 2024-09-10T07:52:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2f840b4aa0 0x7f2f8406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2f840b4aa0 0x7f2f8406ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57820/0 (socket says 172.21.15.3:57820) 2024-09-10T07:52:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 -- 172.21.15.3:0/2724463978 learned_addr learned my addr 172.21.15.3:0/2724463978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:53.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f89400640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f840b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 -- 172.21.15.3:0/2724463978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 msgr2=0x7f2f840b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f840b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 -- 172.21.15.3:0/2724463978 >> v1:172.21.15.70:6789/0 conn(0x7f2f8415b410 legacy=0x7f2f840cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f83e00640 1 -- 172.21.15.3:0/2724463978 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f840cd300 con 0x7f2f840b4aa0 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.116+0000 7f2f89400640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f840b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:53.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.117+0000 7f2f83e00640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f840b4aa0 0x7f2f8406ed10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000bb10 tx=0x7f2f7002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97170f6c79c29df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f83400640 1 -- 172.21.15.3:0/2724463978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70036020 con 0x7f2f840b4aa0 2024-09-10T07:52:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f83400640 1 -- 172.21.15.3:0/2724463978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70032820 con 0x7f2f840b4aa0 2024-09-10T07:52:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f83400640 1 -- 172.21.15.3:0/2724463978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70032bf0 con 0x7f2f840b4aa0 2024-09-10T07:52:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 -- 172.21.15.3:0/2724463978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f840b4aa0 msgr2=0x7f2f8406ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f840b4aa0 0x7f2f8406ed10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000bb10 tx=0x7f2f7002c920 comp rx=0 tx=0).stop 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 -- 172.21.15.3:0/2724463978 shutdown_connections 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f840b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 --2- 172.21.15.3:0/2724463978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f840b4aa0 0x7f2f8406ed10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 -- 172.21.15.3:0/2724463978 >> 172.21.15.3:0/2724463978 conn(0x7f2f8406cfc0 msgr2=0x7f2f8406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 -- 172.21.15.3:0/2724463978 shutdown_connections 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.118+0000 7f2f8b400640 1 -- 172.21.15.3:0/2724463978 wait complete. 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.120+0000 7f2f8b400640 1 Processor -- start 2024-09-10T07:52:53.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.120+0000 7f2f8b400640 1 -- start start 2024-09-10T07:52:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b4aa0 0x7f2f841507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f84150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f840cdba0 con 0x7f2f8415b410 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f840cd8c0 con 0x7f2f840b4aa0 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f8b400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f84072540 con 0x7f2f840cfac0 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f89400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f89400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57830/0 (socket says 172.21.15.3:57830) 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f89400640 1 -- 172.21.15.3:0/1275161567 learned_addr learned my addr 172.21.15.3:0/1275161567 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f88a00640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b4aa0 0x7f2f841507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.121+0000 7f2f83e00640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f84150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 -- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 msgr2=0x7f2f84150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f84150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 -- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b4aa0 msgr2=0x7f2f841507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b4aa0 0x7f2f841507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 -- 172.21.15.3:0/1275161567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f841f0270 con 0x7f2f8415b410 2024-09-10T07:52:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f89400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007ec0 tx=0x7f2f7c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:53.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c01a070 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c015600 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0159f0 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.122+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f84153350 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.123+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f84153910 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.124+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f84153b10 con 0x7f2f8415b410 2024-09-10T07:52:53.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.125+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f7c01d070 con 0x7f2f8415b410 2024-09-10T07:52:53.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.125+0000 7f2f80c00640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 0x7f2f6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:53.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.126+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c08b0a0 con 0x7f2f8415b410 2024-09-10T07:52:53.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.126+0000 7f2f88a00640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 0x7f2f6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:53.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.126+0000 7f2f88a00640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 0x7f2f6406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f78001780 tx=0x7f2f78009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.130+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f841f0270 con 0x7f2f8415b410 2024-09-10T07:52:53.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.383+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2f8415cab0 con 0x7f2f8415b410 2024-09-10T07:52:53.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.384+0000 7f2f80c00640 1 -- 172.21.15.3:0/1275161567 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1959 (secure 0 0 0) 0x7f2f7c010070 con 0x7f2f8415b410 2024-09-10T07:52:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:53.384 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":406,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":270032896,"bytes_avail":772824080384,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-10T07:52:47.574928+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 msgr2=0x7f2f6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 0x7f2f6406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f78001780 tx=0x7f2f78009040 comp rx=0 tx=0).stop 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 msgr2=0x7f2f84152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c007ec0 tx=0x7f2f7c00d4a0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 shutdown_connections 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2f6406be30 0x7f2f6406e2f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f8415b410 0x7f2f84152670 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f840cfac0 0x7f2f84150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 --2- 172.21.15.3:0/1275161567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f840b4aa0 0x7f2f841507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 >> 172.21.15.3:0/1275161567 conn(0x7f2f8406cfc0 msgr2=0x7f2f840b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 shutdown_connections 2024-09-10T07:52:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:53.386+0000 7f2f8b400640 1 -- 172.21.15.3:0/1275161567 wait complete. 2024-09-10T07:52:53.402 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:52:54.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:54.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.570+0000 7fe9bf200640 1 Processor -- start 2024-09-10T07:52:54.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.570+0000 7fe9bf200640 1 -- start start 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9bf200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9c015e860 0x7fe9c00cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9c00d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9bf200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c00cee40 con 0x7fe9c00d4150 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9bf200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c00d3780 con 0x7fe9c015e860 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9bf200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c00cf120 con 0x7fe9c00cfbf0 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9be800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe9c00d4150 0x7fe9c00b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45352/0 (socket says 172.21.15.3:45352) 2024-09-10T07:52:54.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.571+0000 7fe9be800640 1 -- 172.21.15.3:0/3512934819 learned_addr learned my addr 172.21.15.3:0/3512934819 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9b7e00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9c00d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> v2:172.21.15.70:3300/0 conn(0x7fe9c015e860 0x7fe9c00cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4033188142 0 0) 0x7fe9c00cee40 con 0x7fe9c00d4150 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a4003520 con 0x7fe9c00d4150 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3939156364 0 0) 0x7fe9a4003520 con 0x7fe9c00d4150 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 msgr2=0x7fe9c00d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9c00d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 >> v2:172.21.15.70:3300/0 conn(0x7fe9c015e860 msgr2=0x7fe9c00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 --2- 172.21.15.3:0/3512934819 >> v2:172.21.15.70:3300/0 conn(0x7fe9c015e860 0x7fe9c00cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c00d3410 con 0x7fe9c00d4150 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fe9b80019f0 con 0x7fe9c00d4150 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9b7e00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9c00d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> v2:172.21.15.70:3300/0 conn(0x7fe9c015e860 0x7fe9c00cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 >> v1:172.21.15.3:6789/0 conn(0x7fe9c00d4150 legacy=0x7fe9c00b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4003a20 0x7fe9a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9a4004d40 0x7fe9a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00d3780 con 0x7fe9a4003a20 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00cf120 con 0x7fe9a4004d40 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.572+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a400b9c0 con 0x7fe9c00cfbf0 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4003a20 0x7fe9a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9be800640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9a4004d40 0x7fe9a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9b7e00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 -- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 msgr2=0x7fe9a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9c00cfbf0 0x7fe9a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 -- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9a4004d40 msgr2=0x7fe9a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9a4004d40 0x7fe9a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9bde00640 1 -- 172.21.15.3:0/3512934819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a4003520 con 0x7fe9a4003a20 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.573+0000 7fe9be800640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9a4004d40 0x7fe9a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.574+0000 7fe9bde00640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4003a20 0x7fe9a4003e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009c90 tx=0x7fe9a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5969e695099c4862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.574+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0036070 con 0x7fe9a4003a20 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.574+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b0031400 con 0x7fe9a4003a20 2024-09-10T07:52:54.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.574+0000 7fe9bd400640 1 -- 172.21.15.3:0/3512934819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0009860 con 0x7fe9a4003a20 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 -- 172.21.15.3:0/3512934819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4003a20 msgr2=0x7fe9a4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 --2- 172.21.15.3:0/3512934819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4003a20 0x7fe9a4003e40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009c90 tx=0x7fe9a4005830 comp rx=0 tx=0).stop 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9be800640 1 -- 172.21.15.3:0/3512934819 reap_dead start 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 -- 172.21.15.3:0/3512934819 shutdown_connections 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 -- 172.21.15.3:0/3512934819 >> 172.21.15.3:0/3512934819 conn(0x7fe9c00d4650 msgr2=0x7fe9c00d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 -- 172.21.15.3:0/3512934819 shutdown_connections 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.575+0000 7fe9bf200640 1 -- 172.21.15.3:0/3512934819 wait complete. 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.577+0000 7fe9bf200640 1 Processor -- start 2024-09-10T07:52:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.577+0000 7fe9bf200640 1 -- start start 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9a4003a20 0x7fe9c014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c00cc770 0x7fe9c00ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00d3c10 con 0x7fe9c01f40b0 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00d2460 con 0x7fe9c00cc770 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bf200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c00d3d90 con 0x7fe9a4003a20 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9be800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9be800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36084/0 (socket says 172.21.15.3:36084) 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9be800640 1 -- 172.21.15.3:0/1881011263 learned_addr learned my addr 172.21.15.3:0/1881011263 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9b7e00640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c00cc770 0x7fe9c00ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9bde00640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9a4003a20 0x7fe9c014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.578+0000 7fe9be800640 1 -- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9a4003a20 msgr2=0x7fe9c014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9be800640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9a4003a20 0x7fe9c014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9be800640 1 -- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c00cc770 msgr2=0x7fe9c00ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9be800640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c00cc770 0x7fe9c00ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9be800640 1 -- 172.21.15.3:0/1881011263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c00cd1a0 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9be800640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cec30 tx=0x7fe9b800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800ed40 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b8014430 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80147e0 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c00cd480 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.579+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c00cda40 con 0x7fe9c01f40b0 2024-09-10T07:52:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.580+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c00cf6b0 con 0x7fe9c01f40b0 2024-09-10T07:52:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.581+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9b8014980 con 0x7fe9c01f40b0 2024-09-10T07:52:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.582+0000 7fe9b6000640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 0x7fe99c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.582+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fe9b808ac30 con 0x7fe9c01f40b0 2024-09-10T07:52:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.582+0000 7fe9bde00640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 0x7fe99c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.583+0000 7fe9bde00640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 0x7fe99c06e320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0033000 tx=0x7fe9b0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.587+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9b8058ef0 con 0x7fe9c01f40b0 2024-09-10T07:52:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.807+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9c00b4f70 con 0x7fe9c01f40b0 2024-09-10T07:52:54.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.808+0000 7fe9b6000640 1 -- 172.21.15.3:0/1881011263 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7fe9b805d1f0 con 0x7fe9c01f40b0 2024-09-10T07:52:54.808 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:54.808 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:54.808 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 msgr2=0x7fe99c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 0x7fe99c06e320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0033000 tx=0x7fe9b0008030 comp rx=0 tx=0).stop 2024-09-10T07:52:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 msgr2=0x7fe9c01f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cec30 tx=0x7fe9b800e530 comp rx=0 tx=0).stop 2024-09-10T07:52:54.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 shutdown_connections 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe99c06be60 0x7fe99c06e320 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c01f40b0 0x7fe9c01f4530 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c00cc770 0x7fe9c00ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.810+0000 7fe9bf200640 1 --2- 172.21.15.3:0/1881011263 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe9a4003a20 0x7fe9c014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.811+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 >> 172.21.15.3:0/1881011263 conn(0x7fe9c00d4650 msgr2=0x7fe9c00b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.811+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 shutdown_connections 2024-09-10T07:52:54.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.811+0000 7fe9bf200640 1 -- 172.21.15.3:0/1881011263 wait complete. 2024-09-10T07:52:54.827 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:52:54.827 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:52:54.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.982+0000 7fb45b200640 1 Processor -- start 2024-09-10T07:52:54.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.982+0000 7fb45b200640 1 -- start start 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45b200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb45c15e420 0x7fb45c0d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45b200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45c0d2cd0 0x7fb45c06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45b200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c0c9800 con 0x7fb45c06c060 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45b200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb45c06c4b0 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45b200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb45c0d3300 con 0x7fb45c15e420 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb451400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45c0d2cd0 0x7fb45c06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45a800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb45c06c060 0x7fb45c0d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45366/0 (socket says 172.21.15.3:45366) 2024-09-10T07:52:54.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb45a800640 1 -- 172.21.15.3:0/200114143 learned_addr learned my addr 172.21.15.3:0/200114143 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:54.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.983+0000 7fb459e00640 1 --2- 172.21.15.3:0/200114143 >> v2:172.21.15.70:3300/0 conn(0x7fb45c15e420 0x7fb45c0d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.984+0000 7fb451400640 1 -- 172.21.15.3:0/200114143 >> v2:172.21.15.70:3300/0 conn(0x7fb45c15e420 msgr2=0x7fb45c0d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.984+0000 7fb451400640 1 --2- 172.21.15.3:0/200114143 >> v2:172.21.15.70:3300/0 conn(0x7fb45c15e420 0x7fb45c0d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.984+0000 7fb451400640 1 -- 172.21.15.3:0/200114143 >> v1:172.21.15.3:6789/0 conn(0x7fb45c06c060 legacy=0x7fb45c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.984+0000 7fb451400640 1 -- 172.21.15.3:0/200114143 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c0d35e0 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.985+0000 7fb451400640 1 --2- 172.21.15.3:0/200114143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c06bb20 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb45c073880 tx=0x7fb44402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e02fbdcb3c04bea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.985+0000 7fb459400640 1 -- 172.21.15.3:0/200114143 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444037070 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.985+0000 7fb459400640 1 -- 172.21.15.3:0/200114143 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb444032440 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.985+0000 7fb459400640 1 -- 172.21.15.3:0/200114143 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444032810 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 -- 172.21.15.3:0/200114143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 msgr2=0x7fb45c06bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 --2- 172.21.15.3:0/200114143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c06bb20 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb45c073880 tx=0x7fb44402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 -- 172.21.15.3:0/200114143 shutdown_connections 2024-09-10T07:52:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 --2- 172.21.15.3:0/200114143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c06bb20 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 --2- 172.21.15.3:0/200114143 >> v2:172.21.15.70:3300/0 conn(0x7fb45c15e420 0x7fb45c0d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 -- 172.21.15.3:0/200114143 >> 172.21.15.3:0/200114143 conn(0x7fb45c0732e0 msgr2=0x7fb45c062630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 -- 172.21.15.3:0/200114143 shutdown_connections 2024-09-10T07:52:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.986+0000 7fb45b200640 1 -- 172.21.15.3:0/200114143 wait complete. 2024-09-10T07:52:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.987+0000 7fb45b200640 1 Processor -- start 2024-09-10T07:52:54.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 -- start start 2024-09-10T07:52:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb45c06c060 0x7fb45c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb45c15e420 0x7fb45c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0d3e80 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0d3ba0 con 0x7fb45c06c060 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.988+0000 7fb45b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c15f810 con 0x7fb45c15e420 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36102/0 (socket says 172.21.15.3:36102) 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 -- 172.21.15.3:0/4216940674 learned_addr learned my addr 172.21.15.3:0/4216940674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb459e00640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb45c06c060 0x7fb45c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb45a800640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb45c15e420 0x7fb45c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 -- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb45c15e420 msgr2=0x7fb45c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb45c15e420 0x7fb45c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 -- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb45c06c060 msgr2=0x7fb45c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb45c06c060 0x7fb45c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 -- 172.21.15.3:0/4216940674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c151570 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.989+0000 7fb451400640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb444000c00 tx=0x7fb4440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.990+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444037040 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.990+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb444042070 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.990+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c0cfa20 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.990+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444030960 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.990+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c0ce130 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.991+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c15fba0 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.992+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb444030b00 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.992+0000 7fb452a00640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 0x7fb43806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.993+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fb4440ab7c0 con 0x7fb45c0d2cd0 2024-09-10T07:52:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.993+0000 7fb459e00640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 0x7fb43806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.993+0000 7fb459e00640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 0x7fb43806e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009a30 tx=0x7fb44c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:54.997+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb44407a160 con 0x7fb45c0d2cd0 2024-09-10T07:52:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.248+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb45c0cdc00 con 0x7fb45c0d2cd0 2024-09-10T07:52:55.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.249+0000 7fb452a00640 1 -- 172.21.15.3:0/4216940674 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1866 (secure 0 0 0) 0x7fb444079a60 con 0x7fb45c0d2cd0 2024-09-10T07:52:55.249 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:55.249 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":408,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":270036992,"bytes_avail":772824076288,"bytes_total":773094113280,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"d","status":"up:active","gid":6799},{"filesystem_id":6,"rank":1,"name":"a","status":"up:active","gid":7236},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7239}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-09-10T07:52:53.576458+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 msgr2=0x7fb43806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 0x7fb43806e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009a30 tx=0x7fb44c008040 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 msgr2=0x7fb45c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb444000c00 tx=0x7fb4440046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 shutdown_connections 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb43806be30 0x7fb43806e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb45c15e420 0x7fb45c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45c0d2cd0 0x7fb45c0ceeb0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 --2- 172.21.15.3:0/4216940674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb45c06c060 0x7fb45c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 >> 172.21.15.3:0/4216940674 conn(0x7fb45c0732e0 msgr2=0x7fb45c0b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 shutdown_connections 2024-09-10T07:52:55.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.251+0000 7fb45b200640 1 -- 172.21.15.3:0/4216940674 wait complete. 2024-09-10T07:52:55.267 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-10T07:52:55.267 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:52:55.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:52:55.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:52:55.458 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:52:55.458 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:52:55.458 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:52:55.458 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:55.494 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:52:55.495 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:55.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:52:55.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.768+0000 7f2c64c00640 1 Processor -- start 2024-09-10T07:52:55.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.768+0000 7f2c64c00640 1 -- start start 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c600cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c60160af0 0x7f2c600d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c600cf800 0x7f2c600d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2c600cfdd0 con 0x7f2c600cf800 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2c6015cc90 con 0x7f2c60160af0 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c64c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6015cf70 con 0x7f2c60160710 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c5e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c600cf800 0x7f2c600d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c5e800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2c600cf800 0x7f2c600d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36108/0 (socket says 172.21.15.3:36108) 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.769+0000 7f2c5e800640 1 -- 172.21.15.3:0/2082942524 learned_addr learned my addr 172.21.15.3:0/2082942524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5d400640 1 --2- 172.21.15.3:0/2082942524 >> v2:172.21.15.70:3300/0 conn(0x7f2c60160af0 0x7f2c600d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5de00640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c600cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5e800640 1 -- 172.21.15.3:0/2082942524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 msgr2=0x7f2c600cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:55.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5e800640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c600cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5e800640 1 -- 172.21.15.3:0/2082942524 >> v2:172.21.15.70:3300/0 conn(0x7f2c60160af0 msgr2=0x7f2c600d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:55.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5e800640 1 --2- 172.21.15.3:0/2082942524 >> v2:172.21.15.70:3300/0 conn(0x7f2c60160af0 0x7f2c600d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.770+0000 7f2c5e800640 1 -- 172.21.15.3:0/2082942524 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6015d250 con 0x7f2c600cf800 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.771+0000 7f2c5e800640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c600cf800 0x7f2c600d05f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006cd40 tx=0x7f2c5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d041ef15f2c046 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.771+0000 7f2c5ca00640 1 -- 172.21.15.3:0/2082942524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58036070 con 0x7f2c600cf800 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.771+0000 7f2c5ca00640 1 -- 172.21.15.3:0/2082942524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c58031600 con 0x7f2c600cf800 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.771+0000 7f2c5ca00640 1 -- 172.21.15.3:0/2082942524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580319d0 con 0x7f2c600cf800 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 -- 172.21.15.3:0/2082942524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c600cf800 msgr2=0x7f2c600d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c600cf800 0x7f2c600d05f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006cd40 tx=0x7f2c5802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 -- 172.21.15.3:0/2082942524 shutdown_connections 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c600cf800 0x7f2c600d05f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 --2- 172.21.15.3:0/2082942524 >> v2:172.21.15.70:3300/0 conn(0x7f2c60160af0 0x7f2c600d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 --2- 172.21.15.3:0/2082942524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c600cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 -- 172.21.15.3:0/2082942524 >> 172.21.15.3:0/2082942524 conn(0x7f2c6006d000 msgr2=0x7f2c6006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 -- 172.21.15.3:0/2082942524 shutdown_connections 2024-09-10T07:52:55.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.772+0000 7f2c64c00640 1 -- 172.21.15.3:0/2082942524 wait complete. 2024-09-10T07:52:55.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.773+0000 7f2c64c00640 1 Processor -- start 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.774+0000 7f2c64c00640 1 -- start start 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.774+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c600cf800 0x7f2c6014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c601538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c64c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6015d770 con 0x7f2c60160af0 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c64c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6006bb20 con 0x7f2c600cf800 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c64c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2c601551c0 con 0x7f2c60160710 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36122/0 (socket says 172.21.15.3:36122) 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 -- 172.21.15.3:0/965390730 learned_addr learned my addr 172.21.15.3:0/965390730 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:55.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5de00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c600cf800 0x7f2c6014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5d400640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c601538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 -- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 msgr2=0x7f2c601538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c601538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 -- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c600cf800 msgr2=0x7f2c6014acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c600cf800 0x7f2c6014acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5e800640 1 -- 172.21.15.3:0/965390730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6014c7f0 con 0x7f2c60160af0 2024-09-10T07:52:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.775+0000 7f2c5d400640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c601538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:55.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c5e800640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2c5802ac90 tx=0x7f2c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5803f020 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c58037040 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58036070 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c60154c30 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.776+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c60154670 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.777+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c600afd50 con 0x7f2c60160af0 2024-09-10T07:52:55.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.778+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c58042020 con 0x7f2c60160af0 2024-09-10T07:52:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.779+0000 7f2c56000640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 0x7f2c4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.779+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f2c580ab2b0 con 0x7f2c60160af0 2024-09-10T07:52:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.779+0000 7f2c5de00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 0x7f2c4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:55.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.780+0000 7f2c5de00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 0x7f2c4006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:55.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.783+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c58049030 con 0x7f2c60160af0 2024-09-10T07:52:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.979+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c6015c650 con 0x7f2c60160af0 2024-09-10T07:52:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.979+0000 7f2c56000640 1 -- 172.21.15.3:0/965390730 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) ==== 80+0+0 (secure 0 0 0) 0x7f2c6014c7f0 con 0x7f2c60160af0 2024-09-10T07:52:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:52:55.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 msgr2=0x7f2c4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 0x7f2c4006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50008040 comp rx=0 tx=0).stop 2024-09-10T07:52:55.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 msgr2=0x7f2c6015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:55.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2c5802ac90 tx=0x7f2c58004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:55.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 shutdown_connections 2024-09-10T07:52:55.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2c4006be30 0x7f2c4006e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c60160af0 0x7f2c6015eb70 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2c60160710 0x7f2c601538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 --2- 172.21.15.3:0/965390730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c600cf800 0x7f2c6014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:55.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 >> 172.21.15.3:0/965390730 conn(0x7f2c6006d000 msgr2=0x7f2c60152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:55.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 shutdown_connections 2024-09-10T07:52:55.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:55.982+0000 7f2c64c00640 1 -- 172.21.15.3:0/965390730 wait complete. 2024-09-10T07:52:55.998 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:52:55.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:55.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:52:56.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:52:56.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:52:56.064 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:52:56.099 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:52:56.100 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:52:56.100 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:56.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:52:56.181 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:52:56.181 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:52:56.181 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:56.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:52:56.217 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:52:56.217 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:52:56.217 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:52:56.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:52:56.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:56.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.416+0000 7f0086a00640 1 Processor -- start 2024-09-10T07:52:56.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.416+0000 7f0086a00640 1 -- start start 2024-09-10T07:52:56.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00800b6690 0x7f00800cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6a70 0x7f00800d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00800d1510 0x7f00800d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f00800cf540 con 0x7f00800d1510 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f00800cff10 con 0x7f00800b6690 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0086a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00800d1d90 con 0x7f00800b6a70 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0084a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00800d1510 0x7f00800d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0084a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00800d1510 0x7f00800d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36126/0 (socket says 172.21.15.3:36126) 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.417+0000 7f0084a00640 1 -- 172.21.15.3:0/2255006457 learned_addr learned my addr 172.21.15.3:0/2255006457 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f007fe00640 1 --2- 172.21.15.3:0/2255006457 >> v2:172.21.15.70:3300/0 conn(0x7f00800b6690 0x7f00800cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f007f400640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6a70 0x7f00800d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f0084a00640 1 -- 172.21.15.3:0/2255006457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6a70 msgr2=0x7f00800d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f0084a00640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6a70 0x7f00800d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f0084a00640 1 -- 172.21.15.3:0/2255006457 >> v2:172.21.15.70:3300/0 conn(0x7f00800b6690 msgr2=0x7f00800cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f0084a00640 1 --2- 172.21.15.3:0/2255006457 >> v2:172.21.15.70:3300/0 conn(0x7f00800b6690 0x7f00800cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.418+0000 7f0084a00640 1 -- 172.21.15.3:0/2255006457 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00800ceef0 con 0x7f00800d1510 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.419+0000 7f0084a00640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800d1510 0x7f00800d2b50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f00800d2210 tx=0x7f007802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de258428e97ca335 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.419+0000 7f007ea00640 1 -- 172.21.15.3:0/2255006457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078036070 con 0x7f00800d1510 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.419+0000 7f007ea00640 1 -- 172.21.15.3:0/2255006457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078031600 con 0x7f00800d1510 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.419+0000 7f007ea00640 1 -- 172.21.15.3:0/2255006457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00780319d0 con 0x7f00800d1510 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 -- 172.21.15.3:0/2255006457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800d1510 msgr2=0x7f00800d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800d1510 0x7f00800d2b50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f00800d2210 tx=0x7f007802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 -- 172.21.15.3:0/2255006457 shutdown_connections 2024-09-10T07:52:56.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800d1510 0x7f00800d2b50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 --2- 172.21.15.3:0/2255006457 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6a70 0x7f00800d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 --2- 172.21.15.3:0/2255006457 >> v2:172.21.15.70:3300/0 conn(0x7f00800b6690 0x7f00800cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 -- 172.21.15.3:0/2255006457 >> 172.21.15.3:0/2255006457 conn(0x7f0080072560 msgr2=0x7f0080072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 -- 172.21.15.3:0/2255006457 shutdown_connections 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.420+0000 7f0086a00640 1 -- 172.21.15.3:0/2255006457 wait complete. 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.421+0000 7f0086a00640 1 Processor -- start 2024-09-10T07:52:56.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 -- start start 2024-09-10T07:52:56.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6690 0x7f0080157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00800d1510 0x7f00800cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00800d19a0 con 0x7f00800b6a70 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00800b6ed0 con 0x7f00800d1510 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.422+0000 7f0086a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00800d1b20 con 0x7f00800b6690 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36142/0 (socket says 172.21.15.3:36142) 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 -- 172.21.15.3:0/2585845386 learned_addr learned my addr 172.21.15.3:0/2585845386 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007fe00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6690 0x7f0080157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f0084a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00800d1510 0x7f00800cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 -- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6690 msgr2=0x7f0080157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6690 0x7f0080157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 -- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00800d1510 msgr2=0x7f00800cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00800d1510 0x7f00800cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.423+0000 7f007f400640 1 -- 172.21.15.3:0/2585845386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008014cff0 con 0x7f00800b6a70 2024-09-10T07:52:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f007f400640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d7d0 tx=0x7f006c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c004020 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006c0041e0 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0080159090 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0043d0 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.424+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0080156e30 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.425+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008015c530 con 0x7f00800b6a70 2024-09-10T07:52:56.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.426+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f006c01b020 con 0x7f00800b6a70 2024-09-10T07:52:56.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.427+0000 7f0077e00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 0x7f006006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.427+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f006c08ab50 con 0x7f00800b6a70 2024-09-10T07:52:56.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.427+0000 7f007fe00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 0x7f006006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.428+0000 7f007fe00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 0x7f006006e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0080159820 tx=0x7f0070009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.431+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f008014cff0 con 0x7f00800b6a70 2024-09-10T07:52:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.655+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00800b7b80 con 0x7f00800b6a70 2024-09-10T07:52:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.655+0000 7f0077e00640 1 -- 172.21.15.3:0/2585845386 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f006c058ea0 con 0x7f00800b6a70 2024-09-10T07:52:56.655 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:56.655 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.657+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 msgr2=0x7f006006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.657+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 0x7f006006e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0080159820 tx=0x7f0070009190 comp rx=0 tx=0).stop 2024-09-10T07:52:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.657+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 msgr2=0x7f008014cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.657+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d7d0 tx=0x7f006c00dc90 comp rx=0 tx=0).stop 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 shutdown_connections 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f006006be30 0x7f006006e2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00800d1510 0x7f00800cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00800b6a70 0x7f008014cab0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 --2- 172.21.15.3:0/2585845386 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00800b6690 0x7f0080157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 >> 172.21.15.3:0/2585845386 conn(0x7f0080072560 msgr2=0x7f008015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 shutdown_connections 2024-09-10T07:52:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.658+0000 7f0086a00640 1 -- 172.21.15.3:0/2585845386 wait complete. 2024-09-10T07:52:56.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.830+0000 7f5249000640 1 Processor -- start 2024-09-10T07:52:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.831+0000 7f5249000640 1 -- start start 2024-09-10T07:52:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f5249000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f52440b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f5249000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52440b4250 con 0x7f524415b4f0 2024-09-10T07:52:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f5249000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52440ccd80 con 0x7f52440d08f0 2024-09-10T07:52:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f5249000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52440b44f0 con 0x7f524415b110 2024-09-10T07:52:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f523be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f524415b4f0 0x7f52440d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45374/0 (socket says 172.21.15.3:45374) 2024-09-10T07:52:56.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f523be00640 1 -- 172.21.15.3:0/2695364609 learned_addr learned my addr 172.21.15.3:0/2695364609 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.832+0000 7f5241e00640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f52440b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377708502 0 0) 0x7f52440b4250 con 0x7f524415b4f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5228003520 con 0x7f524415b4f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489113005 0 0) 0x7f52440ccd80 con 0x7f52440d08f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52440b4250 con 0x7f52440d08f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 463888857 0 0) 0x7f5228003520 con 0x7f524415b4f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 msgr2=0x7f52440b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f52440b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 >> v1:172.21.15.70:6789/0 conn(0x7f52440d08f0 legacy=0x7f52440cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52440cd060 con 0x7f524415b4f0 2024-09-10T07:52:56.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f522c001690 con 0x7f524415b4f0 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 >> v1:172.21.15.3:6789/0 conn(0x7f524415b4f0 legacy=0x7f52440d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f5228003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52280040d0 0x7f5228004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5228004d40 0x7f52280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52440b44f0 con 0x7f5228004d40 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f522800baa0 con 0x7f52280040d0 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f522800bca0 con 0x7f524415b110 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5242800640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5228004d40 0x7f52280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f523be00640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52280040d0 0x7f5228004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.833+0000 7f5241e00640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f5228003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.834+0000 7f5242800640 1 -- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 msgr2=0x7f5228003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.834+0000 7f5242800640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f524415b110 0x7f5228003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.834+0000 7f5242800640 1 -- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52280040d0 msgr2=0x7f5228004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.834+0000 7f5242800640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52280040d0 0x7f5228004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.834+0000 7f5242800640 1 -- 172.21.15.3:0/2695364609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5228003520 con 0x7f5228004d40 2024-09-10T07:52:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5242800640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5228004d40 0x7f52280051c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f523c001fd0 tx=0x7f5228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dcdb7562270417 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c001bf0 con 0x7f5228004d40 2024-09-10T07:52:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c003b70 con 0x7f5228004d40 2024-09-10T07:52:56.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5241400640 1 -- 172.21.15.3:0/2695364609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0050d0 con 0x7f5228004d40 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5249000640 1 -- 172.21.15.3:0/2695364609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5228004d40 msgr2=0x7f52280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5249000640 1 --2- 172.21.15.3:0/2695364609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5228004d40 0x7f52280051c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f523c001fd0 tx=0x7f5228005830 comp rx=0 tx=0).stop 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.835+0000 7f5242800640 1 -- 172.21.15.3:0/2695364609 reap_dead start 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.836+0000 7f5249000640 1 -- 172.21.15.3:0/2695364609 shutdown_connections 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.836+0000 7f5249000640 1 -- 172.21.15.3:0/2695364609 >> 172.21.15.3:0/2695364609 conn(0x7f524406d000 msgr2=0x7f524406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.836+0000 7f5249000640 1 -- 172.21.15.3:0/2695364609 shutdown_connections 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.836+0000 7f5249000640 1 -- 172.21.15.3:0/2695364609 wait complete. 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.837+0000 7f5249000640 1 Processor -- start 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.837+0000 7f5249000640 1 -- start start 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5244157630 0x7f5244071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52440ae8a0 0x7f52440aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5244073d10 con 0x7f5244157a10 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52440cd620 con 0x7f5244157630 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.838+0000 7f5249000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5244073e90 con 0x7f52440ae8a0 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36162/0 (socket says 172.21.15.3:36162) 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 -- 172.21.15.3:0/2803162190 learned_addr learned my addr 172.21.15.3:0/2803162190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f5241e00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5244157630 0x7f5244071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f5242800640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52440ae8a0 0x7f52440aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 -- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52440ae8a0 msgr2=0x7f52440aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52440ae8a0 0x7f52440aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 -- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5244157630 msgr2=0x7f5244071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5244157630 0x7f5244071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:56.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 -- 172.21.15.3:0/2803162190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52441e6970 con 0x7f5244157a10 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.839+0000 7f523be00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f522c0017e0 tx=0x7f522c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.840+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00f7b0 con 0x7f5244157a10 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.840+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c018070 con 0x7f5244157a10 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.840+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441e58f0 con 0x7f5244157a10 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.840+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00fbb0 con 0x7f5244157a10 2024-09-10T07:52:56.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.840+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441e5eb0 con 0x7f5244157a10 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.841+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52440b3d10 con 0x7f5244157a10 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.842+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f522c00fd50 con 0x7f5244157a10 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.843+0000 7f523a000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 0x7f522006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.843+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f522c08add0 con 0x7f5244157a10 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.843+0000 7f5241e00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 0x7f522006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:56.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.844+0000 7f5241e00640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 0x7f522006e340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f524415f0c0 tx=0x7f5234009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:56.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:56.847+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52441e6970 con 0x7f5244157a10 2024-09-10T07:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.070+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52440d0280 con 0x7f5244157a10 2024-09-10T07:52:57.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.071+0000 7f523a000640 1 -- 172.21.15.3:0/2803162190 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f522c0597f0 con 0x7f5244157a10 2024-09-10T07:52:57.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:57.070 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:57.071 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 msgr2=0x7f522006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 0x7f522006e340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f524415f0c0 tx=0x7f5234009190 comp rx=0 tx=0).stop 2024-09-10T07:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 msgr2=0x7f5244072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f522c0017e0 tx=0x7f522c007430 comp rx=0 tx=0).stop 2024-09-10T07:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 shutdown_connections 2024-09-10T07:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f522006be80 0x7f522006e340 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52440ae8a0 0x7f52440aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5244157a10 0x7f5244072060 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 --2- 172.21.15.3:0/2803162190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5244157630 0x7f5244071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 >> 172.21.15.3:0/2803162190 conn(0x7f524406d000 msgr2=0x7f52440b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 shutdown_connections 2024-09-10T07:52:57.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.073+0000 7f5249000640 1 -- 172.21.15.3:0/2803162190 wait complete. 2024-09-10T07:52:57.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:52:57.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.246+0000 7f7569a00640 1 Processor -- start 2024-09-10T07:52:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.246+0000 7f7569a00640 1 -- start start 2024-09-10T07:52:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7564148130 0x7f75640b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7564072710 0x7f75640d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 0x7f75640ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f75640d1540 con 0x7f7564148130 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7564072f70 con 0x7f7564072710 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7569a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7564073250 con 0x7f75640d10d0 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.247+0000 7f7562a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7564148130 0x7f75640b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7564148130 0x7f75640b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36166/0 (socket says 172.21.15.3:36166) 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 -- 172.21.15.3:0/413209073 learned_addr learned my addr 172.21.15.3:0/413209073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f755a000640 1 --2- 172.21.15.3:0/413209073 >> v2:172.21.15.70:3300/0 conn(0x7f7564072710 0x7f75640d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7563400640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 0x7f75640ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 -- 172.21.15.3:0/413209073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 msgr2=0x7f75640ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 0x7f75640ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 -- 172.21.15.3:0/413209073 >> v2:172.21.15.70:3300/0 conn(0x7f7564072710 msgr2=0x7f75640d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:57.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 --2- 172.21.15.3:0/413209073 >> v2:172.21.15.70:3300/0 conn(0x7f7564072710 0x7f75640d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7562a00640 1 -- 172.21.15.3:0/413209073 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7564073530 con 0x7f7564148130 2024-09-10T07:52:57.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.248+0000 7f7563400640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 0x7f75640ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.249+0000 7f7562a00640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f75640b4a00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7564073180 tx=0x7f755402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8437fc9c5e752c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.249+0000 7f7562000640 1 -- 172.21.15.3:0/413209073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554037070 con 0x7f7564148130 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.249+0000 7f7562000640 1 -- 172.21.15.3:0/413209073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7554032440 con 0x7f7564148130 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.249+0000 7f7562000640 1 -- 172.21.15.3:0/413209073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554032810 con 0x7f7564148130 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 -- 172.21.15.3:0/413209073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 msgr2=0x7f75640b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f75640b4a00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7564073180 tx=0x7f755402a1c0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 -- 172.21.15.3:0/413209073 shutdown_connections 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f75640d10d0 0x7f75640ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 --2- 172.21.15.3:0/413209073 >> v2:172.21.15.70:3300/0 conn(0x7f7564072710 0x7f75640d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 --2- 172.21.15.3:0/413209073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f75640b4a00 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 -- 172.21.15.3:0/413209073 >> 172.21.15.3:0/413209073 conn(0x7f756406bf40 msgr2=0x7f756406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 -- 172.21.15.3:0/413209073 shutdown_connections 2024-09-10T07:52:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.250+0000 7f7569a00640 1 -- 172.21.15.3:0/413209073 wait complete. 2024-09-10T07:52:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.252+0000 7f7569a00640 1 Processor -- start 2024-09-10T07:52:57.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.252+0000 7f7569a00640 1 -- start start 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7564072710 0x7f756415dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 0x7f756413f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f756406d680 con 0x7f7564148130 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7564148510 con 0x7f75640d10d0 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7569a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f756406d800 con 0x7f7564072710 2024-09-10T07:52:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7563400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7563400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36170/0 (socket says 172.21.15.3:36170) 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.253+0000 7f7563400640 1 -- 172.21.15.3:0/487189661 learned_addr learned my addr 172.21.15.3:0/487189661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f755a000640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 0x7f756413f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7562a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7564072710 0x7f756415dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 -- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7564072710 msgr2=0x7f756415dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7564072710 0x7f756415dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 -- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 msgr2=0x7f756413f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 0x7f756413f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 -- 172.21.15.3:0/487189661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641e3f80 con 0x7f7564148130 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f755a000640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 0x7f756413f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7563400640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f755c007ef0 tx=0x7f755c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c01a070 con 0x7f7564148130 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c003b90 con 0x7f7564148130 2024-09-10T07:52:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c004030 con 0x7f7564148130 2024-09-10T07:52:57.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.254+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7564142540 con 0x7f7564148130 2024-09-10T07:52:57.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.255+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7564142b00 con 0x7f7564148130 2024-09-10T07:52:57.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.255+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75641e4430 con 0x7f7564148130 2024-09-10T07:52:57.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.256+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f755c01d070 con 0x7f7564148130 2024-09-10T07:52:57.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.258+0000 7f755b400640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 0x7f754006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.258+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f755c08afe0 con 0x7f7564148130 2024-09-10T07:52:57.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.258+0000 7f7562a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 0x7f754006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.259+0000 7f7562a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 0x7f754006e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7554034000 tx=0x7f755402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.262+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f755c010070 con 0x7f7564148130 2024-09-10T07:52:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.448+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75640b4480 con 0x7f7564148130 2024-09-10T07:52:57.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.449+0000 7f755b400640 1 -- 172.21.15.3:0/487189661 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16660 (secure 0 0 0) 0x7f75640b4480 con 0x7f7564148130 2024-09-10T07:52:57.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:57.449 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:52:02.576089+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:58.800477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T07:52:00.000902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:52:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.451+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 msgr2=0x7f754006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.451+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 0x7f754006e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7554034000 tx=0x7f755402e000 comp rx=0 tx=0).stop 2024-09-10T07:52:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.451+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 msgr2=0x7f7564141530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.451+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f755c007ef0 tx=0x7f755c00d4a0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 shutdown_connections 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f754006be30 0x7f754006e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7564148130 0x7f7564141530 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75640d10d0 0x7f756413f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 --2- 172.21.15.3:0/487189661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7564072710 0x7f756415dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 >> 172.21.15.3:0/487189661 conn(0x7f756406bf40 msgr2=0x7f75640b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 shutdown_connections 2024-09-10T07:52:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.452+0000 7f7569a00640 1 -- 172.21.15.3:0/487189661 wait complete. 2024-09-10T07:52:57.468 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:52:57.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:57.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.629+0000 7fad90c00640 1 Processor -- start 2024-09-10T07:52:57.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.629+0000 7fad90c00640 1 -- start start 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad90c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad8c0b5b20 0x7fad8c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad90c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 0x7fad8c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad90c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8c06d480 con 0x7fad8c148000 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad90c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fad8c06e4c0 con 0x7fad8c0b5b20 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad90c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fad8c1483e0 con 0x7fad8c06d070 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad8aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad8c0b5b20 0x7fad8c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad8aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad8c0b5b20 0x7fad8c06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36184/0 (socket says 172.21.15.3:36184) 2024-09-10T07:52:57.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.630+0000 7fad8aa00640 1 -- 172.21.15.3:0/2142882514 learned_addr learned my addr 172.21.15.3:0/2142882514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8be00640 1 --2- 172.21.15.3:0/2142882514 >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 0x7fad8c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8aa00640 1 -- 172.21.15.3:0/2142882514 >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 msgr2=0x7fad8c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8aa00640 1 --2- 172.21.15.3:0/2142882514 >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 0x7fad8c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8aa00640 1 -- 172.21.15.3:0/2142882514 >> v1:172.21.15.70:6789/0 conn(0x7fad8c148000 legacy=0x7fad8c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8aa00640 1 -- 172.21.15.3:0/2142882514 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0b6290 con 0x7fad8c0b5b20 2024-09-10T07:52:57.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.631+0000 7fad8be00640 1 --2- 172.21.15.3:0/2142882514 >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 0x7fad8c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.632+0000 7fad8aa00640 1 --2- 172.21.15.3:0/2142882514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c06cb30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fad8c15eb50 tx=0x7fad7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea27586bd1ad192 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.632+0000 7fad8a000640 1 -- 172.21.15.3:0/2142882514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78037070 con 0x7fad8c0b5b20 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.632+0000 7fad8a000640 1 -- 172.21.15.3:0/2142882514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad78032440 con 0x7fad8c0b5b20 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.632+0000 7fad8a000640 1 -- 172.21.15.3:0/2142882514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78032810 con 0x7fad8c0b5b20 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 -- 172.21.15.3:0/2142882514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 msgr2=0x7fad8c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 --2- 172.21.15.3:0/2142882514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c06cb30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fad8c15eb50 tx=0x7fad7802a230 comp rx=0 tx=0).stop 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 -- 172.21.15.3:0/2142882514 shutdown_connections 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 --2- 172.21.15.3:0/2142882514 >> v2:172.21.15.70:3300/0 conn(0x7fad8c06d070 0x7fad8c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 --2- 172.21.15.3:0/2142882514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c06cb30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 -- 172.21.15.3:0/2142882514 >> 172.21.15.3:0/2142882514 conn(0x7fad8c071fb0 msgr2=0x7fad8c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 -- 172.21.15.3:0/2142882514 shutdown_connections 2024-09-10T07:52:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.633+0000 7fad90c00640 1 -- 172.21.15.3:0/2142882514 wait complete. 2024-09-10T07:52:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.635+0000 7fad90c00640 1 Processor -- start 2024-09-10T07:52:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.635+0000 7fad90c00640 1 -- start start 2024-09-10T07:52:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 0x7fad8c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 0x7fad8c13e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c06e6f0 con 0x7fad8c0b5b20 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0b6850 con 0x7fad8c06d070 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad90c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0b5f90 con 0x7fad8c148000 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36196/0 (socket says 172.21.15.3:36196) 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 -- 172.21.15.3:0/226375401 learned_addr learned my addr 172.21.15.3:0/226375401 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8b400640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 0x7fad8c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8be00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 0x7fad8c13e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 -- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 msgr2=0x7fad8c13e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 0x7fad8c13e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 -- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 msgr2=0x7fad8c1603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 0x7fad8c1603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8aa00640 1 -- 172.21.15.3:0/226375401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c160d00 con 0x7fad8c0b5b20 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8b400640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 0x7fad8c1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.636+0000 7fad8be00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 0x7fad8c13e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad8aa00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78041020 con 0x7fad8c0b5b20 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad78030970 con 0x7fad8c0b5b20 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78037040 con 0x7fad8c0b5b20 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c160fe0 con 0x7fad8c0b5b20 2024-09-10T07:52:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.637+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c1615a0 con 0x7fad8c0b5b20 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.638+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c06cb30 con 0x7fad8c0b5b20 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.639+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad78044020 con 0x7fad8c0b5b20 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.640+0000 7fad83400640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 0x7fad6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.640+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fad780ab930 con 0x7fad8c0b5b20 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.640+0000 7fad8b400640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 0x7fad6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:57.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.641+0000 7fad8b400640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 0x7fad6c06e320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00a830 tx=0x7fad7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:57.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.644+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad7804b030 con 0x7fad8c0b5b20 2024-09-10T07:52:57.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.867+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad8c147740 con 0x7fad8c0b5b20 2024-09-10T07:52:57.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.868+0000 7fad83400640 1 -- 172.21.15.3:0/226375401 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7fad8c06d480 con 0x7fad8c0b5b20 2024-09-10T07:52:57.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:57.868 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:57.868 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:57.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 msgr2=0x7fad6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 0x7fad6c06e320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00a830 tx=0x7fad7c008040 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 msgr2=0x7fad8c13daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad780046c0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 shutdown_connections 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad6c06be60 0x7fad6c06e320 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad8c148000 0x7fad8c13e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8c0b5b20 0x7fad8c13daf0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 --2- 172.21.15.3:0/226375401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c06d070 0x7fad8c1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 >> 172.21.15.3:0/226375401 conn(0x7fad8c071fb0 msgr2=0x7fad8c072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 shutdown_connections 2024-09-10T07:52:57.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:57.870+0000 7fad90c00640 1 -- 172.21.15.3:0/226375401 wait complete. 2024-09-10T07:52:57.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:52:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.043+0000 7f5588000640 1 Processor -- start 2024-09-10T07:52:58.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.043+0000 7f5588000640 1 -- start start 2024-09-10T07:52:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5580160710 0x7f55800cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160af0 0x7f55800d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55800cf800 0x7f55800d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f55800cfdd0 con 0x7f55800cf800 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f558015cc90 con 0x7f5580160710 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5588000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f558015cf70 con 0x7f5580160af0 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.044+0000 7f5586000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55800cf800 0x7f55800d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55800cf800 0x7f55800d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36212/0 (socket says 172.21.15.3:36212) 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 -- 172.21.15.3:0/4162252109 learned_addr learned my addr 172.21.15.3:0/4162252109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:58.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5584c00640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160af0 0x7f55800d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5585600640 1 --2- 172.21.15.3:0/4162252109 >> v2:172.21.15.70:3300/0 conn(0x7f5580160710 0x7f55800cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 -- 172.21.15.3:0/4162252109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160af0 msgr2=0x7f55800d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160af0 0x7f55800d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 -- 172.21.15.3:0/4162252109 >> v2:172.21.15.70:3300/0 conn(0x7f5580160710 msgr2=0x7f55800cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 --2- 172.21.15.3:0/4162252109 >> v2:172.21.15.70:3300/0 conn(0x7f5580160710 0x7f55800cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.045+0000 7f5586000640 1 -- 172.21.15.3:0/4162252109 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558015d250 con 0x7f55800cf800 2024-09-10T07:52:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.046+0000 7f5586000640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55800cf800 0x7f55800d05f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f558006ccc0 tx=0x7f557802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bc0c3f0ea382f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.046+0000 7f557fe00640 1 -- 172.21.15.3:0/4162252109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578036070 con 0x7f55800cf800 2024-09-10T07:52:58.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.046+0000 7f557fe00640 1 -- 172.21.15.3:0/4162252109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578031600 con 0x7f55800cf800 2024-09-10T07:52:58.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.046+0000 7f557fe00640 1 -- 172.21.15.3:0/4162252109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780319d0 con 0x7f55800cf800 2024-09-10T07:52:58.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 -- 172.21.15.3:0/4162252109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55800cf800 msgr2=0x7f55800d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55800cf800 0x7f55800d05f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f558006ccc0 tx=0x7f557802a6a0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 -- 172.21.15.3:0/4162252109 shutdown_connections 2024-09-10T07:52:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55800cf800 0x7f55800d05f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 --2- 172.21.15.3:0/4162252109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160af0 0x7f55800d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 --2- 172.21.15.3:0/4162252109 >> v2:172.21.15.70:3300/0 conn(0x7f5580160710 0x7f55800cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 -- 172.21.15.3:0/4162252109 >> 172.21.15.3:0/4162252109 conn(0x7f558006d000 msgr2=0x7f558006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 -- 172.21.15.3:0/4162252109 shutdown_connections 2024-09-10T07:52:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.047+0000 7f5588000640 1 -- 172.21.15.3:0/4162252109 wait complete. 2024-09-10T07:52:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.049+0000 7f5588000640 1 Processor -- start 2024-09-10T07:52:58.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.049+0000 7f5588000640 1 -- start start 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800cf800 0x7f558014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160710 0x7f558014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f558015d770 con 0x7f5580160af0 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f558006bb20 con 0x7f55800cf800 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5588000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f558014ef80 con 0x7f5580160710 2024-09-10T07:52:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36216/0 (socket says 172.21.15.3:36216) 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 -- 172.21.15.3:0/3936464202 learned_addr learned my addr 172.21.15.3:0/3936464202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5585600640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800cf800 0x7f558014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5584c00640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160710 0x7f558014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 -- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160710 msgr2=0x7f558014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160710 0x7f558014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 -- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800cf800 msgr2=0x7f558014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800cf800 0x7f558014d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.050+0000 7f5586000640 1 -- 172.21.15.3:0/3936464202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558014f220 con 0x7f5580160af0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f5586000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f557802ac90 tx=0x7f5578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557803f020 con 0x7f5580160af0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578037040 con 0x7f5580160af0 2024-09-10T07:52:58.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558014f500 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578036070 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.051+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55801f38f0 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.052+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558014fa20 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.053+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5578042020 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.054+0000 7f557d600640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 0x7f556006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.054+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f55780ab4b0 con 0x7f5580160af0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.054+0000 7f5585600640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 0x7f556006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.055+0000 7f5585600640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 0x7f556006e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5574009b40 tx=0x7f5574009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.058+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f558014fa20 con 0x7f5580160af0 2024-09-10T07:52:58.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.285+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55801f3af0 con 0x7f5580160af0 2024-09-10T07:52:58.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.285+0000 7f557d600640 1 -- 172.21.15.3:0/3936464202 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5263 (secure 0 0 0) 0x7f5578049020 con 0x7f5580160af0 2024-09-10T07:52:58.286 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:58.286 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":96,"btime":"2024-09-10T07:52:39:678571+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:52:00.997662+0000","modified":"2024-09-10T07:52:39.661612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7239,"mds_1":7236,"mds_2":6799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6799":{"gid":6799,"name":"d","rank":2,"incarnation":89,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3096818521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3096818521},{"type":"v1","addr":"172.21.15.70:6835","nonce":3096818521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7236":{"gid":7236,"name":"a","rank":1,"incarnation":87,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3872292817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3872292817},{"type":"v1","addr":"172.21.15.3:6836","nonce":3872292817}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7239":{"gid":7239,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/2433616379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":2433616379},{"type":"v1","addr":"172.21.15.3:6837","nonce":2433616379}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7239,"qdb_cluster":[6799,7236,7239]},"id":6}]} 2024-09-10T07:52:58.286 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 96 2024-09-10T07:52:58.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 msgr2=0x7f556006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 0x7f556006e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5574009b40 tx=0x7f5574009190 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 msgr2=0x7f5580155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f557802ac90 tx=0x7f5578004a40 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 shutdown_connections 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f556006be30 0x7f556006e2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5580160af0 0x7f5580155e00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5580160710 0x7f558014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 --2- 172.21.15.3:0/3936464202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55800cf800 0x7f558014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 >> 172.21.15.3:0/3936464202 conn(0x7f558006d000 msgr2=0x7f55800b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 shutdown_connections 2024-09-10T07:52:58.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.288+0000 7f5588000640 1 -- 172.21.15.3:0/3936464202 wait complete. 2024-09-10T07:52:58.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:52:58.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.459+0000 7f0f61600640 1 Processor -- start 2024-09-10T07:52:58.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.459+0000 7f0f61600640 1 -- start start 2024-09-10T07:52:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f61600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f5c15b110 0x7f0f5c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f61600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5c0b4250 con 0x7f0f5c0d08f0 2024-09-10T07:52:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f61600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5c0ccd80 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f61600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c0b44f0 con 0x7f0f5c15b110 2024-09-10T07:52:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f5b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f5c0d08f0 0x7f0f5c0cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45384/0 (socket says 172.21.15.3:45384) 2024-09-10T07:52:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f5b400640 1 -- 172.21.15.3:0/1701712063 learned_addr learned my addr 172.21.15.3:0/1701712063 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f5aa00640 1 --2- 172.21.15.3:0/1701712063 >> v2:172.21.15.70:3300/0 conn(0x7f0f5c15b110 0x7f0f5c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f5a000640 1 -- 172.21.15.3:0/1701712063 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1087575271 0 0) 0x7f0f5c0b4250 con 0x7f0f5c0d08f0 2024-09-10T07:52:58.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.461+0000 7f0f5a000640 1 -- 172.21.15.3:0/1701712063 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f40003520 con 0x7f0f5c0d08f0 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.462+0000 7f0f5aa00640 1 -- 172.21.15.3:0/1701712063 >> v1:172.21.15.70:6789/0 conn(0x7f0f5c15b4f0 legacy=0x7f0f5c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.462+0000 7f0f5aa00640 1 -- 172.21.15.3:0/1701712063 >> v1:172.21.15.3:6789/0 conn(0x7f0f5c0d08f0 legacy=0x7f0f5c0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.462+0000 7f0f5aa00640 1 -- 172.21.15.3:0/1701712063 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c0cd060 con 0x7f0f5c15b110 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.463+0000 7f0f5aa00640 1 --2- 172.21.15.3:0/1701712063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c0b3d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097a0 tx=0x7f0f4c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b00119857078386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.463+0000 7f0f5a000640 1 -- 172.21.15.3:0/1701712063 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c037070 con 0x7f0f5c15b110 2024-09-10T07:52:58.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.463+0000 7f0f5a000640 1 -- 172.21.15.3:0/1701712063 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c032440 con 0x7f0f5c15b110 2024-09-10T07:52:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.463+0000 7f0f5a000640 1 -- 172.21.15.3:0/1701712063 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c032810 con 0x7f0f5c15b110 2024-09-10T07:52:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 -- 172.21.15.3:0/1701712063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 msgr2=0x7f0f5c0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 --2- 172.21.15.3:0/1701712063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c0b3d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097a0 tx=0x7f0f4c02a230 comp rx=0 tx=0).stop 2024-09-10T07:52:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 -- 172.21.15.3:0/1701712063 shutdown_connections 2024-09-10T07:52:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 --2- 172.21.15.3:0/1701712063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c0b3d10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 -- 172.21.15.3:0/1701712063 >> 172.21.15.3:0/1701712063 conn(0x7f0f5c06d000 msgr2=0x7f0f5c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 -- 172.21.15.3:0/1701712063 shutdown_connections 2024-09-10T07:52:58.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.464+0000 7f0f61600640 1 -- 172.21.15.3:0/1701712063 wait complete. 2024-09-10T07:52:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.465+0000 7f0f61600640 1 Processor -- start 2024-09-10T07:52:58.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 -- start start 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f5c0d08f0 0x7f0f5c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c073d10 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0cd620 con 0x7f0f5c15b110 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.466+0000 7f0f61600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c073e90 con 0x7f0f5c0d08f0 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36224/0 (socket says 172.21.15.3:36224) 2024-09-10T07:52:58.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 -- 172.21.15.3:0/4179223527 learned_addr learned my addr 172.21.15.3:0/4179223527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5aa00640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f5c0d08f0 0x7f0f5c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f52000640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 -- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f5c0d08f0 msgr2=0x7f0f5c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f5c0d08f0 0x7f0f5c151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 -- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 msgr2=0x7f0f5c154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 -- 172.21.15.3:0/4179223527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c155900 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f52000640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.467+0000 7f0f5b400640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400cb30 tx=0x7f0f5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.468+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017070 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.468+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5400aca0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.468+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54012600 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.468+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c155be0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.468+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c15dfe0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.469+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c0d0f20 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.470+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f5401a070 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.471+0000 7f0f53400640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 0x7f0f3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.471+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f0f5408aba0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.471+0000 7f0f5aa00640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 0x7f0f3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.472+0000 7f0f5aa00640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 0x7f0f3806e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c034000 tx=0x7f0f4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.475+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f54058ef0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.662+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f5c15db60 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.663+0000 7f0f53400640 1 -- 172.21.15.3:0/4179223527 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16660 (secure 0 0 0) 0x7f0f5405d1f0 con 0x7f0f5c15b4f0 2024-09-10T07:52:58.663 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:52:58.663 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:52:02.576089+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:51:58.800477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T07:52:00.000902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 msgr2=0x7f0f3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 0x7f0f3806e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c034000 tx=0x7f0f4c02e000 comp rx=0 tx=0).stop 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 msgr2=0x7f0f5c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400cb30 tx=0x7f0f5400a060 comp rx=0 tx=0).stop 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 shutdown_connections 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f3806be30 0x7f0f3806e2f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f5c15b4f0 0x7f0f5c06bda0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c15b110 0x7f0f5c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 --2- 172.21.15.3:0/4179223527 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f5c0d08f0 0x7f0f5c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 >> 172.21.15.3:0/4179223527 conn(0x7f0f5c06d000 msgr2=0x7f0f5c073320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 shutdown_connections 2024-09-10T07:52:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.666+0000 7f0f61600640 1 -- 172.21.15.3:0/4179223527 wait complete. 2024-09-10T07:52:58.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:52:58.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.840+0000 7fbbd6000640 1 Processor -- start 2024-09-10T07:52:58.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.841+0000 7fbbd6000640 1 -- start start 2024-09-10T07:52:58.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbd6000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbd0160af0 0x7fbbd00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbd6000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd00d05f0 0x7fbbd0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbd6000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd00d0a80 con 0x7fbbd0160710 2024-09-10T07:52:58.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbd6000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd00cfdd0 con 0x7fbbd0160af0 2024-09-10T07:52:58.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbd6000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd015cc90 con 0x7fbbd00d05f0 2024-09-10T07:52:58.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbcf400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbbd0160710 0x7fbbd00cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45392/0 (socket says 172.21.15.3:45392) 2024-09-10T07:52:58.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbcf400640 1 -- 172.21.15.3:0/3385911098 learned_addr learned my addr 172.21.15.3:0/3385911098 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbcea00640 1 --2- 172.21.15.3:0/3385911098 >> v2:172.21.15.70:3300/0 conn(0x7fbbd0160af0 0x7fbbd00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.842+0000 7fbbcfe00640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd00d05f0 0x7fbbd0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780475821 0 0) 0x7fbbd00d0a80 con 0x7fbbd0160710 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbb4003520 con 0x7fbbd0160710 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544885421 0 0) 0x7fbbb4003520 con 0x7fbbd0160710 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd00d05f0 msgr2=0x7fbbd0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd00d05f0 0x7fbbd0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 >> v2:172.21.15.70:3300/0 conn(0x7fbbd0160af0 msgr2=0x7fbbd00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 --2- 172.21.15.3:0/3385911098 >> v2:172.21.15.70:3300/0 conn(0x7fbbd0160af0 0x7fbbd00d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd015cf70 con 0x7fbbd0160710 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fbbc0001670 con 0x7fbbd0160710 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 >> v1:172.21.15.3:6789/0 conn(0x7fbbd0160710 legacy=0x7fbbd00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbb4004800 0x7fbbb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbb40051c0 0x7fbbb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd00cfdd0 con 0x7fbbb40039b0 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd015cc90 con 0x7fbbb40051c0 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb400bf00 con 0x7fbbb4004800 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbcfe00640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd00d05f0 0x7fbbd0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbcea00640 1 --2- 172.21.15.3:0/3385911098 >> v2:172.21.15.70:3300/0 conn(0x7fbbd0160af0 0x7fbbd00d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:52:58.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.843+0000 7fbbcf400640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcea00640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbb4004800 0x7fbbb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcfe00640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbb40051c0 0x7fbbb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcf400640 1 -- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbb4004800 msgr2=0x7fbbb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcf400640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbb4004800 0x7fbbb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcf400640 1 -- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbb40051c0 msgr2=0x7fbbb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcf400640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbb40051c0 0x7fbbb400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcf400640 1 -- 172.21.15.3:0/3385911098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb4003520 con 0x7fbbb40039b0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.844+0000 7fbbcfe00640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbb40051c0 0x7fbbb400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbcf400640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbb4003dd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c190 tx=0x7fbbb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42111299d577862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0007c60 con 0x7fbbb40039b0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0005af0 con 0x7fbbb40039b0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbce000640 1 -- 172.21.15.3:0/3385911098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0007070 con 0x7fbbb40039b0 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbd6000640 1 -- 172.21.15.3:0/3385911098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 msgr2=0x7fbbb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbd6000640 1 --2- 172.21.15.3:0/3385911098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbb4003dd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c190 tx=0x7fbbb4005830 comp rx=0 tx=0).stop 2024-09-10T07:52:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.845+0000 7fbbcfe00640 1 -- 172.21.15.3:0/3385911098 reap_dead start 2024-09-10T07:52:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.846+0000 7fbbd6000640 1 -- 172.21.15.3:0/3385911098 shutdown_connections 2024-09-10T07:52:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.846+0000 7fbbd6000640 1 -- 172.21.15.3:0/3385911098 >> 172.21.15.3:0/3385911098 conn(0x7fbbd006d000 msgr2=0x7fbbd006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.846+0000 7fbbd6000640 1 -- 172.21.15.3:0/3385911098 shutdown_connections 2024-09-10T07:52:58.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.846+0000 7fbbd6000640 1 -- 172.21.15.3:0/3385911098 wait complete. 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.847+0000 7fbbd6000640 1 Processor -- start 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.847+0000 7fbbd6000640 1 -- start start 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbd0150c40 0x7fbbd0152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd0152e70 0x7fbbd00cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd015d770 con 0x7fbbb40039b0 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd015c960 con 0x7fbbd0150c40 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbd6000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd0154640 con 0x7fbbd0152e70 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.848+0000 7fbbcf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36254/0 (socket says 172.21.15.3:36254) 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 -- 172.21.15.3:0/226790243 learned_addr learned my addr 172.21.15.3:0/226790243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:58.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcea00640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbd0150c40 0x7fbbd0152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcfe00640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd0152e70 0x7fbbd00cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 -- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd0152e70 msgr2=0x7fbbd00cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd0152e70 0x7fbbd00cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 -- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbd0150c40 msgr2=0x7fbbd0152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbd0150c40 0x7fbbd0152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 -- 172.21.15.3:0/226790243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd0151ac0 con 0x7fbbb40039b0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbcf400640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c2b0 tx=0x7fbbd015d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.849+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0043070 con 0x7fbbb40039b0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.850+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0038040 con 0x7fbbb40039b0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.850+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0007070 con 0x7fbbb40039b0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.850+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd0151da0 con 0x7fbbb40039b0 2024-09-10T07:52:58.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.850+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd0152360 con 0x7fbbb40039b0 2024-09-10T07:52:58.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.851+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd00cfa60 con 0x7fbbb40039b0 2024-09-10T07:52:58.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.852+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbc0007210 con 0x7fbbb40039b0 2024-09-10T07:52:58.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.853+0000 7fbbc7400640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 0x7fbbb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:58.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.853+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fbbc00aa820 con 0x7fbbb40039b0 2024-09-10T07:52:58.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.853+0000 7fbbcea00640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 0x7fbbb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:58.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.854+0000 7fbbcea00640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 0x7fbbb006e320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0156d00 tx=0x7fbbbc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:58.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:58.857+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbbc0078ae0 con 0x7fbbb40039b0 2024-09-10T07:52:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.080+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbbd015ea70 con 0x7fbbb40039b0 2024-09-10T07:52:59.081 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:52:59.081+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:52:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.083+0000 7fbbc7400640 1 -- 172.21.15.3:0/226790243 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) ==== 161+0+0 (secure 0 0 0) 0x7fbbc007cde0 con 0x7fbbb40039b0 2024-09-10T07:52:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:52:59.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.086+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 msgr2=0x7fbbb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:59.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.086+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 0x7fbbb006e320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0156d00 tx=0x7fbbbc008040 comp rx=0 tx=0).stop 2024-09-10T07:52:59.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.086+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 msgr2=0x7fbbd0151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:59.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.086+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c2b0 tx=0x7fbbd015d140 comp rx=0 tx=0).stop 2024-09-10T07:52:59.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 shutdown_connections 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbbb006be60 0x7fbbb006e320 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbbd0152e70 0x7fbbd00cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbd0150c40 0x7fbbd0152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 --2- 172.21.15.3:0/226790243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbb40039b0 0x7fbbd0151150 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 >> 172.21.15.3:0/226790243 conn(0x7fbbd006d000 msgr2=0x7fbbd00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 shutdown_connections 2024-09-10T07:52:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.087+0000 7fbbd6000640 1 -- 172.21.15.3:0/226790243 wait complete. 2024-09-10T07:52:59.096 INFO:tasks.ceph.mds.d.smithi070.stderr: -1217> 2024-09-10T07:52:34.874+0000 7f3877800640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1) n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e641253b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.429987+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.097 INFO:tasks.ceph.mds.d.smithi070.stderr: -1201> 2024-09-10T07:52:34.874+0000 7f3877800640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/AsLsXsFsx@2},l=7392 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55e641254100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.099 INFO:tasks.ceph.mds.a.smithi003.stderr: -1327> 2024-09-10T07:52:34.873+0000 7f9536000640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1) n(v2 rc2024-09-10T07:52:18.703983+0000 4=0+4) (inest lock) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x5564a90e9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:52:18.703983+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.100 INFO:tasks.ceph.mds.a.smithi003.stderr: -1189> 2024-09-10T07:52:34.875+0000 7f953e200640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1) n(v1 rc2024-09-10T07:52:18.703983+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7392=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x5564a90e8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.429987+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.428987+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:52:18.703983+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:52:59.109 INFO:tasks.ceph.mds.c.smithi003.stderr: -2748> 2024-09-10T07:52:34.873+0000 7f8ca5000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1) n(v3 rc2024-09-10T07:52:18.703983+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7392=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7392 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561e6517b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.426987+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:52:18.703983+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.109 INFO:tasks.ceph.mds.c.smithi003.stderr: -2626> 2024-09-10T07:52:34.875+0000 7f8cad200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1) n(v4 rc2024-09-10T07:52:18.703983+0000 5=0+5) (inest sync->mix g=1,2) caps={7392=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561e6517bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:52:18.703983+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:52:18.427987+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:52:18.703983+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:52:59.144 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:52:59.147 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:52:59.160 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:52:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.272+0000 7f61cac00640 1 Processor -- start 2024-09-10T07:52:59.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.272+0000 7f61cac00640 1 -- start start 2024-09-10T07:52:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61c4161d10 0x7f61c40d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f61c40b76c0 0x7f61c40d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 0x7f61c40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f61c4072140 con 0x7f61c4161d10 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f61c40cf900 con 0x7f61c40b76c0 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61cac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f61c41620f0 con 0x7f61c40cef90 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61c3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61c4161d10 0x7f61c40d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61c3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f61c4161d10 0x7f61c40d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36306/0 (socket says 172.21.15.3:36306) 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61c3e00640 1 -- 172.21.15.3:0/3802296711 learned_addr learned my addr 172.21.15.3:0/3802296711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61c3400640 1 --2- 172.21.15.3:0/3802296711 >> v2:172.21.15.70:3300/0 conn(0x7f61c40b76c0 0x7f61c40d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.273+0000 7f61c8c00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 0x7f61c40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c3e00640 1 -- 172.21.15.3:0/3802296711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 msgr2=0x7f61c40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c3e00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 0x7f61c40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c3e00640 1 -- 172.21.15.3:0/3802296711 >> v2:172.21.15.70:3300/0 conn(0x7f61c40b76c0 msgr2=0x7f61c40d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:52:59.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c3e00640 1 --2- 172.21.15.3:0/3802296711 >> v2:172.21.15.70:3300/0 conn(0x7f61c40b76c0 0x7f61c40d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c3e00640 1 -- 172.21.15.3:0/3802296711 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c40b7b30 con 0x7f61c4161d10 2024-09-10T07:52:59.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.274+0000 7f61c8c00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 0x7f61c40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.275+0000 7f61c3e00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c4161d10 0x7f61c40d3b20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f61c40b7230 tx=0x7f61b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de77972f8115158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.275+0000 7f61c2a00640 1 -- 172.21.15.3:0/3802296711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4037070 con 0x7f61c4161d10 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.275+0000 7f61c2a00640 1 -- 172.21.15.3:0/3802296711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b4032440 con 0x7f61c4161d10 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.275+0000 7f61c2a00640 1 -- 172.21.15.3:0/3802296711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4032810 con 0x7f61c4161d10 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 -- 172.21.15.3:0/3802296711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c4161d10 msgr2=0x7f61c40d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c4161d10 0x7f61c40d3b20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f61c40b7230 tx=0x7f61b402a140 comp rx=0 tx=0).stop 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 -- 172.21.15.3:0/3802296711 shutdown_connections 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40cef90 0x7f61c40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 --2- 172.21.15.3:0/3802296711 >> v2:172.21.15.70:3300/0 conn(0x7f61c40b76c0 0x7f61c40d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 --2- 172.21.15.3:0/3802296711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c4161d10 0x7f61c40d3b20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 -- 172.21.15.3:0/3802296711 >> 172.21.15.3:0/3802296711 conn(0x7f61c40d4730 msgr2=0x7f61c40d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 -- 172.21.15.3:0/3802296711 shutdown_connections 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.276+0000 7f61cac00640 1 -- 172.21.15.3:0/3802296711 wait complete. 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.277+0000 7f61cac00640 1 Processor -- start 2024-09-10T07:52:59.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 -- start start 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40b76c0 0x7f61c415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61c4161d10 0x7f61c40ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4072670 con 0x7f61c40cef90 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4072390 con 0x7f61c4161d10 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.278+0000 7f61cac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f61c4072950 con 0x7f61c40b76c0 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36318/0 (socket says 172.21.15.3:36318) 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 -- 172.21.15.3:0/181702032 learned_addr learned my addr 172.21.15.3:0/181702032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3e00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40b76c0 0x7f61c415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c8c00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61c4161d10 0x7f61c40ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 -- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40b76c0 msgr2=0x7f61c415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40b76c0 0x7f61c415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 -- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61c4161d10 msgr2=0x7f61c40ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:52:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61c4161d10 0x7f61c40ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:52:59.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.279+0000 7f61c3400640 1 -- 172.21.15.3:0/181702032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c415e280 con 0x7f61c40cef90 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61c3400640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e890 tx=0x7f61b000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x7f61c40cef90 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b000c940 con 0x7f61c40cef90 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b000cd10 con 0x7f61c40cef90 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c415e560 con 0x7f61c40cef90 2024-09-10T07:52:59.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.280+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c415df30 con 0x7f61c40cef90 2024-09-10T07:52:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.281+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c415ec20 con 0x7f61c40cef90 2024-09-10T07:52:59.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.282+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61b001a070 con 0x7f61c40cef90 2024-09-10T07:52:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.283+0000 7f61bbe00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 0x7f61a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:52:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.283+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5967+0+0 (secure 0 0 0) 0x7f61b008b290 con 0x7f61c40cef90 2024-09-10T07:52:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.283+0000 7f61c3e00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 0x7f61a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:52:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.284+0000 7f61c3e00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 0x7f61a406e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f61b4034000 tx=0x7f61b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:52:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.287+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f61c415ec20 con 0x7f61c40cef90 2024-09-10T07:52:59.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:52:59.507+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f61c40cf3c0 con 0x7f61c40cef90 2024-09-10T07:53:00.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.085+0000 7f61bbe00640 1 -- 172.21.15.3:0/181702032 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) ==== 104+0+0 (secure 0 0 0) 0x7f61b0059450 con 0x7f61c40cef90 2024-09-10T07:53:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 msgr2=0x7f61a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 0x7f61a406e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f61b4034000 tx=0x7f61b402e000 comp rx=0 tx=0).stop 2024-09-10T07:53:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 msgr2=0x7f61c415c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e890 tx=0x7f61b000ed50 comp rx=0 tx=0).stop 2024-09-10T07:53:00.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 shutdown_connections 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f61a406be60 0x7f61a406e320 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61c4161d10 0x7f61c40ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61c40cef90 0x7f61c415c250 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 --2- 172.21.15.3:0/181702032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f61c40b76c0 0x7f61c415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.088+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 >> 172.21.15.3:0/181702032 conn(0x7f61c40d4730 msgr2=0x7f61c40d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.089+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 shutdown_connections 2024-09-10T07:53:00.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.089+0000 7f61cac00640 1 -- 172.21.15.3:0/181702032 wait complete. 2024-09-10T07:53:00.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:53:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.313+0000 7f5f9c200640 1 Processor -- start 2024-09-10T07:53:00.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.313+0000 7f5f9c200640 1 -- start start 2024-09-10T07:53:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.314+0000 7f5f9c200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f941591b0 0x7f5f9415c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.314+0000 7f5f9c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.314+0000 7f5f9c200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f940cd8d0 con 0x7f5f940b3e10 2024-09-10T07:53:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.314+0000 7f5f9c200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9415b900 con 0x7f5f941591b0 2024-09-10T07:53:00.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.314+0000 7f5f9c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5f940cdbb0 con 0x7f5f940cd360 2024-09-10T07:53:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f9a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f9a200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:41172/0 (socket says 172.21.15.3:41172) 2024-09-10T07:53:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f9a200640 1 -- 172.21.15.3:0/716622056 learned_addr learned my addr 172.21.15.3:0/716622056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f99800640 1 --2- 172.21.15.3:0/716622056 >> v2:172.21.15.70:3300/0 conn(0x7f5f941591b0 0x7f5f9415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f9a200640 1 -- 172.21.15.3:0/716622056 >> v2:172.21.15.70:3300/0 conn(0x7f5f941591b0 msgr2=0x7f5f9415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f9a200640 1 --2- 172.21.15.3:0/716622056 >> v2:172.21.15.70:3300/0 conn(0x7f5f941591b0 0x7f5f9415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.316+0000 7f5f9a200640 1 -- 172.21.15.3:0/716622056 >> v1:172.21.15.70:6789/0 conn(0x7f5f940b3e10 legacy=0x7f5f940ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.316+0000 7f5f9a200640 1 -- 172.21.15.3:0/716622056 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9415bb20 con 0x7f5f940cd360 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.315+0000 7f5f93e00640 1 -- 172.21.15.3:0/716622056 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476844369 0 0) 0x7f5f940cd8d0 con 0x7f5f940b3e10 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9a200640 1 --2- 172.21.15.3:0/716622056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009e50 tx=0x7f5f8c02c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4b4e9db700f8f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f93e00640 1 -- 172.21.15.3:0/716622056 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c032610 con 0x7f5f940cd360 2024-09-10T07:53:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f93e00640 1 -- 172.21.15.3:0/716622056 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c037070 con 0x7f5f940cd360 2024-09-10T07:53:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f93e00640 1 -- 172.21.15.3:0/716622056 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0329d0 con 0x7f5f940cd360 2024-09-10T07:53:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 -- 172.21.15.3:0/716622056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 msgr2=0x7f5f940b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 --2- 172.21.15.3:0/716622056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009e50 tx=0x7f5f8c02c920 comp rx=0 tx=0).stop 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 -- 172.21.15.3:0/716622056 shutdown_connections 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 --2- 172.21.15.3:0/716622056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940cd360 0x7f5f940b42a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 --2- 172.21.15.3:0/716622056 >> v2:172.21.15.70:3300/0 conn(0x7f5f941591b0 0x7f5f9415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.317+0000 7f5f9c200640 1 -- 172.21.15.3:0/716622056 >> 172.21.15.3:0/716622056 conn(0x7f5f94062cc0 msgr2=0x7f5f9406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.318+0000 7f5f9c200640 1 -- 172.21.15.3:0/716622056 shutdown_connections 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.318+0000 7f5f9c200640 1 -- 172.21.15.3:0/716622056 wait complete. 2024-09-10T07:53:00.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.319+0000 7f5f9c200640 1 Processor -- start 2024-09-10T07:53:00.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.319+0000 7f5f9c200640 1 -- start start 2024-09-10T07:53:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940b3e10 0x7f5f940711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f94143b80 0x7f5f941434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940ce410 con 0x7f5f940cd360 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940ce290 con 0x7f5f94143b80 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f9c200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5f940ce590 con 0x7f5f940b3e10 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.320+0000 7f5f98e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36322/0 (socket says 172.21.15.3:36322) 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 -- 172.21.15.3:0/2109505862 learned_addr learned my addr 172.21.15.3:0/2109505862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f99800640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940b3e10 0x7f5f940711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 -- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940b3e10 msgr2=0x7f5f940711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940b3e10 0x7f5f940711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 -- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f94143b80 msgr2=0x7f5f941434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:00.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f94143b80 0x7f5f941434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 -- 172.21.15.3:0/2109505862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f94155b10 con 0x7f5f940cd360 2024-09-10T07:53:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f98e00640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c460 tx=0x7f5f8000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005480 con 0x7f5f940cd360 2024-09-10T07:53:00.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f800035f0 con 0x7f5f940cd360 2024-09-10T07:53:00.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80003a00 con 0x7f5f940cd360 2024-09-10T07:53:00.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.321+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f94155df0 con 0x7f5f940cd360 2024-09-10T07:53:00.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.322+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941563b0 con 0x7f5f940cd360 2024-09-10T07:53:00.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.323+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9415c600 con 0x7f5f940cd360 2024-09-10T07:53:00.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.324+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f80003ba0 con 0x7f5f940cd360 2024-09-10T07:53:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.324+0000 7f5f91600640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 0x7f5f7406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.324+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5967+0+0 (secure 0 0 0) 0x7f5f8008abb0 con 0x7f5f940cd360 2024-09-10T07:53:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.325+0000 7f5f99800640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 0x7f5f7406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:00.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.325+0000 7f5f99800640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 0x7f5f7406e490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009970 tx=0x7f5f88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:00.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.329+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f80059450 con 0x7f5f940cd360 2024-09-10T07:53:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:00.511+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f9415dcf0 con 0x7f5f940cd360 2024-09-10T07:53:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.168+0000 7f5f91600640 1 -- 172.21.15.3:0/2109505862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f5f80058d50 con 0x7f5f940cd360 2024-09-10T07:53:01.168 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:53:01.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 msgr2=0x7f5f7406e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 0x7f5f7406e490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009970 tx=0x7f5f88008040 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 msgr2=0x7f5f94142f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c460 tx=0x7f5f8000c920 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 shutdown_connections 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5f7406bfd0 0x7f5f7406e490 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f94143b80 0x7f5f941434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f940cd360 0x7f5f94142f70 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 --2- 172.21.15.3:0/2109505862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5f940b3e10 0x7f5f940711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 >> 172.21.15.3:0/2109505862 conn(0x7f5f94062cc0 msgr2=0x7f5f9415c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 shutdown_connections 2024-09-10T07:53:01.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.174+0000 7f5f9c200640 1 -- 172.21.15.3:0/2109505862 wait complete. 2024-09-10T07:53:01.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:53:01.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.357+0000 7fea0e200640 1 Processor -- start 2024-09-10T07:53:01.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.358+0000 7fea0e200640 1 -- start start 2024-09-10T07:53:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea0e200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea080d3320 0x7fea080d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea0e200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea080d4ea0 0x7fea080b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea0e200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080cfe30 con 0x7fea0815e860 2024-09-10T07:53:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea0e200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fea080b3f10 con 0x7fea080d4ea0 2024-09-10T07:53:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea0e200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d38c0 con 0x7fea080d3320 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea07e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea080d4ea0 0x7fea080b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea07e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea080d4ea0 0x7fea080b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36334/0 (socket says 172.21.15.3:36334) 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea07e00640 1 -- 172.21.15.3:0/1769654409 learned_addr learned my addr 172.21.15.3:0/1769654409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/1769654409 >> v2:172.21.15.70:3300/0 conn(0x7fea080d3320 0x7fea080d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea07e00640 1 -- 172.21.15.3:0/1769654409 >> v2:172.21.15.70:3300/0 conn(0x7fea080d3320 msgr2=0x7fea080d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.359+0000 7fea07e00640 1 --2- 172.21.15.3:0/1769654409 >> v2:172.21.15.70:3300/0 conn(0x7fea080d3320 0x7fea080d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.360+0000 7fea07e00640 1 -- 172.21.15.3:0/1769654409 >> v1:172.21.15.70:6789/0 conn(0x7fea0815e860 legacy=0x7fea080cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.360+0000 7fea07e00640 1 -- 172.21.15.3:0/1769654409 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080d3ba0 con 0x7fea080d4ea0 2024-09-10T07:53:01.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea07e00640 1 --2- 172.21.15.3:0/1769654409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d4ea0 0x7fea080b61f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fea08162170 tx=0x7fea0002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77f4c978f0169e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea06a00640 1 -- 172.21.15.3:0/1769654409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00036070 con 0x7fea080d4ea0 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea06a00640 1 -- 172.21.15.3:0/1769654409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00031600 con 0x7fea080d4ea0 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea06a00640 1 -- 172.21.15.3:0/1769654409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000319d0 con 0x7fea080d4ea0 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 -- 172.21.15.3:0/1769654409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d4ea0 msgr2=0x7fea080b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 --2- 172.21.15.3:0/1769654409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d4ea0 0x7fea080b61f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fea08162170 tx=0x7fea0002a730 comp rx=0 tx=0).stop 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 -- 172.21.15.3:0/1769654409 shutdown_connections 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 --2- 172.21.15.3:0/1769654409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d4ea0 0x7fea080b61f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 --2- 172.21.15.3:0/1769654409 >> v2:172.21.15.70:3300/0 conn(0x7fea080d3320 0x7fea080d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.361+0000 7fea0e200640 1 -- 172.21.15.3:0/1769654409 >> 172.21.15.3:0/1769654409 conn(0x7fea08072560 msgr2=0x7fea08072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.362+0000 7fea0e200640 1 -- 172.21.15.3:0/1769654409 shutdown_connections 2024-09-10T07:53:01.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.362+0000 7fea0e200640 1 -- 172.21.15.3:0/1769654409 wait complete. 2024-09-10T07:53:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.363+0000 7fea0e200640 1 Processor -- start 2024-09-10T07:53:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.363+0000 7fea0e200640 1 -- start start 2024-09-10T07:53:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea080d4ea0 0x7fea0815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0815e860 0x7fea081f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea080d4440 con 0x7fea080d3320 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fea080d4160 con 0x7fea0815e860 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea0e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea080b6730 con 0x7fea080d4ea0 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea07400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea07400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36344/0 (socket says 172.21.15.3:36344) 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.364+0000 7fea07400640 1 -- 172.21.15.3:0/1282195601 learned_addr learned my addr 172.21.15.3:0/1282195601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07e00640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0815e860 0x7fea081f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea080d4ea0 0x7fea0815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 -- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea080d4ea0 msgr2=0x7fea0815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea080d4ea0 0x7fea0815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 -- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0815e860 msgr2=0x7fea081f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:01.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0815e860 0x7fea081f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:01.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 -- 172.21.15.3:0/1282195601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0815aa90 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.365+0000 7fea07400640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800c3c0 tx=0x7fe9f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.366+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80052d0 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.366+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8005490 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.366+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0815ad70 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.366+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8007260 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.366+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0815b330 con 0x7fea080d3320 2024-09-10T07:53:01.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.367+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea080b6ac0 con 0x7fea080d3320 2024-09-10T07:53:01.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.368+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9f8007400 con 0x7fea080d3320 2024-09-10T07:53:01.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.369+0000 7fe9ff400640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 0x7fe9e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:01.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.369+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5529+0+0 (secure 0 0 0) 0x7fe9f808a1f0 con 0x7fea080d3320 2024-09-10T07:53:01.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.369+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 0x7fe9e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:01.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.370+0000 7fe9ffe00640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 0x7fe9e406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fea080d4980 tx=0x7fe9f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:01.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.376+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9f80585f0 con 0x7fea080d3320 2024-09-10T07:53:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:01.562+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea080b6350 con 0x7fea080d3320 2024-09-10T07:53:02.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.174+0000 7fe9ff400640 1 -- 172.21.15.3:0/1282195601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7fe9f805c8f0 con 0x7fea080d3320 2024-09-10T07:53:02.174 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:53:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.178+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 msgr2=0x7fe9e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.178+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 0x7fe9e406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fea080d4980 tx=0x7fe9f0008040 comp rx=0 tx=0).stop 2024-09-10T07:53:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.179+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 msgr2=0x7fea0814cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.179+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800c3c0 tx=0x7fe9f800c880 comp rx=0 tx=0).stop 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 shutdown_connections 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe9e406be30 0x7fe9e406e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0815e860 0x7fea081f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea080d4ea0 0x7fea0815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 --2- 172.21.15.3:0/1282195601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea080d3320 0x7fea0814cb20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 >> 172.21.15.3:0/1282195601 conn(0x7fea08072560 msgr2=0x7fea080b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 shutdown_connections 2024-09-10T07:53:02.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.180+0000 7fea0e200640 1 -- 172.21.15.3:0/1282195601 wait complete. 2024-09-10T07:53:02.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:53:02.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.374+0000 7fe422800640 1 Processor -- start 2024-09-10T07:53:02.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.374+0000 7fe422800640 1 -- start start 2024-09-10T07:53:02.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe422800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c161b70 0x7fe41c0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe422800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe422800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41c0d3de0 con 0x7fe41c0d3860 2024-09-10T07:53:02.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe422800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe41c161f50 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe422800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c0d40c0 con 0x7fe41c161b70 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe421e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe420a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe41c0d3860 0x7fe41c0d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45414/0 (socket says 172.21.15.3:45414) 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe421e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36350/0 (socket says 172.21.15.3:36350) 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe420a00640 1 -- 172.21.15.3:0/1977432773 learned_addr learned my addr 172.21.15.3:0/1977432773 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.376+0000 7fe421400640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c161b70 0x7fe41c0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.377+0000 7fe421e00640 1 -- 172.21.15.3:0/1977432773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c161b70 msgr2=0x7fe41c0d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.377+0000 7fe421e00640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c161b70 0x7fe41c0d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.377+0000 7fe421e00640 1 -- 172.21.15.3:0/1977432773 >> v1:172.21.15.3:6789/0 conn(0x7fe41c0d3860 legacy=0x7fe41c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.377+0000 7fe41be00640 1 -- 172.21.15.3:0/1977432773 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107692580 0 0) 0x7fe41c0d3de0 con 0x7fe41c0d3860 2024-09-10T07:53:02.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.377+0000 7fe421e00640 1 -- 172.21.15.3:0/1977432773 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0cf690 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.378+0000 7fe421e00640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe414009d90 tx=0x7fe41402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d649d0d5b7a48b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.378+0000 7fe41be00640 1 -- 172.21.15.3:0/1977432773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414007740 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.378+0000 7fe41be00640 1 -- 172.21.15.3:0/1977432773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4140078e0 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.378+0000 7fe41be00640 1 -- 172.21.15.3:0/1977432773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414007cb0 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 -- 172.21.15.3:0/1977432773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0d4cb0 msgr2=0x7fe41c0cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe414009d90 tx=0x7fe41402a660 comp rx=0 tx=0).stop 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 -- 172.21.15.3:0/1977432773 shutdown_connections 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c0cfe90 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 --2- 172.21.15.3:0/1977432773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c161b70 0x7fe41c0d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 -- 172.21.15.3:0/1977432773 >> 172.21.15.3:0/1977432773 conn(0x7fe41c062630 msgr2=0x7fe41c062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 -- 172.21.15.3:0/1977432773 shutdown_connections 2024-09-10T07:53:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.379+0000 7fe422800640 1 -- 172.21.15.3:0/1977432773 wait complete. 2024-09-10T07:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.381+0000 7fe422800640 1 Processor -- start 2024-09-10T07:53:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.381+0000 7fe422800640 1 -- start start 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c0d3860 0x7fe41c1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0cf2d0 con 0x7fe41c161b70 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0c9fc0 con 0x7fe41c0d4cb0 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe422800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c0cf450 con 0x7fe41c0d3860 2024-09-10T07:53:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe421e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe421e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36364/0 (socket says 172.21.15.3:36364) 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe421e00640 1 -- 172.21.15.3:0/2207722863 learned_addr learned my addr 172.21.15.3:0/2207722863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe421400640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c0d3860 0x7fe41c1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.382+0000 7fe420a00640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 -- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c0d3860 msgr2=0x7fe41c1526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c0d3860 0x7fe41c1526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 -- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 msgr2=0x7fe41c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 -- 172.21.15.3:0/2207722863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0cd8b0 con 0x7fe41c161b70 2024-09-10T07:53:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe420a00640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:02.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe421e00640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe414009eb0 tx=0x7fe414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414040070 con 0x7fe41c161b70 2024-09-10T07:53:02.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe414031bc0 con 0x7fe41c161b70 2024-09-10T07:53:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41403c420 con 0x7fe41c161b70 2024-09-10T07:53:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.383+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c0cdb90 con 0x7fe41c161b70 2024-09-10T07:53:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.384+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c154350 con 0x7fe41c161b70 2024-09-10T07:53:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.385+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c0b7540 con 0x7fe41c161b70 2024-09-10T07:53:02.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.385+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe414039040 con 0x7fe41c161b70 2024-09-10T07:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.386+0000 7fe419600640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 0x7fe3fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.386+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7fe4140ac140 con 0x7fe41c161b70 2024-09-10T07:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.387+0000 7fe421400640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 0x7fe3fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.387+0000 7fe421400640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 0x7fe3fc06e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4100079a0 tx=0x7fe410008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.391+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe41407aca0 con 0x7fe41c161b70 2024-09-10T07:53:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.575+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe41c0b77f0 con 0x7fe41c161b70 2024-09-10T07:53:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.579+0000 7fe419600640 1 -- 172.21.15.3:0/2207722863 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fe41407a5a0 con 0x7fe41c161b70 2024-09-10T07:53:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 msgr2=0x7fe3fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 0x7fe3fc06e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4100079a0 tx=0x7fe410008040 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 msgr2=0x7fe41c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe414009eb0 tx=0x7fe414004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 shutdown_connections 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe3fc06be30 0x7fe3fc06e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c161b70 0x7fe41c15b580 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe41c0d4cb0 0x7fe41c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 --2- 172.21.15.3:0/2207722863 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe41c0d3860 0x7fe41c1526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 >> 172.21.15.3:0/2207722863 conn(0x7fe41c062630 msgr2=0x7fe41c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 shutdown_connections 2024-09-10T07:53:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.582+0000 7fe422800640 1 -- 172.21.15.3:0/2207722863 wait complete. 2024-09-10T07:53:02.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-10T07:53:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.755+0000 7ff6db000640 1 Processor -- start 2024-09-10T07:53:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.755+0000 7ff6db000640 1 -- start start 2024-09-10T07:53:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6db000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6d40b6a70 0x7ff6d40d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6db000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d40d1d90 con 0x7ff6d40b6690 2024-09-10T07:53:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6db000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d40ceef0 con 0x7ff6d40d1510 2024-09-10T07:53:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6db000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d40cf540 con 0x7ff6d40b6a70 2024-09-10T07:53:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6d3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff6d40b6690 0x7ff6d40cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45422/0 (socket says 172.21.15.3:45422) 2024-09-10T07:53:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6d3e00640 1 -- 172.21.15.3:0/3377489435 learned_addr learned my addr 172.21.15.3:0/3377489435 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:02.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6cb400640 1 --2- 172.21.15.3:0/3377489435 >> v2:172.21.15.70:3300/0 conn(0x7ff6d40b6a70 0x7ff6d40d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934155555 0 0) 0x7ff6d40d1d90 con 0x7ff6d40b6690 2024-09-10T07:53:02.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.757+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6b8003520 con 0x7ff6d40b6690 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2426273523 0 0) 0x7ff6b8003520 con 0x7ff6d40b6690 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 >> v2:172.21.15.70:3300/0 conn(0x7ff6d40b6a70 msgr2=0x7ff6d40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 --2- 172.21.15.3:0/3377489435 >> v2:172.21.15.70:3300/0 conn(0x7ff6d40b6a70 0x7ff6d40d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 >> v1:172.21.15.70:6789/0 conn(0x7ff6d40d1510 legacy=0x7ff6d40d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d40cf1d0 con 0x7ff6d40b6690 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7ff6c4001670 con 0x7ff6d40b6690 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6cb400640 1 --2- 172.21.15.3:0/3377489435 >> v2:172.21.15.70:3300/0 conn(0x7ff6d40b6a70 0x7ff6d40d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:02.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 >> v1:172.21.15.3:6789/0 conn(0x7ff6d40b6690 legacy=0x7ff6d40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b8004800 0x7ff6b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b80051c0 0x7ff6b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d40cf540 con 0x7ff6b80039b0 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b800c060 con 0x7ff6b80051c0 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b800c260 con 0x7ff6b8004800 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6cb400640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b8004800 0x7ff6b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.758+0000 7ff6d9000640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b80051c0 0x7ff6b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d3e00640 1 -- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b8004800 msgr2=0x7ff6b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b8004800 0x7ff6b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d3e00640 1 -- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b80051c0 msgr2=0x7ff6b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b80051c0 0x7ff6b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d3e00640 1 -- 172.21.15.3:0/3377489435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b8003520 con 0x7ff6b80039b0 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.759+0000 7ff6d9000640 1 -- 172.21.15.3:0/3377489435 reap_dead start 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6b8003dd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400c210 tx=0x7ff6b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a815083b594dcf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007d30 con 0x7ff6b80039b0 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c4005ad0 con 0x7ff6b80039b0 2024-09-10T07:53:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6d3400640 1 -- 172.21.15.3:0/3377489435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007070 con 0x7ff6b80039b0 2024-09-10T07:53:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6db000640 1 -- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 msgr2=0x7ff6b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.760+0000 7ff6db000640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6b8003dd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400c210 tx=0x7ff6b8005830 comp rx=0 tx=0).stop 2024-09-10T07:53:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.761+0000 7ff6db000640 1 -- 172.21.15.3:0/3377489435 shutdown_connections 2024-09-10T07:53:02.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.761+0000 7ff6db000640 1 --2- 172.21.15.3:0/3377489435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6b8003dd0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.761+0000 7ff6db000640 1 -- 172.21.15.3:0/3377489435 >> 172.21.15.3:0/3377489435 conn(0x7ff6d4072560 msgr2=0x7ff6d4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.761+0000 7ff6db000640 1 -- 172.21.15.3:0/3377489435 shutdown_connections 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.761+0000 7ff6db000640 1 -- 172.21.15.3:0/3377489435 wait complete. 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.762+0000 7ff6db000640 1 Processor -- start 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.762+0000 7ff6db000640 1 -- start start 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6d40ce380 0x7ff6d40cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 0x7ff6d41619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d415c6b0 con 0x7ff6b80039b0 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d415c530 con 0x7ff6d4153e50 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.763+0000 7ff6db000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d40d1940 con 0x7ff6d40ce380 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36390/0 (socket says 172.21.15.3:36390) 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 -- 172.21.15.3:0/975512666 learned_addr learned my addr 172.21.15.3:0/975512666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6cb400640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6d40ce380 0x7ff6d40cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d9000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 0x7ff6d41619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 -- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6d40ce380 msgr2=0x7ff6d40cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6d40ce380 0x7ff6d40cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 -- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 msgr2=0x7ff6d41619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 0x7ff6d41619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 -- 172.21.15.3:0/975512666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d4161440 con 0x7ff6b80039b0 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d9000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 0x7ff6d41619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.764+0000 7ff6d3e00640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4034040 tx=0x7ff6c4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.765+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4000ed0 con 0x7ff6b80039b0 2024-09-10T07:53:02.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.765+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c4042070 con 0x7ff6b80039b0 2024-09-10T07:53:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.765+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007290 con 0x7ff6b80039b0 2024-09-10T07:53:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.765+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d40cbe60 con 0x7ff6b80039b0 2024-09-10T07:53:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.765+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d40cc420 con 0x7ff6b80039b0 2024-09-10T07:53:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.766+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40d0fd0 con 0x7ff6b80039b0 2024-09-10T07:53:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.767+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6c403b040 con 0x7ff6b80039b0 2024-09-10T07:53:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.768+0000 7ff6d0c00640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 0x7ff6b006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.768+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7ff6c40aada0 con 0x7ff6b80039b0 2024-09-10T07:53:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.768+0000 7ff6cb400640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 0x7ff6b006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:02.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.769+0000 7ff6cb400640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 0x7ff6b006e4c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff6d41f7b50 tx=0x7ff6bc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:02.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:02.772+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff6d4161440 con 0x7ff6b80039b0 2024-09-10T07:53:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.027+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7ff6d4069140 con 0x7ff6b80039b0 2024-09-10T07:53:03.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.176+0000 7ff6d0c00640 1 -- 172.21.15.3:0/975512666 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v339) ==== 154+0+0 (secure 0 0 0) 0x7ff6c4079ab0 con 0x7ff6b80039b0 2024-09-10T07:53:03.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 msgr2=0x7ff6b006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 0x7ff6b006e4c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff6d41f7b50 tx=0x7ff6bc009190 comp rx=0 tx=0).stop 2024-09-10T07:53:03.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 msgr2=0x7ff6d40ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4034040 tx=0x7ff6c4001a40 comp rx=0 tx=0).stop 2024-09-10T07:53:03.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 shutdown_connections 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff6b006c000 0x7ff6b006e4c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4153e50 0x7ff6d41619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6d40ce380 0x7ff6d40cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 --2- 172.21.15.3:0/975512666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6b80039b0 0x7ff6d40ccf00 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.178+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 >> 172.21.15.3:0/975512666 conn(0x7ff6d4072560 msgr2=0x7ff6d4143410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.179+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 shutdown_connections 2024-09-10T07:53:03.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.179+0000 7ff6db000640 1 -- 172.21.15.3:0/975512666 wait complete. 2024-09-10T07:53:03.195 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:53:03.196 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-10T07:53:03.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:53:03.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.353+0000 7f4043600640 1 Processor -- start 2024-09-10T07:53:03.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.353+0000 7f4043600640 1 -- start start 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4043600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f403c0d08f0 0x7f403c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4043600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0ccd80 con 0x7f403c15b4f0 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4043600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0cd060 con 0x7f403c15b110 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4043600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f403c0b44f0 con 0x7f403c0d08f0 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4033e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f403c15b4f0 0x7f403c0d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45434/0 (socket says 172.21.15.3:45434) 2024-09-10T07:53:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.354+0000 7f4033e00640 1 -- 172.21.15.3:0/3964306451 learned_addr learned my addr 172.21.15.3:0/3964306451 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f4041600640 1 --2- 172.21.15.3:0/3964306451 >> v2:172.21.15.70:3300/0 conn(0x7f403c0d08f0 0x7f403c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151710338 0 0) 0x7f403c0ccd80 con 0x7f403c15b4f0 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4020003520 con 0x7f403c15b4f0 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3015919021 0 0) 0x7f403c0cd060 con 0x7f403c15b110 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f403c0ccd80 con 0x7f403c15b110 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f4041600640 1 -- 172.21.15.3:0/3964306451 >> v1:172.21.15.70:6789/0 conn(0x7f403c15b110 legacy=0x7f403c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f4041600640 1 -- 172.21.15.3:0/3964306451 >> v1:172.21.15.3:6789/0 conn(0x7f403c15b4f0 legacy=0x7f403c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.355+0000 7f4041600640 1 -- 172.21.15.3:0/3964306451 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0cd340 con 0x7f403c0d08f0 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.356+0000 7f4041600640 1 --2- 172.21.15.3:0/3964306451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c0ceb50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc63521287f0492 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.356+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034007740 con 0x7f403c0d08f0 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.356+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340078e0 con 0x7f403c0d08f0 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f403be00640 1 -- 172.21.15.3:0/3964306451 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034007cb0 con 0x7f403c0d08f0 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 -- 172.21.15.3:0/3964306451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 msgr2=0x7f403c0ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 --2- 172.21.15.3:0/3964306451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c0ceb50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a6f0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 -- 172.21.15.3:0/3964306451 shutdown_connections 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 --2- 172.21.15.3:0/3964306451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c0ceb50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 -- 172.21.15.3:0/3964306451 >> 172.21.15.3:0/3964306451 conn(0x7f403c06d000 msgr2=0x7f403c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 -- 172.21.15.3:0/3964306451 shutdown_connections 2024-09-10T07:53:03.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.357+0000 7f4043600640 1 -- 172.21.15.3:0/3964306451 wait complete. 2024-09-10T07:53:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.359+0000 7f4043600640 1 Processor -- start 2024-09-10T07:53:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.359+0000 7f4043600640 1 -- start start 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f403c15b4f0 0x7f403c072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f403c073e90 con 0x7f403c15b110 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f403c073d10 con 0x7f403c0d08f0 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4043600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f403c074010 con 0x7f403c15b4f0 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36404/0 (socket says 172.21.15.3:36404) 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 -- 172.21.15.3:0/4028776303 learned_addr learned my addr 172.21.15.3:0/4028776303 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4040c00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4041600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f403c15b4f0 0x7f403c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 -- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f403c15b4f0 msgr2=0x7f403c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f403c15b4f0 0x7f403c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 -- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 msgr2=0x7f403c151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.360+0000 7f4033e00640 1 -- 172.21.15.3:0/4028776303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c15b8d0 con 0x7f403c15b110 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4040c00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4033e00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f402400c530 tx=0x7f402400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024017070 con 0x7f403c15b110 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4024005ce0 con 0x7f403c15b110 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0730e0 con 0x7f403c15b110 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024007270 con 0x7f403c15b110 2024-09-10T07:53:03.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.361+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c15bb90 con 0x7f403c15b110 2024-09-10T07:53:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.362+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0b3d10 con 0x7f403c15b110 2024-09-10T07:53:03.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.363+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f402401a070 con 0x7f403c15b110 2024-09-10T07:53:03.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.364+0000 7f4039600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 0x7f401806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.364+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7f4024089f50 con 0x7f403c15b110 2024-09-10T07:53:03.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.364+0000 7f4040c00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 0x7f401806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.365+0000 7f4040c00640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 0x7f401806e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f402c0011e0 tx=0x7f402c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.369+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4024058460 con 0x7f403c15b110 2024-09-10T07:53:03.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.556+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f403c0d0640 con 0x7f403c15b110 2024-09-10T07:53:03.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.557+0000 7f4039600640 1 -- 172.21.15.3:0/4028776303 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+13217 (secure 0 0 0) 0x7f402405c760 con 0x7f403c15b110 2024-09-10T07:53:03.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:03.557 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":93,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:02.170289+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6837/2433616379":"2024-09-11T07:52:59.082421+0000","172.21.15.3:6835/2433616379":"2024-09-11T07:52:59.082421+0000","172.21.15.3:6836/3872292817":"2024-09-11T07:52:59.082416+0000","172.21.15.3:6834/3872292817":"2024-09-11T07:52:59.082416+0000","172.21.15.70:6835/3096818521":"2024-09-11T07:52:59.082403+0000","172.21.15.70:6833/3096818521":"2024-09-11T07:52:59.082403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 msgr2=0x7f401806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 0x7f401806e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f402c0011e0 tx=0x7f402c009040 comp rx=0 tx=0).stop 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 msgr2=0x7f403c0721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f402400c530 tx=0x7f402400c9f0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 shutdown_connections 2024-09-10T07:53:03.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f401806be30 0x7f401806e2f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f403c15b4f0 0x7f403c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f403c15b110 0x7f403c0721d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 --2- 172.21.15.3:0/4028776303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c0d08f0 0x7f403c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 >> 172.21.15.3:0/4028776303 conn(0x7f403c06d000 msgr2=0x7f403c06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 shutdown_connections 2024-09-10T07:53:03.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.559+0000 7f4043600640 1 -- 172.21.15.3:0/4028776303 wait complete. 2024-09-10T07:53:03.576 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T07:46:21.545953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T07:53:03.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.736+0000 7f3919800640 1 Processor -- start 2024-09-10T07:53:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.736+0000 7f3919800640 1 -- start start 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3919800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3914147ce0 0x7f39140ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3919800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140aede0 0x7f3914072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3919800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391415faf0 con 0x7f39140728d0 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3919800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f391415eaf0 con 0x7f3914147ce0 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3919800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f39140730c0 con 0x7f39140aede0 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3914147ce0 0x7f39140ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3914147ce0 0x7f39140ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36414/0 (socket says 172.21.15.3:36414) 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 -- 172.21.15.3:0/3241718205 learned_addr learned my addr 172.21.15.3:0/3241718205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f390be00640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140aede0 0x7f3914072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 -- 172.21.15.3:0/3241718205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140aede0 msgr2=0x7f3914072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140aede0 0x7f3914072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 -- 172.21.15.3:0/3241718205 >> v1:172.21.15.3:6789/0 conn(0x7f39140728d0 legacy=0x7f3914070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.738+0000 7f3912a00640 1 -- 172.21.15.3:0/3241718205 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391415ed30 con 0x7f3914147ce0 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.739+0000 7f3912a00640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914147ce0 0x7f39140ae8a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3904009870 tx=0x7f390402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3aad8bbbc53b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3912000640 1 -- 172.21.15.3:0/3241718205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904037070 con 0x7f3914147ce0 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3912000640 1 -- 172.21.15.3:0/3241718205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904032440 con 0x7f3914147ce0 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3912000640 1 -- 172.21.15.3:0/3241718205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904032810 con 0x7f3914147ce0 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 -- 172.21.15.3:0/3241718205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914147ce0 msgr2=0x7f39140ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914147ce0 0x7f39140ae8a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3904009870 tx=0x7f390402a250 comp rx=0 tx=0).stop 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 -- 172.21.15.3:0/3241718205 shutdown_connections 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140aede0 0x7f3914072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 --2- 172.21.15.3:0/3241718205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3914147ce0 0x7f39140ae8a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 -- 172.21.15.3:0/3241718205 >> 172.21.15.3:0/3241718205 conn(0x7f39140b5cb0 msgr2=0x7f391406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 -- 172.21.15.3:0/3241718205 shutdown_connections 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.740+0000 7f3919800640 1 -- 172.21.15.3:0/3241718205 wait complete. 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.742+0000 7f3919800640 1 Processor -- start 2024-09-10T07:53:03.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.742+0000 7f3919800640 1 -- start start 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140728d0 0x7f39140b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3914147ce0 0x7f391413a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39140af220 con 0x7f39140aede0 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f391415f660 con 0x7f3914147ce0 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3919800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f391415f2f0 con 0x7f39140728d0 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f390be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f390be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36428/0 (socket says 172.21.15.3:36428) 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f390be00640 1 -- 172.21.15.3:0/3671525298 learned_addr learned my addr 172.21.15.3:0/3671525298 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3912a00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140728d0 0x7f39140b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.743+0000 7f3913400640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3914147ce0 0x7f391413a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 -- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140728d0 msgr2=0x7f39140b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140728d0 0x7f39140b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 -- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3914147ce0 msgr2=0x7f391413a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3914147ce0 0x7f391413a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 -- 172.21.15.3:0/3671525298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914146410 con 0x7f39140aede0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390be00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d830 tx=0x7f38fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00b660 con 0x7f39140aede0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc019070 con 0x7f39140aede0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3914146750 con 0x7f39140aede0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00ba20 con 0x7f39140aede0 2024-09-10T07:53:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.744+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3914146f90 con 0x7f39140aede0 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.745+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3914147510 con 0x7f39140aede0 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.746+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38fc01e460 con 0x7f39140aede0 2024-09-10T07:53:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.747+0000 7f390aa00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 0x7f38f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.747+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7f38fc08b270 con 0x7f39140aede0 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.747+0000 7f3912a00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 0x7f38f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:03.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.748+0000 7f3912a00640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 0x7f38f006e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3904034000 tx=0x7f390402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:03.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.751+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3914146750 con 0x7f39140aede0 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.938+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3914072d10 con 0x7f39140aede0 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.939+0000 7f390aa00640 1 -- 172.21.15.3:0/3671525298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f38fc059dd0 con 0x7f39140aede0 2024-09-10T07:53:03.938 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-10T07:53:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 msgr2=0x7f38f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 0x7f38f006e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3904034000 tx=0x7f390402e000 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 msgr2=0x7f3914145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d830 tx=0x7f38fc00dcf0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 shutdown_connections 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f38f006be30 0x7f38f006e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3914147ce0 0x7f391413a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39140aede0 0x7f3914145aa0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 --2- 172.21.15.3:0/3671525298 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f39140728d0 0x7f39140b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 >> 172.21.15.3:0/3671525298 conn(0x7f39140b5cb0 msgr2=0x7f391415fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 shutdown_connections 2024-09-10T07:53:03.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:03.941+0000 7f3919800640 1 -- 172.21.15.3:0/3671525298 wait complete. 2024-09-10T07:53:03.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.112+0000 7f2326a00640 1 Processor -- start 2024-09-10T07:53:04.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.112+0000 7f2326a00640 1 -- start start 2024-09-10T07:53:04.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.113+0000 7f2326a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f232015e860 0x7f23200cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.113+0000 7f2326a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2320073150 0x7f2320073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.113+0000 7f2326a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23200cee40 con 0x7f23200cfbf0 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.113+0000 7f2326a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f23200d3780 con 0x7f232015e860 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.113+0000 7f2326a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f23200cf120 con 0x7f2320073150 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f232015e860 0x7f23200cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f232015e860 0x7f23200cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60844/0 (socket says 172.21.15.3:60844) 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 -- 172.21.15.3:0/3392403137 learned_addr learned my addr 172.21.15.3:0/3392403137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2326000640 1 --2- 172.21.15.3:0/3392403137 >> v2:172.21.15.70:3300/0 conn(0x7f2320073150 0x7f2320073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 -- 172.21.15.3:0/3392403137 >> v2:172.21.15.70:3300/0 conn(0x7f2320073150 msgr2=0x7f2320073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 --2- 172.21.15.3:0/3392403137 >> v2:172.21.15.70:3300/0 conn(0x7f2320073150 0x7f2320073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 -- 172.21.15.3:0/3392403137 >> v1:172.21.15.70:6789/0 conn(0x7f23200cfbf0 legacy=0x7f23200d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:04.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.114+0000 7f2325600640 1 -- 172.21.15.3:0/3392403137 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200d3410 con 0x7f232015e860 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.115+0000 7f2325600640 1 --2- 172.21.15.3:0/3392403137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f232015e860 0x7f23200cf6b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f23200b4060 tx=0x7f231402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53969d473c066982 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.115+0000 7f231fe00640 1 -- 172.21.15.3:0/3392403137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314037070 con 0x7f232015e860 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.115+0000 7f231fe00640 1 -- 172.21.15.3:0/3392403137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2314032440 con 0x7f232015e860 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.115+0000 7f231fe00640 1 -- 172.21.15.3:0/3392403137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314032810 con 0x7f232015e860 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 -- 172.21.15.3:0/3392403137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f232015e860 msgr2=0x7f23200cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 --2- 172.21.15.3:0/3392403137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f232015e860 0x7f23200cf6b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f23200b4060 tx=0x7f231402a270 comp rx=0 tx=0).stop 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 -- 172.21.15.3:0/3392403137 shutdown_connections 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 --2- 172.21.15.3:0/3392403137 >> v2:172.21.15.70:3300/0 conn(0x7f2320073150 0x7f2320073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:04.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 --2- 172.21.15.3:0/3392403137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f232015e860 0x7f23200cf6b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 -- 172.21.15.3:0/3392403137 >> 172.21.15.3:0/3392403137 conn(0x7f23200d4650 msgr2=0x7f23200d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 -- 172.21.15.3:0/3392403137 shutdown_connections 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.116+0000 7f2326a00640 1 -- 172.21.15.3:0/3392403137 wait complete. 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.118+0000 7f2326a00640 1 Processor -- start 2024-09-10T07:53:04.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.118+0000 7f2326a00640 1 -- start start 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 0x7f23201549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f232015e860 0x7f23200abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23200d2460 con 0x7f23200cfbf0 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f23200cf400 con 0x7f2320073150 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f23200b7880 con 0x7f232015e860 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60860/0 (socket says 172.21.15.3:60860) 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 -- 172.21.15.3:0/651726658 learned_addr learned my addr 172.21.15.3:0/651726658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:04.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2325600640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 0x7f23201549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2326000640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f232015e860 0x7f23200abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 -- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f232015e860 msgr2=0x7f23200abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f232015e860 0x7f23200abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 -- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 msgr2=0x7f23201549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 0x7f23201549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2324c00640 1 -- 172.21.15.3:0/651726658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200adaf0 con 0x7f23200cfbf0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.119+0000 7f2325600640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 0x7f23201549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f2324c00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c400 tx=0x7f230c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0052d0 con 0x7f23200cfbf0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c005490 con 0x7f23200cfbf0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23200ac510 con 0x7f23200cfbf0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c007260 con 0x7f23200cfbf0 2024-09-10T07:53:04.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.120+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23200acad0 con 0x7f23200cfbf0 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.121+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23200cf6b0 con 0x7f23200cfbf0 2024-09-10T07:53:04.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.122+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f230c007400 con 0x7f23200cfbf0 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.123+0000 7f231d600640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 0x7f230006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.124+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7f230c08a140 con 0x7f23200cfbf0 2024-09-10T07:53:04.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.124+0000 7f2325600640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 0x7f230006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:04.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.124+0000 7f2325600640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 0x7f230006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2314034000 tx=0x7f231402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:04.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.128+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f230c058650 con 0x7f23200cfbf0 2024-09-10T07:53:04.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:04.377+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7f23200b5a50 con 0x7f23200cfbf0 2024-09-10T07:53:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.184+0000 7f231d600640 1 -- 172.21.15.3:0/651726658 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v341) ==== 168+0+0 (secure 0 0 0) 0x7f230c05c950 con 0x7f23200cfbf0 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 msgr2=0x7f230006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 0x7f230006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2314034000 tx=0x7f231402e000 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 msgr2=0x7f23200ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c400 tx=0x7f230c00c8c0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 shutdown_connections 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f230006be30 0x7f230006e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f232015e860 0x7f23200abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23200cfbf0 0x7f23200ad5b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 --2- 172.21.15.3:0/651726658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2320073150 0x7f23201549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 >> 172.21.15.3:0/651726658 conn(0x7f23200d4650 msgr2=0x7f2320072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 shutdown_connections 2024-09-10T07:53:05.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.187+0000 7f2326a00640 1 -- 172.21.15.3:0/651726658 wait complete. 2024-09-10T07:53:05.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T07:53:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.372+0000 7fb900400640 1 Processor -- start 2024-09-10T07:53:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.372+0000 7fb900400640 1 -- start start 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.373+0000 7fb900400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb900400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8f806c060 0x7fb8f80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb900400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f806e4d0 con 0x7fb8f8155750 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb900400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f806c4d0 con 0x7fb8f806c060 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb900400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f8155b30 con 0x7fb8f8162270 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8f806c060 0x7fb8f80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb8f806c060 0x7fb8f80cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60870/0 (socket says 172.21.15.3:60870) 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 -- 172.21.15.3:0/390987580 learned_addr learned my addr 172.21.15.3:0/390987580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fda00640 1 --1- 172.21.15.3:0/390987580 >> v1:172.21.15.3:6789/0 conn(0x7fb8f8155750 0x7fb8f8162aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36734/0 (socket says 172.21.15.3:36734) 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fd000640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 -- 172.21.15.3:0/390987580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 msgr2=0x7fb8f806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 -- 172.21.15.3:0/390987580 >> v1:172.21.15.3:6789/0 conn(0x7fb8f8155750 legacy=0x7fb8f8162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.374+0000 7fb8fe400640 1 -- 172.21.15.3:0/390987580 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f806e7b0 con 0x7fb8f806c060 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.375+0000 7fb8fe400640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f80cefb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1fd613fffa314 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb8f7e00640 1 -- 172.21.15.3:0/390987580 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0036070 con 0x7fb8f806c060 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb8f7e00640 1 -- 172.21.15.3:0/390987580 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f0031600 con 0x7fb8f806c060 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb8f7e00640 1 -- 172.21.15.3:0/390987580 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00319d0 con 0x7fb8f806c060 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 -- 172.21.15.3:0/390987580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 msgr2=0x7fb8f80cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f80cefb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f002a710 comp rx=0 tx=0).stop 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 -- 172.21.15.3:0/390987580 shutdown_connections 2024-09-10T07:53:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f80cefb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 --2- 172.21.15.3:0/390987580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 -- 172.21.15.3:0/390987580 >> 172.21.15.3:0/390987580 conn(0x7fb8f80b2a50 msgr2=0x7fb8f80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 -- 172.21.15.3:0/390987580 shutdown_connections 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.376+0000 7fb900400640 1 -- 172.21.15.3:0/390987580 wait complete. 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.378+0000 7fb900400640 1 Processor -- start 2024-09-10T07:53:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.378+0000 7fb900400640 1 -- start start 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8f8155750 0x7fb8f8150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f806f0e0 con 0x7fb8f806c060 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f806ed70 con 0x7fb8f8155750 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb900400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f806f570 con 0x7fb8f8162270 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb8fda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb8fda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60876/0 (socket says 172.21.15.3:60876) 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.379+0000 7fb8fda00640 1 -- 172.21.15.3:0/55998852 learned_addr learned my addr 172.21.15.3:0/55998852 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 -- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 msgr2=0x7fb8f80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 -- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8f8155750 msgr2=0x7fb8f8150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8f8155750 0x7fb8f8150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 -- 172.21.15.3:0/55998852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f8150c30 con 0x7fb8f806c060 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8fda00640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c430 tx=0x7fb8ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0074e0 con 0x7fb8f806c060 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec007680 con 0x7fb8f806c060 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec004a00 con 0x7fb8f806c060 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f8150f10 con 0x7fb8f806c060 2024-09-10T07:53:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.380+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f81514d0 con 0x7fb8f806c060 2024-09-10T07:53:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.381+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f80cefb0 con 0x7fb8f806c060 2024-09-10T07:53:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.382+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8ec01a020 con 0x7fb8f806c060 2024-09-10T07:53:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.383+0000 7fb8f5600640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 0x7fb8d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.383+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7fb8ec08a460 con 0x7fb8f806c060 2024-09-10T07:53:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.384+0000 7fb8fd000640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 0x7fb8d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:05.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.384+0000 7fb8fd000640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 0x7fb8d806e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006fd0 tx=0x7fb8e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:05.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.387+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb8ec059050 con 0x7fb8f806c060 2024-09-10T07:53:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.571+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb8f80b6070 con 0x7fb8f806c060 2024-09-10T07:53:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.574+0000 7fb8f5600640 1 -- 172.21.15.3:0/55998852 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fb8ec058950 con 0x7fb8f806c060 2024-09-10T07:53:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.576+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 msgr2=0x7fb8d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.576+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 0x7fb8d806e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006fd0 tx=0x7fb8e4008040 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.576+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 msgr2=0x7fb8f81483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.576+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c430 tx=0x7fb8ec00c8f0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 shutdown_connections 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb8d806be30 0x7fb8d806e2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8f8162270 0x7fb8f80b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8f8155750 0x7fb8f8150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 --2- 172.21.15.3:0/55998852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8f806c060 0x7fb8f81483e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 >> 172.21.15.3:0/55998852 conn(0x7fb8f80b2a50 msgr2=0x7fb8f815df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 shutdown_connections 2024-09-10T07:53:05.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:05.577+0000 7fb900400640 1 -- 172.21.15.3:0/55998852 wait complete. 2024-09-10T07:53:05.592 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:53:05.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:53:05.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:53:05.771 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:53:05.771 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:53:05.771 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:05.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:53:05.806 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:53:05.806 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:05.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:53:05.844 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:53:05.844 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:05.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:53:05.881 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:53:05.882 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:05.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:53:05.920 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T07:53:05.920 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.077+0000 7f66fa200640 1 Processor -- start 2024-09-10T07:53:06.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.077+0000 7f66fa200640 1 -- start start 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66fa200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 0x7f66f415f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66fa200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66f415fdb0 0x7f66f406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66fa200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f415fbb0 con 0x7f66f4160220 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66fa200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f66f40b36a0 con 0x7f66f415fdb0 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66fa200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f66f40b3980 con 0x7f66f4161b30 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.078+0000 7f66f2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66f415fdb0 0x7f66f406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66f415fdb0 0x7f66f406bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60878/0 (socket says 172.21.15.3:60878) 2024-09-10T07:53:06.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f3e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66f4160220 0x7f66f40b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36740/0 (socket says 172.21.15.3:36740) 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 -- 172.21.15.3:0/931007529 learned_addr learned my addr 172.21.15.3:0/931007529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f3400640 1 --2- 172.21.15.3:0/931007529 >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 0x7f66f415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 -- 172.21.15.3:0/931007529 >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 msgr2=0x7f66f415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 --2- 172.21.15.3:0/931007529 >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 0x7f66f415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 -- 172.21.15.3:0/931007529 >> v1:172.21.15.3:6789/0 conn(0x7f66f4160220 legacy=0x7f66f40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2a00640 1 -- 172.21.15.3:0/931007529 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f40b3c60 con 0x7f66f415fdb0 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f3400640 1 --2- 172.21.15.3:0/931007529 >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 0x7f66f415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.079+0000 7f66f2000640 1 -- 172.21.15.3:0/931007529 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769302932 0 0) 0x7f66ec001490 con 0x7f66f4160220 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.080+0000 7f66f2a00640 1 --2- 172.21.15.3:0/931007529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f406bb20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f66f406e300 tx=0x7f66e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00138a8605b09e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.080+0000 7f66f2000640 1 -- 172.21.15.3:0/931007529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0037070 con 0x7f66f415fdb0 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.080+0000 7f66f2000640 1 -- 172.21.15.3:0/931007529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0032440 con 0x7f66f415fdb0 2024-09-10T07:53:06.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.080+0000 7f66f2000640 1 -- 172.21.15.3:0/931007529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0032810 con 0x7f66f415fdb0 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 -- 172.21.15.3:0/931007529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 msgr2=0x7f66f406bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 --2- 172.21.15.3:0/931007529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f406bb20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f66f406e300 tx=0x7f66e002a270 comp rx=0 tx=0).stop 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 -- 172.21.15.3:0/931007529 shutdown_connections 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 --2- 172.21.15.3:0/931007529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f406bb20 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 --2- 172.21.15.3:0/931007529 >> v2:172.21.15.70:3300/0 conn(0x7f66f4161b30 0x7f66f415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 -- 172.21.15.3:0/931007529 >> 172.21.15.3:0/931007529 conn(0x7f66f406d000 msgr2=0x7f66f406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 -- 172.21.15.3:0/931007529 shutdown_connections 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.081+0000 7f66fa200640 1 -- 172.21.15.3:0/931007529 wait complete. 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.082+0000 7f66fa200640 1 Processor -- start 2024-09-10T07:53:06.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.083+0000 7f66fa200640 1 -- start start 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66f4160220 0x7f66f40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 0x7f66f40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4161f10 con 0x7f66f415fdb0 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40b4220 con 0x7f66f4161b30 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66fa200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40aefa0 con 0x7f66f4160220 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60890/0 (socket says 172.21.15.3:60890) 2024-09-10T07:53:06.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 -- 172.21.15.3:0/200097284 learned_addr learned my addr 172.21.15.3:0/200097284 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f2a00640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66f4160220 0x7f66f40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3e00640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 0x7f66f40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 -- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66f4160220 msgr2=0x7f66f40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66f4160220 0x7f66f40cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 -- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 msgr2=0x7f66f40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 0x7f66f40cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3400640 1 -- 172.21.15.3:0/200097284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f414d290 con 0x7f66f415fdb0 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.084+0000 7f66f3e00640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 0x7f66f40cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66f3400640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e7e0 tx=0x7f66e400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e400f040 con 0x7f66f415fdb0 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e4004030 con 0x7f66f415fdb0 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f414d770 con 0x7f66f415fdb0 2024-09-10T07:53:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4004400 con 0x7f66f415fdb0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.085+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f40cff20 con 0x7f66f415fdb0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.086+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f40aff30 con 0x7f66f415fdb0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.087+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66e400c6d0 con 0x7f66f415fdb0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.088+0000 7f66eb400640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.088+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5115+0+0 (secure 0 0 0) 0x7f66e408a9a0 con 0x7f66f415fdb0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.088+0000 7f66f2a00640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.089+0000 7f66f2a00640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 0x7f66d406e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.092+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66e4017020 con 0x7f66f415fdb0 2024-09-10T07:53:06.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.313+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f415a4c0 con 0x7f66f415fdb0 2024-09-10T07:53:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.313+0000 7f66eb400640 1 -- 172.21.15.3:0/200097284 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+3899 (secure 0 0 0) 0x7f66e4058eb0 con 0x7f66f415fdb0 2024-09-10T07:53:06.313 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:06.313 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":98,"btime":"2024-09-10T07:53:00:084297+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6796,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/484170457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":484170457},{"type":"v1","addr":"172.21.15.70:6834","nonce":484170457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/595718475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":595718475},{"type":"v1","addr":"172.21.15.70:6835","nonce":595718475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6821,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3313931713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3313931713},{"type":"v1","addr":"172.21.15.3:6837","nonce":3313931713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7947,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2736004829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2736004829},{"type":"v1","addr":"172.21.15.3:6835","nonce":2736004829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[]} 2024-09-10T07:53:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 98 2024-09-10T07:53:06.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 msgr2=0x7f66d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 0x7f66d406e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e002e000 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 msgr2=0x7f66f414cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e7e0 tx=0x7f66e400eca0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 shutdown_connections 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f4161b30 0x7f66f40cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66f4160220 0x7f66f40cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 --2- 172.21.15.3:0/200097284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f415fdb0 0x7f66f414cd50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 >> 172.21.15.3:0/200097284 conn(0x7f66f406d000 msgr2=0x7f66f415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 shutdown_connections 2024-09-10T07:53:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.316+0000 7f66fa200640 1 -- 172.21.15.3:0/200097284 wait complete. 2024-09-10T07:53:06.332 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T07:53:06.333 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T07:53:06.333 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:53:06.335 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:53:06.336 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T07:53:06.336 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T07:53:06.336 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:53:06.336 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:53:06.336 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T07:53:06.336 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T07:53:06.337 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:53:06.337 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:53:06.337 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T07:53:06.337 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T07:53:06.337 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:53:06.337 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:53:06.348 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:53:06.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T07:53:06.350 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:53:06.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T07:53:06.352 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:53:06.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T07:53:06.353 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:53:06.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T07:53:06.394 INFO:tasks.ceph.mds.a:Started 2024-09-10T07:53:06.395 DEBUG:teuthology.parallel:result is None 2024-09-10T07:53:06.397 INFO:tasks.ceph.mds.b:Started 2024-09-10T07:53:06.397 DEBUG:teuthology.parallel:result is None 2024-09-10T07:53:06.398 INFO:tasks.ceph.mds.c:Started 2024-09-10T07:53:06.398 DEBUG:teuthology.parallel:result is None 2024-09-10T07:53:06.400 INFO:tasks.ceph.mds.d:Started 2024-09-10T07:53:06.401 DEBUG:teuthology.parallel:result is None 2024-09-10T07:53:06.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T07:53:06.526 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:53:06.527 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-09-10T07:53:06.527 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:53:06.527 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:53:06.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.565+0000 7fa496000640 1 Processor -- start 2024-09-10T07:53:06.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.565+0000 7fa496000640 1 -- start start 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.566+0000 7fa496000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa4900d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa496000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4900cfdd0 con 0x7fa490160710 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa496000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa49015cc90 con 0x7fa4900d05f0 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa496000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa49015cf70 con 0x7fa490160af0 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa490160710 0x7fa4900cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36748/0 (socket says 172.21.15.3:36748) 2024-09-10T07:53:06.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48f400640 1 -- 172.21.15.3:0/1777645482 learned_addr learned my addr 172.21.15.3:0/1777645482 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:06.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa487e00640 1 --2- 172.21.15.3:0/1777645482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa4900d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769517818 0 0) 0x7fa4900cfdd0 con 0x7fa490160710 2024-09-10T07:53:06.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa474003520 con 0x7fa490160710 2024-09-10T07:53:06.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327897899 0 0) 0x7fa49015cc90 con 0x7fa4900d05f0 2024-09-10T07:53:06.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4900cfdd0 con 0x7fa4900d05f0 2024-09-10T07:53:06.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa487e00640 1 -- 172.21.15.3:0/1777645482 >> v1:172.21.15.70:6789/0 conn(0x7fa4900d05f0 legacy=0x7fa490072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa487e00640 1 -- 172.21.15.3:0/1777645482 >> v1:172.21.15.3:6789/0 conn(0x7fa490160710 legacy=0x7fa4900cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.567+0000 7fa487e00640 1 -- 172.21.15.3:0/1777645482 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49015d250 con 0x7fa490160af0 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.568+0000 7fa487e00640 1 --2- 172.21.15.3:0/1777645482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa4900d00b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4780097d0 tx=0x7fa47802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6004b87c2bc67294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478037070 con 0x7fa490160af0 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa478032440 con 0x7fa490160af0 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa48ea00640 1 -- 172.21.15.3:0/1777645482 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478032810 con 0x7fa490160af0 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa496000640 1 -- 172.21.15.3:0/1777645482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 msgr2=0x7fa4900d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa496000640 1 --2- 172.21.15.3:0/1777645482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa4900d00b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4780097d0 tx=0x7fa47802a210 comp rx=0 tx=0).stop 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa496000640 1 -- 172.21.15.3:0/1777645482 shutdown_connections 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa496000640 1 --2- 172.21.15.3:0/1777645482 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa4900d00b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.569+0000 7fa496000640 1 -- 172.21.15.3:0/1777645482 >> 172.21.15.3:0/1777645482 conn(0x7fa49006d000 msgr2=0x7fa49006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:06.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.570+0000 7fa496000640 1 -- 172.21.15.3:0/1777645482 shutdown_connections 2024-09-10T07:53:06.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.570+0000 7fa496000640 1 -- 172.21.15.3:0/1777645482 wait complete. 2024-09-10T07:53:06.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.571+0000 7fa496000640 1 Processor -- start 2024-09-10T07:53:06.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.571+0000 7fa496000640 1 -- start start 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 0x7fa4901f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa49014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa49015d770 con 0x7fa4900d05f0 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900d0a10 con 0x7fa490160710 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa496000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa490150a50 con 0x7fa490160af0 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa48f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa48f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60916/0 (socket says 172.21.15.3:60916) 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa48f400640 1 -- 172.21.15.3:0/2952161320 learned_addr learned my addr 172.21.15.3:0/2952161320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.572+0000 7fa487e00640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 0x7fa4901f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48fe00640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa49014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 -- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 msgr2=0x7fa49014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa49014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 -- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 msgr2=0x7fa4901f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 0x7fa4901f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 -- 172.21.15.3:0/2952161320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4901501b0 con 0x7fa4900d05f0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa487e00640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 0x7fa4901f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa48f400640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c530 tx=0x7fa48000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.573+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa4900d05f0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.574+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480005ce0 con 0x7fa4900d05f0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.574+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480007270 con 0x7fa4900d05f0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.574+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa490150490 con 0x7fa4900d05f0 2024-09-10T07:53:06.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.574+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49014dbd0 con 0x7fa4900d05f0 2024-09-10T07:53:06.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.575+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900cfaf0 con 0x7fa4900d05f0 2024-09-10T07:53:06.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.576+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa48001a070 con 0x7fa4900d05f0 2024-09-10T07:53:06.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.577+0000 7fa487400640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 0x7fa46c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:06.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.577+0000 7fa487e00640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 0x7fa46c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:06.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.577+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5115+0+0 (secure 0 0 0) 0x7fa480088720 con 0x7fa4900d05f0 2024-09-10T07:53:06.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.578+0000 7fa487e00640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 0x7fa46c06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa478034000 tx=0x7fa47802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:06.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.583+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4800584d0 con 0x7fa4900d05f0 2024-09-10T07:53:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:06.783+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa49015d8f0 con 0x7fa4900d05f0 2024-09-10T07:53:07.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.593+0000 7fa487400640 1 -- 172.21.15.3:0/2952161320 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) ==== 97+0+0 (secure 0 0 0) 0x7fa48005c7d0 con 0x7fa4900d05f0 2024-09-10T07:53:07.593 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-10T07:53:07.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.596+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 msgr2=0x7fa46c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.596+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 0x7fa46c06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa478034000 tx=0x7fa47802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.596+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 msgr2=0x7fa49014d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.596+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c530 tx=0x7fa48000c9f0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 shutdown_connections 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa46c06be30 0x7fa46c06e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa490160af0 0x7fa49014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa490160710 0x7fa4901f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 --2- 172.21.15.3:0/2952161320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4900d05f0 0x7fa49014d460 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 >> 172.21.15.3:0/2952161320 conn(0x7fa49006d000 msgr2=0x7fa4900cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 shutdown_connections 2024-09-10T07:53:07.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.597+0000 7fa496000640 1 -- 172.21.15.3:0/2952161320 wait complete. 2024-09-10T07:53:07.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T07:53:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.779+0000 7fed25000640 1 Processor -- start 2024-09-10T07:53:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.779+0000 7fed25000640 1 -- start start 2024-09-10T07:53:07.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.780+0000 7fed25000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed2015b110 0x7fed200b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.780+0000 7fed25000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed200d08f0 0x7fed200cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.780+0000 7fed25000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed200b4250 con 0x7fed2015b4f0 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.780+0000 7fed25000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fed200b44f0 con 0x7fed2015b110 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.780+0000 7fed25000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed200ccd80 con 0x7fed200d08f0 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed2015b110 0x7fed200b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed2015b110 0x7fed200b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60930/0 (socket says 172.21.15.3:60930) 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 -- 172.21.15.3:0/1027492319 learned_addr learned my addr 172.21.15.3:0/1027492319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1e800640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed200d08f0 0x7fed200cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 -- 172.21.15.3:0/1027492319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed200d08f0 msgr2=0x7fed200cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed200d08f0 0x7fed200cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 -- 172.21.15.3:0/1027492319 >> v1:172.21.15.70:6789/0 conn(0x7fed2015b4f0 legacy=0x7fed200d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.781+0000 7fed1de00640 1 -- 172.21.15.3:0/1027492319 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed200cd060 con 0x7fed2015b110 2024-09-10T07:53:07.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.782+0000 7fed1de00640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2015b110 0x7fed200b3d10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fed200b4670 tx=0x7fed1002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa1bc72c832b7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.782+0000 7fed1d400640 1 -- 172.21.15.3:0/1027492319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10037070 con 0x7fed2015b110 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.782+0000 7fed1d400640 1 -- 172.21.15.3:0/1027492319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10032440 con 0x7fed2015b110 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.782+0000 7fed1d400640 1 -- 172.21.15.3:0/1027492319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10032810 con 0x7fed2015b110 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 -- 172.21.15.3:0/1027492319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2015b110 msgr2=0x7fed200b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2015b110 0x7fed200b3d10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fed200b4670 tx=0x7fed1002a250 comp rx=0 tx=0).stop 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 -- 172.21.15.3:0/1027492319 shutdown_connections 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed200d08f0 0x7fed200cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 --2- 172.21.15.3:0/1027492319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed2015b110 0x7fed200b3d10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 -- 172.21.15.3:0/1027492319 >> 172.21.15.3:0/1027492319 conn(0x7fed2006d000 msgr2=0x7fed2006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 -- 172.21.15.3:0/1027492319 shutdown_connections 2024-09-10T07:53:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.783+0000 7fed25000640 1 -- 172.21.15.3:0/1027492319 wait complete. 2024-09-10T07:53:07.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.785+0000 7fed25000640 1 Processor -- start 2024-09-10T07:53:07.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.785+0000 7fed25000640 1 -- start start 2024-09-10T07:53:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed2015b110 0x7fed2014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2015b4f0 0x7fed201677e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed20073d10 con 0x7fed200d08f0 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed200cd620 con 0x7fed2015b4f0 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed25000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fed20073e90 con 0x7fed2015b110 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60932/0 (socket says 172.21.15.3:60932) 2024-09-10T07:53:07.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 -- 172.21.15.3:0/629634851 learned_addr learned my addr 172.21.15.3:0/629634851 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed17e00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed2015b110 0x7fed2014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1e800640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2015b4f0 0x7fed201677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 -- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed2015b110 msgr2=0x7fed2014eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed2015b110 0x7fed2014eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 -- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2015b4f0 msgr2=0x7fed201677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:07.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2015b4f0 0x7fed201677e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.786+0000 7fed1de00640 1 -- 172.21.15.3:0/629634851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed20166740 con 0x7fed200d08f0 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed1de00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fed1002a290 tx=0x7fed100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10041020 con 0x7fed200d08f0 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed100306c0 con 0x7fed200d08f0 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10037040 con 0x7fed200d08f0 2024-09-10T07:53:07.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed20166a20 con 0x7fed200d08f0 2024-09-10T07:53:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.787+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed20166fe0 con 0x7fed200d08f0 2024-09-10T07:53:07.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.788+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed200b3d10 con 0x7fed200d08f0 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.789+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed10044020 con 0x7fed200d08f0 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.790+0000 7fed16000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 0x7fecfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.790+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7fed100ab190 con 0x7fed200d08f0 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.790+0000 7fed17e00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 0x7fecfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.791+0000 7fed17e00640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 0x7fecfc06e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fed2015c010 tx=0x7fed08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:07.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:07.794+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed1004b030 con 0x7fed200d08f0 2024-09-10T07:53:08.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.047+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fed2006da40 con 0x7fed200d08f0 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.047+0000 7fed16000640 1 -- 172.21.15.3:0/629634851 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fed10079830 con 0x7fed200d08f0 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mPJQgOhAAvstWWUKrafZMGUe74ycexg==", 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.047 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mRBo9EhAAoU/Glhi+9gfULqJoQRuEVQ==", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9moCVXCBAAA8t/A2+RbkCPKzlWXFbwYg==", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m7NkCHBAA9uaAON5KjD413UAUnWLtYA==", 2024-09-10T07:53:08.049 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAX+d9mij2nMRAAMR9CxHrpAsip2iROWRVZKg==", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.050 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAc+d9m3ZqlARAANal8h3/2X5xy4lPxf4ZDBw==", 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-10T07:53:08.051 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAg+d9muovmCxAAcuqycW47T9gkspQAd/vAHw==", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAk+d9mFHPeFRAAEAotRtX9SVCcQhFs+f6uig==", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.052 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp+d9mYOVKLBAA3DN8abVk7ytKPetjLZOfKg==", 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-10T07:53:08.053 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAt+d9motloNhAAgdMdTDRBVhv4JausbOaw6g==", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy+d9mN3coBBAAunK4Hs2imDw5aDhhqk270g==", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA1+d9muCh0OxAAikKRQ3TjIa8FY4JThOqjsQ==", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ==", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.055 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mtzGIMRAAGH0a0ZRZ0EPvulRU1pleKw==", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mfvkXKxAA7vKgKNd2i85JNVv70iOxrQ==", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAU+d9mv38GOBAAgtwE9liW+23l00jyX4wAZA==", 2024-09-10T07:53:08.057 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mKZfOChAAV7GBMQvE0GwRUpbljk+GZg==", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-10T07:53:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mNJdsMxAA8Yk5wzM8BFbjFuItMfJwqQ==", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAT+d9mxlZ6LBAAC/p0eKJZ1BB6waaRkFQXyQ==", 2024-09-10T07:53:08.059 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-10T07:53:08.060 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 msgr2=0x7fecfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 0x7fecfc06e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fed2015c010 tx=0x7fed08009040 comp rx=0 tx=0).stop 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 msgr2=0x7fed20165fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fed1002a290 tx=0x7fed100046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 shutdown_connections 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fecfc06be30 0x7fecfc06e2f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2015b4f0 0x7fed201677e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed2015b110 0x7fed2014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 --2- 172.21.15.3:0/629634851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed200d08f0 0x7fed20165fd0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 >> 172.21.15.3:0/629634851 conn(0x7fed2006d000 msgr2=0x7fed20073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.049+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 shutdown_connections 2024-09-10T07:53:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.050+0000 7fed25000640 1 -- 172.21.15.3:0/629634851 wait complete. 2024-09-10T07:53:08.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T07:53:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.221+0000 7f8850e00640 1 Processor -- start 2024-09-10T07:53:08.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.222+0000 7f8850e00640 1 -- start start 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8850e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f884c070f30 0x7f884c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8850e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f884c0cfdc0 0x7f884c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8850e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f884c0cd7f0 con 0x7f884c15c720 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8850e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f884c072210 con 0x7f884c070f30 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8850e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f884c0cf880 con 0x7f884c0cfdc0 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8841400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f884c070f30 0x7f884c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8841400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f884c070f30 0x7f884c071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60944/0 (socket says 172.21.15.3:60944) 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f8841400640 1 -- 172.21.15.3:0/3130550544 learned_addr learned my addr 172.21.15.3:0/3130550544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:08.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.223+0000 7f884a800640 1 --2- 172.21.15.3:0/3130550544 >> v2:172.21.15.70:3300/0 conn(0x7f884c0cfdc0 0x7f884c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.224+0000 7f8841400640 1 -- 172.21.15.3:0/3130550544 >> v2:172.21.15.70:3300/0 conn(0x7f884c0cfdc0 msgr2=0x7f884c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.224+0000 7f8841400640 1 --2- 172.21.15.3:0/3130550544 >> v2:172.21.15.70:3300/0 conn(0x7f884c0cfdc0 0x7f884c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.224+0000 7f8841400640 1 -- 172.21.15.3:0/3130550544 >> v1:172.21.15.70:6789/0 conn(0x7f884c15c720 legacy=0x7f884c073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.224+0000 7f8841400640 1 -- 172.21.15.3:0/3130550544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c0cda90 con 0x7f884c070f30 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8841400640 1 --2- 172.21.15.3:0/3130550544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c070f30 0x7f884c071cd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f884c06dd40 tx=0x7f883402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c129675e05fa79f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8849400640 1 -- 172.21.15.3:0/3130550544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834037070 con 0x7f884c070f30 2024-09-10T07:53:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8849400640 1 -- 172.21.15.3:0/3130550544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834032440 con 0x7f884c070f30 2024-09-10T07:53:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8849400640 1 -- 172.21.15.3:0/3130550544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834032810 con 0x7f884c070f30 2024-09-10T07:53:08.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8850e00640 1 -- 172.21.15.3:0/3130550544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c070f30 msgr2=0x7f884c071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.225+0000 7f8850e00640 1 --2- 172.21.15.3:0/3130550544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c070f30 0x7f884c071cd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f884c06dd40 tx=0x7f883402a270 comp rx=0 tx=0).stop 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 -- 172.21.15.3:0/3130550544 shutdown_connections 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 --2- 172.21.15.3:0/3130550544 >> v2:172.21.15.70:3300/0 conn(0x7f884c0cfdc0 0x7f884c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 --2- 172.21.15.3:0/3130550544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c070f30 0x7f884c071cd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 -- 172.21.15.3:0/3130550544 >> 172.21.15.3:0/3130550544 conn(0x7f884c06cfc0 msgr2=0x7f884c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 -- 172.21.15.3:0/3130550544 shutdown_connections 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.226+0000 7f8850e00640 1 -- 172.21.15.3:0/3130550544 wait complete. 2024-09-10T07:53:08.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.227+0000 7f8850e00640 1 Processor -- start 2024-09-10T07:53:08.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.227+0000 7f8850e00640 1 -- start start 2024-09-10T07:53:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f884c070f30 0x7f884c1e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f884c15c720 0x7f884c1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f884c073d20 con 0x7f884c0cfdc0 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f884c06ed10 con 0x7f884c070f30 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8850e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f884c073ea0 con 0x7f884c15c720 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8841400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8841400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60950/0 (socket says 172.21.15.3:60950) 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.228+0000 7f8841400640 1 -- 172.21.15.3:0/1242042554 learned_addr learned my addr 172.21.15.3:0/1242042554 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 -- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f884c15c720 msgr2=0x7f884c1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f884c15c720 0x7f884c1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 -- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f884c070f30 msgr2=0x7f884c1e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f884c070f30 0x7f884c1e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 -- 172.21.15.3:0/1242042554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c1e6870 con 0x7f884c0cfdc0 2024-09-10T07:53:08.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8841400640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f883402a2b0 tx=0x7f88340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834037040 con 0x7f884c0cfdc0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834042070 con 0x7f884c0cfdc0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883403b870 con 0x7f884c0cfdc0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c1e6b50 con 0x7f884c0cfdc0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.229+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1e7110 con 0x7f884c0cfdc0 2024-09-10T07:53:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.230+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c0cf2b0 con 0x7f884c0cfdc0 2024-09-10T07:53:08.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.231+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f883403ba10 con 0x7f884c0cfdc0 2024-09-10T07:53:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.233+0000 7f8842a00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 0x7f882806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.233+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f88340ab340 con 0x7f884c0cfdc0 2024-09-10T07:53:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.233+0000 7f8849e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 0x7f882806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.233+0000 7f8849e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 0x7f882806e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f883c000f60 tx=0x7f883c0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.237+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f883407a030 con 0x7f884c0cfdc0 2024-09-10T07:53:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.422+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f884c158920 con 0x7f884c0cfdc0 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.422+0000 7f8842a00640 1 -- 172.21.15.3:0/1242042554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) ==== 84+0+65 (secure 0 0 0) 0x7f8834079930 con 0x7f884c0cfdc0 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:08.422 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-10T07:53:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 msgr2=0x7f882806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 0x7f882806e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f883c000f60 tx=0x7f883c0079b0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 msgr2=0x7f884c158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f883402a2b0 tx=0x7f88340046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 shutdown_connections 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f882806be60 0x7f882806e320 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f884c15c720 0x7f884c1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0cfdc0 0x7f884c158b80 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 --2- 172.21.15.3:0/1242042554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f884c070f30 0x7f884c1e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 >> 172.21.15.3:0/1242042554 conn(0x7f884c06cfc0 msgr2=0x7f884c15ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.424+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 shutdown_connections 2024-09-10T07:53:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.425+0000 7f8850e00640 1 -- 172.21.15.3:0/1242042554 wait complete. 2024-09-10T07:53:08.439 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T07:53:08.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T07:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.600+0000 7fe498c00640 1 Processor -- start 2024-09-10T07:53:08.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.601+0000 7fe498c00640 1 -- start start 2024-09-10T07:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe498c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4940b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe498c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4940cf1f0 con 0x7fe49415c530 2024-09-10T07:53:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe498c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4940b2930 con 0x7fe4940b24d0 2024-09-10T07:53:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe498c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4940d00c0 con 0x7fe4940cd910 2024-09-10T07:53:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe491e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe49415c530 0x7fe4940cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36758/0 (socket says 172.21.15.3:36758) 2024-09-10T07:53:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe491e00640 1 -- 172.21.15.3:0/869852455 learned_addr learned my addr 172.21.15.3:0/869852455 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:08.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe48be00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4940b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.602+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325324046 0 0) 0x7fe4940cf1f0 con 0x7fe49415c530 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe478003520 con 0x7fe49415c530 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258769202 0 0) 0x7fe4940b2930 con 0x7fe4940b24d0 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4940cf1f0 con 0x7fe4940b24d0 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2926336965 0 0) 0x7fe478003520 con 0x7fe49415c530 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 msgr2=0x7fe4940b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4940b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 >> v1:172.21.15.70:6789/0 conn(0x7fe4940b24d0 legacy=0x7fe4940722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4940ce150 con 0x7fe49415c530 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fe484001670 con 0x7fe49415c530 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 >> v1:172.21.15.3:6789/0 conn(0x7fe49415c530 legacy=0x7fe4940cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe478003b70 0x7fe478003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe47800ba70 con 0x7fe478004800 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe47800bbf0 con 0x7fe478003b70 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe47800bda0 con 0x7fe4940cd910 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe491e00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe478003b70 0x7fe478003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe48be00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.603+0000 7fe492800640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.604+0000 7fe48be00640 1 -- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 msgr2=0x7fe4780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.604+0000 7fe48be00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940cd910 0x7fe4780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.604+0000 7fe48be00640 1 -- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe478003b70 msgr2=0x7fe478003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.604+0000 7fe48be00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe478003b70 0x7fe478003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.604+0000 7fe48be00640 1 -- 172.21.15.3:0/869852455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478003520 con 0x7fe478004800 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe48be00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe478004c80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009a90 tx=0x7fe478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4d5defc36fab32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c036070 con 0x7fe478004800 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47c031400 con 0x7fe478004800 2024-09-10T07:53:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe491400640 1 -- 172.21.15.3:0/869852455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c009690 con 0x7fe478004800 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe498c00640 1 -- 172.21.15.3:0/869852455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 msgr2=0x7fe478004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.605+0000 7fe498c00640 1 --2- 172.21.15.3:0/869852455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe478004c80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009a90 tx=0x7fe478005830 comp rx=0 tx=0).stop 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.606+0000 7fe492800640 1 -- 172.21.15.3:0/869852455 reap_dead start 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.606+0000 7fe498c00640 1 -- 172.21.15.3:0/869852455 shutdown_connections 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.606+0000 7fe498c00640 1 -- 172.21.15.3:0/869852455 >> 172.21.15.3:0/869852455 conn(0x7fe494062cc0 msgr2=0x7fe49406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.606+0000 7fe498c00640 1 -- 172.21.15.3:0/869852455 shutdown_connections 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.606+0000 7fe498c00640 1 -- 172.21.15.3:0/869852455 wait complete. 2024-09-10T07:53:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.607+0000 7fe498c00640 1 Processor -- start 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.607+0000 7fe498c00640 1 -- start start 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940b4780 0x7fe494155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 0x7fe4941f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940ce9f0 con 0x7fe478004800 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940ce710 con 0x7fe4941564f0 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe498c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe4940cecd0 con 0x7fe4940b4780 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.608+0000 7fe491e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60954/0 (socket says 172.21.15.3:60954) 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 -- 172.21.15.3:0/24062553 learned_addr learned my addr 172.21.15.3:0/24062553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe492800640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 0x7fe4941f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe48be00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940b4780 0x7fe494155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 -- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940b4780 msgr2=0x7fe494155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940b4780 0x7fe494155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 -- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 msgr2=0x7fe4941f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 0x7fe4941f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 -- 172.21.15.3:0/24062553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4940b3400 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe492800640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 0x7fe4941f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe491e00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe4840072f0 tx=0x7fe4840077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48401a070 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48400f9c0 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4940b36e0 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.609+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484001670 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.610+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4940b3ca0 con 0x7fe478004800 2024-09-10T07:53:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.611+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4940b1f90 con 0x7fe478004800 2024-09-10T07:53:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.611+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe48401d070 con 0x7fe478004800 2024-09-10T07:53:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.612+0000 7fe48a000640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 0x7fe47006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.612+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7fe48408a650 con 0x7fe478004800 2024-09-10T07:53:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.613+0000 7fe48be00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 0x7fe47006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.613+0000 7fe48be00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 0x7fe47006e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe47c033000 tx=0x7fe47c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:08.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.616+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4840592b0 con 0x7fe478004800 2024-09-10T07:53:08.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:08.800+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4940cff60 con 0x7fe478004800 2024-09-10T07:53:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.198+0000 7fe48a000640 1 -- 172.21.15.3:0/24062553 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) ==== 138+0+0 (secure 0 0 0) 0x7fe484058bb0 con 0x7fe478004800 2024-09-10T07:53:09.197 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 msgr2=0x7fe47006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 0x7fe47006e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe47c033000 tx=0x7fe47c008030 comp rx=0 tx=0).stop 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 msgr2=0x7fe49414c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe4840072f0 tx=0x7fe4840077b0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 shutdown_connections 2024-09-10T07:53:09.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe47006be30 0x7fe47006e2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4941564f0 0x7fe4941f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4940b4780 0x7fe494155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 --2- 172.21.15.3:0/24062553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe478004800 0x7fe49414c070 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 >> 172.21.15.3:0/24062553 conn(0x7fe494062cc0 msgr2=0x7fe494072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 shutdown_connections 2024-09-10T07:53:09.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.201+0000 7fe498c00640 1 -- 172.21.15.3:0/24062553 wait complete. 2024-09-10T07:53:09.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T07:53:09.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.386+0000 7f6c54a00640 1 Processor -- start 2024-09-10T07:53:09.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.386+0000 7f6c54a00640 1 -- start start 2024-09-10T07:53:09.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.387+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 0x7f6c5015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.387+0000 7f6c54a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 0x7f6c50071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.387+0000 7f6c54a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c5015eff0 con 0x7f6c5015b230 2024-09-10T07:53:09.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.387+0000 7f6c54a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6c5015fbe0 con 0x7f6c50072c50 2024-09-10T07:53:09.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.387+0000 7f6c54a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5015c8d0 con 0x7f6c500b4c90 2024-09-10T07:53:09.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.388+0000 7f6c4e800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 0x7f6c50071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.388+0000 7f6c45400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 0x7f6c5015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.388+0000 7f6c4de00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c5015b230 0x7f6c500b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:37804/0 (socket says 172.21.15.3:37804) 2024-09-10T07:53:09.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.388+0000 7f6c4de00640 1 -- 172.21.15.3:0/1242157574 learned_addr learned my addr 172.21.15.3:0/1242157574 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.388+0000 7f6c4d400640 1 -- 172.21.15.3:0/1242157574 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723522703 0 0) 0x7f6c5015eff0 con 0x7f6c5015b230 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c4d400640 1 -- 172.21.15.3:0/1242157574 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c34005730 con 0x7f6c5015b230 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c45400640 1 -- 172.21.15.3:0/1242157574 >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 msgr2=0x7f6c50071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c45400640 1 --2- 172.21.15.3:0/1242157574 >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 0x7f6c50071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c45400640 1 -- 172.21.15.3:0/1242157574 >> v1:172.21.15.70:6789/0 conn(0x7f6c5015b230 legacy=0x7f6c500b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c45400640 1 -- 172.21.15.3:0/1242157574 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5015f2b0 con 0x7f6c500b4c90 2024-09-10T07:53:09.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.389+0000 7f6c4e800640 1 --2- 172.21.15.3:0/1242157574 >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 0x7f6c50071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c45400640 1 --2- 172.21.15.3:0/1242157574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 0x7f6c5015c390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c380097a0 tx=0x7f6c3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cae4d74dc3e3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c4d400640 1 -- 172.21.15.3:0/1242157574 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38037070 con 0x7f6c500b4c90 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c4d400640 1 -- 172.21.15.3:0/1242157574 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38032440 con 0x7f6c500b4c90 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c4d400640 1 -- 172.21.15.3:0/1242157574 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380327f0 con 0x7f6c500b4c90 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c54a00640 1 -- 172.21.15.3:0/1242157574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 msgr2=0x7f6c5015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.390+0000 7f6c54a00640 1 --2- 172.21.15.3:0/1242157574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 0x7f6c5015c390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c380097a0 tx=0x7f6c3802a180 comp rx=0 tx=0).stop 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 -- 172.21.15.3:0/1242157574 shutdown_connections 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 --2- 172.21.15.3:0/1242157574 >> v2:172.21.15.70:3300/0 conn(0x7f6c50072c50 0x7f6c50071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 --2- 172.21.15.3:0/1242157574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c500b4c90 0x7f6c5015c390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 -- 172.21.15.3:0/1242157574 >> 172.21.15.3:0/1242157574 conn(0x7f6c500b5cb0 msgr2=0x7f6c5006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:09.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 -- 172.21.15.3:0/1242157574 shutdown_connections 2024-09-10T07:53:09.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.391+0000 7f6c54a00640 1 -- 172.21.15.3:0/1242157574 wait complete. 2024-09-10T07:53:09.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.392+0000 7f6c54a00640 1 Processor -- start 2024-09-10T07:53:09.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.392+0000 7f6c54a00640 1 -- start start 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c50072c50 0x7f6c5013db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 0x7f6c500ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5015b610 con 0x7f6c500b4c90 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5015f870 con 0x7f6c50072c50 2024-09-10T07:53:09.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.393+0000 7f6c54a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5015b790 con 0x7f6c5015b230 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60966/0 (socket says 172.21.15.3:60966) 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 -- 172.21.15.3:0/2927487531 learned_addr learned my addr 172.21.15.3:0/2927487531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c4e800640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 0x7f6c500ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c4de00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c50072c50 0x7f6c5013db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 -- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 msgr2=0x7f6c500ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 0x7f6c500ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 -- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c50072c50 msgr2=0x7f6c5013db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c50072c50 0x7f6c5013db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c45400640 1 -- 172.21.15.3:0/2927487531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500b0bb0 con 0x7f6c500b4c90 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.394+0000 7f6c4e800640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 0x7f6c500ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:09.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c45400640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6c38035040 tx=0x7f6c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:09.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38037040 con 0x7f6c500b4c90 2024-09-10T07:53:09.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c380044c0 con 0x7f6c500b4c90 2024-09-10T07:53:09.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5013e050 con 0x7f6c500b4c90 2024-09-10T07:53:09.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380325f0 con 0x7f6c500b4c90 2024-09-10T07:53:09.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.395+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c500b1130 con 0x7f6c500b4c90 2024-09-10T07:53:09.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.396+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50158dc0 con 0x7f6c500b4c90 2024-09-10T07:53:09.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.397+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c38044020 con 0x7f6c500b4c90 2024-09-10T07:53:09.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.398+0000 7f6c46a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 0x7f6c2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:09.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.398+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5227+0+0 (secure 0 0 0) 0x7f6c380aade0 con 0x7f6c500b4c90 2024-09-10T07:53:09.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.398+0000 7f6c4de00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 0x7f6c2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:09.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.399+0000 7f6c4de00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 0x7f6c2c06e320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005880 tx=0x7f6c40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:09.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.403+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c50158dc0 con 0x7f6c500b4c90 2024-09-10T07:53:09.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:09.588+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6c5006e1e0 con 0x7f6c500b4c90 2024-09-10T07:53:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.201+0000 7f6c46a00640 1 -- 172.21.15.3:0/2927487531 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) ==== 170+0+0 (secure 0 0 0) 0x7f6c38079280 con 0x7f6c500b4c90 2024-09-10T07:53:10.200 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-10T07:53:10.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 msgr2=0x7f6c2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 0x7f6c2c06e320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005880 tx=0x7f6c40009040 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 msgr2=0x7f6c50157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6c38035040 tx=0x7f6c380046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 shutdown_connections 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6c2c06be60 0x7f6c2c06e320 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c5015b230 0x7f6c500ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c500b4c90 0x7f6c50157250 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 --2- 172.21.15.3:0/2927487531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c50072c50 0x7f6c5013db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.205+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 >> 172.21.15.3:0/2927487531 conn(0x7f6c500b5cb0 msgr2=0x7f6c500ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.206+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 shutdown_connections 2024-09-10T07:53:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.207+0000 7f6c54a00640 1 -- 172.21.15.3:0/2927487531 wait complete. 2024-09-10T07:53:10.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T07:53:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.397+0000 7f0408000640 1 Processor -- start 2024-09-10T07:53:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.397+0000 7f0408000640 1 -- start start 2024-09-10T07:53:10.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.398+0000 7f0408000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04001605a0 0x7f04000cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.398+0000 7f0408000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04000b3a80 0x7f0400166490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.398+0000 7f0408000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0400071be0 con 0x7f04000720c0 2024-09-10T07:53:10.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.398+0000 7f0408000640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f040006dbe0 con 0x7f04000b3a80 2024-09-10T07:53:10.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.398+0000 7f0408000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f040006dec0 con 0x7f04001605a0 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04000b3a80 0x7f0400166490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04000b3a80 0x7f0400166490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60974/0 (socket says 172.21.15.3:60974) 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 -- 172.21.15.3:0/213232882 learned_addr learned my addr 172.21.15.3:0/213232882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0405600640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04001605a0 0x7f04000cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 -- 172.21.15.3:0/213232882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04001605a0 msgr2=0x7f04000cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04001605a0 0x7f04000cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 -- 172.21.15.3:0/213232882 >> v1:172.21.15.70:6789/0 conn(0x7f04000720c0 legacy=0x7f04000b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.399+0000 7f0406000640 1 -- 172.21.15.3:0/213232882 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040006e1a0 con 0x7f04000b3a80 2024-09-10T07:53:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.400+0000 7f0406000640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000b3a80 0x7f0400166490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009e50 tx=0x7f03f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e451d6de609f7281 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.400+0000 7f03ffe00640 1 -- 172.21.15.3:0/213232882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8036070 con 0x7f04000b3a80 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f03ffe00640 1 -- 172.21.15.3:0/213232882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f8031600 con 0x7f04000b3a80 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f03ffe00640 1 -- 172.21.15.3:0/213232882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f80319d0 con 0x7f04000b3a80 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 -- 172.21.15.3:0/213232882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000b3a80 msgr2=0x7f0400166490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000b3a80 0x7f0400166490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009e50 tx=0x7f03f802a680 comp rx=0 tx=0).stop 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 -- 172.21.15.3:0/213232882 shutdown_connections 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000b3a80 0x7f0400166490 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 --2- 172.21.15.3:0/213232882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04001605a0 0x7f04000cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 -- 172.21.15.3:0/213232882 >> 172.21.15.3:0/213232882 conn(0x7f04000b5cb0 msgr2=0x7f040006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 -- 172.21.15.3:0/213232882 shutdown_connections 2024-09-10T07:53:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.401+0000 7f0408000640 1 -- 172.21.15.3:0/213232882 wait complete. 2024-09-10T07:53:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.403+0000 7f0408000640 1 Processor -- start 2024-09-10T07:53:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.403+0000 7f0408000640 1 -- start start 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 0x7f04000ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04001605a0 0x7f04000abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f04001669d0 con 0x7f04000720c0 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f04000b4180 con 0x7f04001605a0 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0408000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0400166b50 con 0x7f04000b3a80 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60984/0 (socket says 172.21.15.3:60984) 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 -- 172.21.15.3:0/4236026537 learned_addr learned my addr 172.21.15.3:0/4236026537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0406000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04001605a0 0x7f04000abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0404c00640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 0x7f04000ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 -- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 msgr2=0x7f04000ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 0x7f04000ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 -- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04001605a0 msgr2=0x7f04000abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.404+0000 7f0405600640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04001605a0 0x7f04000abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f0405600640 1 -- 172.21.15.3:0/4236026537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04000ae3c0 con 0x7f04000720c0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f0404c00640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 0x7f04000ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f0405600640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e870 tx=0x7f03f400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4017070 con 0x7f04000720c0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f400c940 con 0x7f04000720c0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f400cd30 con 0x7f04000720c0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04000ae6a0 con 0x7f04000720c0 2024-09-10T07:53:10.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.405+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04000aec60 con 0x7f04000720c0 2024-09-10T07:53:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.406+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400166490 con 0x7f04000720c0 2024-09-10T07:53:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.407+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f401a070 con 0x7f04000720c0 2024-09-10T07:53:10.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.408+0000 7f03fd600640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 0x7f03e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:10.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.408+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5609+0+0 (secure 0 0 0) 0x7f03f408b060 con 0x7f04000720c0 2024-09-10T07:53:10.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.409+0000 7f0404c00640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 0x7f03e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:10.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.409+0000 7f0404c00640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 0x7f03e006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f04000d01a0 tx=0x7f03ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:10.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.413+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03f4059380 con 0x7f04000720c0 2024-09-10T07:53:10.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:10.635+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f040006a050 con 0x7f04000720c0 2024-09-10T07:53:11.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.240+0000 7f03fd600640 1 -- 172.21.15.3:0/4236026537 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v100) ==== 174+0+0 (secure 0 0 0) 0x7f03f405d680 con 0x7f04000720c0 2024-09-10T07:53:11.243 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 14 and data pool 15 2024-09-10T07:53:11.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 msgr2=0x7f03e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 0x7f03e006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f04000d01a0 tx=0x7f03ec009040 comp rx=0 tx=0).stop 2024-09-10T07:53:11.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 msgr2=0x7f040015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e870 tx=0x7f03f400ed30 comp rx=0 tx=0).stop 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 shutdown_connections 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f03e006be30 0x7f03e006e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04001605a0 0x7f04000abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04000b3a80 0x7f04000ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 --2- 172.21.15.3:0/4236026537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04000720c0 0x7f040015d700 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.249+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 >> 172.21.15.3:0/4236026537 conn(0x7f04000b5cb0 msgr2=0x7f0400070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.250+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 shutdown_connections 2024-09-10T07:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.251+0000 7f0408000640 1 -- 172.21.15.3:0/4236026537 wait complete. 2024-09-10T07:53:11.279 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:53:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.437+0000 7f5161200640 1 Processor -- start 2024-09-10T07:53:11.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.437+0000 7f5161200640 1 -- start start 2024-09-10T07:53:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.438+0000 7f5161200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.438+0000 7f5161200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0b4250 con 0x7f515c15b4f0 2024-09-10T07:53:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.438+0000 7f5161200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0ccd80 con 0x7f515c0d08f0 2024-09-10T07:53:11.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.438+0000 7f5161200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f515c0b44f0 con 0x7f515c15b110 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f515c15b4f0 0x7f515c0d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36774/0 (socket says 172.21.15.3:36774) 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159400640 1 -- 172.21.15.3:0/1679478619 learned_addr learned my addr 172.21.15.3:0/1679478619 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159e00640 1 --2- 172.21.15.3:0/1679478619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386546504 0 0) 0x7f515c0b4250 con 0x7f515c15b4f0 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5140003520 con 0x7f515c15b4f0 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762249851 0 0) 0x7f515c0ccd80 con 0x7f515c0d08f0 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159e00640 1 -- 172.21.15.3:0/1679478619 >> v1:172.21.15.70:6789/0 conn(0x7f515c0d08f0 legacy=0x7f515c0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159e00640 1 -- 172.21.15.3:0/1679478619 >> v1:172.21.15.3:6789/0 conn(0x7f515c15b4f0 legacy=0x7f515c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.439+0000 7f5159e00640 1 -- 172.21.15.3:0/1679478619 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0cd060 con 0x7f515c15b110 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.440+0000 7f5159e00640 1 --2- 172.21.15.3:0/1679478619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0b3d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f514c0097d0 tx=0x7f514c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cd34dd575fb12e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c037070 con 0x7f515c15b110 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c032440 con 0x7f515c15b110 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5158a00640 1 -- 172.21.15.3:0/1679478619 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c032810 con 0x7f515c15b110 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 -- 172.21.15.3:0/1679478619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 msgr2=0x7f515c0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 --2- 172.21.15.3:0/1679478619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0b3d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f514c0097d0 tx=0x7f514c02a210 comp rx=0 tx=0).stop 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 -- 172.21.15.3:0/1679478619 shutdown_connections 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 --2- 172.21.15.3:0/1679478619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0b3d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 -- 172.21.15.3:0/1679478619 >> 172.21.15.3:0/1679478619 conn(0x7f515c06d000 msgr2=0x7f515c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 -- 172.21.15.3:0/1679478619 shutdown_connections 2024-09-10T07:53:11.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.441+0000 7f5161200640 1 -- 172.21.15.3:0/1679478619 wait complete. 2024-09-10T07:53:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.443+0000 7f5161200640 1 Processor -- start 2024-09-10T07:53:11.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.443+0000 7f5161200640 1 -- start start 2024-09-10T07:53:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515c0d08f0 0x7f515c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f515c073d10 con 0x7f515c15b4f0 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f515c0cd620 con 0x7f515c0d08f0 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5161200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f515c073e90 con 0x7f515c15b110 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f515a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5159400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f5159e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515c0d08f0 0x7f515c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f515a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60986/0 (socket says 172.21.15.3:60986) 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.444+0000 7f515a800640 1 -- 172.21.15.3:0/3640166509 learned_addr learned my addr 172.21.15.3:0/3640166509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 -- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 msgr2=0x7f515c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 -- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515c0d08f0 msgr2=0x7f515c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515c0d08f0 0x7f515c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 -- 172.21.15.3:0/3640166509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c15cc20 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f515a800640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f515400c790 tx=0x7f515400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154003da0 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5154005070 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154005460 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c15cf00 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.445+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c15d4c0 con 0x7f515c15b4f0 2024-09-10T07:53:11.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.447+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c0b3d10 con 0x7f515c15b4f0 2024-09-10T07:53:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.447+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5154005600 con 0x7f515c15b4f0 2024-09-10T07:53:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.448+0000 7f5152000640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 0x7f513c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.448+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f515408acb0 con 0x7f515c15b4f0 2024-09-10T07:53:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.449+0000 7f5159e00640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 0x7f513c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.449+0000 7f5159e00640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 0x7f513c06e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f514c034000 tx=0x7f514c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.453+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5154059000 con 0x7f515c15b4f0 2024-09-10T07:53:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.641+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f515c15d690 con 0x7f515c15b4f0 2024-09-10T07:53:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.642+0000 7f5152000640 1 -- 172.21.15.3:0/3640166509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16620 (secure 0 0 0) 0x7f515405d300 con 0x7f515c15b4f0 2024-09-10T07:53:11.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:11.643 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":101,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:10.636650+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:53:08.843582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T07:53:09.632431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 msgr2=0x7f513c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 0x7f513c06e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f514c034000 tx=0x7f514c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 msgr2=0x7f515c1588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f515400c790 tx=0x7f515400cc50 comp rx=0 tx=0).stop 2024-09-10T07:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 shutdown_connections 2024-09-10T07:53:11.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f513c06be30 0x7f513c06e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f515c15b4f0 0x7f515c1588f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f515c15b110 0x7f515c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 --2- 172.21.15.3:0/3640166509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515c0d08f0 0x7f515c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 >> 172.21.15.3:0/3640166509 conn(0x7f515c06d000 msgr2=0x7f515c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 shutdown_connections 2024-09-10T07:53:11.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.645+0000 7f5161200640 1 -- 172.21.15.3:0/3640166509 wait complete. 2024-09-10T07:53:11.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:53:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.831+0000 7ff34e600640 1 Processor -- start 2024-09-10T07:53:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.831+0000 7ff34e600640 1 -- start start 2024-09-10T07:53:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34e600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34815e860 0x7ff3480cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34e600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 0x7ff3480d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34e600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3480cee40 con 0x7ff3480d4150 2024-09-10T07:53:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34e600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480d3780 con 0x7ff34815e860 2024-09-10T07:53:11.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34e600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480cf120 con 0x7ff3480cfbf0 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34815e860 0x7ff3480cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff34815e860 0x7ff3480cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32768/0 (socket says 172.21.15.3:32768) 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.832+0000 7ff34d200640 1 -- 172.21.15.3:0/3831479308 learned_addr learned my addr 172.21.15.3:0/3831479308 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff347e00640 1 --2- 172.21.15.3:0/3831479308 >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 0x7ff3480d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff34d200640 1 -- 172.21.15.3:0/3831479308 >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 msgr2=0x7ff3480d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff34d200640 1 --2- 172.21.15.3:0/3831479308 >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 0x7ff3480d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff34d200640 1 -- 172.21.15.3:0/3831479308 >> v1:172.21.15.70:6789/0 conn(0x7ff3480d4150 legacy=0x7ff3480b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff34d200640 1 -- 172.21.15.3:0/3831479308 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480d3410 con 0x7ff34815e860 2024-09-10T07:53:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.833+0000 7ff347e00640 1 --2- 172.21.15.3:0/3831479308 >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 0x7ff3480d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.834+0000 7ff34d200640 1 --2- 172.21.15.3:0/3831479308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34815e860 0x7ff3480cf6b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff3480d0ad0 tx=0x7ff33c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95dd553994d368f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.834+0000 7ff347400640 1 -- 172.21.15.3:0/3831479308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c037070 con 0x7ff34815e860 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.834+0000 7ff347400640 1 -- 172.21.15.3:0/3831479308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33c032440 con 0x7ff34815e860 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.834+0000 7ff347400640 1 -- 172.21.15.3:0/3831479308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c032810 con 0x7ff34815e860 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 -- 172.21.15.3:0/3831479308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34815e860 msgr2=0x7ff3480cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 --2- 172.21.15.3:0/3831479308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34815e860 0x7ff3480cf6b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff3480d0ad0 tx=0x7ff33c02a230 comp rx=0 tx=0).stop 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 -- 172.21.15.3:0/3831479308 shutdown_connections 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 --2- 172.21.15.3:0/3831479308 >> v2:172.21.15.70:3300/0 conn(0x7ff3480cfbf0 0x7ff3480d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 --2- 172.21.15.3:0/3831479308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff34815e860 0x7ff3480cf6b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 -- 172.21.15.3:0/3831479308 >> 172.21.15.3:0/3831479308 conn(0x7ff3480d4650 msgr2=0x7ff3480d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 -- 172.21.15.3:0/3831479308 shutdown_connections 2024-09-10T07:53:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.835+0000 7ff34e600640 1 -- 172.21.15.3:0/3831479308 wait complete. 2024-09-10T07:53:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.836+0000 7ff34e600640 1 Processor -- start 2024-09-10T07:53:11.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 -- start start 2024-09-10T07:53:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3480cfbf0 0x7ff3480cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 0x7ff3480ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480b7f90 con 0x7ff3480d4150 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480d2460 con 0x7ff3480cfbf0 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.837+0000 7ff34e600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480b8110 con 0x7ff34815e860 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32784/0 (socket says 172.21.15.3:32784) 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 -- 172.21.15.3:0/3588151228 learned_addr learned my addr 172.21.15.3:0/3588151228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff34d200640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3480cfbf0 0x7ff3480cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff34dc00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 0x7ff3480ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 -- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 msgr2=0x7ff3480ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 0x7ff3480ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 -- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3480cfbf0 msgr2=0x7ff3480cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3480cfbf0 0x7ff3480cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 -- 172.21.15.3:0/3588151228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480cd200 con 0x7ff3480d4150 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff34dc00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 0x7ff3480ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.838+0000 7ff347e00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c7f0 tx=0x7ff33400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.839+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334015070 con 0x7ff3480d4150 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.839+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff334007a30 con 0x7ff3480d4150 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.839+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334007e00 con 0x7ff3480d4150 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.839+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3480cbc20 con 0x7ff3480d4150 2024-09-10T07:53:11.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.839+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3480cc1e0 con 0x7ff3480d4150 2024-09-10T07:53:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.840+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480d3bf0 con 0x7ff3480d4150 2024-09-10T07:53:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.841+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff334018070 con 0x7ff3480d4150 2024-09-10T07:53:11.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.841+0000 7ff344c00640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 0x7ff32806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.842+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff33408aef0 con 0x7ff3480d4150 2024-09-10T07:53:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.842+0000 7ff34d200640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 0x7ff32806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:11.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.842+0000 7ff34d200640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 0x7ff32806e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff33c034000 tx=0x7ff33c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:11.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:11.846+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff334059800 con 0x7ff3480d4150 2024-09-10T07:53:12.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.030+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3480d2c70 con 0x7ff3480d4150 2024-09-10T07:53:12.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.031+0000 7ff344c00640 1 -- 172.21.15.3:0/3588151228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16620 (secure 0 0 0) 0x7ff334059100 con 0x7ff3480d4150 2024-09-10T07:53:12.031 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:12.031 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":101,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:10.636650+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:53:08.843582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T07:53:09.632431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 msgr2=0x7ff32806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 0x7ff32806e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff33c034000 tx=0x7ff33c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 msgr2=0x7ff34814fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c7f0 tx=0x7ff33400cbf0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 shutdown_connections 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff32806be30 0x7ff32806e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff34815e860 0x7ff3480ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3480d4150 0x7ff34814fa10 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 --2- 172.21.15.3:0/3588151228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3480cfbf0 0x7ff3480cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 >> 172.21.15.3:0/3588151228 conn(0x7ff3480d4650 msgr2=0x7ff3480b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.033+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 shutdown_connections 2024-09-10T07:53:12.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.034+0000 7ff34e600640 1 -- 172.21.15.3:0/3588151228 wait complete. 2024-09-10T07:53:12.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T07:53:12.198 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:53:12.198+0000 7f2cd0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T07:53:12.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.210+0000 7f083f800640 1 Processor -- start 2024-09-10T07:53:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.210+0000 7f083f800640 1 -- start start 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083f800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083f800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08380d2a90 con 0x7f08380d3b30 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083f800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08380d2d70 con 0x7f083815e7e0 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083f800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08380b8120 con 0x7f08380b7780 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083d800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f08380d3b30 0x7f08380d3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36776/0 (socket says 172.21.15.3:36776) 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.211+0000 7f083d800640 1 -- 172.21.15.3:0/133077311 learned_addr learned my addr 172.21.15.3:0/133077311 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837e00640 1 --2- 172.21.15.3:0/133077311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183997481 0 0) 0x7f08380d2a90 con 0x7f08380d3b30 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0820003520 con 0x7f08380d3b30 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625853916 0 0) 0x7f08380d2d70 con 0x7f083815e7e0 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837e00640 1 -- 172.21.15.3:0/133077311 >> v1:172.21.15.70:6789/0 conn(0x7f083815e7e0 legacy=0x7f08380cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837e00640 1 -- 172.21.15.3:0/133077311 >> v1:172.21.15.3:6789/0 conn(0x7f08380d3b30 legacy=0x7f08380d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.212+0000 7f0837e00640 1 -- 172.21.15.3:0/133077311 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380d3050 con 0x7f08380b7780 2024-09-10T07:53:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.213+0000 7f0837e00640 1 --2- 172.21.15.3:0/133077311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380b7be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08240097d0 tx=0x7f082402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1183001118e70b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824037070 con 0x7f08380b7780 2024-09-10T07:53:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824032440 con 0x7f08380b7780 2024-09-10T07:53:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f0837400640 1 -- 172.21.15.3:0/133077311 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824032810 con 0x7f08380b7780 2024-09-10T07:53:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 -- 172.21.15.3:0/133077311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 msgr2=0x7f08380b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 --2- 172.21.15.3:0/133077311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380b7be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08240097d0 tx=0x7f082402a210 comp rx=0 tx=0).stop 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 -- 172.21.15.3:0/133077311 shutdown_connections 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 --2- 172.21.15.3:0/133077311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380b7be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 -- 172.21.15.3:0/133077311 >> 172.21.15.3:0/133077311 conn(0x7f0838072560 msgr2=0x7f0838072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 -- 172.21.15.3:0/133077311 shutdown_connections 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.214+0000 7f083f800640 1 -- 172.21.15.3:0/133077311 wait complete. 2024-09-10T07:53:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.216+0000 7f083f800640 1 Processor -- start 2024-09-10T07:53:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.216+0000 7f083f800640 1 -- start start 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 0x7f08381f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08380b3940 con 0x7f08380d3b30 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f08380d3610 con 0x7f083815e7e0 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f083f800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f08380d26b0 con 0x7f08380b7780 2024-09-10T07:53:12.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f0837e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f0837e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32794/0 (socket says 172.21.15.3:32794) 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.217+0000 7f0837e00640 1 -- 172.21.15.3:0/1357498406 learned_addr learned my addr 172.21.15.3:0/1357498406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f083ce00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 -- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 msgr2=0x7f08380cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f083d800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 0x7f08381f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 -- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 msgr2=0x7f08381f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 0x7f08381f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 -- 172.21.15.3:0/1357498406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380ce920 con 0x7f08380d3b30 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f083ce00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f083d800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 0x7f08381f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.218+0000 7f0837e00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0824035040 tx=0x7f08240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.219+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824041070 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.219+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824004230 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.219+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381f5000 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.219+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824037040 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.219+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08380ce640 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.220+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08380b7be0 con 0x7f08380d3b30 2024-09-10T07:53:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.221+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0824003610 con 0x7f08380d3b30 2024-09-10T07:53:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.221+0000 7f0834c00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.222+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f08240ab640 con 0x7f08380d3b30 2024-09-10T07:53:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.222+0000 7f083ce00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.222+0000 7f083ce00640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 0x7f081806e320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f082c000f60 tx=0x7f082c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:12.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.226+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08381f5000 con 0x7f08380d3b30 2024-09-10T07:53:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:12.449+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f08380b8560 con 0x7f08380d3b30 2024-09-10T07:53:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.239+0000 7f0834c00640 1 -- 172.21.15.3:0/1357498406 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) ==== 118+0+0 (secure 0 0 0) 0x7f0824079f50 con 0x7f08380d3b30 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 msgr2=0x7f081806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 0x7f081806e320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f082c000f60 tx=0x7f082c009040 comp rx=0 tx=0).stop 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 msgr2=0x7f0838158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0824035040 tx=0x7f08240046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 shutdown_connections 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f081806be60 0x7f081806e320 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f083815e7e0 0x7f08381f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08380d3b30 0x7f0838158ef0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 --2- 172.21.15.3:0/1357498406 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f08380b7780 0x7f08380cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 >> 172.21.15.3:0/1357498406 conn(0x7f0838072560 msgr2=0x7f08380b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:13.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 shutdown_connections 2024-09-10T07:53:13.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.242+0000 7f083f800640 1 -- 172.21.15.3:0/1357498406 wait complete. 2024-09-10T07:53:13.260 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T07:53:13.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T07:53:13.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.424+0000 7f65eca00640 1 Processor -- start 2024-09-10T07:53:13.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.424+0000 7f65eca00640 1 -- start start 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65e8161b70 0x7f65e80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65e80d4ae0 0x7f65e80d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f65e8161f50 con 0x7f65e80d4ae0 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f65e80d3de0 con 0x7f65e8161b70 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.425+0000 7f65eca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f65e80d40c0 con 0x7f65e80d3860 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65e80d4ae0 0x7f65e80d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65e80d4ae0 0x7f65e80d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32804/0 (socket says 172.21.15.3:32804) 2024-09-10T07:53:13.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 -- 172.21.15.3:0/3305853073 learned_addr learned my addr 172.21.15.3:0/3305853073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e6a00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7400640 1 --2- 172.21.15.3:0/3305853073 >> v2:172.21.15.70:3300/0 conn(0x7f65e8161b70 0x7f65e80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 -- 172.21.15.3:0/3305853073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 msgr2=0x7f65e80d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e80d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 -- 172.21.15.3:0/3305853073 >> v2:172.21.15.70:3300/0 conn(0x7f65e8161b70 msgr2=0x7f65e80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 --2- 172.21.15.3:0/3305853073 >> v2:172.21.15.70:3300/0 conn(0x7f65e8161b70 0x7f65e80d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.426+0000 7f65e7e00640 1 -- 172.21.15.3:0/3305853073 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e80cf690 con 0x7f65e80d4ae0 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.427+0000 7f65e7e00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e80d4ae0 0x7f65e80d1020 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f65e80d55c0 tx=0x7f65e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf6758f536820f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.427+0000 7f65e6000640 1 -- 172.21.15.3:0/3305853073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0036070 con 0x7f65e80d4ae0 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.427+0000 7f65e6000640 1 -- 172.21.15.3:0/3305853073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e0031600 con 0x7f65e80d4ae0 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.427+0000 7f65e6000640 1 -- 172.21.15.3:0/3305853073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00319d0 con 0x7f65e80d4ae0 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 -- 172.21.15.3:0/3305853073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e80d4ae0 msgr2=0x7f65e80d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e80d4ae0 0x7f65e80d1020 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f65e80d55c0 tx=0x7f65e002a6a0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 -- 172.21.15.3:0/3305853073 shutdown_connections 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e80d4ae0 0x7f65e80d1020 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 --2- 172.21.15.3:0/3305853073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e80d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 --2- 172.21.15.3:0/3305853073 >> v2:172.21.15.70:3300/0 conn(0x7f65e8161b70 0x7f65e80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 -- 172.21.15.3:0/3305853073 >> 172.21.15.3:0/3305853073 conn(0x7f65e8062630 msgr2=0x7f65e8062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:13.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 -- 172.21.15.3:0/3305853073 shutdown_connections 2024-09-10T07:53:13.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.428+0000 7f65eca00640 1 -- 172.21.15.3:0/3305853073 wait complete. 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.430+0000 7f65eca00640 1 Processor -- start 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.430+0000 7f65eca00640 1 -- start start 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e80d4ae0 0x7f65e80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80cf2d0 con 0x7f65e8161b70 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80c9fc0 con 0x7f65e80d4ae0 2024-09-10T07:53:13.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65eca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f65e80cf450 con 0x7f65e80d3860 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32808/0 (socket says 172.21.15.3:32808) 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 -- 172.21.15.3:0/2333863190 learned_addr learned my addr 172.21.15.3:0/2333863190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7400640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e6a00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e80d4ae0 0x7f65e80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 -- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 msgr2=0x7f65e8150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e8150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 -- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e80d4ae0 msgr2=0x7f65e80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e80d4ae0 0x7f65e80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.431+0000 7f65e7e00640 1 -- 172.21.15.3:0/2333863190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81595b0 con 0x7f65e8161b70 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65e7e00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f65e0034040 tx=0x7f65e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:13.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0036050 con 0x7f65e8161b70 2024-09-10T07:53:13.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e0042070 con 0x7f65e8161b70 2024-09-10T07:53:13.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e80cb6e0 con 0x7f65e8161b70 2024-09-10T07:53:13.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0009510 con 0x7f65e8161b70 2024-09-10T07:53:13.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.432+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e80cbca0 con 0x7f65e8161b70 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.433+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e80b7540 con 0x7f65e8161b70 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.434+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65e0007bd0 con 0x7f65e8161b70 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.435+0000 7f65df400640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 0x7f65c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.435+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f65e00abc40 con 0x7f65e8161b70 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.435+0000 7f65e7400640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 0x7f65c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:13.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.436+0000 7f65e7400640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 0x7f65c806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65d8006fd0 tx=0x7f65d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:13.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.439+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f65e0079f00 con 0x7f65e8161b70 2024-09-10T07:53:13.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:13.661+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f65e80761a0 con 0x7f65e8161b70 2024-09-10T07:53:14.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.245+0000 7f65df400640 1 -- 172.21.15.3:0/2333863190 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) ==== 122+0+0 (secure 0 0 0) 0x7f65e007e200 con 0x7f65e8161b70 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 msgr2=0x7f65c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 0x7f65c806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65d8006fd0 tx=0x7f65d8008040 comp rx=0 tx=0).stop 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 msgr2=0x7f65e80cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f65e0034040 tx=0x7f65e0004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 shutdown_connections 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f65c806be30 0x7f65c806e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65e8161b70 0x7f65e80cdd60 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e80d4ae0 0x7f65e80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 --2- 172.21.15.3:0/2333863190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65e80d3860 0x7f65e8150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 >> 172.21.15.3:0/2333863190 conn(0x7f65e8062630 msgr2=0x7f65e80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 shutdown_connections 2024-09-10T07:53:14.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.248+0000 7f65eca00640 1 -- 172.21.15.3:0/2333863190 wait complete. 2024-09-10T07:53:14.265 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T07:53:14.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:14.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.423+0000 7f0f77e00640 1 Processor -- start 2024-09-10T07:53:14.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.424+0000 7f0f77e00640 1 -- start start 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.425+0000 7f0f77e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f78145010 0x7f0f78073410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.425+0000 7f0f77e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.425+0000 7f0f77e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f780ce730 con 0x7f0f78073950 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.425+0000 7f0f77e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f0f780ced10 con 0x7f0f78145010 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.425+0000 7f0f77e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f78073e60 con 0x7f0f7815ca80 2024-09-10T07:53:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f78145010 0x7f0f78073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f6fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f6fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:38840/0 (socket says 172.21.15.3:38840) 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f78145010 0x7f0f78073410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:45524/0 (socket says 172.21.15.3:45524) 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f77400640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0f78073950 0x7f0f78160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:36194/0 (socket says 172.21.15.3:36194) 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f6fe00640 1 -- 172.21.15.3:0/3167374173 learned_addr learned my addr 172.21.15.3:0/3167374173 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 -- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 msgr2=0x7f0f780b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 -- 172.21.15.3:0/3167374173 >> v1:172.21.15.70:6789/0 conn(0x7f0f78073950 legacy=0x7f0f78160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76a00640 1 -- 172.21.15.3:0/3167374173 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f780ce9a0 con 0x7f0f78145010 2024-09-10T07:53:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f76000640 1 -- 172.21.15.3:0/3167374173 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212481110 0 0) 0x7f0f780ce730 con 0x7f0f78073950 2024-09-10T07:53:14.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.426+0000 7f0f6fe00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.427+0000 7f0f76a00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f78145010 0x7f0f78073410 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f680097a0 tx=0x7f0f6802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b953a6d560a50b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.427+0000 7f0f76000640 1 -- 172.21.15.3:0/3167374173 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68037070 con 0x7f0f78145010 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.427+0000 7f0f76000640 1 -- 172.21.15.3:0/3167374173 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f68032440 con 0x7f0f78145010 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.427+0000 7f0f76000640 1 -- 172.21.15.3:0/3167374173 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68032810 con 0x7f0f78145010 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 -- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f78145010 msgr2=0x7f0f78073410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f78145010 0x7f0f78073410 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f680097a0 tx=0x7f0f6802a180 comp rx=0 tx=0).stop 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 -- 172.21.15.3:0/3167374173 shutdown_connections 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f7815ca80 0x7f0f780b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 --2- 172.21.15.3:0/3167374173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f78145010 0x7f0f78073410 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 -- 172.21.15.3:0/3167374173 >> 172.21.15.3:0/3167374173 conn(0x7f0f7806bb20 msgr2=0x7f0f7806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:14.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 -- 172.21.15.3:0/3167374173 shutdown_connections 2024-09-10T07:53:14.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.428+0000 7f0f77e00640 1 -- 172.21.15.3:0/3167374173 wait complete. 2024-09-10T07:53:14.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.430+0000 7f0f77e00640 1 Processor -- start 2024-09-10T07:53:14.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.430+0000 7f0f77e00640 1 -- start start 2024-09-10T07:53:14.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f78073950 0x7f0f781f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f7815ca80 0x7f0f780b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78145570 con 0x7f0f78145010 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781453f0 con 0x7f0f7815ca80 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0f78071cd0 con 0x7f0f78073950 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35854/0 (socket says 172.21.15.3:35854) 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f6fe00640 1 -- 172.21.15.3:0/2507857354 learned_addr learned my addr 172.21.15.3:0/2507857354 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f76a00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f78073950 0x7f0f781f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.431+0000 7f0f77400640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f7815ca80 0x7f0f780b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 -- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f78073950 msgr2=0x7f0f781f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f78073950 0x7f0f781f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 -- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f7815ca80 msgr2=0x7f0f780b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f7815ca80 0x7f0f780b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 -- 172.21.15.3:0/2507857354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7815ff90 con 0x7f0f78145010 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6fe00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9b0 tx=0x7f0f6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60017070 con 0x7f0f78145010 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6000c9a0 con 0x7f0f78145010 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f78160270 con 0x7f0f78145010 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000cd50 con 0x7f0f78145010 2024-09-10T07:53:14.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.432+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781f51e0 con 0x7f0f78145010 2024-09-10T07:53:14.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.433+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f78073410 con 0x7f0f78145010 2024-09-10T07:53:14.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.434+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f6001a070 con 0x7f0f78145010 2024-09-10T07:53:14.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.435+0000 7f0f6ea00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 0x7f0f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.435+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6008b110 con 0x7f0f78145010 2024-09-10T07:53:14.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.435+0000 7f0f76a00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 0x7f0f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.436+0000 7f0f76a00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 0x7f0f5406e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f68034000 tx=0x7f0f6802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.439+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f60059350 con 0x7f0f78145010 2024-09-10T07:53:14.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.658+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f780cdbd0 con 0x7f0f78145010 2024-09-10T07:53:14.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.659+0000 7f0f6ea00640 1 -- 172.21.15.3:0/2507857354 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5222 (secure 0 0 0) 0x7f0f6005d650 con 0x7f0f78145010 2024-09-10T07:53:14.658 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:14.659 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":103,"btime":"2024-09-10T07:53:14:241064+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:13.662468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[7543]},"id":7}]} 2024-09-10T07:53:14.659 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 103 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 msgr2=0x7f0f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 0x7f0f5406e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f68034000 tx=0x7f0f6802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 msgr2=0x7f0f780b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9b0 tx=0x7f0f6000ee70 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 shutdown_connections 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f0f5406be30 0x7f0f5406e2f0 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f68034000 tx=0x7f0f6802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f7815ca80 0x7f0f780b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f78145010 0x7f0f780b44c0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 --2- 172.21.15.3:0/2507857354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0f78073950 0x7f0f781f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 >> 172.21.15.3:0/2507857354 conn(0x7f0f7806bb20 msgr2=0x7f0f780b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 shutdown_connections 2024-09-10T07:53:14.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.661+0000 7f0f77e00640 1 -- 172.21.15.3:0/2507857354 wait complete. 2024-09-10T07:53:14.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:53:14.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.833+0000 7f6e32400640 1 Processor -- start 2024-09-10T07:53:14.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.833+0000 7f6e32400640 1 -- start start 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.834+0000 7f6e32400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 0x7f6e2c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.834+0000 7f6e32400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0b5d80 con 0x7f6e2c0d0fd0 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e32400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0b6060 con 0x7f6e2c162c80 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e32400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c1630e0 con 0x7f6e2c0737f0 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6e2c0d0fd0 0x7f6e2c0b52c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56680/0 (socket says 172.21.15.3:56680) 2024-09-10T07:53:14.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2b400640 1 -- 172.21.15.3:0/1592488858 learned_addr learned my addr 172.21.15.3:0/1592488858 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2be00640 1 --2- 172.21.15.3:0/1592488858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 0x7f6e2c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428980703 0 0) 0x7f6e2c0b5d80 con 0x7f6e2c0d0fd0 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e10003520 con 0x7f6e2c0d0fd0 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965900415 0 0) 0x7f6e2c0b6060 con 0x7f6e2c162c80 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e2c0b5d80 con 0x7f6e2c162c80 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2be00640 1 -- 172.21.15.3:0/1592488858 >> v1:172.21.15.70:6789/0 conn(0x7f6e2c162c80 legacy=0x7f6e2c0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2be00640 1 -- 172.21.15.3:0/1592488858 >> v1:172.21.15.3:6789/0 conn(0x7f6e2c0d0fd0 legacy=0x7f6e2c0b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.835+0000 7f6e2be00640 1 -- 172.21.15.3:0/1592488858 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1621a0 con 0x7f6e2c0737f0 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e2be00640 1 --2- 172.21.15.3:0/1592488858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 0x7f6e2c154e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009dc0 tx=0x7f6e2402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de05de632e8dae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24007740 con 0x7f6e2c0737f0 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e240078e0 con 0x7f6e2c0737f0 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e2aa00640 1 -- 172.21.15.3:0/1592488858 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24007cb0 con 0x7f6e2c0737f0 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e32400640 1 -- 172.21.15.3:0/1592488858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 msgr2=0x7f6e2c154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.837+0000 7f6e32400640 1 --2- 172.21.15.3:0/1592488858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 0x7f6e2c154e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009dc0 tx=0x7f6e2402a6d0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.838+0000 7f6e32400640 1 -- 172.21.15.3:0/1592488858 shutdown_connections 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.838+0000 7f6e32400640 1 --2- 172.21.15.3:0/1592488858 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0737f0 0x7f6e2c154e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.838+0000 7f6e32400640 1 -- 172.21.15.3:0/1592488858 >> 172.21.15.3:0/1592488858 conn(0x7f6e2c06bb20 msgr2=0x7f6e2c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.838+0000 7f6e32400640 1 -- 172.21.15.3:0/1592488858 shutdown_connections 2024-09-10T07:53:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.838+0000 7f6e32400640 1 -- 172.21.15.3:0/1592488858 wait complete. 2024-09-10T07:53:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.839+0000 7f6e32400640 1 Processor -- start 2024-09-10T07:53:14.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.839+0000 7f6e32400640 1 -- start start 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 0x7f6e2c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0d0fd0 0x7f6e2c1e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c0d13b0 con 0x7f6e2c162c80 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c162760 con 0x7f6e2c0737f0 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.840+0000 7f6e32400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c159270 con 0x7f6e2c0d0fd0 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35864/0 (socket says 172.21.15.3:35864) 2024-09-10T07:53:14.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 -- 172.21.15.3:0/707449785 learned_addr learned my addr 172.21.15.3:0/707449785 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2b400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 0x7f6e2c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e23e00640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0d0fd0 0x7f6e2c1e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 -- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0d0fd0 msgr2=0x7f6e2c1e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0d0fd0 0x7f6e2c1e89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 -- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 msgr2=0x7f6e2c1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 0x7f6e2c1567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 -- 172.21.15.3:0/707449785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1e9050 con 0x7f6e2c162c80 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2b400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 0x7f6e2c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:14.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.841+0000 7f6e2be00640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ee0 tx=0x7f6e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.842+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2403f070 con 0x7f6e2c162c80 2024-09-10T07:53:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.842+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e24031bc0 con 0x7f6e2c162c80 2024-09-10T07:53:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.842+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24009450 con 0x7f6e2c162c80 2024-09-10T07:53:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.842+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1e9330 con 0x7f6e2c162c80 2024-09-10T07:53:14.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.842+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1e98f0 con 0x7f6e2c162c80 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.843+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c0b5a60 con 0x7f6e2c162c80 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.844+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e2403b410 con 0x7f6e2c162c80 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.845+0000 7f6e23400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 0x7f6e0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.845+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f6e240ac440 con 0x7f6e2c162c80 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.845+0000 7f6e2b400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 0x7f6e0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.846+0000 7f6e2b400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 0x7f6e0806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c007d70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:14.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:14.849+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e2407a700 con 0x7f6e2c162c80 2024-09-10T07:53:15.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.032+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e2c09ee90 con 0x7f6e2c162c80 2024-09-10T07:53:15.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.034+0000 7f6e23400640 1 -- 172.21.15.3:0/707449785 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16607 (secure 0 0 0) 0x7f6e2407ea00 con 0x7f6e2c162c80 2024-09-10T07:53:15.033 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:15.034 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":102,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:12.227124+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:53:08.843582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T07:53:09.632431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 msgr2=0x7f6e0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 0x7f6e0806e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f60 tx=0x7f6e1c007d70 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 msgr2=0x7f6e2c15f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ee0 tx=0x7f6e24004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 shutdown_connections 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6e0806be30 0x7f6e0806e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e2c162c80 0x7f6e2c15f3a0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6e2c0d0fd0 0x7f6e2c1e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 --2- 172.21.15.3:0/707449785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e2c0737f0 0x7f6e2c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 >> 172.21.15.3:0/707449785 conn(0x7f6e2c06bb20 msgr2=0x7f6e2c1577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:15.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 shutdown_connections 2024-09-10T07:53:15.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.036+0000 7f6e32400640 1 -- 172.21.15.3:0/707449785 wait complete. 2024-09-10T07:53:15.053 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:53:15.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:53:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.211+0000 7f2a81c00640 1 Processor -- start 2024-09-10T07:53:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.211+0000 7f2a81c00640 1 -- start start 2024-09-10T07:53:15.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a81c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a7c073660 0x7f2a7c175ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a81c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c06f4f0 con 0x7f2a7c16b5d0 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a81c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0b4020 con 0x7f2a7c0cfe70 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a81c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7c08cc80 con 0x7f2a7c073660 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a81200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a7c073660 0x7f2a7c175ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a7be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2a7c16b5d0 0x7f2a7c06e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56686/0 (socket says 172.21.15.3:56686) 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a7be00640 1 -- 172.21.15.3:0/2259851539 learned_addr learned my addr 172.21.15.3:0/2259851539 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.212+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159666431 0 0) 0x7f2a7c06f4f0 con 0x7f2a7c16b5d0 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.223+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a60003500 con 0x7f2a7c16b5d0 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.223+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630798496 0 0) 0x7f2a7c0b4020 con 0x7f2a7c0cfe70 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.223+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a7c06f4f0 con 0x7f2a7c0cfe70 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.223+0000 7f2a81200640 1 -- 172.21.15.3:0/2259851539 >> v1:172.21.15.70:6789/0 conn(0x7f2a7c0cfe70 legacy=0x7f2a7c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.224+0000 7f2a81200640 1 -- 172.21.15.3:0/2259851539 >> v1:172.21.15.3:6789/0 conn(0x7f2a7c16b5d0 legacy=0x7f2a7c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.224+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523704684 0 0) 0x7f2a60003500 con 0x7f2a7c16b5d0 2024-09-10T07:53:15.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.224+0000 7f2a81200640 1 -- 172.21.15.3:0/2259851539 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c0b4300 con 0x7f2a7c073660 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81200640 1 --2- 172.21.15.3:0/2259851539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c175ff0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007670 tx=0x7f2a7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6328ccf2b874d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c001670 con 0x7f2a7c073660 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a740092e0 con 0x7f2a7c073660 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a7b400640 1 -- 172.21.15.3:0/2259851539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74009490 con 0x7f2a7c073660 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81c00640 1 -- 172.21.15.3:0/2259851539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 msgr2=0x7f2a7c175ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81c00640 1 --2- 172.21.15.3:0/2259851539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c175ff0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007670 tx=0x7f2a7402a6a0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81c00640 1 -- 172.21.15.3:0/2259851539 shutdown_connections 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81c00640 1 --2- 172.21.15.3:0/2259851539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c175ff0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.225+0000 7f2a81c00640 1 -- 172.21.15.3:0/2259851539 >> 172.21.15.3:0/2259851539 conn(0x7f2a7c06d000 msgr2=0x7f2a7c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.226+0000 7f2a81c00640 1 -- 172.21.15.3:0/2259851539 shutdown_connections 2024-09-10T07:53:15.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.226+0000 7f2a81c00640 1 -- 172.21.15.3:0/2259851539 wait complete. 2024-09-10T07:53:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.227+0000 7f2a81c00640 1 Processor -- start 2024-09-10T07:53:15.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.227+0000 7f2a81c00640 1 -- start start 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a7c0cfe70 0x7f2a7c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 0x7f2a7c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c072130 con 0x7f2a7c073660 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c071e50 con 0x7f2a7c0cfe70 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a81c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0cfba0 con 0x7f2a7c16b5d0 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.228+0000 7f2a7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35882/0 (socket says 172.21.15.3:35882) 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 -- 172.21.15.3:0/1466695848 learned_addr learned my addr 172.21.15.3:0/1466695848 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a81200640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 0x7f2a7c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a73e00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a7c0cfe70 0x7f2a7c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 -- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 msgr2=0x7f2a7c157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 0x7f2a7c157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 -- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a7c0cfe70 msgr2=0x7f2a7c157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a7c0cfe70 0x7f2a7c157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 -- 172.21.15.3:0/1466695848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c158d30 con 0x7f2a7c073660 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a81200640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 0x7f2a7c157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a7be00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c001eb0 tx=0x7f2a6c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.229+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c017070 con 0x7f2a7c073660 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.230+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c005ad0 con 0x7f2a7c073660 2024-09-10T07:53:15.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.230+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c007070 con 0x7f2a7c073660 2024-09-10T07:53:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.230+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c15ad10 con 0x7f2a7c073660 2024-09-10T07:53:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.230+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c0aa8a0 con 0x7f2a7c073660 2024-09-10T07:53:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.231+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c063610 con 0x7f2a7c073660 2024-09-10T07:53:15.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.232+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a6c01a070 con 0x7f2a7c073660 2024-09-10T07:53:15.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.232+0000 7f2a78c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.232+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2a6c08a3e0 con 0x7f2a7c073660 2024-09-10T07:53:15.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.233+0000 7f2a73e00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.233+0000 7f2a73e00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 0x7f2a5806e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c168580 tx=0x7f2a64009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.237+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a6c058d80 con 0x7f2a7c073660 2024-09-10T07:53:15.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.417+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a7c15aff0 con 0x7f2a5806be30 2024-09-10T07:53:15.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.424+0000 7f2a78c00640 1 -- 172.21.15.3:0/1466695848 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250124 (secure 0 0 0) 0x7f2a7c15aff0 con 0x7f2a5806be30 2024-09-10T07:53:15.427 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 msgr2=0x7f2a5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 0x7f2a5806e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c168580 tx=0x7f2a64009040 comp rx=0 tx=0).stop 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 msgr2=0x7f2a7c0ab0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c001eb0 tx=0x7f2a6c00cbe0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 shutdown_connections 2024-09-10T07:53:15.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2a7c16b5d0 0x7f2a7c157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a7c0cfe70 0x7f2a7c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 --2- 172.21.15.3:0/1466695848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a7c073660 0x7f2a7c0ab0a0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 >> 172.21.15.3:0/1466695848 conn(0x7f2a7c06d000 msgr2=0x7f2a7c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 shutdown_connections 2024-09-10T07:53:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.428+0000 7f2a81c00640 1 -- 172.21.15.3:0/1466695848 wait complete. 2024-09-10T07:53:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-10T07:53:13.580059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264436,"kb_used_data":10740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710284,"statfs":{"total":773094113280,"available":772823330816,"internally_reserved":0,"allocated":10997760,"data_stored":5410639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.380148"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214073+0000","last_change":"2024-09-10T07:53:10.214073+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214073+0000","last_undegraded":"2024-09-10T07:53:10.214073+0000","last_fullsized":"2024-09-10T07:53:10.214073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215296+0000","last_change":"2024-09-10T07:53:10.215296+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215296+0000","last_undegraded":"2024-09-10T07:53:10.215296+0000","last_fullsized":"2024-09-10T07:53:10.215296+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233484+0000","last_change":"2024-09-10T07:53:11.252106+0000","last_active":"2024-09-10T07:53:12.233484+0000","last_peered":"2024-09-10T07:53:12.233484+0000","last_clean":"2024-09-10T07:53:12.233484+0000","last_became_active":"2024-09-10T07:53:11.251807+0000","last_became_peered":"2024-09-10T07:53:11.251807+0000","last_unstale":"2024-09-10T07:53:12.233484+0000","last_undegraded":"2024-09-10T07:53:12.233484+0000","last_fullsized":"2024-09-10T07:53:12.233484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:01.780830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215936+0000","last_change":"2024-09-10T07:53:10.215936+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215936+0000","last_undegraded":"2024-09-10T07:53:10.215936+0000","last_fullsized":"2024-09-10T07:53:10.215936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.237820+0000","last_change":"2024-09-10T07:53:11.234549+0000","last_active":"2024-09-10T07:53:11.237820+0000","last_peered":"2024-09-10T07:53:11.237820+0000","last_clean":"2024-09-10T07:53:11.237820+0000","last_became_active":"2024-09-10T07:53:11.234201+0000","last_became_peered":"2024-09-10T07:53:11.234201+0000","last_unstale":"2024-09-10T07:53:11.237820+0000","last_undegraded":"2024-09-10T07:53:11.237820+0000","last_fullsized":"2024-09-10T07:53:11.237820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:49.298649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.243698+0000","last_change":"2024-09-10T07:53:11.236870+0000","last_active":"2024-09-10T07:53:11.243698+0000","last_peered":"2024-09-10T07:53:11.243698+0000","last_clean":"2024-09-10T07:53:11.243698+0000","last_became_active":"2024-09-10T07:53:11.235843+0000","last_became_peered":"2024-09-10T07:53:11.235843+0000","last_unstale":"2024-09-10T07:53:11.243698+0000","last_undegraded":"2024-09-10T07:53:11.243698+0000","last_fullsized":"2024-09-10T07:53:11.243698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.860697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261073+0000","last_change":"2024-09-10T07:53:11.255394+0000","last_active":"2024-09-10T07:53:11.261073+0000","last_peered":"2024-09-10T07:53:11.261073+0000","last_clean":"2024-09-10T07:53:11.261073+0000","last_became_active":"2024-09-10T07:53:11.254136+0000","last_became_peered":"2024-09-10T07:53:11.254136+0000","last_unstale":"2024-09-10T07:53:11.261073+0000","last_undegraded":"2024-09-10T07:53:11.261073+0000","last_fullsized":"2024-09-10T07:53:11.261073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:35.472507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233314+0000","last_change":"2024-09-10T07:53:11.250032+0000","last_active":"2024-09-10T07:53:12.233314+0000","last_peered":"2024-09-10T07:53:12.233314+0000","last_clean":"2024-09-10T07:53:12.233314+0000","last_became_active":"2024-09-10T07:53:11.249626+0000","last_became_peered":"2024-09-10T07:53:11.249626+0000","last_unstale":"2024-09-10T07:53:12.233314+0000","last_undegraded":"2024-09-10T07:53:12.233314+0000","last_fullsized":"2024-09-10T07:53:12.233314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.283338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248152+0000","last_change":"2024-09-10T07:53:11.243615+0000","last_active":"2024-09-10T07:53:11.248152+0000","last_peered":"2024-09-10T07:53:11.248152+0000","last_clean":"2024-09-10T07:53:11.248152+0000","last_became_active":"2024-09-10T07:53:11.243351+0000","last_became_peered":"2024-09-10T07:53:11.243351+0000","last_unstale":"2024-09-10T07:53:11.248152+0000","last_undegraded":"2024-09-10T07:53:11.248152+0000","last_fullsized":"2024-09-10T07:53:11.248152+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:53.710317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268481+0000","last_change":"2024-09-10T07:53:11.267004+0000","last_active":"2024-09-10T07:53:11.268481+0000","last_peered":"2024-09-10T07:53:11.268481+0000","last_clean":"2024-09-10T07:53:11.268481+0000","last_became_active":"2024-09-10T07:53:11.266699+0000","last_became_peered":"2024-09-10T07:53:11.266699+0000","last_unstale":"2024-09-10T07:53:11.268481+0000","last_undegraded":"2024-09-10T07:53:11.268481+0000","last_fullsized":"2024-09-10T07:53:11.268481+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:08.715930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239275+0000","last_change":"2024-09-10T07:53:11.236475+0000","last_active":"2024-09-10T07:53:11.239275+0000","last_peered":"2024-09-10T07:53:11.239275+0000","last_clean":"2024-09-10T07:53:11.239275+0000","last_became_active":"2024-09-10T07:53:11.235672+0000","last_became_peered":"2024-09-10T07:53:11.235672+0000","last_unstale":"2024-09-10T07:53:11.239275+0000","last_undegraded":"2024-09-10T07:53:11.239275+0000","last_fullsized":"2024-09-10T07:53:11.239275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:31:33.473232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269070+0000","last_change":"2024-09-10T07:53:11.267139+0000","last_active":"2024-09-10T07:53:11.269070+0000","last_peered":"2024-09-10T07:53:11.269070+0000","last_clean":"2024-09-10T07:53:11.269070+0000","last_became_active":"2024-09-10T07:53:11.266829+0000","last_became_peered":"2024-09-10T07:53:11.266829+0000","last_unstale":"2024-09-10T07:53:11.269070+0000","last_undegraded":"2024-09-10T07:53:11.269070+0000","last_fullsized":"2024-09-10T07:53:11.269070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.218726+0000","last_change":"2024-09-10T07:53:10.218726+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.218726+0000","last_undegraded":"2024-09-10T07:53:10.218726+0000","last_fullsized":"2024-09-10T07:53:10.218726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212879+0000","last_change":"2024-09-10T07:53:10.212879+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212879+0000","last_undegraded":"2024-09-10T07:53:10.212879+0000","last_fullsized":"2024-09-10T07:53:10.212879+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211714+0000","last_change":"2024-09-10T07:53:10.211714+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211714+0000","last_undegraded":"2024-09-10T07:53:10.211714+0000","last_fullsized":"2024-09-10T07:53:10.211714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260661+0000","last_change":"2024-09-10T07:53:11.254673+0000","last_active":"2024-09-10T07:53:11.260661+0000","last_peered":"2024-09-10T07:53:11.260661+0000","last_clean":"2024-09-10T07:53:11.260661+0000","last_became_active":"2024-09-10T07:53:11.251614+0000","last_became_peered":"2024-09-10T07:53:11.251614+0000","last_unstale":"2024-09-10T07:53:11.260661+0000","last_undegraded":"2024-09-10T07:53:11.260661+0000","last_fullsized":"2024-09-10T07:53:11.260661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:50.934355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212721+0000","last_change":"2024-09-10T07:53:10.212721+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212721+0000","last_undegraded":"2024-09-10T07:53:10.212721+0000","last_fullsized":"2024-09-10T07:53:10.212721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214734+0000","last_change":"2024-09-10T07:53:10.214734+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214734+0000","last_undegraded":"2024-09-10T07:53:10.214734+0000","last_fullsized":"2024-09-10T07:53:10.214734+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214070+0000","last_change":"2024-09-10T07:53:10.214070+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214070+0000","last_undegraded":"2024-09-10T07:53:10.214070+0000","last_fullsized":"2024-09-10T07:53:10.214070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239700+0000","last_change":"2024-09-10T07:53:11.234958+0000","last_active":"2024-09-10T07:53:11.239700+0000","last_peered":"2024-09-10T07:53:11.239700+0000","last_clean":"2024-09-10T07:53:11.239700+0000","last_became_active":"2024-09-10T07:53:11.234678+0000","last_became_peered":"2024-09-10T07:53:11.234678+0000","last_unstale":"2024-09-10T07:53:11.239700+0000","last_undegraded":"2024-09-10T07:53:11.239700+0000","last_fullsized":"2024-09-10T07:53:11.239700+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:47.186154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233477+0000","last_change":"2024-09-10T07:53:11.248147+0000","last_active":"2024-09-10T07:53:12.233477+0000","last_peered":"2024-09-10T07:53:12.233477+0000","last_clean":"2024-09-10T07:53:12.233477+0000","last_became_active":"2024-09-10T07:53:11.247829+0000","last_became_peered":"2024-09-10T07:53:11.247829+0000","last_unstale":"2024-09-10T07:53:12.233477+0000","last_undegraded":"2024-09-10T07:53:12.233477+0000","last_fullsized":"2024-09-10T07:53:12.233477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:54.042192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232852+0000","last_change":"2024-09-10T07:53:11.249206+0000","last_active":"2024-09-10T07:53:12.232852+0000","last_peered":"2024-09-10T07:53:12.232852+0000","last_clean":"2024-09-10T07:53:12.232852+0000","last_became_active":"2024-09-10T07:53:11.248315+0000","last_became_peered":"2024-09-10T07:53:11.248315+0000","last_unstale":"2024-09-10T07:53:12.232852+0000","last_undegraded":"2024-09-10T07:53:12.232852+0000","last_fullsized":"2024-09-10T07:53:12.232852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:31.145698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211200+0000","last_change":"2024-09-10T07:53:10.211200+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211200+0000","last_undegraded":"2024-09-10T07:53:10.211200+0000","last_fullsized":"2024-09-10T07:53:10.211200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.263806+0000","last_change":"2024-09-10T07:53:11.262775+0000","last_active":"2024-09-10T07:53:11.263806+0000","last_peered":"2024-09-10T07:53:11.263806+0000","last_clean":"2024-09-10T07:53:11.263806+0000","last_became_active":"2024-09-10T07:53:11.262438+0000","last_became_peered":"2024-09-10T07:53:11.262438+0000","last_unstale":"2024-09-10T07:53:11.263806+0000","last_undegraded":"2024-09-10T07:53:11.263806+0000","last_fullsized":"2024-09-10T07:53:11.263806+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:12.859790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.257372+0000","last_change":"2024-09-10T07:53:11.255844+0000","last_active":"2024-09-10T07:53:11.257372+0000","last_peered":"2024-09-10T07:53:11.257372+0000","last_clean":"2024-09-10T07:53:11.257372+0000","last_became_active":"2024-09-10T07:53:11.255548+0000","last_became_peered":"2024-09-10T07:53:11.255548+0000","last_unstale":"2024-09-10T07:53:11.257372+0000","last_undegraded":"2024-09-10T07:53:11.257372+0000","last_fullsized":"2024-09-10T07:53:11.257372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:06.407725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214175+0000","last_change":"2024-09-10T07:53:10.214175+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214175+0000","last_undegraded":"2024-09-10T07:53:10.214175+0000","last_fullsized":"2024-09-10T07:53:10.214175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248744+0000","last_change":"2024-09-10T07:53:11.246590+0000","last_active":"2024-09-10T07:53:11.248744+0000","last_peered":"2024-09-10T07:53:11.248744+0000","last_clean":"2024-09-10T07:53:11.248744+0000","last_became_active":"2024-09-10T07:53:11.245943+0000","last_became_peered":"2024-09-10T07:53:11.245943+0000","last_unstale":"2024-09-10T07:53:11.248744+0000","last_undegraded":"2024-09-10T07:53:11.248744+0000","last_fullsized":"2024-09-10T07:53:11.248744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258576+0000","last_change":"2024-09-10T07:53:11.256724+0000","last_active":"2024-09-10T07:53:11.258576+0000","last_peered":"2024-09-10T07:53:11.258576+0000","last_clean":"2024-09-10T07:53:11.258576+0000","last_became_active":"2024-09-10T07:53:11.256427+0000","last_became_peered":"2024-09-10T07:53:11.256427+0000","last_unstale":"2024-09-10T07:53:11.258576+0000","last_undegraded":"2024-09-10T07:53:11.258576+0000","last_fullsized":"2024-09-10T07:53:11.258576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:19.541855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214213+0000","last_change":"2024-09-10T07:53:10.214213+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214213+0000","last_undegraded":"2024-09-10T07:53:10.214213+0000","last_fullsized":"2024-09-10T07:53:10.214213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.224154+0000","last_change":"2024-09-10T07:53:10.224154+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.224154+0000","last_undegraded":"2024-09-10T07:53:10.224154+0000","last_fullsized":"2024-09-10T07:53:10.224154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234095+0000","last_change":"2024-09-10T07:53:11.256491+0000","last_active":"2024-09-10T07:53:12.234095+0000","last_peered":"2024-09-10T07:53:12.234095+0000","last_clean":"2024-09-10T07:53:12.234095+0000","last_became_active":"2024-09-10T07:53:11.255961+0000","last_became_peered":"2024-09-10T07:53:11.255961+0000","last_unstale":"2024-09-10T07:53:12.234095+0000","last_undegraded":"2024-09-10T07:53:12.234095+0000","last_fullsized":"2024-09-10T07:53:12.234095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:11.297862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214870+0000","last_change":"2024-09-10T07:53:10.214870+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214870+0000","last_undegraded":"2024-09-10T07:53:10.214870+0000","last_fullsized":"2024-09-10T07:53:10.214870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217871+0000","last_change":"2024-09-10T07:53:10.217871+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217871+0000","last_undegraded":"2024-09-10T07:53:10.217871+0000","last_fullsized":"2024-09-10T07:53:10.217871+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236893+0000","last_change":"2024-09-10T07:53:10.235925+0000","last_active":"2024-09-10T07:53:10.236893+0000","last_peered":"2024-09-10T07:53:10.236893+0000","last_clean":"2024-09-10T07:53:10.236893+0000","last_became_active":"2024-09-10T07:53:10.235350+0000","last_became_peered":"2024-09-10T07:53:10.235350+0000","last_unstale":"2024-09-10T07:53:10.236893+0000","last_undegraded":"2024-09-10T07:53:10.236893+0000","last_fullsized":"2024-09-10T07:53:10.236893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215028+0000","last_change":"2024-09-10T07:53:10.215028+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215028+0000","last_undegraded":"2024-09-10T07:53:10.215028+0000","last_fullsized":"2024-09-10T07:53:10.215028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267919+0000","last_change":"2024-09-10T07:53:10.232379+0000","last_active":"2024-09-10T07:53:11.267919+0000","last_peered":"2024-09-10T07:53:11.267919+0000","last_clean":"2024-09-10T07:53:11.267919+0000","last_became_active":"2024-09-10T07:53:10.231931+0000","last_became_peered":"2024-09-10T07:53:10.231931+0000","last_unstale":"2024-09-10T07:53:11.267919+0000","last_undegraded":"2024-09-10T07:53:11.267919+0000","last_fullsized":"2024-09-10T07:53:11.267919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:22:58.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215453+0000","last_change":"2024-09-10T07:53:10.215453+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215453+0000","last_undegraded":"2024-09-10T07:53:10.215453+0000","last_fullsized":"2024-09-10T07:53:10.215453+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233603+0000","last_change":"2024-09-10T07:53:10.227323+0000","last_active":"2024-09-10T07:53:12.233603+0000","last_peered":"2024-09-10T07:53:12.233603+0000","last_clean":"2024-09-10T07:53:12.233603+0000","last_became_active":"2024-09-10T07:53:10.227012+0000","last_became_peered":"2024-09-10T07:53:10.227012+0000","last_unstale":"2024-09-10T07:53:12.233603+0000","last_undegraded":"2024-09-10T07:53:12.233603+0000","last_fullsized":"2024-09-10T07:53:12.233603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:56.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.249667+0000","last_change":"2024-09-10T07:53:11.246551+0000","last_active":"2024-09-10T07:53:11.249667+0000","last_peered":"2024-09-10T07:53:11.249667+0000","last_clean":"2024-09-10T07:53:11.249667+0000","last_became_active":"2024-09-10T07:53:11.246300+0000","last_became_peered":"2024-09-10T07:53:11.246300+0000","last_unstale":"2024-09-10T07:53:11.249667+0000","last_undegraded":"2024-09-10T07:53:11.249667+0000","last_fullsized":"2024-09-10T07:53:11.249667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234150+0000","last_change":"2024-09-10T07:53:10.231193+0000","last_active":"2024-09-10T07:53:10.234150+0000","last_peered":"2024-09-10T07:53:10.234150+0000","last_clean":"2024-09-10T07:53:10.234150+0000","last_became_active":"2024-09-10T07:53:10.226288+0000","last_became_peered":"2024-09-10T07:53:10.226288+0000","last_unstale":"2024-09-10T07:53:10.234150+0000","last_undegraded":"2024-09-10T07:53:10.234150+0000","last_fullsized":"2024-09-10T07:53:10.234150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:31.899936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.241212+0000","last_change":"2024-09-10T07:53:11.235832+0000","last_active":"2024-09-10T07:53:11.241212+0000","last_peered":"2024-09-10T07:53:11.241212+0000","last_clean":"2024-09-10T07:53:11.241212+0000","last_became_active":"2024-09-10T07:53:11.235506+0000","last_became_peered":"2024-09-10T07:53:11.235506+0000","last_unstale":"2024-09-10T07:53:11.241212+0000","last_undegraded":"2024-09-10T07:53:11.241212+0000","last_fullsized":"2024-09-10T07:53:11.241212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:05.491743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233697+0000","last_change":"2024-09-10T07:53:10.228445+0000","last_active":"2024-09-10T07:53:10.233697+0000","last_peered":"2024-09-10T07:53:10.233697+0000","last_clean":"2024-09-10T07:53:10.233697+0000","last_became_active":"2024-09-10T07:53:10.227806+0000","last_became_peered":"2024-09-10T07:53:10.227806+0000","last_unstale":"2024-09-10T07:53:10.233697+0000","last_undegraded":"2024-09-10T07:53:10.233697+0000","last_fullsized":"2024-09-10T07:53:10.233697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:25:41.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.238929+0000","last_change":"2024-09-10T07:53:11.235298+0000","last_active":"2024-09-10T07:53:11.238929+0000","last_peered":"2024-09-10T07:53:11.238929+0000","last_clean":"2024-09-10T07:53:11.238929+0000","last_became_active":"2024-09-10T07:53:11.234953+0000","last_became_peered":"2024-09-10T07:53:11.234953+0000","last_unstale":"2024-09-10T07:53:11.238929+0000","last_undegraded":"2024-09-10T07:53:11.238929+0000","last_fullsized":"2024-09-10T07:53:11.238929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:14.411009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232971+0000","last_change":"2024-09-10T07:53:10.229226+0000","last_active":"2024-09-10T07:53:12.232971+0000","last_peered":"2024-09-10T07:53:12.232971+0000","last_clean":"2024-09-10T07:53:12.232971+0000","last_became_active":"2024-09-10T07:53:10.228904+0000","last_became_peered":"2024-09-10T07:53:10.228904+0000","last_unstale":"2024-09-10T07:53:12.232971+0000","last_undegraded":"2024-09-10T07:53:12.232971+0000","last_fullsized":"2024-09-10T07:53:12.232971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:29.688893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214033+0000","last_change":"2024-09-10T07:53:10.214033+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214033+0000","last_undegraded":"2024-09-10T07:53:10.214033+0000","last_fullsized":"2024-09-10T07:53:10.214033+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.229770+0000","last_change":"2024-09-10T07:53:10.228005+0000","last_active":"2024-09-10T07:53:11.229770+0000","last_peered":"2024-09-10T07:53:11.229770+0000","last_clean":"2024-09-10T07:53:11.229770+0000","last_became_active":"2024-09-10T07:53:10.227661+0000","last_became_peered":"2024-09-10T07:53:10.227661+0000","last_unstale":"2024-09-10T07:53:11.229770+0000","last_undegraded":"2024-09-10T07:53:11.229770+0000","last_fullsized":"2024-09-10T07:53:11.229770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:45.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.240934+0000","last_change":"2024-09-10T07:53:11.235959+0000","last_active":"2024-09-10T07:53:11.240934+0000","last_peered":"2024-09-10T07:53:11.240934+0000","last_clean":"2024-09-10T07:53:11.240934+0000","last_became_active":"2024-09-10T07:53:11.235687+0000","last_became_peered":"2024-09-10T07:53:11.235687+0000","last_unstale":"2024-09-10T07:53:11.240934+0000","last_undegraded":"2024-09-10T07:53:11.240934+0000","last_fullsized":"2024-09-10T07:53:11.240934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:00.667191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235952+0000","last_change":"2024-09-10T07:53:10.233196+0000","last_active":"2024-09-10T07:53:10.235952+0000","last_peered":"2024-09-10T07:53:10.235952+0000","last_clean":"2024-09-10T07:53:10.235952+0000","last_became_active":"2024-09-10T07:53:10.232510+0000","last_became_peered":"2024-09-10T07:53:10.232510+0000","last_unstale":"2024-09-10T07:53:10.235952+0000","last_undegraded":"2024-09-10T07:53:10.235952+0000","last_fullsized":"2024-09-10T07:53:10.235952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:11.857423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.213811+0000","last_change":"2024-09-10T07:53:10.213811+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.213811+0000","last_undegraded":"2024-09-10T07:53:10.213811+0000","last_fullsized":"2024-09-10T07:53:10.213811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234051+0000","last_change":"2024-09-10T07:53:10.213607+0000","last_active":"2024-09-10T07:53:12.234051+0000","last_peered":"2024-09-10T07:53:12.234051+0000","last_clean":"2024-09-10T07:53:12.234051+0000","last_became_active":"2024-09-10T07:53:10.213317+0000","last_became_peered":"2024-09-10T07:53:10.213317+0000","last_unstale":"2024-09-10T07:53:12.234051+0000","last_undegraded":"2024-09-10T07:53:12.234051+0000","last_fullsized":"2024-09-10T07:53:12.234051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:19.086909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.251040+0000","last_change":"2024-09-10T07:53:11.250003+0000","last_active":"2024-09-10T07:53:11.251040+0000","last_peered":"2024-09-10T07:53:11.251040+0000","last_clean":"2024-09-10T07:53:11.251040+0000","last_became_active":"2024-09-10T07:53:11.249735+0000","last_became_peered":"2024-09-10T07:53:11.249735+0000","last_unstale":"2024-09-10T07:53:11.251040+0000","last_undegraded":"2024-09-10T07:53:11.251040+0000","last_fullsized":"2024-09-10T07:53:11.251040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:53.440934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233009+0000","last_change":"2024-09-10T07:53:10.226296+0000","last_active":"2024-09-10T07:53:10.233009+0000","last_peered":"2024-09-10T07:53:10.233009+0000","last_clean":"2024-09-10T07:53:10.233009+0000","last_became_active":"2024-09-10T07:53:10.225937+0000","last_became_peered":"2024-09-10T07:53:10.225937+0000","last_unstale":"2024-09-10T07:53:10.233009+0000","last_undegraded":"2024-09-10T07:53:10.233009+0000","last_fullsized":"2024-09-10T07:53:10.233009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269185+0000","last_change":"2024-09-10T07:53:11.266667+0000","last_active":"2024-09-10T07:53:11.269185+0000","last_peered":"2024-09-10T07:53:11.269185+0000","last_clean":"2024-09-10T07:53:11.269185+0000","last_became_active":"2024-09-10T07:53:11.266400+0000","last_became_peered":"2024-09-10T07:53:11.266400+0000","last_unstale":"2024-09-10T07:53:11.269185+0000","last_undegraded":"2024-09-10T07:53:11.269185+0000","last_fullsized":"2024-09-10T07:53:11.269185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:39.890020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269567+0000","last_change":"2024-09-10T07:53:10.223127+0000","last_active":"2024-09-10T07:53:11.269567+0000","last_peered":"2024-09-10T07:53:11.269567+0000","last_clean":"2024-09-10T07:53:11.269567+0000","last_became_active":"2024-09-10T07:53:10.222729+0000","last_became_peered":"2024-09-10T07:53:10.222729+0000","last_unstale":"2024-09-10T07:53:11.269567+0000","last_undegraded":"2024-09-10T07:53:11.269567+0000","last_fullsized":"2024-09-10T07:53:11.269567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.995404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261874+0000","last_change":"2024-09-10T07:53:11.260871+0000","last_active":"2024-09-10T07:53:11.261874+0000","last_peered":"2024-09-10T07:53:11.261874+0000","last_clean":"2024-09-10T07:53:11.261874+0000","last_became_active":"2024-09-10T07:53:11.260613+0000","last_became_peered":"2024-09-10T07:53:11.260613+0000","last_unstale":"2024-09-10T07:53:11.261874+0000","last_undegraded":"2024-09-10T07:53:11.261874+0000","last_fullsized":"2024-09-10T07:53:11.261874+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:15.810230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.277289+0000","last_change":"2024-09-10T07:53:10.235054+0000","last_active":"2024-09-10T07:53:11.277289+0000","last_peered":"2024-09-10T07:53:11.277289+0000","last_clean":"2024-09-10T07:53:11.277289+0000","last_became_active":"2024-09-10T07:53:10.234764+0000","last_became_peered":"2024-09-10T07:53:10.234764+0000","last_unstale":"2024-09-10T07:53:11.277289+0000","last_undegraded":"2024-09-10T07:53:11.277289+0000","last_fullsized":"2024-09-10T07:53:11.277289+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212467+0000","last_change":"2024-09-10T07:53:10.212467+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212467+0000","last_undegraded":"2024-09-10T07:53:10.212467+0000","last_fullsized":"2024-09-10T07:53:10.212467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234989+0000","last_change":"2024-09-10T07:53:10.222685+0000","last_active":"2024-09-10T07:53:10.234989+0000","last_peered":"2024-09-10T07:53:10.234989+0000","last_clean":"2024-09-10T07:53:10.234989+0000","last_became_active":"2024-09-10T07:53:10.222344+0000","last_became_peered":"2024-09-10T07:53:10.222344+0000","last_unstale":"2024-09-10T07:53:10.234989+0000","last_undegraded":"2024-09-10T07:53:10.234989+0000","last_fullsized":"2024-09-10T07:53:10.234989+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:18.539489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233639+0000","last_change":"2024-09-10T07:53:11.257850+0000","last_active":"2024-09-10T07:53:12.233639+0000","last_peered":"2024-09-10T07:53:12.233639+0000","last_clean":"2024-09-10T07:53:12.233639+0000","last_became_active":"2024-09-10T07:53:11.255958+0000","last_became_peered":"2024-09-10T07:53:11.255958+0000","last_unstale":"2024-09-10T07:53:12.233639+0000","last_undegraded":"2024-09-10T07:53:12.233639+0000","last_fullsized":"2024-09-10T07:53:12.233639+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:05.576611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233362+0000","last_change":"2024-09-10T07:53:10.229874+0000","last_active":"2024-09-10T07:53:10.233362+0000","last_peered":"2024-09-10T07:53:10.233362+0000","last_clean":"2024-09-10T07:53:10.233362+0000","last_became_active":"2024-09-10T07:53:10.229597+0000","last_became_peered":"2024-09-10T07:53:10.229597+0000","last_unstale":"2024-09-10T07:53:10.233362+0000","last_undegraded":"2024-09-10T07:53:10.233362+0000","last_fullsized":"2024-09-10T07:53:10.233362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239396+0000","last_change":"2024-09-10T07:53:11.236857+0000","last_active":"2024-09-10T07:53:11.239396+0000","last_peered":"2024-09-10T07:53:11.239396+0000","last_clean":"2024-09-10T07:53:11.239396+0000","last_became_active":"2024-09-10T07:53:11.236085+0000","last_became_peered":"2024-09-10T07:53:11.236085+0000","last_unstale":"2024-09-10T07:53:11.239396+0000","last_undegraded":"2024-09-10T07:53:11.239396+0000","last_fullsized":"2024-09-10T07:53:11.239396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:25.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231957+0000","last_change":"2024-09-10T07:53:10.223641+0000","last_active":"2024-09-10T07:53:11.231957+0000","last_peered":"2024-09-10T07:53:11.231957+0000","last_clean":"2024-09-10T07:53:11.231957+0000","last_became_active":"2024-09-10T07:53:10.223373+0000","last_became_peered":"2024-09-10T07:53:10.223373+0000","last_unstale":"2024-09-10T07:53:11.231957+0000","last_undegraded":"2024-09-10T07:53:11.231957+0000","last_fullsized":"2024-09-10T07:53:11.231957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233338+0000","last_change":"2024-09-10T07:53:11.256226+0000","last_active":"2024-09-10T07:53:12.233338+0000","last_peered":"2024-09-10T07:53:12.233338+0000","last_clean":"2024-09-10T07:53:12.233338+0000","last_became_active":"2024-09-10T07:53:11.247430+0000","last_became_peered":"2024-09-10T07:53:11.247430+0000","last_unstale":"2024-09-10T07:53:12.233338+0000","last_undegraded":"2024-09-10T07:53:12.233338+0000","last_fullsized":"2024-09-10T07:53:12.233338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:22.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.232748+0000","last_change":"2024-09-10T07:53:10.223678+0000","last_active":"2024-09-10T07:53:10.232748+0000","last_peered":"2024-09-10T07:53:10.232748+0000","last_clean":"2024-09-10T07:53:10.232748+0000","last_became_active":"2024-09-10T07:53:10.223390+0000","last_became_peered":"2024-09-10T07:53:10.223390+0000","last_unstale":"2024-09-10T07:53:10.232748+0000","last_undegraded":"2024-09-10T07:53:10.232748+0000","last_fullsized":"2024-09-10T07:53:10.232748+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:35.321389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.252550+0000","last_change":"2024-09-10T07:53:11.251829+0000","last_active":"2024-09-10T07:53:11.252550+0000","last_peered":"2024-09-10T07:53:11.252550+0000","last_clean":"2024-09-10T07:53:11.252550+0000","last_became_active":"2024-09-10T07:53:11.251547+0000","last_became_peered":"2024-09-10T07:53:11.251547+0000","last_unstale":"2024-09-10T07:53:11.252550+0000","last_undegraded":"2024-09-10T07:53:11.252550+0000","last_fullsized":"2024-09-10T07:53:11.252550+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:43.617740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271292+0000","last_change":"2024-09-10T07:53:10.232882+0000","last_active":"2024-09-10T07:53:11.271292+0000","last_peered":"2024-09-10T07:53:11.271292+0000","last_clean":"2024-09-10T07:53:11.271292+0000","last_became_active":"2024-09-10T07:53:10.232094+0000","last_became_peered":"2024-09-10T07:53:10.232094+0000","last_unstale":"2024-09-10T07:53:11.271292+0000","last_undegraded":"2024-09-10T07:53:11.271292+0000","last_fullsized":"2024-09-10T07:53:11.271292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.250788+0000","last_change":"2024-09-10T07:53:11.248994+0000","last_active":"2024-09-10T07:53:11.250788+0000","last_peered":"2024-09-10T07:53:11.250788+0000","last_clean":"2024-09-10T07:53:11.250788+0000","last_became_active":"2024-09-10T07:53:11.248544+0000","last_became_peered":"2024-09-10T07:53:11.248544+0000","last_unstale":"2024-09-10T07:53:11.250788+0000","last_undegraded":"2024-09-10T07:53:11.250788+0000","last_fullsized":"2024-09-10T07:53:11.250788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:07.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268745+0000","last_change":"2024-09-10T07:53:10.213371+0000","last_active":"2024-09-10T07:53:11.268745+0000","last_peered":"2024-09-10T07:53:11.268745+0000","last_clean":"2024-09-10T07:53:11.268745+0000","last_became_active":"2024-09-10T07:53:10.211436+0000","last_became_peered":"2024-09-10T07:53:10.211436+0000","last_unstale":"2024-09-10T07:53:11.268745+0000","last_undegraded":"2024-09-10T07:53:11.268745+0000","last_fullsized":"2024-09-10T07:53:11.268745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:54.890133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239996+0000","last_change":"2024-09-10T07:53:11.235023+0000","last_active":"2024-09-10T07:53:11.239996+0000","last_peered":"2024-09-10T07:53:11.239996+0000","last_clean":"2024-09-10T07:53:11.239996+0000","last_became_active":"2024-09-10T07:53:11.234748+0000","last_became_peered":"2024-09-10T07:53:11.234748+0000","last_unstale":"2024-09-10T07:53:11.239996+0000","last_undegraded":"2024-09-10T07:53:11.239996+0000","last_fullsized":"2024-09-10T07:53:11.239996+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.136947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235891+0000","last_change":"2024-09-10T07:53:10.232440+0000","last_active":"2024-09-10T07:53:10.235891+0000","last_peered":"2024-09-10T07:53:10.235891+0000","last_clean":"2024-09-10T07:53:10.235891+0000","last_became_active":"2024-09-10T07:53:10.229715+0000","last_became_peered":"2024-09-10T07:53:10.229715+0000","last_unstale":"2024-09-10T07:53:10.235891+0000","last_undegraded":"2024-09-10T07:53:10.235891+0000","last_fullsized":"2024-09-10T07:53:10.235891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:05.012224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212060+0000","last_change":"2024-09-10T07:53:10.212060+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212060+0000","last_undegraded":"2024-09-10T07:53:10.212060+0000","last_fullsized":"2024-09-10T07:53:10.212060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233676+0000","last_change":"2024-09-10T07:53:10.227747+0000","last_active":"2024-09-10T07:53:10.233676+0000","last_peered":"2024-09-10T07:53:10.233676+0000","last_clean":"2024-09-10T07:53:10.233676+0000","last_became_active":"2024-09-10T07:53:10.225865+0000","last_became_peered":"2024-09-10T07:53:10.225865+0000","last_unstale":"2024-09-10T07:53:10.233676+0000","last_undegraded":"2024-09-10T07:53:10.233676+0000","last_fullsized":"2024-09-10T07:53:10.233676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211732+0000","last_change":"2024-09-10T07:53:10.211732+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211732+0000","last_undegraded":"2024-09-10T07:53:10.211732+0000","last_fullsized":"2024-09-10T07:53:10.211732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.247861+0000","last_change":"2024-09-10T07:53:10.228787+0000","last_active":"2024-09-10T07:53:11.247861+0000","last_peered":"2024-09-10T07:53:11.247861+0000","last_clean":"2024-09-10T07:53:11.247861+0000","last_became_active":"2024-09-10T07:53:10.228445+0000","last_became_peered":"2024-09-10T07:53:10.228445+0000","last_unstale":"2024-09-10T07:53:11.247861+0000","last_undegraded":"2024-09-10T07:53:11.247861+0000","last_fullsized":"2024-09-10T07:53:11.247861+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:55.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211105+0000","last_change":"2024-09-10T07:53:10.211105+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211105+0000","last_undegraded":"2024-09-10T07:53:10.211105+0000","last_fullsized":"2024-09-10T07:53:10.211105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.245589+0000","last_change":"2024-09-10T07:53:10.228240+0000","last_active":"2024-09-10T07:53:11.245589+0000","last_peered":"2024-09-10T07:53:11.245589+0000","last_clean":"2024-09-10T07:53:11.245589+0000","last_became_active":"2024-09-10T07:53:10.227508+0000","last_became_peered":"2024-09-10T07:53:10.227508+0000","last_unstale":"2024-09-10T07:53:11.245589+0000","last_undegraded":"2024-09-10T07:53:11.245589+0000","last_fullsized":"2024-09-10T07:53:11.245589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:39.308304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261353+0000","last_change":"2024-09-10T07:53:11.259531+0000","last_active":"2024-09-10T07:53:11.261353+0000","last_peered":"2024-09-10T07:53:11.261353+0000","last_clean":"2024-09-10T07:53:11.261353+0000","last_became_active":"2024-09-10T07:53:11.258000+0000","last_became_peered":"2024-09-10T07:53:11.258000+0000","last_unstale":"2024-09-10T07:53:11.261353+0000","last_undegraded":"2024-09-10T07:53:11.261353+0000","last_fullsized":"2024-09-10T07:53:11.261353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:06.014591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232682+0000","last_change":"2024-09-10T07:53:10.231034+0000","last_active":"2024-09-10T07:53:12.232682+0000","last_peered":"2024-09-10T07:53:12.232682+0000","last_clean":"2024-09-10T07:53:12.232682+0000","last_became_active":"2024-09-10T07:53:10.230111+0000","last_became_peered":"2024-09-10T07:53:10.230111+0000","last_unstale":"2024-09-10T07:53:12.232682+0000","last_undegraded":"2024-09-10T07:53:12.232682+0000","last_fullsized":"2024-09-10T07:53:12.232682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:58.029264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217413+0000","last_change":"2024-09-10T07:53:10.217413+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217413+0000","last_undegraded":"2024-09-10T07:53:10.217413+0000","last_fullsized":"2024-09-10T07:53:10.217413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267444+0000","last_change":"2024-09-10T07:53:10.222067+0000","last_active":"2024-09-10T07:53:11.267444+0000","last_peered":"2024-09-10T07:53:11.267444+0000","last_clean":"2024-09-10T07:53:11.267444+0000","last_became_active":"2024-09-10T07:53:10.221728+0000","last_became_peered":"2024-09-10T07:53:10.221728+0000","last_unstale":"2024-09-10T07:53:11.267444+0000","last_undegraded":"2024-09-10T07:53:11.267444+0000","last_fullsized":"2024-09-10T07:53:11.267444+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212195+0000","last_change":"2024-09-10T07:53:10.212195+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212195+0000","last_undegraded":"2024-09-10T07:53:10.212195+0000","last_fullsized":"2024-09-10T07:53:10.212195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236035+0000","last_change":"2024-09-10T07:53:10.232887+0000","last_active":"2024-09-10T07:53:10.236035+0000","last_peered":"2024-09-10T07:53:10.236035+0000","last_clean":"2024-09-10T07:53:10.236035+0000","last_became_active":"2024-09-10T07:53:10.230235+0000","last_became_peered":"2024-09-10T07:53:10.230235+0000","last_unstale":"2024-09-10T07:53:10.236035+0000","last_undegraded":"2024-09-10T07:53:10.236035+0000","last_fullsized":"2024-09-10T07:53:10.236035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:14.807864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233395+0000","last_change":"2024-09-10T07:53:11.254067+0000","last_active":"2024-09-10T07:53:12.233395+0000","last_peered":"2024-09-10T07:53:12.233395+0000","last_clean":"2024-09-10T07:53:12.233395+0000","last_became_active":"2024-09-10T07:53:11.253708+0000","last_became_peered":"2024-09-10T07:53:11.253708+0000","last_unstale":"2024-09-10T07:53:12.233395+0000","last_undegraded":"2024-09-10T07:53:12.233395+0000","last_fullsized":"2024-09-10T07:53:12.233395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:01.851731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235237+0000","last_change":"2024-09-10T07:53:10.223013+0000","last_active":"2024-09-10T07:53:10.235237+0000","last_peered":"2024-09-10T07:53:10.235237+0000","last_clean":"2024-09-10T07:53:10.235237+0000","last_became_active":"2024-09-10T07:53:10.222053+0000","last_became_peered":"2024-09-10T07:53:10.222053+0000","last_unstale":"2024-09-10T07:53:10.235237+0000","last_undegraded":"2024-09-10T07:53:10.235237+0000","last_fullsized":"2024-09-10T07:53:10.235237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:42.615374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258362+0000","last_change":"2024-09-10T07:53:11.247894+0000","last_active":"2024-09-10T07:53:11.258362+0000","last_peered":"2024-09-10T07:53:11.258362+0000","last_clean":"2024-09-10T07:53:11.258362+0000","last_became_active":"2024-09-10T07:53:11.247569+0000","last_became_peered":"2024-09-10T07:53:11.247569+0000","last_unstale":"2024-09-10T07:53:11.258362+0000","last_undegraded":"2024-09-10T07:53:11.258362+0000","last_fullsized":"2024-09-10T07:53:11.258362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:37.700469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248875+0000","last_change":"2024-09-10T07:53:10.223877+0000","last_active":"2024-09-10T07:53:11.248875+0000","last_peered":"2024-09-10T07:53:11.248875+0000","last_clean":"2024-09-10T07:53:11.248875+0000","last_became_active":"2024-09-10T07:53:10.222951+0000","last_became_peered":"2024-09-10T07:53:10.222951+0000","last_unstale":"2024-09-10T07:53:11.248875+0000","last_undegraded":"2024-09-10T07:53:11.248875+0000","last_fullsized":"2024-09-10T07:53:11.248875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:16.490800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211556+0000","last_change":"2024-09-10T07:53:10.211556+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211556+0000","last_undegraded":"2024-09-10T07:53:10.211556+0000","last_fullsized":"2024-09-10T07:53:10.211556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233746+0000","last_change":"2024-09-10T07:53:10.227674+0000","last_active":"2024-09-10T07:53:12.233746+0000","last_peered":"2024-09-10T07:53:12.233746+0000","last_clean":"2024-09-10T07:53:12.233746+0000","last_became_active":"2024-09-10T07:53:10.227321+0000","last_became_peered":"2024-09-10T07:53:10.227321+0000","last_unstale":"2024-09-10T07:53:12.233746+0000","last_undegraded":"2024-09-10T07:53:12.233746+0000","last_fullsized":"2024-09-10T07:53:12.233746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:58.837406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260406+0000","last_change":"2024-09-10T07:53:11.252691+0000","last_active":"2024-09-10T07:53:11.260406+0000","last_peered":"2024-09-10T07:53:11.260406+0000","last_clean":"2024-09-10T07:53:11.260406+0000","last_became_active":"2024-09-10T07:53:11.252471+0000","last_became_peered":"2024-09-10T07:53:11.252471+0000","last_unstale":"2024-09-10T07:53:11.260406+0000","last_undegraded":"2024-09-10T07:53:11.260406+0000","last_fullsized":"2024-09-10T07:53:11.260406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:41.229077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248309+0000","last_change":"2024-09-10T07:53:10.230718+0000","last_active":"2024-09-10T07:53:11.248309+0000","last_peered":"2024-09-10T07:53:11.248309+0000","last_clean":"2024-09-10T07:53:11.248309+0000","last_became_active":"2024-09-10T07:53:10.230504+0000","last_became_peered":"2024-09-10T07:53:10.230504+0000","last_unstale":"2024-09-10T07:53:11.248309+0000","last_undegraded":"2024-09-10T07:53:11.248309+0000","last_fullsized":"2024-09-10T07:53:11.248309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:14.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212681+0000","last_change":"2024-09-10T07:53:10.212681+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212681+0000","last_undegraded":"2024-09-10T07:53:10.212681+0000","last_fullsized":"2024-09-10T07:53:10.212681+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236579+0000","last_change":"2024-09-10T07:53:10.234562+0000","last_active":"2024-09-10T07:53:10.236579+0000","last_peered":"2024-09-10T07:53:10.236579+0000","last_clean":"2024-09-10T07:53:10.236579+0000","last_became_active":"2024-09-10T07:53:10.234276+0000","last_became_peered":"2024-09-10T07:53:10.234276+0000","last_unstale":"2024-09-10T07:53:10.236579+0000","last_undegraded":"2024-09-10T07:53:10.236579+0000","last_fullsized":"2024-09-10T07:53:10.236579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268617+0000","last_change":"2024-09-10T07:53:11.264027+0000","last_active":"2024-09-10T07:53:11.268617+0000","last_peered":"2024-09-10T07:53:11.268617+0000","last_clean":"2024-09-10T07:53:11.268617+0000","last_became_active":"2024-09-10T07:53:11.263741+0000","last_became_peered":"2024-09-10T07:53:11.263741+0000","last_unstale":"2024-09-10T07:53:11.268617+0000","last_undegraded":"2024-09-10T07:53:11.268617+0000","last_fullsized":"2024-09-10T07:53:11.268617+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:04.728438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232795+0000","last_change":"2024-09-10T07:53:10.224089+0000","last_active":"2024-09-10T07:53:12.232795+0000","last_peered":"2024-09-10T07:53:12.232795+0000","last_clean":"2024-09-10T07:53:12.232795+0000","last_became_active":"2024-09-10T07:53:10.222933+0000","last_became_peered":"2024-09-10T07:53:10.222933+0000","last_unstale":"2024-09-10T07:53:12.232795+0000","last_undegraded":"2024-09-10T07:53:12.232795+0000","last_fullsized":"2024-09-10T07:53:12.232795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:01.604342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234021+0000","last_change":"2024-09-10T07:53:11.266735+0000","last_active":"2024-09-10T07:53:12.234021+0000","last_peered":"2024-09-10T07:53:12.234021+0000","last_clean":"2024-09-10T07:53:12.234021+0000","last_became_active":"2024-09-10T07:53:11.266398+0000","last_became_peered":"2024-09-10T07:53:11.266398+0000","last_unstale":"2024-09-10T07:53:12.234021+0000","last_undegraded":"2024-09-10T07:53:12.234021+0000","last_fullsized":"2024-09-10T07:53:12.234021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:05.878447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271008+0000","last_change":"2024-09-10T07:53:10.235291+0000","last_active":"2024-09-10T07:53:11.271008+0000","last_peered":"2024-09-10T07:53:11.271008+0000","last_clean":"2024-09-10T07:53:11.271008+0000","last_became_active":"2024-09-10T07:53:10.235015+0000","last_became_peered":"2024-09-10T07:53:10.235015+0000","last_unstale":"2024-09-10T07:53:11.271008+0000","last_undegraded":"2024-09-10T07:53:11.271008+0000","last_fullsized":"2024-09-10T07:53:11.271008+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":314,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231633+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:53:11.231633+0000","last_peered":"2024-09-10T07:53:11.231633+0000","last_clean":"2024-09-10T07:53:11.231633+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:53:11.231633+0000","last_undegraded":"2024-09-10T07:53:11.231633+0000","last_fullsized":"2024-09-10T07:53:11.231633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476822,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476822,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":1127976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1113202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1241088,"data_stored":536466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:53:15.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:53:15.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.641+0000 7fa85b200640 1 Processor -- start 2024-09-10T07:53:15.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.641+0000 7fa85b200640 1 -- start start 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85b200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8540d3c10 0x7fa854073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85b200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8540d4050 0x7fa854073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85b200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540cfdc0 con 0x7fa85415e860 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85b200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540d3780 con 0x7fa8540d4050 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85b200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540cee40 con 0x7fa8540d3c10 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa859e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa85415e860 0x7fa8540cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56696/0 (socket says 172.21.15.3:56696) 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa859e00640 1 -- 172.21.15.3:0/1554529135 learned_addr learned my addr 172.21.15.3:0/1554529135 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:15.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.642+0000 7fa85a800640 1 --2- 172.21.15.3:0/1554529135 >> v2:172.21.15.3:3300/0 conn(0x7fa8540d4050 0x7fa854073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.643+0000 7fa858a00640 1 -- 172.21.15.3:0/1554529135 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3217990295 0 0) 0x7fa8540cfdc0 con 0x7fa85415e860 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.643+0000 7fa85a800640 1 -- 172.21.15.3:0/1554529135 >> v2:172.21.15.70:3300/0 conn(0x7fa8540d3c10 msgr2=0x7fa854073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.643+0000 7fa85a800640 1 --2- 172.21.15.3:0/1554529135 >> v2:172.21.15.70:3300/0 conn(0x7fa8540d3c10 0x7fa854073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.643+0000 7fa85a800640 1 -- 172.21.15.3:0/1554529135 >> v1:172.21.15.3:6789/0 conn(0x7fa85415e860 legacy=0x7fa8540cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.643+0000 7fa85a800640 1 -- 172.21.15.3:0/1554529135 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540cf120 con 0x7fa8540d4050 2024-09-10T07:53:15.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa85a800640 1 --2- 172.21.15.3:0/1554529135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d4050 0x7fa854073690 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa8540d0ad0 tx=0x7fa85002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28845f0cf2e7cd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa858a00640 1 -- 172.21.15.3:0/1554529135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850007740 con 0x7fa8540d4050 2024-09-10T07:53:15.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa858a00640 1 -- 172.21.15.3:0/1554529135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500078e0 con 0x7fa8540d4050 2024-09-10T07:53:15.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa858a00640 1 -- 172.21.15.3:0/1554529135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850007cb0 con 0x7fa8540d4050 2024-09-10T07:53:15.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa85b200640 1 -- 172.21.15.3:0/1554529135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d4050 msgr2=0x7fa854073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.644+0000 7fa85b200640 1 --2- 172.21.15.3:0/1554529135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d4050 0x7fa854073690 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa8540d0ad0 tx=0x7fa85002a680 comp rx=0 tx=0).stop 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 -- 172.21.15.3:0/1554529135 shutdown_connections 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 --2- 172.21.15.3:0/1554529135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d4050 0x7fa854073690 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 --2- 172.21.15.3:0/1554529135 >> v2:172.21.15.70:3300/0 conn(0x7fa8540d3c10 0x7fa854073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 -- 172.21.15.3:0/1554529135 >> 172.21.15.3:0/1554529135 conn(0x7fa8540d4650 msgr2=0x7fa8540d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 -- 172.21.15.3:0/1554529135 shutdown_connections 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.645+0000 7fa85b200640 1 -- 172.21.15.3:0/1554529135 wait complete. 2024-09-10T07:53:15.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.646+0000 7fa85b200640 1 Processor -- start 2024-09-10T07:53:15.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.646+0000 7fa85b200640 1 -- start start 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 0x7fa8541614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 0x7fa8540acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540d2460 con 0x7fa8540d3c10 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540cf400 con 0x7fa8540d4050 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.647+0000 7fa85b200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa8540b7880 con 0x7fa85415e860 2024-09-10T07:53:15.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35906/0 (socket says 172.21.15.3:35906) 2024-09-10T07:53:15.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 -- 172.21.15.3:0/767055578 learned_addr learned my addr 172.21.15.3:0/767055578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859400640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 0x7fa8541614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 -- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 msgr2=0x7fa8540acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa85a800640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 0x7fa8540acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 0x7fa8540acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 -- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 msgr2=0x7fa8541614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 0x7fa8541614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 -- 172.21.15.3:0/767055578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85415c2c0 con 0x7fa8540d3c10 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859400640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 0x7fa8541614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa85a800640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 0x7fa8540acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.648+0000 7fa859e00640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c430 tx=0x7fa84800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.649+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480074e0 con 0x7fa8540d3c10 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.649+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8480047e0 con 0x7fa8540d3c10 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.649+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004bd0 con 0x7fa8540d3c10 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.649+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8540b2cd0 con 0x7fa8540d3c10 2024-09-10T07:53:15.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.649+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8540b3290 con 0x7fa8540d3c10 2024-09-10T07:53:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.650+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8540b7c40 con 0x7fa8540d3c10 2024-09-10T07:53:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.651+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa848011070 con 0x7fa8540d3c10 2024-09-10T07:53:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.651+0000 7fa84e000640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.652+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa84808ab10 con 0x7fa8540d3c10 2024-09-10T07:53:15.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.652+0000 7fa859400640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:15.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.652+0000 7fa859400640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 0x7fa83806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa844006fd0 tx=0x7fa844008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:15.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.656+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa8480594b0 con 0x7fa8540d3c10 2024-09-10T07:53:15.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.839+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8540d08c0 con 0x7fa83806be30 2024-09-10T07:53:15.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.845+0000 7fa84e000640 1 -- 172.21.15.3:0/767055578 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250124 (secure 0 0 0) 0x7fa8540d08c0 con 0x7fa83806be30 2024-09-10T07:53:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:15.849 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:53:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 msgr2=0x7fa83806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 0x7fa83806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa844006fd0 tx=0x7fa844008040 comp rx=0 tx=0).stop 2024-09-10T07:53:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 msgr2=0x7fa85415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c430 tx=0x7fa84800c8f0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 shutdown_connections 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa83806be30 0x7fa83806e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa85415e860 0x7fa8540acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8540d4050 0x7fa8541614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 --2- 172.21.15.3:0/767055578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8540d3c10 0x7fa85415bd80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.849+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 >> 172.21.15.3:0/767055578 conn(0x7fa8540d4650 msgr2=0x7fa8540b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.850+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 shutdown_connections 2024-09-10T07:53:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:15.850+0000 7fa85b200640 1 -- 172.21.15.3:0/767055578 wait complete. 2024-09-10T07:53:15.865 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-10T07:53:15.580430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264436,"kb_used_data":10740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710284,"statfs":{"total":773094113280,"available":772823330816,"internally_reserved":0,"allocated":10997760,"data_stored":5410639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000952"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214073+0000","last_change":"2024-09-10T07:53:10.214073+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214073+0000","last_undegraded":"2024-09-10T07:53:10.214073+0000","last_fullsized":"2024-09-10T07:53:10.214073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215296+0000","last_change":"2024-09-10T07:53:10.215296+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215296+0000","last_undegraded":"2024-09-10T07:53:10.215296+0000","last_fullsized":"2024-09-10T07:53:10.215296+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233484+0000","last_change":"2024-09-10T07:53:11.252106+0000","last_active":"2024-09-10T07:53:12.233484+0000","last_peered":"2024-09-10T07:53:12.233484+0000","last_clean":"2024-09-10T07:53:12.233484+0000","last_became_active":"2024-09-10T07:53:11.251807+0000","last_became_peered":"2024-09-10T07:53:11.251807+0000","last_unstale":"2024-09-10T07:53:12.233484+0000","last_undegraded":"2024-09-10T07:53:12.233484+0000","last_fullsized":"2024-09-10T07:53:12.233484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:01.780830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215936+0000","last_change":"2024-09-10T07:53:10.215936+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215936+0000","last_undegraded":"2024-09-10T07:53:10.215936+0000","last_fullsized":"2024-09-10T07:53:10.215936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.237820+0000","last_change":"2024-09-10T07:53:11.234549+0000","last_active":"2024-09-10T07:53:11.237820+0000","last_peered":"2024-09-10T07:53:11.237820+0000","last_clean":"2024-09-10T07:53:11.237820+0000","last_became_active":"2024-09-10T07:53:11.234201+0000","last_became_peered":"2024-09-10T07:53:11.234201+0000","last_unstale":"2024-09-10T07:53:11.237820+0000","last_undegraded":"2024-09-10T07:53:11.237820+0000","last_fullsized":"2024-09-10T07:53:11.237820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:49.298649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.243698+0000","last_change":"2024-09-10T07:53:11.236870+0000","last_active":"2024-09-10T07:53:11.243698+0000","last_peered":"2024-09-10T07:53:11.243698+0000","last_clean":"2024-09-10T07:53:11.243698+0000","last_became_active":"2024-09-10T07:53:11.235843+0000","last_became_peered":"2024-09-10T07:53:11.235843+0000","last_unstale":"2024-09-10T07:53:11.243698+0000","last_undegraded":"2024-09-10T07:53:11.243698+0000","last_fullsized":"2024-09-10T07:53:11.243698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.860697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261073+0000","last_change":"2024-09-10T07:53:11.255394+0000","last_active":"2024-09-10T07:53:11.261073+0000","last_peered":"2024-09-10T07:53:11.261073+0000","last_clean":"2024-09-10T07:53:11.261073+0000","last_became_active":"2024-09-10T07:53:11.254136+0000","last_became_peered":"2024-09-10T07:53:11.254136+0000","last_unstale":"2024-09-10T07:53:11.261073+0000","last_undegraded":"2024-09-10T07:53:11.261073+0000","last_fullsized":"2024-09-10T07:53:11.261073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:35.472507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233314+0000","last_change":"2024-09-10T07:53:11.250032+0000","last_active":"2024-09-10T07:53:12.233314+0000","last_peered":"2024-09-10T07:53:12.233314+0000","last_clean":"2024-09-10T07:53:12.233314+0000","last_became_active":"2024-09-10T07:53:11.249626+0000","last_became_peered":"2024-09-10T07:53:11.249626+0000","last_unstale":"2024-09-10T07:53:12.233314+0000","last_undegraded":"2024-09-10T07:53:12.233314+0000","last_fullsized":"2024-09-10T07:53:12.233314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.283338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248152+0000","last_change":"2024-09-10T07:53:11.243615+0000","last_active":"2024-09-10T07:53:11.248152+0000","last_peered":"2024-09-10T07:53:11.248152+0000","last_clean":"2024-09-10T07:53:11.248152+0000","last_became_active":"2024-09-10T07:53:11.243351+0000","last_became_peered":"2024-09-10T07:53:11.243351+0000","last_unstale":"2024-09-10T07:53:11.248152+0000","last_undegraded":"2024-09-10T07:53:11.248152+0000","last_fullsized":"2024-09-10T07:53:11.248152+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:53.710317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268481+0000","last_change":"2024-09-10T07:53:11.267004+0000","last_active":"2024-09-10T07:53:11.268481+0000","last_peered":"2024-09-10T07:53:11.268481+0000","last_clean":"2024-09-10T07:53:11.268481+0000","last_became_active":"2024-09-10T07:53:11.266699+0000","last_became_peered":"2024-09-10T07:53:11.266699+0000","last_unstale":"2024-09-10T07:53:11.268481+0000","last_undegraded":"2024-09-10T07:53:11.268481+0000","last_fullsized":"2024-09-10T07:53:11.268481+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:08.715930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239275+0000","last_change":"2024-09-10T07:53:11.236475+0000","last_active":"2024-09-10T07:53:11.239275+0000","last_peered":"2024-09-10T07:53:11.239275+0000","last_clean":"2024-09-10T07:53:11.239275+0000","last_became_active":"2024-09-10T07:53:11.235672+0000","last_became_peered":"2024-09-10T07:53:11.235672+0000","last_unstale":"2024-09-10T07:53:11.239275+0000","last_undegraded":"2024-09-10T07:53:11.239275+0000","last_fullsized":"2024-09-10T07:53:11.239275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:31:33.473232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269070+0000","last_change":"2024-09-10T07:53:11.267139+0000","last_active":"2024-09-10T07:53:11.269070+0000","last_peered":"2024-09-10T07:53:11.269070+0000","last_clean":"2024-09-10T07:53:11.269070+0000","last_became_active":"2024-09-10T07:53:11.266829+0000","last_became_peered":"2024-09-10T07:53:11.266829+0000","last_unstale":"2024-09-10T07:53:11.269070+0000","last_undegraded":"2024-09-10T07:53:11.269070+0000","last_fullsized":"2024-09-10T07:53:11.269070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.218726+0000","last_change":"2024-09-10T07:53:10.218726+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.218726+0000","last_undegraded":"2024-09-10T07:53:10.218726+0000","last_fullsized":"2024-09-10T07:53:10.218726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212879+0000","last_change":"2024-09-10T07:53:10.212879+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212879+0000","last_undegraded":"2024-09-10T07:53:10.212879+0000","last_fullsized":"2024-09-10T07:53:10.212879+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211714+0000","last_change":"2024-09-10T07:53:10.211714+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211714+0000","last_undegraded":"2024-09-10T07:53:10.211714+0000","last_fullsized":"2024-09-10T07:53:10.211714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260661+0000","last_change":"2024-09-10T07:53:11.254673+0000","last_active":"2024-09-10T07:53:11.260661+0000","last_peered":"2024-09-10T07:53:11.260661+0000","last_clean":"2024-09-10T07:53:11.260661+0000","last_became_active":"2024-09-10T07:53:11.251614+0000","last_became_peered":"2024-09-10T07:53:11.251614+0000","last_unstale":"2024-09-10T07:53:11.260661+0000","last_undegraded":"2024-09-10T07:53:11.260661+0000","last_fullsized":"2024-09-10T07:53:11.260661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:50.934355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212721+0000","last_change":"2024-09-10T07:53:10.212721+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212721+0000","last_undegraded":"2024-09-10T07:53:10.212721+0000","last_fullsized":"2024-09-10T07:53:10.212721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214734+0000","last_change":"2024-09-10T07:53:10.214734+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214734+0000","last_undegraded":"2024-09-10T07:53:10.214734+0000","last_fullsized":"2024-09-10T07:53:10.214734+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214070+0000","last_change":"2024-09-10T07:53:10.214070+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214070+0000","last_undegraded":"2024-09-10T07:53:10.214070+0000","last_fullsized":"2024-09-10T07:53:10.214070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239700+0000","last_change":"2024-09-10T07:53:11.234958+0000","last_active":"2024-09-10T07:53:11.239700+0000","last_peered":"2024-09-10T07:53:11.239700+0000","last_clean":"2024-09-10T07:53:11.239700+0000","last_became_active":"2024-09-10T07:53:11.234678+0000","last_became_peered":"2024-09-10T07:53:11.234678+0000","last_unstale":"2024-09-10T07:53:11.239700+0000","last_undegraded":"2024-09-10T07:53:11.239700+0000","last_fullsized":"2024-09-10T07:53:11.239700+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:47.186154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233477+0000","last_change":"2024-09-10T07:53:11.248147+0000","last_active":"2024-09-10T07:53:12.233477+0000","last_peered":"2024-09-10T07:53:12.233477+0000","last_clean":"2024-09-10T07:53:12.233477+0000","last_became_active":"2024-09-10T07:53:11.247829+0000","last_became_peered":"2024-09-10T07:53:11.247829+0000","last_unstale":"2024-09-10T07:53:12.233477+0000","last_undegraded":"2024-09-10T07:53:12.233477+0000","last_fullsized":"2024-09-10T07:53:12.233477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:54.042192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232852+0000","last_change":"2024-09-10T07:53:11.249206+0000","last_active":"2024-09-10T07:53:12.232852+0000","last_peered":"2024-09-10T07:53:12.232852+0000","last_clean":"2024-09-10T07:53:12.232852+0000","last_became_active":"2024-09-10T07:53:11.248315+0000","last_became_peered":"2024-09-10T07:53:11.248315+0000","last_unstale":"2024-09-10T07:53:12.232852+0000","last_undegraded":"2024-09-10T07:53:12.232852+0000","last_fullsized":"2024-09-10T07:53:12.232852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:31.145698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211200+0000","last_change":"2024-09-10T07:53:10.211200+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211200+0000","last_undegraded":"2024-09-10T07:53:10.211200+0000","last_fullsized":"2024-09-10T07:53:10.211200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.263806+0000","last_change":"2024-09-10T07:53:11.262775+0000","last_active":"2024-09-10T07:53:11.263806+0000","last_peered":"2024-09-10T07:53:11.263806+0000","last_clean":"2024-09-10T07:53:11.263806+0000","last_became_active":"2024-09-10T07:53:11.262438+0000","last_became_peered":"2024-09-10T07:53:11.262438+0000","last_unstale":"2024-09-10T07:53:11.263806+0000","last_undegraded":"2024-09-10T07:53:11.263806+0000","last_fullsized":"2024-09-10T07:53:11.263806+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:12.859790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.257372+0000","last_change":"2024-09-10T07:53:11.255844+0000","last_active":"2024-09-10T07:53:11.257372+0000","last_peered":"2024-09-10T07:53:11.257372+0000","last_clean":"2024-09-10T07:53:11.257372+0000","last_became_active":"2024-09-10T07:53:11.255548+0000","last_became_peered":"2024-09-10T07:53:11.255548+0000","last_unstale":"2024-09-10T07:53:11.257372+0000","last_undegraded":"2024-09-10T07:53:11.257372+0000","last_fullsized":"2024-09-10T07:53:11.257372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:06.407725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214175+0000","last_change":"2024-09-10T07:53:10.214175+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214175+0000","last_undegraded":"2024-09-10T07:53:10.214175+0000","last_fullsized":"2024-09-10T07:53:10.214175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248744+0000","last_change":"2024-09-10T07:53:11.246590+0000","last_active":"2024-09-10T07:53:11.248744+0000","last_peered":"2024-09-10T07:53:11.248744+0000","last_clean":"2024-09-10T07:53:11.248744+0000","last_became_active":"2024-09-10T07:53:11.245943+0000","last_became_peered":"2024-09-10T07:53:11.245943+0000","last_unstale":"2024-09-10T07:53:11.248744+0000","last_undegraded":"2024-09-10T07:53:11.248744+0000","last_fullsized":"2024-09-10T07:53:11.248744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258576+0000","last_change":"2024-09-10T07:53:11.256724+0000","last_active":"2024-09-10T07:53:11.258576+0000","last_peered":"2024-09-10T07:53:11.258576+0000","last_clean":"2024-09-10T07:53:11.258576+0000","last_became_active":"2024-09-10T07:53:11.256427+0000","last_became_peered":"2024-09-10T07:53:11.256427+0000","last_unstale":"2024-09-10T07:53:11.258576+0000","last_undegraded":"2024-09-10T07:53:11.258576+0000","last_fullsized":"2024-09-10T07:53:11.258576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:19.541855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214213+0000","last_change":"2024-09-10T07:53:10.214213+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214213+0000","last_undegraded":"2024-09-10T07:53:10.214213+0000","last_fullsized":"2024-09-10T07:53:10.214213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.224154+0000","last_change":"2024-09-10T07:53:10.224154+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.224154+0000","last_undegraded":"2024-09-10T07:53:10.224154+0000","last_fullsized":"2024-09-10T07:53:10.224154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234095+0000","last_change":"2024-09-10T07:53:11.256491+0000","last_active":"2024-09-10T07:53:12.234095+0000","last_peered":"2024-09-10T07:53:12.234095+0000","last_clean":"2024-09-10T07:53:12.234095+0000","last_became_active":"2024-09-10T07:53:11.255961+0000","last_became_peered":"2024-09-10T07:53:11.255961+0000","last_unstale":"2024-09-10T07:53:12.234095+0000","last_undegraded":"2024-09-10T07:53:12.234095+0000","last_fullsized":"2024-09-10T07:53:12.234095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:11.297862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214870+0000","last_change":"2024-09-10T07:53:10.214870+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214870+0000","last_undegraded":"2024-09-10T07:53:10.214870+0000","last_fullsized":"2024-09-10T07:53:10.214870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217871+0000","last_change":"2024-09-10T07:53:10.217871+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217871+0000","last_undegraded":"2024-09-10T07:53:10.217871+0000","last_fullsized":"2024-09-10T07:53:10.217871+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236893+0000","last_change":"2024-09-10T07:53:10.235925+0000","last_active":"2024-09-10T07:53:10.236893+0000","last_peered":"2024-09-10T07:53:10.236893+0000","last_clean":"2024-09-10T07:53:10.236893+0000","last_became_active":"2024-09-10T07:53:10.235350+0000","last_became_peered":"2024-09-10T07:53:10.235350+0000","last_unstale":"2024-09-10T07:53:10.236893+0000","last_undegraded":"2024-09-10T07:53:10.236893+0000","last_fullsized":"2024-09-10T07:53:10.236893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215028+0000","last_change":"2024-09-10T07:53:10.215028+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215028+0000","last_undegraded":"2024-09-10T07:53:10.215028+0000","last_fullsized":"2024-09-10T07:53:10.215028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267919+0000","last_change":"2024-09-10T07:53:10.232379+0000","last_active":"2024-09-10T07:53:11.267919+0000","last_peered":"2024-09-10T07:53:11.267919+0000","last_clean":"2024-09-10T07:53:11.267919+0000","last_became_active":"2024-09-10T07:53:10.231931+0000","last_became_peered":"2024-09-10T07:53:10.231931+0000","last_unstale":"2024-09-10T07:53:11.267919+0000","last_undegraded":"2024-09-10T07:53:11.267919+0000","last_fullsized":"2024-09-10T07:53:11.267919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:22:58.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215453+0000","last_change":"2024-09-10T07:53:10.215453+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215453+0000","last_undegraded":"2024-09-10T07:53:10.215453+0000","last_fullsized":"2024-09-10T07:53:10.215453+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233603+0000","last_change":"2024-09-10T07:53:10.227323+0000","last_active":"2024-09-10T07:53:12.233603+0000","last_peered":"2024-09-10T07:53:12.233603+0000","last_clean":"2024-09-10T07:53:12.233603+0000","last_became_active":"2024-09-10T07:53:10.227012+0000","last_became_peered":"2024-09-10T07:53:10.227012+0000","last_unstale":"2024-09-10T07:53:12.233603+0000","last_undegraded":"2024-09-10T07:53:12.233603+0000","last_fullsized":"2024-09-10T07:53:12.233603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:56.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.249667+0000","last_change":"2024-09-10T07:53:11.246551+0000","last_active":"2024-09-10T07:53:11.249667+0000","last_peered":"2024-09-10T07:53:11.249667+0000","last_clean":"2024-09-10T07:53:11.249667+0000","last_became_active":"2024-09-10T07:53:11.246300+0000","last_became_peered":"2024-09-10T07:53:11.246300+0000","last_unstale":"2024-09-10T07:53:11.249667+0000","last_undegraded":"2024-09-10T07:53:11.249667+0000","last_fullsized":"2024-09-10T07:53:11.249667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234150+0000","last_change":"2024-09-10T07:53:10.231193+0000","last_active":"2024-09-10T07:53:10.234150+0000","last_peered":"2024-09-10T07:53:10.234150+0000","last_clean":"2024-09-10T07:53:10.234150+0000","last_became_active":"2024-09-10T07:53:10.226288+0000","last_became_peered":"2024-09-10T07:53:10.226288+0000","last_unstale":"2024-09-10T07:53:10.234150+0000","last_undegraded":"2024-09-10T07:53:10.234150+0000","last_fullsized":"2024-09-10T07:53:10.234150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:31.899936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.241212+0000","last_change":"2024-09-10T07:53:11.235832+0000","last_active":"2024-09-10T07:53:11.241212+0000","last_peered":"2024-09-10T07:53:11.241212+0000","last_clean":"2024-09-10T07:53:11.241212+0000","last_became_active":"2024-09-10T07:53:11.235506+0000","last_became_peered":"2024-09-10T07:53:11.235506+0000","last_unstale":"2024-09-10T07:53:11.241212+0000","last_undegraded":"2024-09-10T07:53:11.241212+0000","last_fullsized":"2024-09-10T07:53:11.241212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:05.491743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233697+0000","last_change":"2024-09-10T07:53:10.228445+0000","last_active":"2024-09-10T07:53:10.233697+0000","last_peered":"2024-09-10T07:53:10.233697+0000","last_clean":"2024-09-10T07:53:10.233697+0000","last_became_active":"2024-09-10T07:53:10.227806+0000","last_became_peered":"2024-09-10T07:53:10.227806+0000","last_unstale":"2024-09-10T07:53:10.233697+0000","last_undegraded":"2024-09-10T07:53:10.233697+0000","last_fullsized":"2024-09-10T07:53:10.233697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:25:41.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.238929+0000","last_change":"2024-09-10T07:53:11.235298+0000","last_active":"2024-09-10T07:53:11.238929+0000","last_peered":"2024-09-10T07:53:11.238929+0000","last_clean":"2024-09-10T07:53:11.238929+0000","last_became_active":"2024-09-10T07:53:11.234953+0000","last_became_peered":"2024-09-10T07:53:11.234953+0000","last_unstale":"2024-09-10T07:53:11.238929+0000","last_undegraded":"2024-09-10T07:53:11.238929+0000","last_fullsized":"2024-09-10T07:53:11.238929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:14.411009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232971+0000","last_change":"2024-09-10T07:53:10.229226+0000","last_active":"2024-09-10T07:53:12.232971+0000","last_peered":"2024-09-10T07:53:12.232971+0000","last_clean":"2024-09-10T07:53:12.232971+0000","last_became_active":"2024-09-10T07:53:10.228904+0000","last_became_peered":"2024-09-10T07:53:10.228904+0000","last_unstale":"2024-09-10T07:53:12.232971+0000","last_undegraded":"2024-09-10T07:53:12.232971+0000","last_fullsized":"2024-09-10T07:53:12.232971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:29.688893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214033+0000","last_change":"2024-09-10T07:53:10.214033+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214033+0000","last_undegraded":"2024-09-10T07:53:10.214033+0000","last_fullsized":"2024-09-10T07:53:10.214033+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.229770+0000","last_change":"2024-09-10T07:53:10.228005+0000","last_active":"2024-09-10T07:53:11.229770+0000","last_peered":"2024-09-10T07:53:11.229770+0000","last_clean":"2024-09-10T07:53:11.229770+0000","last_became_active":"2024-09-10T07:53:10.227661+0000","last_became_peered":"2024-09-10T07:53:10.227661+0000","last_unstale":"2024-09-10T07:53:11.229770+0000","last_undegraded":"2024-09-10T07:53:11.229770+0000","last_fullsized":"2024-09-10T07:53:11.229770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:45.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.240934+0000","last_change":"2024-09-10T07:53:11.235959+0000","last_active":"2024-09-10T07:53:11.240934+0000","last_peered":"2024-09-10T07:53:11.240934+0000","last_clean":"2024-09-10T07:53:11.240934+0000","last_became_active":"2024-09-10T07:53:11.235687+0000","last_became_peered":"2024-09-10T07:53:11.235687+0000","last_unstale":"2024-09-10T07:53:11.240934+0000","last_undegraded":"2024-09-10T07:53:11.240934+0000","last_fullsized":"2024-09-10T07:53:11.240934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:00.667191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235952+0000","last_change":"2024-09-10T07:53:10.233196+0000","last_active":"2024-09-10T07:53:10.235952+0000","last_peered":"2024-09-10T07:53:10.235952+0000","last_clean":"2024-09-10T07:53:10.235952+0000","last_became_active":"2024-09-10T07:53:10.232510+0000","last_became_peered":"2024-09-10T07:53:10.232510+0000","last_unstale":"2024-09-10T07:53:10.235952+0000","last_undegraded":"2024-09-10T07:53:10.235952+0000","last_fullsized":"2024-09-10T07:53:10.235952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:11.857423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.213811+0000","last_change":"2024-09-10T07:53:10.213811+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.213811+0000","last_undegraded":"2024-09-10T07:53:10.213811+0000","last_fullsized":"2024-09-10T07:53:10.213811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234051+0000","last_change":"2024-09-10T07:53:10.213607+0000","last_active":"2024-09-10T07:53:12.234051+0000","last_peered":"2024-09-10T07:53:12.234051+0000","last_clean":"2024-09-10T07:53:12.234051+0000","last_became_active":"2024-09-10T07:53:10.213317+0000","last_became_peered":"2024-09-10T07:53:10.213317+0000","last_unstale":"2024-09-10T07:53:12.234051+0000","last_undegraded":"2024-09-10T07:53:12.234051+0000","last_fullsized":"2024-09-10T07:53:12.234051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:19.086909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.251040+0000","last_change":"2024-09-10T07:53:11.250003+0000","last_active":"2024-09-10T07:53:11.251040+0000","last_peered":"2024-09-10T07:53:11.251040+0000","last_clean":"2024-09-10T07:53:11.251040+0000","last_became_active":"2024-09-10T07:53:11.249735+0000","last_became_peered":"2024-09-10T07:53:11.249735+0000","last_unstale":"2024-09-10T07:53:11.251040+0000","last_undegraded":"2024-09-10T07:53:11.251040+0000","last_fullsized":"2024-09-10T07:53:11.251040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:53.440934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233009+0000","last_change":"2024-09-10T07:53:10.226296+0000","last_active":"2024-09-10T07:53:10.233009+0000","last_peered":"2024-09-10T07:53:10.233009+0000","last_clean":"2024-09-10T07:53:10.233009+0000","last_became_active":"2024-09-10T07:53:10.225937+0000","last_became_peered":"2024-09-10T07:53:10.225937+0000","last_unstale":"2024-09-10T07:53:10.233009+0000","last_undegraded":"2024-09-10T07:53:10.233009+0000","last_fullsized":"2024-09-10T07:53:10.233009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269185+0000","last_change":"2024-09-10T07:53:11.266667+0000","last_active":"2024-09-10T07:53:11.269185+0000","last_peered":"2024-09-10T07:53:11.269185+0000","last_clean":"2024-09-10T07:53:11.269185+0000","last_became_active":"2024-09-10T07:53:11.266400+0000","last_became_peered":"2024-09-10T07:53:11.266400+0000","last_unstale":"2024-09-10T07:53:11.269185+0000","last_undegraded":"2024-09-10T07:53:11.269185+0000","last_fullsized":"2024-09-10T07:53:11.269185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:39.890020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269567+0000","last_change":"2024-09-10T07:53:10.223127+0000","last_active":"2024-09-10T07:53:11.269567+0000","last_peered":"2024-09-10T07:53:11.269567+0000","last_clean":"2024-09-10T07:53:11.269567+0000","last_became_active":"2024-09-10T07:53:10.222729+0000","last_became_peered":"2024-09-10T07:53:10.222729+0000","last_unstale":"2024-09-10T07:53:11.269567+0000","last_undegraded":"2024-09-10T07:53:11.269567+0000","last_fullsized":"2024-09-10T07:53:11.269567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.995404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261874+0000","last_change":"2024-09-10T07:53:11.260871+0000","last_active":"2024-09-10T07:53:11.261874+0000","last_peered":"2024-09-10T07:53:11.261874+0000","last_clean":"2024-09-10T07:53:11.261874+0000","last_became_active":"2024-09-10T07:53:11.260613+0000","last_became_peered":"2024-09-10T07:53:11.260613+0000","last_unstale":"2024-09-10T07:53:11.261874+0000","last_undegraded":"2024-09-10T07:53:11.261874+0000","last_fullsized":"2024-09-10T07:53:11.261874+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:15.810230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.277289+0000","last_change":"2024-09-10T07:53:10.235054+0000","last_active":"2024-09-10T07:53:11.277289+0000","last_peered":"2024-09-10T07:53:11.277289+0000","last_clean":"2024-09-10T07:53:11.277289+0000","last_became_active":"2024-09-10T07:53:10.234764+0000","last_became_peered":"2024-09-10T07:53:10.234764+0000","last_unstale":"2024-09-10T07:53:11.277289+0000","last_undegraded":"2024-09-10T07:53:11.277289+0000","last_fullsized":"2024-09-10T07:53:11.277289+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212467+0000","last_change":"2024-09-10T07:53:10.212467+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212467+0000","last_undegraded":"2024-09-10T07:53:10.212467+0000","last_fullsized":"2024-09-10T07:53:10.212467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234989+0000","last_change":"2024-09-10T07:53:10.222685+0000","last_active":"2024-09-10T07:53:10.234989+0000","last_peered":"2024-09-10T07:53:10.234989+0000","last_clean":"2024-09-10T07:53:10.234989+0000","last_became_active":"2024-09-10T07:53:10.222344+0000","last_became_peered":"2024-09-10T07:53:10.222344+0000","last_unstale":"2024-09-10T07:53:10.234989+0000","last_undegraded":"2024-09-10T07:53:10.234989+0000","last_fullsized":"2024-09-10T07:53:10.234989+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:18.539489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233639+0000","last_change":"2024-09-10T07:53:11.257850+0000","last_active":"2024-09-10T07:53:12.233639+0000","last_peered":"2024-09-10T07:53:12.233639+0000","last_clean":"2024-09-10T07:53:12.233639+0000","last_became_active":"2024-09-10T07:53:11.255958+0000","last_became_peered":"2024-09-10T07:53:11.255958+0000","last_unstale":"2024-09-10T07:53:12.233639+0000","last_undegraded":"2024-09-10T07:53:12.233639+0000","last_fullsized":"2024-09-10T07:53:12.233639+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:05.576611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233362+0000","last_change":"2024-09-10T07:53:10.229874+0000","last_active":"2024-09-10T07:53:10.233362+0000","last_peered":"2024-09-10T07:53:10.233362+0000","last_clean":"2024-09-10T07:53:10.233362+0000","last_became_active":"2024-09-10T07:53:10.229597+0000","last_became_peered":"2024-09-10T07:53:10.229597+0000","last_unstale":"2024-09-10T07:53:10.233362+0000","last_undegraded":"2024-09-10T07:53:10.233362+0000","last_fullsized":"2024-09-10T07:53:10.233362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239396+0000","last_change":"2024-09-10T07:53:11.236857+0000","last_active":"2024-09-10T07:53:11.239396+0000","last_peered":"2024-09-10T07:53:11.239396+0000","last_clean":"2024-09-10T07:53:11.239396+0000","last_became_active":"2024-09-10T07:53:11.236085+0000","last_became_peered":"2024-09-10T07:53:11.236085+0000","last_unstale":"2024-09-10T07:53:11.239396+0000","last_undegraded":"2024-09-10T07:53:11.239396+0000","last_fullsized":"2024-09-10T07:53:11.239396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:25.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231957+0000","last_change":"2024-09-10T07:53:10.223641+0000","last_active":"2024-09-10T07:53:11.231957+0000","last_peered":"2024-09-10T07:53:11.231957+0000","last_clean":"2024-09-10T07:53:11.231957+0000","last_became_active":"2024-09-10T07:53:10.223373+0000","last_became_peered":"2024-09-10T07:53:10.223373+0000","last_unstale":"2024-09-10T07:53:11.231957+0000","last_undegraded":"2024-09-10T07:53:11.231957+0000","last_fullsized":"2024-09-10T07:53:11.231957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233338+0000","last_change":"2024-09-10T07:53:11.256226+0000","last_active":"2024-09-10T07:53:12.233338+0000","last_peered":"2024-09-10T07:53:12.233338+0000","last_clean":"2024-09-10T07:53:12.233338+0000","last_became_active":"2024-09-10T07:53:11.247430+0000","last_became_peered":"2024-09-10T07:53:11.247430+0000","last_unstale":"2024-09-10T07:53:12.233338+0000","last_undegraded":"2024-09-10T07:53:12.233338+0000","last_fullsized":"2024-09-10T07:53:12.233338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:22.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.232748+0000","last_change":"2024-09-10T07:53:10.223678+0000","last_active":"2024-09-10T07:53:10.232748+0000","last_peered":"2024-09-10T07:53:10.232748+0000","last_clean":"2024-09-10T07:53:10.232748+0000","last_became_active":"2024-09-10T07:53:10.223390+0000","last_became_peered":"2024-09-10T07:53:10.223390+0000","last_unstale":"2024-09-10T07:53:10.232748+0000","last_undegraded":"2024-09-10T07:53:10.232748+0000","last_fullsized":"2024-09-10T07:53:10.232748+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:35.321389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.252550+0000","last_change":"2024-09-10T07:53:11.251829+0000","last_active":"2024-09-10T07:53:11.252550+0000","last_peered":"2024-09-10T07:53:11.252550+0000","last_clean":"2024-09-10T07:53:11.252550+0000","last_became_active":"2024-09-10T07:53:11.251547+0000","last_became_peered":"2024-09-10T07:53:11.251547+0000","last_unstale":"2024-09-10T07:53:11.252550+0000","last_undegraded":"2024-09-10T07:53:11.252550+0000","last_fullsized":"2024-09-10T07:53:11.252550+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:43.617740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271292+0000","last_change":"2024-09-10T07:53:10.232882+0000","last_active":"2024-09-10T07:53:11.271292+0000","last_peered":"2024-09-10T07:53:11.271292+0000","last_clean":"2024-09-10T07:53:11.271292+0000","last_became_active":"2024-09-10T07:53:10.232094+0000","last_became_peered":"2024-09-10T07:53:10.232094+0000","last_unstale":"2024-09-10T07:53:11.271292+0000","last_undegraded":"2024-09-10T07:53:11.271292+0000","last_fullsized":"2024-09-10T07:53:11.271292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.250788+0000","last_change":"2024-09-10T07:53:11.248994+0000","last_active":"2024-09-10T07:53:11.250788+0000","last_peered":"2024-09-10T07:53:11.250788+0000","last_clean":"2024-09-10T07:53:11.250788+0000","last_became_active":"2024-09-10T07:53:11.248544+0000","last_became_peered":"2024-09-10T07:53:11.248544+0000","last_unstale":"2024-09-10T07:53:11.250788+0000","last_undegraded":"2024-09-10T07:53:11.250788+0000","last_fullsized":"2024-09-10T07:53:11.250788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:07.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268745+0000","last_change":"2024-09-10T07:53:10.213371+0000","last_active":"2024-09-10T07:53:11.268745+0000","last_peered":"2024-09-10T07:53:11.268745+0000","last_clean":"2024-09-10T07:53:11.268745+0000","last_became_active":"2024-09-10T07:53:10.211436+0000","last_became_peered":"2024-09-10T07:53:10.211436+0000","last_unstale":"2024-09-10T07:53:11.268745+0000","last_undegraded":"2024-09-10T07:53:11.268745+0000","last_fullsized":"2024-09-10T07:53:11.268745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:54.890133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239996+0000","last_change":"2024-09-10T07:53:11.235023+0000","last_active":"2024-09-10T07:53:11.239996+0000","last_peered":"2024-09-10T07:53:11.239996+0000","last_clean":"2024-09-10T07:53:11.239996+0000","last_became_active":"2024-09-10T07:53:11.234748+0000","last_became_peered":"2024-09-10T07:53:11.234748+0000","last_unstale":"2024-09-10T07:53:11.239996+0000","last_undegraded":"2024-09-10T07:53:11.239996+0000","last_fullsized":"2024-09-10T07:53:11.239996+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.136947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235891+0000","last_change":"2024-09-10T07:53:10.232440+0000","last_active":"2024-09-10T07:53:10.235891+0000","last_peered":"2024-09-10T07:53:10.235891+0000","last_clean":"2024-09-10T07:53:10.235891+0000","last_became_active":"2024-09-10T07:53:10.229715+0000","last_became_peered":"2024-09-10T07:53:10.229715+0000","last_unstale":"2024-09-10T07:53:10.235891+0000","last_undegraded":"2024-09-10T07:53:10.235891+0000","last_fullsized":"2024-09-10T07:53:10.235891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:05.012224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212060+0000","last_change":"2024-09-10T07:53:10.212060+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212060+0000","last_undegraded":"2024-09-10T07:53:10.212060+0000","last_fullsized":"2024-09-10T07:53:10.212060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233676+0000","last_change":"2024-09-10T07:53:10.227747+0000","last_active":"2024-09-10T07:53:10.233676+0000","last_peered":"2024-09-10T07:53:10.233676+0000","last_clean":"2024-09-10T07:53:10.233676+0000","last_became_active":"2024-09-10T07:53:10.225865+0000","last_became_peered":"2024-09-10T07:53:10.225865+0000","last_unstale":"2024-09-10T07:53:10.233676+0000","last_undegraded":"2024-09-10T07:53:10.233676+0000","last_fullsized":"2024-09-10T07:53:10.233676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211732+0000","last_change":"2024-09-10T07:53:10.211732+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211732+0000","last_undegraded":"2024-09-10T07:53:10.211732+0000","last_fullsized":"2024-09-10T07:53:10.211732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.247861+0000","last_change":"2024-09-10T07:53:10.228787+0000","last_active":"2024-09-10T07:53:11.247861+0000","last_peered":"2024-09-10T07:53:11.247861+0000","last_clean":"2024-09-10T07:53:11.247861+0000","last_became_active":"2024-09-10T07:53:10.228445+0000","last_became_peered":"2024-09-10T07:53:10.228445+0000","last_unstale":"2024-09-10T07:53:11.247861+0000","last_undegraded":"2024-09-10T07:53:11.247861+0000","last_fullsized":"2024-09-10T07:53:11.247861+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:55.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211105+0000","last_change":"2024-09-10T07:53:10.211105+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211105+0000","last_undegraded":"2024-09-10T07:53:10.211105+0000","last_fullsized":"2024-09-10T07:53:10.211105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.245589+0000","last_change":"2024-09-10T07:53:10.228240+0000","last_active":"2024-09-10T07:53:11.245589+0000","last_peered":"2024-09-10T07:53:11.245589+0000","last_clean":"2024-09-10T07:53:11.245589+0000","last_became_active":"2024-09-10T07:53:10.227508+0000","last_became_peered":"2024-09-10T07:53:10.227508+0000","last_unstale":"2024-09-10T07:53:11.245589+0000","last_undegraded":"2024-09-10T07:53:11.245589+0000","last_fullsized":"2024-09-10T07:53:11.245589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:39.308304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261353+0000","last_change":"2024-09-10T07:53:11.259531+0000","last_active":"2024-09-10T07:53:11.261353+0000","last_peered":"2024-09-10T07:53:11.261353+0000","last_clean":"2024-09-10T07:53:11.261353+0000","last_became_active":"2024-09-10T07:53:11.258000+0000","last_became_peered":"2024-09-10T07:53:11.258000+0000","last_unstale":"2024-09-10T07:53:11.261353+0000","last_undegraded":"2024-09-10T07:53:11.261353+0000","last_fullsized":"2024-09-10T07:53:11.261353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:06.014591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232682+0000","last_change":"2024-09-10T07:53:10.231034+0000","last_active":"2024-09-10T07:53:12.232682+0000","last_peered":"2024-09-10T07:53:12.232682+0000","last_clean":"2024-09-10T07:53:12.232682+0000","last_became_active":"2024-09-10T07:53:10.230111+0000","last_became_peered":"2024-09-10T07:53:10.230111+0000","last_unstale":"2024-09-10T07:53:12.232682+0000","last_undegraded":"2024-09-10T07:53:12.232682+0000","last_fullsized":"2024-09-10T07:53:12.232682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:58.029264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217413+0000","last_change":"2024-09-10T07:53:10.217413+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217413+0000","last_undegraded":"2024-09-10T07:53:10.217413+0000","last_fullsized":"2024-09-10T07:53:10.217413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267444+0000","last_change":"2024-09-10T07:53:10.222067+0000","last_active":"2024-09-10T07:53:11.267444+0000","last_peered":"2024-09-10T07:53:11.267444+0000","last_clean":"2024-09-10T07:53:11.267444+0000","last_became_active":"2024-09-10T07:53:10.221728+0000","last_became_peered":"2024-09-10T07:53:10.221728+0000","last_unstale":"2024-09-10T07:53:11.267444+0000","last_undegraded":"2024-09-10T07:53:11.267444+0000","last_fullsized":"2024-09-10T07:53:11.267444+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212195+0000","last_change":"2024-09-10T07:53:10.212195+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212195+0000","last_undegraded":"2024-09-10T07:53:10.212195+0000","last_fullsized":"2024-09-10T07:53:10.212195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236035+0000","last_change":"2024-09-10T07:53:10.232887+0000","last_active":"2024-09-10T07:53:10.236035+0000","last_peered":"2024-09-10T07:53:10.236035+0000","last_clean":"2024-09-10T07:53:10.236035+0000","last_became_active":"2024-09-10T07:53:10.230235+0000","last_became_peered":"2024-09-10T07:53:10.230235+0000","last_unstale":"2024-09-10T07:53:10.236035+0000","last_undegraded":"2024-09-10T07:53:10.236035+0000","last_fullsized":"2024-09-10T07:53:10.236035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:14.807864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233395+0000","last_change":"2024-09-10T07:53:11.254067+0000","last_active":"2024-09-10T07:53:12.233395+0000","last_peered":"2024-09-10T07:53:12.233395+0000","last_clean":"2024-09-10T07:53:12.233395+0000","last_became_active":"2024-09-10T07:53:11.253708+0000","last_became_peered":"2024-09-10T07:53:11.253708+0000","last_unstale":"2024-09-10T07:53:12.233395+0000","last_undegraded":"2024-09-10T07:53:12.233395+0000","last_fullsized":"2024-09-10T07:53:12.233395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:01.851731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235237+0000","last_change":"2024-09-10T07:53:10.223013+0000","last_active":"2024-09-10T07:53:10.235237+0000","last_peered":"2024-09-10T07:53:10.235237+0000","last_clean":"2024-09-10T07:53:10.235237+0000","last_became_active":"2024-09-10T07:53:10.222053+0000","last_became_peered":"2024-09-10T07:53:10.222053+0000","last_unstale":"2024-09-10T07:53:10.235237+0000","last_undegraded":"2024-09-10T07:53:10.235237+0000","last_fullsized":"2024-09-10T07:53:10.235237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:42.615374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258362+0000","last_change":"2024-09-10T07:53:11.247894+0000","last_active":"2024-09-10T07:53:11.258362+0000","last_peered":"2024-09-10T07:53:11.258362+0000","last_clean":"2024-09-10T07:53:11.258362+0000","last_became_active":"2024-09-10T07:53:11.247569+0000","last_became_peered":"2024-09-10T07:53:11.247569+0000","last_unstale":"2024-09-10T07:53:11.258362+0000","last_undegraded":"2024-09-10T07:53:11.258362+0000","last_fullsized":"2024-09-10T07:53:11.258362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:37.700469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248875+0000","last_change":"2024-09-10T07:53:10.223877+0000","last_active":"2024-09-10T07:53:11.248875+0000","last_peered":"2024-09-10T07:53:11.248875+0000","last_clean":"2024-09-10T07:53:11.248875+0000","last_became_active":"2024-09-10T07:53:10.222951+0000","last_became_peered":"2024-09-10T07:53:10.222951+0000","last_unstale":"2024-09-10T07:53:11.248875+0000","last_undegraded":"2024-09-10T07:53:11.248875+0000","last_fullsized":"2024-09-10T07:53:11.248875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:16.490800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211556+0000","last_change":"2024-09-10T07:53:10.211556+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211556+0000","last_undegraded":"2024-09-10T07:53:10.211556+0000","last_fullsized":"2024-09-10T07:53:10.211556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233746+0000","last_change":"2024-09-10T07:53:10.227674+0000","last_active":"2024-09-10T07:53:12.233746+0000","last_peered":"2024-09-10T07:53:12.233746+0000","last_clean":"2024-09-10T07:53:12.233746+0000","last_became_active":"2024-09-10T07:53:10.227321+0000","last_became_peered":"2024-09-10T07:53:10.227321+0000","last_unstale":"2024-09-10T07:53:12.233746+0000","last_undegraded":"2024-09-10T07:53:12.233746+0000","last_fullsized":"2024-09-10T07:53:12.233746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:58.837406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260406+0000","last_change":"2024-09-10T07:53:11.252691+0000","last_active":"2024-09-10T07:53:11.260406+0000","last_peered":"2024-09-10T07:53:11.260406+0000","last_clean":"2024-09-10T07:53:11.260406+0000","last_became_active":"2024-09-10T07:53:11.252471+0000","last_became_peered":"2024-09-10T07:53:11.252471+0000","last_unstale":"2024-09-10T07:53:11.260406+0000","last_undegraded":"2024-09-10T07:53:11.260406+0000","last_fullsized":"2024-09-10T07:53:11.260406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:41.229077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248309+0000","last_change":"2024-09-10T07:53:10.230718+0000","last_active":"2024-09-10T07:53:11.248309+0000","last_peered":"2024-09-10T07:53:11.248309+0000","last_clean":"2024-09-10T07:53:11.248309+0000","last_became_active":"2024-09-10T07:53:10.230504+0000","last_became_peered":"2024-09-10T07:53:10.230504+0000","last_unstale":"2024-09-10T07:53:11.248309+0000","last_undegraded":"2024-09-10T07:53:11.248309+0000","last_fullsized":"2024-09-10T07:53:11.248309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:14.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212681+0000","last_change":"2024-09-10T07:53:10.212681+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212681+0000","last_undegraded":"2024-09-10T07:53:10.212681+0000","last_fullsized":"2024-09-10T07:53:10.212681+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236579+0000","last_change":"2024-09-10T07:53:10.234562+0000","last_active":"2024-09-10T07:53:10.236579+0000","last_peered":"2024-09-10T07:53:10.236579+0000","last_clean":"2024-09-10T07:53:10.236579+0000","last_became_active":"2024-09-10T07:53:10.234276+0000","last_became_peered":"2024-09-10T07:53:10.234276+0000","last_unstale":"2024-09-10T07:53:10.236579+0000","last_undegraded":"2024-09-10T07:53:10.236579+0000","last_fullsized":"2024-09-10T07:53:10.236579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268617+0000","last_change":"2024-09-10T07:53:11.264027+0000","last_active":"2024-09-10T07:53:11.268617+0000","last_peered":"2024-09-10T07:53:11.268617+0000","last_clean":"2024-09-10T07:53:11.268617+0000","last_became_active":"2024-09-10T07:53:11.263741+0000","last_became_peered":"2024-09-10T07:53:11.263741+0000","last_unstale":"2024-09-10T07:53:11.268617+0000","last_undegraded":"2024-09-10T07:53:11.268617+0000","last_fullsized":"2024-09-10T07:53:11.268617+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:04.728438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232795+0000","last_change":"2024-09-10T07:53:10.224089+0000","last_active":"2024-09-10T07:53:12.232795+0000","last_peered":"2024-09-10T07:53:12.232795+0000","last_clean":"2024-09-10T07:53:12.232795+0000","last_became_active":"2024-09-10T07:53:10.222933+0000","last_became_peered":"2024-09-10T07:53:10.222933+0000","last_unstale":"2024-09-10T07:53:12.232795+0000","last_undegraded":"2024-09-10T07:53:12.232795+0000","last_fullsized":"2024-09-10T07:53:12.232795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:01.604342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234021+0000","last_change":"2024-09-10T07:53:11.266735+0000","last_active":"2024-09-10T07:53:12.234021+0000","last_peered":"2024-09-10T07:53:12.234021+0000","last_clean":"2024-09-10T07:53:12.234021+0000","last_became_active":"2024-09-10T07:53:11.266398+0000","last_became_peered":"2024-09-10T07:53:11.266398+0000","last_unstale":"2024-09-10T07:53:12.234021+0000","last_undegraded":"2024-09-10T07:53:12.234021+0000","last_fullsized":"2024-09-10T07:53:12.234021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:05.878447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271008+0000","last_change":"2024-09-10T07:53:10.235291+0000","last_active":"2024-09-10T07:53:11.271008+0000","last_peered":"2024-09-10T07:53:11.271008+0000","last_clean":"2024-09-10T07:53:11.271008+0000","last_became_active":"2024-09-10T07:53:10.235015+0000","last_became_peered":"2024-09-10T07:53:10.235015+0000","last_unstale":"2024-09-10T07:53:11.271008+0000","last_undegraded":"2024-09-10T07:53:11.271008+0000","last_fullsized":"2024-09-10T07:53:11.271008+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":314,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231633+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:53:11.231633+0000","last_peered":"2024-09-10T07:53:11.231633+0000","last_clean":"2024-09-10T07:53:11.231633+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:53:11.231633+0000","last_undegraded":"2024-09-10T07:53:11.231633+0000","last_fullsized":"2024-09-10T07:53:11.231633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476822,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476822,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":1127976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1113202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1241088,"data_stored":536466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:53:15.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:53:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.063+0000 7f2ecdc00640 1 Processor -- start 2024-09-10T07:53:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.063+0000 7f2ecdc00640 1 -- start start 2024-09-10T07:53:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ecdc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ec815e480 0x7f2ec80d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ecdc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ecdc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec80d1f20 con 0x7f2ec815e860 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ecdc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec80d2fc0 con 0x7f2ec815e480 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ecdc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80d2200 con 0x7f2ec80b71c0 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ec7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ec815e480 0x7f2ec80d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ec7400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ec815e860 0x7f2ec80b6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56706/0 (socket says 172.21.15.3:56706) 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.064+0000 7f2ec7400640 1 -- 172.21.15.3:0/2025871301 learned_addr learned my addr 172.21.15.3:0/2025871301 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ecd200640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ec7e00640 1 -- 172.21.15.3:0/2025871301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 msgr2=0x7f2ec806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ec7e00640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ec7e00640 1 -- 172.21.15.3:0/2025871301 >> v1:172.21.15.3:6789/0 conn(0x7f2ec815e860 legacy=0x7f2ec80b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ec6a00640 1 -- 172.21.15.3:0/2025871301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904885742 0 0) 0x7f2ec80d1f20 con 0x7f2ec815e860 2024-09-10T07:53:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.065+0000 7f2ec7e00640 1 -- 172.21.15.3:0/2025871301 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80d24e0 con 0x7f2ec815e480 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.066+0000 7f2ec7e00640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e480 0x7f2ec80d15a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80097a0 tx=0x7f2eb802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7a0cec6fe689ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.066+0000 7f2ec6a00640 1 -- 172.21.15.3:0/2025871301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037070 con 0x7f2ec815e480 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.066+0000 7f2ec6a00640 1 -- 172.21.15.3:0/2025871301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb802ae50 con 0x7f2ec815e480 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.066+0000 7f2ec6a00640 1 -- 172.21.15.3:0/2025871301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8032660 con 0x7f2ec815e480 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 -- 172.21.15.3:0/2025871301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e480 msgr2=0x7f2ec80d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e480 0x7f2ec80d15a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80097a0 tx=0x7f2eb802a100 comp rx=0 tx=0).stop 2024-09-10T07:53:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 -- 172.21.15.3:0/2025871301 shutdown_connections 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/2025871301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e480 0x7f2ec80d15a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 -- 172.21.15.3:0/2025871301 >> 172.21.15.3:0/2025871301 conn(0x7f2ec806f210 msgr2=0x7f2ec806f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 -- 172.21.15.3:0/2025871301 shutdown_connections 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.067+0000 7f2ecdc00640 1 -- 172.21.15.3:0/2025871301 wait complete. 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.068+0000 7f2ecdc00640 1 Processor -- start 2024-09-10T07:53:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 -- start start 2024-09-10T07:53:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec8159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec815e480 0x7f2ec8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80d1c60 con 0x7f2ec815e860 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80d1ae0 con 0x7f2ec815e480 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.069+0000 7f2ecdc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec8073d10 con 0x7f2ec80b71c0 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35914/0 (socket says 172.21.15.3:35914) 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 -- 172.21.15.3:0/3830458665 learned_addr learned my addr 172.21.15.3:0/3830458665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ec7400640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec815e480 0x7f2ec8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ec7e00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 -- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 msgr2=0x7f2ec8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec8159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 -- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec815e480 msgr2=0x7f2ec8142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec815e480 0x7f2ec8142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 -- 172.21.15.3:0/3830458665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec8143360 con 0x7f2ec815e860 2024-09-10T07:53:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.070+0000 7f2ecd200640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0007e60 tx=0x7f2ec000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.071+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0014420 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.071+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0019070 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.071+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0014820 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.071+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec8143640 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.071+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec815b290 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.072+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec81407a0 con 0x7f2ec815e860 2024-09-10T07:53:16.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.073+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ec00149c0 con 0x7f2ec815e860 2024-09-10T07:53:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.074+0000 7f2ebfe00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.074+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2ec008b690 con 0x7f2ec815e860 2024-09-10T07:53:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.074+0000 7f2ec7e00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.075+0000 7f2ec7e00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 0x7f2ea806e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8034000 tx=0x7f2eb802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.078+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ec81407a0 con 0x7f2ec815e860 2024-09-10T07:53:16.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.259+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ec815b460 con 0x7f2ea806be30 2024-09-10T07:53:16.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.265+0000 7f2ebfe00640 1 -- 172.21.15.3:0/3830458665 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250124 (secure 0 0 0) 0x7f2ec815b460 con 0x7f2ea806be30 2024-09-10T07:53:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:16.269 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 msgr2=0x7f2ea806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 0x7f2ea806e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8034000 tx=0x7f2eb802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 msgr2=0x7f2ec8140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0007e60 tx=0x7f2ec000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 shutdown_connections 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ec815e860 0x7f2ec8140a80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec815e480 0x7f2ec8142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 --2- 172.21.15.3:0/3830458665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ec80b71c0 0x7f2ec8159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.269+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 >> 172.21.15.3:0/3830458665 conn(0x7f2ec806f210 msgr2=0x7f2ec806f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.270+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 shutdown_connections 2024-09-10T07:53:16.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.270+0000 7f2ecdc00640 1 -- 172.21.15.3:0/3830458665 wait complete. 2024-09-10T07:53:16.285 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-10T07:53:15.580430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264436,"kb_used_data":10740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710284,"statfs":{"total":773094113280,"available":772823330816,"internally_reserved":0,"allocated":10997760,"data_stored":5410639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000952"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214073+0000","last_change":"2024-09-10T07:53:10.214073+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214073+0000","last_undegraded":"2024-09-10T07:53:10.214073+0000","last_fullsized":"2024-09-10T07:53:10.214073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215296+0000","last_change":"2024-09-10T07:53:10.215296+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215296+0000","last_undegraded":"2024-09-10T07:53:10.215296+0000","last_fullsized":"2024-09-10T07:53:10.215296+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233484+0000","last_change":"2024-09-10T07:53:11.252106+0000","last_active":"2024-09-10T07:53:12.233484+0000","last_peered":"2024-09-10T07:53:12.233484+0000","last_clean":"2024-09-10T07:53:12.233484+0000","last_became_active":"2024-09-10T07:53:11.251807+0000","last_became_peered":"2024-09-10T07:53:11.251807+0000","last_unstale":"2024-09-10T07:53:12.233484+0000","last_undegraded":"2024-09-10T07:53:12.233484+0000","last_fullsized":"2024-09-10T07:53:12.233484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:01.780830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215936+0000","last_change":"2024-09-10T07:53:10.215936+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215936+0000","last_undegraded":"2024-09-10T07:53:10.215936+0000","last_fullsized":"2024-09-10T07:53:10.215936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.237820+0000","last_change":"2024-09-10T07:53:11.234549+0000","last_active":"2024-09-10T07:53:11.237820+0000","last_peered":"2024-09-10T07:53:11.237820+0000","last_clean":"2024-09-10T07:53:11.237820+0000","last_became_active":"2024-09-10T07:53:11.234201+0000","last_became_peered":"2024-09-10T07:53:11.234201+0000","last_unstale":"2024-09-10T07:53:11.237820+0000","last_undegraded":"2024-09-10T07:53:11.237820+0000","last_fullsized":"2024-09-10T07:53:11.237820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:49.298649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.243698+0000","last_change":"2024-09-10T07:53:11.236870+0000","last_active":"2024-09-10T07:53:11.243698+0000","last_peered":"2024-09-10T07:53:11.243698+0000","last_clean":"2024-09-10T07:53:11.243698+0000","last_became_active":"2024-09-10T07:53:11.235843+0000","last_became_peered":"2024-09-10T07:53:11.235843+0000","last_unstale":"2024-09-10T07:53:11.243698+0000","last_undegraded":"2024-09-10T07:53:11.243698+0000","last_fullsized":"2024-09-10T07:53:11.243698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.860697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261073+0000","last_change":"2024-09-10T07:53:11.255394+0000","last_active":"2024-09-10T07:53:11.261073+0000","last_peered":"2024-09-10T07:53:11.261073+0000","last_clean":"2024-09-10T07:53:11.261073+0000","last_became_active":"2024-09-10T07:53:11.254136+0000","last_became_peered":"2024-09-10T07:53:11.254136+0000","last_unstale":"2024-09-10T07:53:11.261073+0000","last_undegraded":"2024-09-10T07:53:11.261073+0000","last_fullsized":"2024-09-10T07:53:11.261073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:35.472507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233314+0000","last_change":"2024-09-10T07:53:11.250032+0000","last_active":"2024-09-10T07:53:12.233314+0000","last_peered":"2024-09-10T07:53:12.233314+0000","last_clean":"2024-09-10T07:53:12.233314+0000","last_became_active":"2024-09-10T07:53:11.249626+0000","last_became_peered":"2024-09-10T07:53:11.249626+0000","last_unstale":"2024-09-10T07:53:12.233314+0000","last_undegraded":"2024-09-10T07:53:12.233314+0000","last_fullsized":"2024-09-10T07:53:12.233314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.283338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248152+0000","last_change":"2024-09-10T07:53:11.243615+0000","last_active":"2024-09-10T07:53:11.248152+0000","last_peered":"2024-09-10T07:53:11.248152+0000","last_clean":"2024-09-10T07:53:11.248152+0000","last_became_active":"2024-09-10T07:53:11.243351+0000","last_became_peered":"2024-09-10T07:53:11.243351+0000","last_unstale":"2024-09-10T07:53:11.248152+0000","last_undegraded":"2024-09-10T07:53:11.248152+0000","last_fullsized":"2024-09-10T07:53:11.248152+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:53.710317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268481+0000","last_change":"2024-09-10T07:53:11.267004+0000","last_active":"2024-09-10T07:53:11.268481+0000","last_peered":"2024-09-10T07:53:11.268481+0000","last_clean":"2024-09-10T07:53:11.268481+0000","last_became_active":"2024-09-10T07:53:11.266699+0000","last_became_peered":"2024-09-10T07:53:11.266699+0000","last_unstale":"2024-09-10T07:53:11.268481+0000","last_undegraded":"2024-09-10T07:53:11.268481+0000","last_fullsized":"2024-09-10T07:53:11.268481+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:08.715930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239275+0000","last_change":"2024-09-10T07:53:11.236475+0000","last_active":"2024-09-10T07:53:11.239275+0000","last_peered":"2024-09-10T07:53:11.239275+0000","last_clean":"2024-09-10T07:53:11.239275+0000","last_became_active":"2024-09-10T07:53:11.235672+0000","last_became_peered":"2024-09-10T07:53:11.235672+0000","last_unstale":"2024-09-10T07:53:11.239275+0000","last_undegraded":"2024-09-10T07:53:11.239275+0000","last_fullsized":"2024-09-10T07:53:11.239275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:31:33.473232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269070+0000","last_change":"2024-09-10T07:53:11.267139+0000","last_active":"2024-09-10T07:53:11.269070+0000","last_peered":"2024-09-10T07:53:11.269070+0000","last_clean":"2024-09-10T07:53:11.269070+0000","last_became_active":"2024-09-10T07:53:11.266829+0000","last_became_peered":"2024-09-10T07:53:11.266829+0000","last_unstale":"2024-09-10T07:53:11.269070+0000","last_undegraded":"2024-09-10T07:53:11.269070+0000","last_fullsized":"2024-09-10T07:53:11.269070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.218726+0000","last_change":"2024-09-10T07:53:10.218726+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.218726+0000","last_undegraded":"2024-09-10T07:53:10.218726+0000","last_fullsized":"2024-09-10T07:53:10.218726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212879+0000","last_change":"2024-09-10T07:53:10.212879+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212879+0000","last_undegraded":"2024-09-10T07:53:10.212879+0000","last_fullsized":"2024-09-10T07:53:10.212879+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211714+0000","last_change":"2024-09-10T07:53:10.211714+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211714+0000","last_undegraded":"2024-09-10T07:53:10.211714+0000","last_fullsized":"2024-09-10T07:53:10.211714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260661+0000","last_change":"2024-09-10T07:53:11.254673+0000","last_active":"2024-09-10T07:53:11.260661+0000","last_peered":"2024-09-10T07:53:11.260661+0000","last_clean":"2024-09-10T07:53:11.260661+0000","last_became_active":"2024-09-10T07:53:11.251614+0000","last_became_peered":"2024-09-10T07:53:11.251614+0000","last_unstale":"2024-09-10T07:53:11.260661+0000","last_undegraded":"2024-09-10T07:53:11.260661+0000","last_fullsized":"2024-09-10T07:53:11.260661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:50.934355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212721+0000","last_change":"2024-09-10T07:53:10.212721+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212721+0000","last_undegraded":"2024-09-10T07:53:10.212721+0000","last_fullsized":"2024-09-10T07:53:10.212721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214734+0000","last_change":"2024-09-10T07:53:10.214734+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214734+0000","last_undegraded":"2024-09-10T07:53:10.214734+0000","last_fullsized":"2024-09-10T07:53:10.214734+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214070+0000","last_change":"2024-09-10T07:53:10.214070+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214070+0000","last_undegraded":"2024-09-10T07:53:10.214070+0000","last_fullsized":"2024-09-10T07:53:10.214070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239700+0000","last_change":"2024-09-10T07:53:11.234958+0000","last_active":"2024-09-10T07:53:11.239700+0000","last_peered":"2024-09-10T07:53:11.239700+0000","last_clean":"2024-09-10T07:53:11.239700+0000","last_became_active":"2024-09-10T07:53:11.234678+0000","last_became_peered":"2024-09-10T07:53:11.234678+0000","last_unstale":"2024-09-10T07:53:11.239700+0000","last_undegraded":"2024-09-10T07:53:11.239700+0000","last_fullsized":"2024-09-10T07:53:11.239700+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:47.186154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233477+0000","last_change":"2024-09-10T07:53:11.248147+0000","last_active":"2024-09-10T07:53:12.233477+0000","last_peered":"2024-09-10T07:53:12.233477+0000","last_clean":"2024-09-10T07:53:12.233477+0000","last_became_active":"2024-09-10T07:53:11.247829+0000","last_became_peered":"2024-09-10T07:53:11.247829+0000","last_unstale":"2024-09-10T07:53:12.233477+0000","last_undegraded":"2024-09-10T07:53:12.233477+0000","last_fullsized":"2024-09-10T07:53:12.233477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:54.042192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232852+0000","last_change":"2024-09-10T07:53:11.249206+0000","last_active":"2024-09-10T07:53:12.232852+0000","last_peered":"2024-09-10T07:53:12.232852+0000","last_clean":"2024-09-10T07:53:12.232852+0000","last_became_active":"2024-09-10T07:53:11.248315+0000","last_became_peered":"2024-09-10T07:53:11.248315+0000","last_unstale":"2024-09-10T07:53:12.232852+0000","last_undegraded":"2024-09-10T07:53:12.232852+0000","last_fullsized":"2024-09-10T07:53:12.232852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:31.145698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211200+0000","last_change":"2024-09-10T07:53:10.211200+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211200+0000","last_undegraded":"2024-09-10T07:53:10.211200+0000","last_fullsized":"2024-09-10T07:53:10.211200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.263806+0000","last_change":"2024-09-10T07:53:11.262775+0000","last_active":"2024-09-10T07:53:11.263806+0000","last_peered":"2024-09-10T07:53:11.263806+0000","last_clean":"2024-09-10T07:53:11.263806+0000","last_became_active":"2024-09-10T07:53:11.262438+0000","last_became_peered":"2024-09-10T07:53:11.262438+0000","last_unstale":"2024-09-10T07:53:11.263806+0000","last_undegraded":"2024-09-10T07:53:11.263806+0000","last_fullsized":"2024-09-10T07:53:11.263806+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:12.859790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.257372+0000","last_change":"2024-09-10T07:53:11.255844+0000","last_active":"2024-09-10T07:53:11.257372+0000","last_peered":"2024-09-10T07:53:11.257372+0000","last_clean":"2024-09-10T07:53:11.257372+0000","last_became_active":"2024-09-10T07:53:11.255548+0000","last_became_peered":"2024-09-10T07:53:11.255548+0000","last_unstale":"2024-09-10T07:53:11.257372+0000","last_undegraded":"2024-09-10T07:53:11.257372+0000","last_fullsized":"2024-09-10T07:53:11.257372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:06.407725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214175+0000","last_change":"2024-09-10T07:53:10.214175+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214175+0000","last_undegraded":"2024-09-10T07:53:10.214175+0000","last_fullsized":"2024-09-10T07:53:10.214175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248744+0000","last_change":"2024-09-10T07:53:11.246590+0000","last_active":"2024-09-10T07:53:11.248744+0000","last_peered":"2024-09-10T07:53:11.248744+0000","last_clean":"2024-09-10T07:53:11.248744+0000","last_became_active":"2024-09-10T07:53:11.245943+0000","last_became_peered":"2024-09-10T07:53:11.245943+0000","last_unstale":"2024-09-10T07:53:11.248744+0000","last_undegraded":"2024-09-10T07:53:11.248744+0000","last_fullsized":"2024-09-10T07:53:11.248744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258576+0000","last_change":"2024-09-10T07:53:11.256724+0000","last_active":"2024-09-10T07:53:11.258576+0000","last_peered":"2024-09-10T07:53:11.258576+0000","last_clean":"2024-09-10T07:53:11.258576+0000","last_became_active":"2024-09-10T07:53:11.256427+0000","last_became_peered":"2024-09-10T07:53:11.256427+0000","last_unstale":"2024-09-10T07:53:11.258576+0000","last_undegraded":"2024-09-10T07:53:11.258576+0000","last_fullsized":"2024-09-10T07:53:11.258576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:19.541855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214213+0000","last_change":"2024-09-10T07:53:10.214213+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214213+0000","last_undegraded":"2024-09-10T07:53:10.214213+0000","last_fullsized":"2024-09-10T07:53:10.214213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.224154+0000","last_change":"2024-09-10T07:53:10.224154+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.224154+0000","last_undegraded":"2024-09-10T07:53:10.224154+0000","last_fullsized":"2024-09-10T07:53:10.224154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234095+0000","last_change":"2024-09-10T07:53:11.256491+0000","last_active":"2024-09-10T07:53:12.234095+0000","last_peered":"2024-09-10T07:53:12.234095+0000","last_clean":"2024-09-10T07:53:12.234095+0000","last_became_active":"2024-09-10T07:53:11.255961+0000","last_became_peered":"2024-09-10T07:53:11.255961+0000","last_unstale":"2024-09-10T07:53:12.234095+0000","last_undegraded":"2024-09-10T07:53:12.234095+0000","last_fullsized":"2024-09-10T07:53:12.234095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:11.297862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214870+0000","last_change":"2024-09-10T07:53:10.214870+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214870+0000","last_undegraded":"2024-09-10T07:53:10.214870+0000","last_fullsized":"2024-09-10T07:53:10.214870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217871+0000","last_change":"2024-09-10T07:53:10.217871+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217871+0000","last_undegraded":"2024-09-10T07:53:10.217871+0000","last_fullsized":"2024-09-10T07:53:10.217871+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236893+0000","last_change":"2024-09-10T07:53:10.235925+0000","last_active":"2024-09-10T07:53:10.236893+0000","last_peered":"2024-09-10T07:53:10.236893+0000","last_clean":"2024-09-10T07:53:10.236893+0000","last_became_active":"2024-09-10T07:53:10.235350+0000","last_became_peered":"2024-09-10T07:53:10.235350+0000","last_unstale":"2024-09-10T07:53:10.236893+0000","last_undegraded":"2024-09-10T07:53:10.236893+0000","last_fullsized":"2024-09-10T07:53:10.236893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215028+0000","last_change":"2024-09-10T07:53:10.215028+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215028+0000","last_undegraded":"2024-09-10T07:53:10.215028+0000","last_fullsized":"2024-09-10T07:53:10.215028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267919+0000","last_change":"2024-09-10T07:53:10.232379+0000","last_active":"2024-09-10T07:53:11.267919+0000","last_peered":"2024-09-10T07:53:11.267919+0000","last_clean":"2024-09-10T07:53:11.267919+0000","last_became_active":"2024-09-10T07:53:10.231931+0000","last_became_peered":"2024-09-10T07:53:10.231931+0000","last_unstale":"2024-09-10T07:53:11.267919+0000","last_undegraded":"2024-09-10T07:53:11.267919+0000","last_fullsized":"2024-09-10T07:53:11.267919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:22:58.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215453+0000","last_change":"2024-09-10T07:53:10.215453+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215453+0000","last_undegraded":"2024-09-10T07:53:10.215453+0000","last_fullsized":"2024-09-10T07:53:10.215453+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233603+0000","last_change":"2024-09-10T07:53:10.227323+0000","last_active":"2024-09-10T07:53:12.233603+0000","last_peered":"2024-09-10T07:53:12.233603+0000","last_clean":"2024-09-10T07:53:12.233603+0000","last_became_active":"2024-09-10T07:53:10.227012+0000","last_became_peered":"2024-09-10T07:53:10.227012+0000","last_unstale":"2024-09-10T07:53:12.233603+0000","last_undegraded":"2024-09-10T07:53:12.233603+0000","last_fullsized":"2024-09-10T07:53:12.233603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:56.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.249667+0000","last_change":"2024-09-10T07:53:11.246551+0000","last_active":"2024-09-10T07:53:11.249667+0000","last_peered":"2024-09-10T07:53:11.249667+0000","last_clean":"2024-09-10T07:53:11.249667+0000","last_became_active":"2024-09-10T07:53:11.246300+0000","last_became_peered":"2024-09-10T07:53:11.246300+0000","last_unstale":"2024-09-10T07:53:11.249667+0000","last_undegraded":"2024-09-10T07:53:11.249667+0000","last_fullsized":"2024-09-10T07:53:11.249667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234150+0000","last_change":"2024-09-10T07:53:10.231193+0000","last_active":"2024-09-10T07:53:10.234150+0000","last_peered":"2024-09-10T07:53:10.234150+0000","last_clean":"2024-09-10T07:53:10.234150+0000","last_became_active":"2024-09-10T07:53:10.226288+0000","last_became_peered":"2024-09-10T07:53:10.226288+0000","last_unstale":"2024-09-10T07:53:10.234150+0000","last_undegraded":"2024-09-10T07:53:10.234150+0000","last_fullsized":"2024-09-10T07:53:10.234150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:31.899936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.241212+0000","last_change":"2024-09-10T07:53:11.235832+0000","last_active":"2024-09-10T07:53:11.241212+0000","last_peered":"2024-09-10T07:53:11.241212+0000","last_clean":"2024-09-10T07:53:11.241212+0000","last_became_active":"2024-09-10T07:53:11.235506+0000","last_became_peered":"2024-09-10T07:53:11.235506+0000","last_unstale":"2024-09-10T07:53:11.241212+0000","last_undegraded":"2024-09-10T07:53:11.241212+0000","last_fullsized":"2024-09-10T07:53:11.241212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:05.491743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233697+0000","last_change":"2024-09-10T07:53:10.228445+0000","last_active":"2024-09-10T07:53:10.233697+0000","last_peered":"2024-09-10T07:53:10.233697+0000","last_clean":"2024-09-10T07:53:10.233697+0000","last_became_active":"2024-09-10T07:53:10.227806+0000","last_became_peered":"2024-09-10T07:53:10.227806+0000","last_unstale":"2024-09-10T07:53:10.233697+0000","last_undegraded":"2024-09-10T07:53:10.233697+0000","last_fullsized":"2024-09-10T07:53:10.233697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:25:41.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.238929+0000","last_change":"2024-09-10T07:53:11.235298+0000","last_active":"2024-09-10T07:53:11.238929+0000","last_peered":"2024-09-10T07:53:11.238929+0000","last_clean":"2024-09-10T07:53:11.238929+0000","last_became_active":"2024-09-10T07:53:11.234953+0000","last_became_peered":"2024-09-10T07:53:11.234953+0000","last_unstale":"2024-09-10T07:53:11.238929+0000","last_undegraded":"2024-09-10T07:53:11.238929+0000","last_fullsized":"2024-09-10T07:53:11.238929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:14.411009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232971+0000","last_change":"2024-09-10T07:53:10.229226+0000","last_active":"2024-09-10T07:53:12.232971+0000","last_peered":"2024-09-10T07:53:12.232971+0000","last_clean":"2024-09-10T07:53:12.232971+0000","last_became_active":"2024-09-10T07:53:10.228904+0000","last_became_peered":"2024-09-10T07:53:10.228904+0000","last_unstale":"2024-09-10T07:53:12.232971+0000","last_undegraded":"2024-09-10T07:53:12.232971+0000","last_fullsized":"2024-09-10T07:53:12.232971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:29.688893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214033+0000","last_change":"2024-09-10T07:53:10.214033+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214033+0000","last_undegraded":"2024-09-10T07:53:10.214033+0000","last_fullsized":"2024-09-10T07:53:10.214033+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.229770+0000","last_change":"2024-09-10T07:53:10.228005+0000","last_active":"2024-09-10T07:53:11.229770+0000","last_peered":"2024-09-10T07:53:11.229770+0000","last_clean":"2024-09-10T07:53:11.229770+0000","last_became_active":"2024-09-10T07:53:10.227661+0000","last_became_peered":"2024-09-10T07:53:10.227661+0000","last_unstale":"2024-09-10T07:53:11.229770+0000","last_undegraded":"2024-09-10T07:53:11.229770+0000","last_fullsized":"2024-09-10T07:53:11.229770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:45.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.240934+0000","last_change":"2024-09-10T07:53:11.235959+0000","last_active":"2024-09-10T07:53:11.240934+0000","last_peered":"2024-09-10T07:53:11.240934+0000","last_clean":"2024-09-10T07:53:11.240934+0000","last_became_active":"2024-09-10T07:53:11.235687+0000","last_became_peered":"2024-09-10T07:53:11.235687+0000","last_unstale":"2024-09-10T07:53:11.240934+0000","last_undegraded":"2024-09-10T07:53:11.240934+0000","last_fullsized":"2024-09-10T07:53:11.240934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:00.667191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235952+0000","last_change":"2024-09-10T07:53:10.233196+0000","last_active":"2024-09-10T07:53:10.235952+0000","last_peered":"2024-09-10T07:53:10.235952+0000","last_clean":"2024-09-10T07:53:10.235952+0000","last_became_active":"2024-09-10T07:53:10.232510+0000","last_became_peered":"2024-09-10T07:53:10.232510+0000","last_unstale":"2024-09-10T07:53:10.235952+0000","last_undegraded":"2024-09-10T07:53:10.235952+0000","last_fullsized":"2024-09-10T07:53:10.235952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:11.857423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.213811+0000","last_change":"2024-09-10T07:53:10.213811+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.213811+0000","last_undegraded":"2024-09-10T07:53:10.213811+0000","last_fullsized":"2024-09-10T07:53:10.213811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234051+0000","last_change":"2024-09-10T07:53:10.213607+0000","last_active":"2024-09-10T07:53:12.234051+0000","last_peered":"2024-09-10T07:53:12.234051+0000","last_clean":"2024-09-10T07:53:12.234051+0000","last_became_active":"2024-09-10T07:53:10.213317+0000","last_became_peered":"2024-09-10T07:53:10.213317+0000","last_unstale":"2024-09-10T07:53:12.234051+0000","last_undegraded":"2024-09-10T07:53:12.234051+0000","last_fullsized":"2024-09-10T07:53:12.234051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:19.086909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.251040+0000","last_change":"2024-09-10T07:53:11.250003+0000","last_active":"2024-09-10T07:53:11.251040+0000","last_peered":"2024-09-10T07:53:11.251040+0000","last_clean":"2024-09-10T07:53:11.251040+0000","last_became_active":"2024-09-10T07:53:11.249735+0000","last_became_peered":"2024-09-10T07:53:11.249735+0000","last_unstale":"2024-09-10T07:53:11.251040+0000","last_undegraded":"2024-09-10T07:53:11.251040+0000","last_fullsized":"2024-09-10T07:53:11.251040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:53.440934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233009+0000","last_change":"2024-09-10T07:53:10.226296+0000","last_active":"2024-09-10T07:53:10.233009+0000","last_peered":"2024-09-10T07:53:10.233009+0000","last_clean":"2024-09-10T07:53:10.233009+0000","last_became_active":"2024-09-10T07:53:10.225937+0000","last_became_peered":"2024-09-10T07:53:10.225937+0000","last_unstale":"2024-09-10T07:53:10.233009+0000","last_undegraded":"2024-09-10T07:53:10.233009+0000","last_fullsized":"2024-09-10T07:53:10.233009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269185+0000","last_change":"2024-09-10T07:53:11.266667+0000","last_active":"2024-09-10T07:53:11.269185+0000","last_peered":"2024-09-10T07:53:11.269185+0000","last_clean":"2024-09-10T07:53:11.269185+0000","last_became_active":"2024-09-10T07:53:11.266400+0000","last_became_peered":"2024-09-10T07:53:11.266400+0000","last_unstale":"2024-09-10T07:53:11.269185+0000","last_undegraded":"2024-09-10T07:53:11.269185+0000","last_fullsized":"2024-09-10T07:53:11.269185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:39.890020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269567+0000","last_change":"2024-09-10T07:53:10.223127+0000","last_active":"2024-09-10T07:53:11.269567+0000","last_peered":"2024-09-10T07:53:11.269567+0000","last_clean":"2024-09-10T07:53:11.269567+0000","last_became_active":"2024-09-10T07:53:10.222729+0000","last_became_peered":"2024-09-10T07:53:10.222729+0000","last_unstale":"2024-09-10T07:53:11.269567+0000","last_undegraded":"2024-09-10T07:53:11.269567+0000","last_fullsized":"2024-09-10T07:53:11.269567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.995404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261874+0000","last_change":"2024-09-10T07:53:11.260871+0000","last_active":"2024-09-10T07:53:11.261874+0000","last_peered":"2024-09-10T07:53:11.261874+0000","last_clean":"2024-09-10T07:53:11.261874+0000","last_became_active":"2024-09-10T07:53:11.260613+0000","last_became_peered":"2024-09-10T07:53:11.260613+0000","last_unstale":"2024-09-10T07:53:11.261874+0000","last_undegraded":"2024-09-10T07:53:11.261874+0000","last_fullsized":"2024-09-10T07:53:11.261874+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:15.810230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.277289+0000","last_change":"2024-09-10T07:53:10.235054+0000","last_active":"2024-09-10T07:53:11.277289+0000","last_peered":"2024-09-10T07:53:11.277289+0000","last_clean":"2024-09-10T07:53:11.277289+0000","last_became_active":"2024-09-10T07:53:10.234764+0000","last_became_peered":"2024-09-10T07:53:10.234764+0000","last_unstale":"2024-09-10T07:53:11.277289+0000","last_undegraded":"2024-09-10T07:53:11.277289+0000","last_fullsized":"2024-09-10T07:53:11.277289+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212467+0000","last_change":"2024-09-10T07:53:10.212467+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212467+0000","last_undegraded":"2024-09-10T07:53:10.212467+0000","last_fullsized":"2024-09-10T07:53:10.212467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234989+0000","last_change":"2024-09-10T07:53:10.222685+0000","last_active":"2024-09-10T07:53:10.234989+0000","last_peered":"2024-09-10T07:53:10.234989+0000","last_clean":"2024-09-10T07:53:10.234989+0000","last_became_active":"2024-09-10T07:53:10.222344+0000","last_became_peered":"2024-09-10T07:53:10.222344+0000","last_unstale":"2024-09-10T07:53:10.234989+0000","last_undegraded":"2024-09-10T07:53:10.234989+0000","last_fullsized":"2024-09-10T07:53:10.234989+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:18.539489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233639+0000","last_change":"2024-09-10T07:53:11.257850+0000","last_active":"2024-09-10T07:53:12.233639+0000","last_peered":"2024-09-10T07:53:12.233639+0000","last_clean":"2024-09-10T07:53:12.233639+0000","last_became_active":"2024-09-10T07:53:11.255958+0000","last_became_peered":"2024-09-10T07:53:11.255958+0000","last_unstale":"2024-09-10T07:53:12.233639+0000","last_undegraded":"2024-09-10T07:53:12.233639+0000","last_fullsized":"2024-09-10T07:53:12.233639+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:05.576611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233362+0000","last_change":"2024-09-10T07:53:10.229874+0000","last_active":"2024-09-10T07:53:10.233362+0000","last_peered":"2024-09-10T07:53:10.233362+0000","last_clean":"2024-09-10T07:53:10.233362+0000","last_became_active":"2024-09-10T07:53:10.229597+0000","last_became_peered":"2024-09-10T07:53:10.229597+0000","last_unstale":"2024-09-10T07:53:10.233362+0000","last_undegraded":"2024-09-10T07:53:10.233362+0000","last_fullsized":"2024-09-10T07:53:10.233362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239396+0000","last_change":"2024-09-10T07:53:11.236857+0000","last_active":"2024-09-10T07:53:11.239396+0000","last_peered":"2024-09-10T07:53:11.239396+0000","last_clean":"2024-09-10T07:53:11.239396+0000","last_became_active":"2024-09-10T07:53:11.236085+0000","last_became_peered":"2024-09-10T07:53:11.236085+0000","last_unstale":"2024-09-10T07:53:11.239396+0000","last_undegraded":"2024-09-10T07:53:11.239396+0000","last_fullsized":"2024-09-10T07:53:11.239396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:25.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231957+0000","last_change":"2024-09-10T07:53:10.223641+0000","last_active":"2024-09-10T07:53:11.231957+0000","last_peered":"2024-09-10T07:53:11.231957+0000","last_clean":"2024-09-10T07:53:11.231957+0000","last_became_active":"2024-09-10T07:53:10.223373+0000","last_became_peered":"2024-09-10T07:53:10.223373+0000","last_unstale":"2024-09-10T07:53:11.231957+0000","last_undegraded":"2024-09-10T07:53:11.231957+0000","last_fullsized":"2024-09-10T07:53:11.231957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233338+0000","last_change":"2024-09-10T07:53:11.256226+0000","last_active":"2024-09-10T07:53:12.233338+0000","last_peered":"2024-09-10T07:53:12.233338+0000","last_clean":"2024-09-10T07:53:12.233338+0000","last_became_active":"2024-09-10T07:53:11.247430+0000","last_became_peered":"2024-09-10T07:53:11.247430+0000","last_unstale":"2024-09-10T07:53:12.233338+0000","last_undegraded":"2024-09-10T07:53:12.233338+0000","last_fullsized":"2024-09-10T07:53:12.233338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:22.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.232748+0000","last_change":"2024-09-10T07:53:10.223678+0000","last_active":"2024-09-10T07:53:10.232748+0000","last_peered":"2024-09-10T07:53:10.232748+0000","last_clean":"2024-09-10T07:53:10.232748+0000","last_became_active":"2024-09-10T07:53:10.223390+0000","last_became_peered":"2024-09-10T07:53:10.223390+0000","last_unstale":"2024-09-10T07:53:10.232748+0000","last_undegraded":"2024-09-10T07:53:10.232748+0000","last_fullsized":"2024-09-10T07:53:10.232748+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:35.321389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.252550+0000","last_change":"2024-09-10T07:53:11.251829+0000","last_active":"2024-09-10T07:53:11.252550+0000","last_peered":"2024-09-10T07:53:11.252550+0000","last_clean":"2024-09-10T07:53:11.252550+0000","last_became_active":"2024-09-10T07:53:11.251547+0000","last_became_peered":"2024-09-10T07:53:11.251547+0000","last_unstale":"2024-09-10T07:53:11.252550+0000","last_undegraded":"2024-09-10T07:53:11.252550+0000","last_fullsized":"2024-09-10T07:53:11.252550+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:43.617740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271292+0000","last_change":"2024-09-10T07:53:10.232882+0000","last_active":"2024-09-10T07:53:11.271292+0000","last_peered":"2024-09-10T07:53:11.271292+0000","last_clean":"2024-09-10T07:53:11.271292+0000","last_became_active":"2024-09-10T07:53:10.232094+0000","last_became_peered":"2024-09-10T07:53:10.232094+0000","last_unstale":"2024-09-10T07:53:11.271292+0000","last_undegraded":"2024-09-10T07:53:11.271292+0000","last_fullsized":"2024-09-10T07:53:11.271292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.250788+0000","last_change":"2024-09-10T07:53:11.248994+0000","last_active":"2024-09-10T07:53:11.250788+0000","last_peered":"2024-09-10T07:53:11.250788+0000","last_clean":"2024-09-10T07:53:11.250788+0000","last_became_active":"2024-09-10T07:53:11.248544+0000","last_became_peered":"2024-09-10T07:53:11.248544+0000","last_unstale":"2024-09-10T07:53:11.250788+0000","last_undegraded":"2024-09-10T07:53:11.250788+0000","last_fullsized":"2024-09-10T07:53:11.250788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:07.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268745+0000","last_change":"2024-09-10T07:53:10.213371+0000","last_active":"2024-09-10T07:53:11.268745+0000","last_peered":"2024-09-10T07:53:11.268745+0000","last_clean":"2024-09-10T07:53:11.268745+0000","last_became_active":"2024-09-10T07:53:10.211436+0000","last_became_peered":"2024-09-10T07:53:10.211436+0000","last_unstale":"2024-09-10T07:53:11.268745+0000","last_undegraded":"2024-09-10T07:53:11.268745+0000","last_fullsized":"2024-09-10T07:53:11.268745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:54.890133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239996+0000","last_change":"2024-09-10T07:53:11.235023+0000","last_active":"2024-09-10T07:53:11.239996+0000","last_peered":"2024-09-10T07:53:11.239996+0000","last_clean":"2024-09-10T07:53:11.239996+0000","last_became_active":"2024-09-10T07:53:11.234748+0000","last_became_peered":"2024-09-10T07:53:11.234748+0000","last_unstale":"2024-09-10T07:53:11.239996+0000","last_undegraded":"2024-09-10T07:53:11.239996+0000","last_fullsized":"2024-09-10T07:53:11.239996+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.136947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235891+0000","last_change":"2024-09-10T07:53:10.232440+0000","last_active":"2024-09-10T07:53:10.235891+0000","last_peered":"2024-09-10T07:53:10.235891+0000","last_clean":"2024-09-10T07:53:10.235891+0000","last_became_active":"2024-09-10T07:53:10.229715+0000","last_became_peered":"2024-09-10T07:53:10.229715+0000","last_unstale":"2024-09-10T07:53:10.235891+0000","last_undegraded":"2024-09-10T07:53:10.235891+0000","last_fullsized":"2024-09-10T07:53:10.235891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:05.012224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212060+0000","last_change":"2024-09-10T07:53:10.212060+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212060+0000","last_undegraded":"2024-09-10T07:53:10.212060+0000","last_fullsized":"2024-09-10T07:53:10.212060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233676+0000","last_change":"2024-09-10T07:53:10.227747+0000","last_active":"2024-09-10T07:53:10.233676+0000","last_peered":"2024-09-10T07:53:10.233676+0000","last_clean":"2024-09-10T07:53:10.233676+0000","last_became_active":"2024-09-10T07:53:10.225865+0000","last_became_peered":"2024-09-10T07:53:10.225865+0000","last_unstale":"2024-09-10T07:53:10.233676+0000","last_undegraded":"2024-09-10T07:53:10.233676+0000","last_fullsized":"2024-09-10T07:53:10.233676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211732+0000","last_change":"2024-09-10T07:53:10.211732+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211732+0000","last_undegraded":"2024-09-10T07:53:10.211732+0000","last_fullsized":"2024-09-10T07:53:10.211732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.247861+0000","last_change":"2024-09-10T07:53:10.228787+0000","last_active":"2024-09-10T07:53:11.247861+0000","last_peered":"2024-09-10T07:53:11.247861+0000","last_clean":"2024-09-10T07:53:11.247861+0000","last_became_active":"2024-09-10T07:53:10.228445+0000","last_became_peered":"2024-09-10T07:53:10.228445+0000","last_unstale":"2024-09-10T07:53:11.247861+0000","last_undegraded":"2024-09-10T07:53:11.247861+0000","last_fullsized":"2024-09-10T07:53:11.247861+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:55.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211105+0000","last_change":"2024-09-10T07:53:10.211105+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211105+0000","last_undegraded":"2024-09-10T07:53:10.211105+0000","last_fullsized":"2024-09-10T07:53:10.211105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.245589+0000","last_change":"2024-09-10T07:53:10.228240+0000","last_active":"2024-09-10T07:53:11.245589+0000","last_peered":"2024-09-10T07:53:11.245589+0000","last_clean":"2024-09-10T07:53:11.245589+0000","last_became_active":"2024-09-10T07:53:10.227508+0000","last_became_peered":"2024-09-10T07:53:10.227508+0000","last_unstale":"2024-09-10T07:53:11.245589+0000","last_undegraded":"2024-09-10T07:53:11.245589+0000","last_fullsized":"2024-09-10T07:53:11.245589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:39.308304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261353+0000","last_change":"2024-09-10T07:53:11.259531+0000","last_active":"2024-09-10T07:53:11.261353+0000","last_peered":"2024-09-10T07:53:11.261353+0000","last_clean":"2024-09-10T07:53:11.261353+0000","last_became_active":"2024-09-10T07:53:11.258000+0000","last_became_peered":"2024-09-10T07:53:11.258000+0000","last_unstale":"2024-09-10T07:53:11.261353+0000","last_undegraded":"2024-09-10T07:53:11.261353+0000","last_fullsized":"2024-09-10T07:53:11.261353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:06.014591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232682+0000","last_change":"2024-09-10T07:53:10.231034+0000","last_active":"2024-09-10T07:53:12.232682+0000","last_peered":"2024-09-10T07:53:12.232682+0000","last_clean":"2024-09-10T07:53:12.232682+0000","last_became_active":"2024-09-10T07:53:10.230111+0000","last_became_peered":"2024-09-10T07:53:10.230111+0000","last_unstale":"2024-09-10T07:53:12.232682+0000","last_undegraded":"2024-09-10T07:53:12.232682+0000","last_fullsized":"2024-09-10T07:53:12.232682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:58.029264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217413+0000","last_change":"2024-09-10T07:53:10.217413+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217413+0000","last_undegraded":"2024-09-10T07:53:10.217413+0000","last_fullsized":"2024-09-10T07:53:10.217413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267444+0000","last_change":"2024-09-10T07:53:10.222067+0000","last_active":"2024-09-10T07:53:11.267444+0000","last_peered":"2024-09-10T07:53:11.267444+0000","last_clean":"2024-09-10T07:53:11.267444+0000","last_became_active":"2024-09-10T07:53:10.221728+0000","last_became_peered":"2024-09-10T07:53:10.221728+0000","last_unstale":"2024-09-10T07:53:11.267444+0000","last_undegraded":"2024-09-10T07:53:11.267444+0000","last_fullsized":"2024-09-10T07:53:11.267444+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212195+0000","last_change":"2024-09-10T07:53:10.212195+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212195+0000","last_undegraded":"2024-09-10T07:53:10.212195+0000","last_fullsized":"2024-09-10T07:53:10.212195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236035+0000","last_change":"2024-09-10T07:53:10.232887+0000","last_active":"2024-09-10T07:53:10.236035+0000","last_peered":"2024-09-10T07:53:10.236035+0000","last_clean":"2024-09-10T07:53:10.236035+0000","last_became_active":"2024-09-10T07:53:10.230235+0000","last_became_peered":"2024-09-10T07:53:10.230235+0000","last_unstale":"2024-09-10T07:53:10.236035+0000","last_undegraded":"2024-09-10T07:53:10.236035+0000","last_fullsized":"2024-09-10T07:53:10.236035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:14.807864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233395+0000","last_change":"2024-09-10T07:53:11.254067+0000","last_active":"2024-09-10T07:53:12.233395+0000","last_peered":"2024-09-10T07:53:12.233395+0000","last_clean":"2024-09-10T07:53:12.233395+0000","last_became_active":"2024-09-10T07:53:11.253708+0000","last_became_peered":"2024-09-10T07:53:11.253708+0000","last_unstale":"2024-09-10T07:53:12.233395+0000","last_undegraded":"2024-09-10T07:53:12.233395+0000","last_fullsized":"2024-09-10T07:53:12.233395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:01.851731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235237+0000","last_change":"2024-09-10T07:53:10.223013+0000","last_active":"2024-09-10T07:53:10.235237+0000","last_peered":"2024-09-10T07:53:10.235237+0000","last_clean":"2024-09-10T07:53:10.235237+0000","last_became_active":"2024-09-10T07:53:10.222053+0000","last_became_peered":"2024-09-10T07:53:10.222053+0000","last_unstale":"2024-09-10T07:53:10.235237+0000","last_undegraded":"2024-09-10T07:53:10.235237+0000","last_fullsized":"2024-09-10T07:53:10.235237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:42.615374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258362+0000","last_change":"2024-09-10T07:53:11.247894+0000","last_active":"2024-09-10T07:53:11.258362+0000","last_peered":"2024-09-10T07:53:11.258362+0000","last_clean":"2024-09-10T07:53:11.258362+0000","last_became_active":"2024-09-10T07:53:11.247569+0000","last_became_peered":"2024-09-10T07:53:11.247569+0000","last_unstale":"2024-09-10T07:53:11.258362+0000","last_undegraded":"2024-09-10T07:53:11.258362+0000","last_fullsized":"2024-09-10T07:53:11.258362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:37.700469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248875+0000","last_change":"2024-09-10T07:53:10.223877+0000","last_active":"2024-09-10T07:53:11.248875+0000","last_peered":"2024-09-10T07:53:11.248875+0000","last_clean":"2024-09-10T07:53:11.248875+0000","last_became_active":"2024-09-10T07:53:10.222951+0000","last_became_peered":"2024-09-10T07:53:10.222951+0000","last_unstale":"2024-09-10T07:53:11.248875+0000","last_undegraded":"2024-09-10T07:53:11.248875+0000","last_fullsized":"2024-09-10T07:53:11.248875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:16.490800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211556+0000","last_change":"2024-09-10T07:53:10.211556+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211556+0000","last_undegraded":"2024-09-10T07:53:10.211556+0000","last_fullsized":"2024-09-10T07:53:10.211556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233746+0000","last_change":"2024-09-10T07:53:10.227674+0000","last_active":"2024-09-10T07:53:12.233746+0000","last_peered":"2024-09-10T07:53:12.233746+0000","last_clean":"2024-09-10T07:53:12.233746+0000","last_became_active":"2024-09-10T07:53:10.227321+0000","last_became_peered":"2024-09-10T07:53:10.227321+0000","last_unstale":"2024-09-10T07:53:12.233746+0000","last_undegraded":"2024-09-10T07:53:12.233746+0000","last_fullsized":"2024-09-10T07:53:12.233746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:58.837406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260406+0000","last_change":"2024-09-10T07:53:11.252691+0000","last_active":"2024-09-10T07:53:11.260406+0000","last_peered":"2024-09-10T07:53:11.260406+0000","last_clean":"2024-09-10T07:53:11.260406+0000","last_became_active":"2024-09-10T07:53:11.252471+0000","last_became_peered":"2024-09-10T07:53:11.252471+0000","last_unstale":"2024-09-10T07:53:11.260406+0000","last_undegraded":"2024-09-10T07:53:11.260406+0000","last_fullsized":"2024-09-10T07:53:11.260406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:41.229077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248309+0000","last_change":"2024-09-10T07:53:10.230718+0000","last_active":"2024-09-10T07:53:11.248309+0000","last_peered":"2024-09-10T07:53:11.248309+0000","last_clean":"2024-09-10T07:53:11.248309+0000","last_became_active":"2024-09-10T07:53:10.230504+0000","last_became_peered":"2024-09-10T07:53:10.230504+0000","last_unstale":"2024-09-10T07:53:11.248309+0000","last_undegraded":"2024-09-10T07:53:11.248309+0000","last_fullsized":"2024-09-10T07:53:11.248309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:14.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212681+0000","last_change":"2024-09-10T07:53:10.212681+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212681+0000","last_undegraded":"2024-09-10T07:53:10.212681+0000","last_fullsized":"2024-09-10T07:53:10.212681+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236579+0000","last_change":"2024-09-10T07:53:10.234562+0000","last_active":"2024-09-10T07:53:10.236579+0000","last_peered":"2024-09-10T07:53:10.236579+0000","last_clean":"2024-09-10T07:53:10.236579+0000","last_became_active":"2024-09-10T07:53:10.234276+0000","last_became_peered":"2024-09-10T07:53:10.234276+0000","last_unstale":"2024-09-10T07:53:10.236579+0000","last_undegraded":"2024-09-10T07:53:10.236579+0000","last_fullsized":"2024-09-10T07:53:10.236579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268617+0000","last_change":"2024-09-10T07:53:11.264027+0000","last_active":"2024-09-10T07:53:11.268617+0000","last_peered":"2024-09-10T07:53:11.268617+0000","last_clean":"2024-09-10T07:53:11.268617+0000","last_became_active":"2024-09-10T07:53:11.263741+0000","last_became_peered":"2024-09-10T07:53:11.263741+0000","last_unstale":"2024-09-10T07:53:11.268617+0000","last_undegraded":"2024-09-10T07:53:11.268617+0000","last_fullsized":"2024-09-10T07:53:11.268617+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:04.728438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232795+0000","last_change":"2024-09-10T07:53:10.224089+0000","last_active":"2024-09-10T07:53:12.232795+0000","last_peered":"2024-09-10T07:53:12.232795+0000","last_clean":"2024-09-10T07:53:12.232795+0000","last_became_active":"2024-09-10T07:53:10.222933+0000","last_became_peered":"2024-09-10T07:53:10.222933+0000","last_unstale":"2024-09-10T07:53:12.232795+0000","last_undegraded":"2024-09-10T07:53:12.232795+0000","last_fullsized":"2024-09-10T07:53:12.232795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:01.604342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234021+0000","last_change":"2024-09-10T07:53:11.266735+0000","last_active":"2024-09-10T07:53:12.234021+0000","last_peered":"2024-09-10T07:53:12.234021+0000","last_clean":"2024-09-10T07:53:12.234021+0000","last_became_active":"2024-09-10T07:53:11.266398+0000","last_became_peered":"2024-09-10T07:53:11.266398+0000","last_unstale":"2024-09-10T07:53:12.234021+0000","last_undegraded":"2024-09-10T07:53:12.234021+0000","last_fullsized":"2024-09-10T07:53:12.234021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:05.878447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271008+0000","last_change":"2024-09-10T07:53:10.235291+0000","last_active":"2024-09-10T07:53:11.271008+0000","last_peered":"2024-09-10T07:53:11.271008+0000","last_clean":"2024-09-10T07:53:11.271008+0000","last_became_active":"2024-09-10T07:53:10.235015+0000","last_became_peered":"2024-09-10T07:53:10.235015+0000","last_unstale":"2024-09-10T07:53:11.271008+0000","last_undegraded":"2024-09-10T07:53:11.271008+0000","last_fullsized":"2024-09-10T07:53:11.271008+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":314,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231633+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:53:11.231633+0000","last_peered":"2024-09-10T07:53:11.231633+0000","last_clean":"2024-09-10T07:53:11.231633+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:53:11.231633+0000","last_undegraded":"2024-09-10T07:53:11.231633+0000","last_fullsized":"2024-09-10T07:53:11.231633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476822,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476822,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":1127976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1113202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1241088,"data_stored":536466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3f is not active+clean 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214073+0000', 'last_change': '2024-09-10T07:53:10.214073+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214073+0000', 'last_undegraded': '2024-09-10T07:53:10.214073+0000', 'last_fullsized': '2024-09-10T07:53:10.214073+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3e is not active+clean 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.215296+0000', 'last_change': '2024-09-10T07:53:10.215296+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.215296+0000', 'last_undegraded': '2024-09-10T07:53:10.215296+0000', 'last_fullsized': '2024-09-10T07:53:10.215296+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3c is not active+clean 2024-09-10T07:53:16.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.215936+0000', 'last_change': '2024-09-10T07:53:10.215936+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.215936+0000', 'last_undegraded': '2024-09-10T07:53:10.215936+0000', 'last_fullsized': '2024-09-10T07:53:10.215936+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:PG 15.33 is not active+clean 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.218726+0000', 'last_change': '2024-09-10T07:53:10.218726+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.218726+0000', 'last_undegraded': '2024-09-10T07:53:10.218726+0000', 'last_fullsized': '2024-09-10T07:53:10.218726+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:PG 15.32 is not active+clean 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212879+0000', 'last_change': '2024-09-10T07:53:10.212879+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212879+0000', 'last_undegraded': '2024-09-10T07:53:10.212879+0000', 'last_fullsized': '2024-09-10T07:53:10.212879+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:PG 15.31 is not active+clean 2024-09-10T07:53:16.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.211714+0000', 'last_change': '2024-09-10T07:53:10.211714+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.211714+0000', 'last_undegraded': '2024-09-10T07:53:10.211714+0000', 'last_fullsized': '2024-09-10T07:53:10.211714+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2f is not active+clean 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212721+0000', 'last_change': '2024-09-10T07:53:10.212721+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212721+0000', 'last_undegraded': '2024-09-10T07:53:10.212721+0000', 'last_fullsized': '2024-09-10T07:53:10.212721+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2e is not active+clean 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214734+0000', 'last_change': '2024-09-10T07:53:10.214734+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214734+0000', 'last_undegraded': '2024-09-10T07:53:10.214734+0000', 'last_fullsized': '2024-09-10T07:53:10.214734+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214070+0000', 'last_change': '2024-09-10T07:53:10.214070+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214070+0000', 'last_undegraded': '2024-09-10T07:53:10.214070+0000', 'last_fullsized': '2024-09-10T07:53:10.214070+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:PG 15.29 is not active+clean 2024-09-10T07:53:16.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.211200+0000', 'last_change': '2024-09-10T07:53:10.211200+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.211200+0000', 'last_undegraded': '2024-09-10T07:53:10.211200+0000', 'last_fullsized': '2024-09-10T07:53:10.211200+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.26 is not active+clean 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214175+0000', 'last_change': '2024-09-10T07:53:10.214175+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214175+0000', 'last_undegraded': '2024-09-10T07:53:10.214175+0000', 'last_fullsized': '2024-09-10T07:53:10.214175+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.23 is not active+clean 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214213+0000', 'last_change': '2024-09-10T07:53:10.214213+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214213+0000', 'last_undegraded': '2024-09-10T07:53:10.214213+0000', 'last_fullsized': '2024-09-10T07:53:10.214213+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.22 is not active+clean 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.224154+0000', 'last_change': '2024-09-10T07:53:10.224154+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.224154+0000', 'last_undegraded': '2024-09-10T07:53:10.224154+0000', 'last_fullsized': '2024-09-10T07:53:10.224154+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.20 is not active+clean 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214870+0000', 'last_change': '2024-09-10T07:53:10.214870+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214870+0000', 'last_undegraded': '2024-09-10T07:53:10.214870+0000', 'last_fullsized': '2024-09-10T07:53:10.214870+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.217871+0000', 'last_change': '2024-09-10T07:53:10.217871+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.217871+0000', 'last_undegraded': '2024-09-10T07:53:10.217871+0000', 'last_fullsized': '2024-09-10T07:53:10.217871+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.292 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1f is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.215028+0000', 'last_change': '2024-09-10T07:53:10.215028+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.215028+0000', 'last_undegraded': '2024-09-10T07:53:10.215028+0000', 'last_fullsized': '2024-09-10T07:53:10.215028+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1c is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.215453+0000', 'last_change': '2024-09-10T07:53:10.215453+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.215453+0000', 'last_undegraded': '2024-09-10T07:53:10.215453+0000', 'last_fullsized': '2024-09-10T07:53:10.215453+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.18 is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.214033+0000', 'last_change': '2024-09-10T07:53:10.214033+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.214033+0000', 'last_undegraded': '2024-09-10T07:53:10.214033+0000', 'last_fullsized': '2024-09-10T07:53:10.214033+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.16 is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.213811+0000', 'last_change': '2024-09-10T07:53:10.213811+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.213811+0000', 'last_undegraded': '2024-09-10T07:53:10.213811+0000', 'last_fullsized': '2024-09-10T07:53:10.213811+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.12 is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212467+0000', 'last_change': '2024-09-10T07:53:10.212467+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212467+0000', 'last_undegraded': '2024-09-10T07:53:10.212467+0000', 'last_fullsized': '2024-09-10T07:53:10.212467+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212060+0000', 'last_change': '2024-09-10T07:53:10.212060+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212060+0000', 'last_undegraded': '2024-09-10T07:53:10.212060+0000', 'last_fullsized': '2024-09-10T07:53:10.212060+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.293 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.211732+0000', 'last_change': '2024-09-10T07:53:10.211732+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.211732+0000', 'last_undegraded': '2024-09-10T07:53:10.211732+0000', 'last_fullsized': '2024-09-10T07:53:10.211732+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:PG 15.8 is not active+clean 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.211105+0000', 'last_change': '2024-09-10T07:53:10.211105+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.211105+0000', 'last_undegraded': '2024-09-10T07:53:10.211105+0000', 'last_fullsized': '2024-09-10T07:53:10.211105+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:PG 15.6 is not active+clean 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.217413+0000', 'last_change': '2024-09-10T07:53:10.217413+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.217413+0000', 'last_undegraded': '2024-09-10T07:53:10.217413+0000', 'last_fullsized': '2024-09-10T07:53:10.217413+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-09-10T07:53:16.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212195+0000', 'last_change': '2024-09-10T07:53:10.212195+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212195+0000', 'last_undegraded': '2024-09-10T07:53:10.212195+0000', 'last_fullsized': '2024-09-10T07:53:10.212195+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.295 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-09-10T07:53:16.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.211556+0000', 'last_change': '2024-09-10T07:53:10.211556+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.211556+0000', 'last_undegraded': '2024-09-10T07:53:10.211556+0000', 'last_fullsized': '2024-09-10T07:53:10.211556+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.295 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-09-10T07:53:16.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-10T07:53:10.212681+0000', 'last_change': '2024-09-10T07:53:10.212681+0000', 'last_active': '2024-09-10T07:53:10.199182+0000', 'last_peered': '2024-09-10T07:53:10.199182+0000', 'last_clean': '2024-09-10T07:53:10.199182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T07:53:10.212681+0000', 'last_undegraded': '2024-09-10T07:53:10.212681+0000', 'last_fullsized': '2024-09-10T07:53:10.212681+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'last_clean_scrub_stamp': '2024-09-10T07:53:10.199182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T07:53:16.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.481+0000 7fd151a00640 1 Processor -- start 2024-09-10T07:53:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.481+0000 7fd151a00640 1 -- start start 2024-09-10T07:53:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd151a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd151a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c074200 con 0x7fd14c145d00 2024-09-10T07:53:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd151a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c073c40 con 0x7fd14c0714b0 2024-09-10T07:53:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd151a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd14c073f20 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd14be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd14c145d00 0x7fd14c073330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56712/0 (socket says 172.21.15.3:56712) 2024-09-10T07:53:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd14be00640 1 -- 172.21.15.3:0/1700908312 learned_addr learned my addr 172.21.15.3:0/1700908312 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.482+0000 7fd143400640 1 --2- 172.21.15.3:0/1700908312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1256257527 0 0) 0x7fd14c074200 con 0x7fd14c145d00 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd130003520 con 0x7fd14c145d00 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929966077 0 0) 0x7fd14c073c40 con 0x7fd14c0714b0 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd14c074200 con 0x7fd14c0714b0 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1676180062 0 0) 0x7fd130003520 con 0x7fd14c145d00 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd143400640 1 -- 172.21.15.3:0/1700908312 >> v1:172.21.15.70:6789/0 conn(0x7fd14c0714b0 legacy=0x7fd14c0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd143400640 1 -- 172.21.15.3:0/1700908312 >> v1:172.21.15.3:6789/0 conn(0x7fd14c145d00 legacy=0x7fd14c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.483+0000 7fd143400640 1 -- 172.21.15.3:0/1700908312 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c15ff30 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.484+0000 7fd143400640 1 --2- 172.21.15.3:0/1700908312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c160730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1340097d0 tx=0x7fd13402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20514ffaf371414 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.484+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134037070 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.484+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13402ac30 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.484+0000 7fd14b400640 1 -- 172.21.15.3:0/1700908312 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c001670 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 -- 172.21.15.3:0/1700908312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 msgr2=0x7fd14c160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 --2- 172.21.15.3:0/1700908312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c160730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1340097d0 tx=0x7fd13402a1a0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 -- 172.21.15.3:0/1700908312 shutdown_connections 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 --2- 172.21.15.3:0/1700908312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c160730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 -- 172.21.15.3:0/1700908312 >> 172.21.15.3:0/1700908312 conn(0x7fd14c06bb20 msgr2=0x7fd14c06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 -- 172.21.15.3:0/1700908312 shutdown_connections 2024-09-10T07:53:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.485+0000 7fd151a00640 1 -- 172.21.15.3:0/1700908312 wait complete. 2024-09-10T07:53:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.487+0000 7fd151a00640 1 Processor -- start 2024-09-10T07:53:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.487+0000 7fd151a00640 1 -- start start 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.487+0000 7fd151a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.487+0000 7fd151a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 0x7fd14c13cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c160df0 con 0x7fd14c0714b0 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c160c70 con 0x7fd14c145d00 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd14c0ce910 con 0x7fd14c0cfbe0 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35930/0 (socket says 172.21.15.3:35930) 2024-09-10T07:53:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 -- 172.21.15.3:0/1672654116 learned_addr learned my addr 172.21.15.3:0/1672654116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd143400640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 -- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 msgr2=0x7fd14c13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c13c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151000640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 0x7fd14c13cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 -- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 msgr2=0x7fd14c13cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 0x7fd14c13cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd14be00640 1 -- 172.21.15.3:0/1672654116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c15e7f0 con 0x7fd14c0714b0 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd151000640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 0x7fd14c13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.488+0000 7fd143400640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd14be00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00c750 tx=0x7fd13c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c005ad0 con 0x7fd14c0714b0 2024-09-10T07:53:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c005c70 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c005e60 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c15ead0 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.489+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c15f090 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.490+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14c0cdd10 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.491+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd13c0072b0 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.492+0000 7fd148c00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 0x7fd12806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.492+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd13c08ac80 con 0x7fd14c0714b0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.492+0000 7fd143400640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 0x7fd12806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.493+0000 7fd143400640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 0x7fd12806e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd134034000 tx=0x7fd13402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.497+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd13c0596e0 con 0x7fd14c0714b0 2024-09-10T07:53:16.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.750+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd14c160730 con 0x7fd14c0714b0 2024-09-10T07:53:16.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.751+0000 7fd148c00640 1 -- 172.21.15.3:0/1672654116 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1144 (secure 0 0 0) 0x7fd13c058fe0 con 0x7fd14c0714b0 2024-09-10T07:53:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:16.750 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":430,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":70},{"state_name":"creating+peering","count":27}],"num_pgs":97,"num_pools":3,"num_objects":13,"data_bytes":592162,"bytes_used":270782464,"bytes_avail":772823330816,"bytes_total":773094113280,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":103,"btime":"2024-09-10T07:53:14:241064+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-09-10T07:52:59.577824+0000","services":{}},"progress_events":{}} 2024-09-10T07:53:16.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 msgr2=0x7fd12806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 0x7fd12806e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd134034000 tx=0x7fd13402e000 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 msgr2=0x7fd14c14f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00c750 tx=0x7fd13c00cc10 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 shutdown_connections 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd12806be30 0x7fd12806e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd14c145d00 0x7fd14c13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd14c0cfbe0 0x7fd14c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 --2- 172.21.15.3:0/1672654116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd14c0714b0 0x7fd14c14f830 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 >> 172.21.15.3:0/1672654116 conn(0x7fd14c06bb20 msgr2=0x7fd14c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 shutdown_connections 2024-09-10T07:53:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.753+0000 7fd151a00640 1 -- 172.21.15.3:0/1672654116 wait complete. 2024-09-10T07:53:16.769 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T07:53:16.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:53:16.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.926+0000 7f2ce8800640 1 Processor -- start 2024-09-10T07:53:16.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.926+0000 7f2ce8800640 1 -- start start 2024-09-10T07:53:16.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.927+0000 7f2ce8800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce015ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.927+0000 7f2ce8800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce0160470 0x7f2ce015da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.927+0000 7f2ce8800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce00af1d0 con 0x7f2ce0156a80 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.927+0000 7f2ce8800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce00af4b0 con 0x7f2ce0160470 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.927+0000 7f2ce8800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00af790 con 0x7f2ce0160b80 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce0160470 0x7f2ce015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2ce0160470 0x7f2ce015da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35932/0 (socket says 172.21.15.3:35932) 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 -- 172.21.15.3:0/2735895389 learned_addr learned my addr 172.21.15.3:0/2735895389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce5400640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce015ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 -- 172.21.15.3:0/2735895389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 msgr2=0x7f2ce015ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce015ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 -- 172.21.15.3:0/2735895389 >> v1:172.21.15.70:6789/0 conn(0x7f2ce0156a80 legacy=0x7f2ce0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.928+0000 7f2ce6800640 1 -- 172.21.15.3:0/2735895389 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce00afa70 con 0x7f2ce0160470 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.929+0000 7f2ce6800640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0160470 0x7f2ce015da50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009e50 tx=0x7f2cdc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f3bc578fd7f249 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.929+0000 7f2ce4a00640 1 -- 172.21.15.3:0/2735895389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc036070 con 0x7f2ce0160470 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.929+0000 7f2ce4a00640 1 -- 172.21.15.3:0/2735895389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cdc031600 con 0x7f2ce0160470 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.929+0000 7f2ce4a00640 1 -- 172.21.15.3:0/2735895389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0319d0 con 0x7f2ce0160470 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 -- 172.21.15.3:0/2735895389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0160470 msgr2=0x7f2ce015da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0160470 0x7f2ce015da50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009e50 tx=0x7f2cdc02a6a0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 -- 172.21.15.3:0/2735895389 shutdown_connections 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0160470 0x7f2ce015da50 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2735895389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce015ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 -- 172.21.15.3:0/2735895389 >> 172.21.15.3:0/2735895389 conn(0x7f2ce006bb20 msgr2=0x7f2ce006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:16.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 -- 172.21.15.3:0/2735895389 shutdown_connections 2024-09-10T07:53:16.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.930+0000 7f2ce8800640 1 -- 172.21.15.3:0/2735895389 wait complete. 2024-09-10T07:53:16.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.932+0000 7f2ce8800640 1 Processor -- start 2024-09-10T07:53:16.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.932+0000 7f2ce8800640 1 -- start start 2024-09-10T07:53:16.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 0x7f2ce015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce0150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00b0270 con 0x7f2ce0156a80 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00aeee0 con 0x7f2ce0160470 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce8800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00b2eb0 con 0x7f2ce0160b80 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35936/0 (socket says 172.21.15.3:35936) 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 -- 172.21.15.3:0/2880199793 learned_addr learned my addr 172.21.15.3:0/2880199793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5400640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 0x7f2ce015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce6800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce0150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 -- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 msgr2=0x7f2ce0150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce0150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 -- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 msgr2=0x7f2ce015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 0x7f2ce015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:16.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5e00640 1 -- 172.21.15.3:0/2880199793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce00b22a0 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce6800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce0150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.933+0000 7f2ce5400640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 0x7f2ce015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2ce5e00640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c400 tx=0x7f2cd400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40052d0 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd4005490 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce00b2580 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4007260 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.934+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce00b2b40 con 0x7f2ce0156a80 2024-09-10T07:53:16.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.935+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce015ff30 con 0x7f2ce0156a80 2024-09-10T07:53:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.936+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cd4007400 con 0x7f2ce0156a80 2024-09-10T07:53:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.937+0000 7f2cda000640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 0x7f2cc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.937+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2cd408a490 con 0x7f2ce0156a80 2024-09-10T07:53:16.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.937+0000 7f2ce5400640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 0x7f2cc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:16.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.938+0000 7f2ce5400640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 0x7f2cc406e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:16.941+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cd4058da0 con 0x7f2ce0156a80 2024-09-10T07:53:17.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.125+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ce00b05d0 con 0x7f2cc406be30 2024-09-10T07:53:17.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.132+0000 7f2cda000640 1 -- 172.21.15.3:0/2880199793 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250124 (secure 0 0 0) 0x7f2ce00b05d0 con 0x7f2cc406be30 2024-09-10T07:53:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:17.135 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:53:17.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 msgr2=0x7f2cc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 0x7f2cc406e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0008040 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 msgr2=0x7f2ce006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c400 tx=0x7f2cd400c8c0 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 shutdown_connections 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f2cc406be30 0x7f2cc406e2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2ce0160b80 0x7f2ce0150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ce0160470 0x7f2ce015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 --2- 172.21.15.3:0/2880199793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ce0156a80 0x7f2ce006ef60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 >> 172.21.15.3:0/2880199793 conn(0x7f2ce006bb20 msgr2=0x7f2ce015e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:17.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 shutdown_connections 2024-09-10T07:53:17.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:17.136+0000 7f2ce8800640 1 -- 172.21.15.3:0/2880199793 wait complete. 2024-09-10T07:53:17.152 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-10T07:53:15.580430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264436,"kb_used_data":10740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710284,"statfs":{"total":773094113280,"available":772823330816,"internally_reserved":0,"allocated":10997760,"data_stored":5410639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000952"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214073+0000","last_change":"2024-09-10T07:53:10.214073+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214073+0000","last_undegraded":"2024-09-10T07:53:10.214073+0000","last_fullsized":"2024-09-10T07:53:10.214073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215296+0000","last_change":"2024-09-10T07:53:10.215296+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215296+0000","last_undegraded":"2024-09-10T07:53:10.215296+0000","last_fullsized":"2024-09-10T07:53:10.215296+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233484+0000","last_change":"2024-09-10T07:53:11.252106+0000","last_active":"2024-09-10T07:53:12.233484+0000","last_peered":"2024-09-10T07:53:12.233484+0000","last_clean":"2024-09-10T07:53:12.233484+0000","last_became_active":"2024-09-10T07:53:11.251807+0000","last_became_peered":"2024-09-10T07:53:11.251807+0000","last_unstale":"2024-09-10T07:53:12.233484+0000","last_undegraded":"2024-09-10T07:53:12.233484+0000","last_fullsized":"2024-09-10T07:53:12.233484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:01.780830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215936+0000","last_change":"2024-09-10T07:53:10.215936+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215936+0000","last_undegraded":"2024-09-10T07:53:10.215936+0000","last_fullsized":"2024-09-10T07:53:10.215936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.237820+0000","last_change":"2024-09-10T07:53:11.234549+0000","last_active":"2024-09-10T07:53:11.237820+0000","last_peered":"2024-09-10T07:53:11.237820+0000","last_clean":"2024-09-10T07:53:11.237820+0000","last_became_active":"2024-09-10T07:53:11.234201+0000","last_became_peered":"2024-09-10T07:53:11.234201+0000","last_unstale":"2024-09-10T07:53:11.237820+0000","last_undegraded":"2024-09-10T07:53:11.237820+0000","last_fullsized":"2024-09-10T07:53:11.237820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:49.298649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.243698+0000","last_change":"2024-09-10T07:53:11.236870+0000","last_active":"2024-09-10T07:53:11.243698+0000","last_peered":"2024-09-10T07:53:11.243698+0000","last_clean":"2024-09-10T07:53:11.243698+0000","last_became_active":"2024-09-10T07:53:11.235843+0000","last_became_peered":"2024-09-10T07:53:11.235843+0000","last_unstale":"2024-09-10T07:53:11.243698+0000","last_undegraded":"2024-09-10T07:53:11.243698+0000","last_fullsized":"2024-09-10T07:53:11.243698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.860697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261073+0000","last_change":"2024-09-10T07:53:11.255394+0000","last_active":"2024-09-10T07:53:11.261073+0000","last_peered":"2024-09-10T07:53:11.261073+0000","last_clean":"2024-09-10T07:53:11.261073+0000","last_became_active":"2024-09-10T07:53:11.254136+0000","last_became_peered":"2024-09-10T07:53:11.254136+0000","last_unstale":"2024-09-10T07:53:11.261073+0000","last_undegraded":"2024-09-10T07:53:11.261073+0000","last_fullsized":"2024-09-10T07:53:11.261073+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:35.472507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233314+0000","last_change":"2024-09-10T07:53:11.250032+0000","last_active":"2024-09-10T07:53:12.233314+0000","last_peered":"2024-09-10T07:53:12.233314+0000","last_clean":"2024-09-10T07:53:12.233314+0000","last_became_active":"2024-09-10T07:53:11.249626+0000","last_became_peered":"2024-09-10T07:53:11.249626+0000","last_unstale":"2024-09-10T07:53:12.233314+0000","last_undegraded":"2024-09-10T07:53:12.233314+0000","last_fullsized":"2024-09-10T07:53:12.233314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.283338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248152+0000","last_change":"2024-09-10T07:53:11.243615+0000","last_active":"2024-09-10T07:53:11.248152+0000","last_peered":"2024-09-10T07:53:11.248152+0000","last_clean":"2024-09-10T07:53:11.248152+0000","last_became_active":"2024-09-10T07:53:11.243351+0000","last_became_peered":"2024-09-10T07:53:11.243351+0000","last_unstale":"2024-09-10T07:53:11.248152+0000","last_undegraded":"2024-09-10T07:53:11.248152+0000","last_fullsized":"2024-09-10T07:53:11.248152+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:53.710317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268481+0000","last_change":"2024-09-10T07:53:11.267004+0000","last_active":"2024-09-10T07:53:11.268481+0000","last_peered":"2024-09-10T07:53:11.268481+0000","last_clean":"2024-09-10T07:53:11.268481+0000","last_became_active":"2024-09-10T07:53:11.266699+0000","last_became_peered":"2024-09-10T07:53:11.266699+0000","last_unstale":"2024-09-10T07:53:11.268481+0000","last_undegraded":"2024-09-10T07:53:11.268481+0000","last_fullsized":"2024-09-10T07:53:11.268481+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:08.715930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239275+0000","last_change":"2024-09-10T07:53:11.236475+0000","last_active":"2024-09-10T07:53:11.239275+0000","last_peered":"2024-09-10T07:53:11.239275+0000","last_clean":"2024-09-10T07:53:11.239275+0000","last_became_active":"2024-09-10T07:53:11.235672+0000","last_became_peered":"2024-09-10T07:53:11.235672+0000","last_unstale":"2024-09-10T07:53:11.239275+0000","last_undegraded":"2024-09-10T07:53:11.239275+0000","last_fullsized":"2024-09-10T07:53:11.239275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:31:33.473232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269070+0000","last_change":"2024-09-10T07:53:11.267139+0000","last_active":"2024-09-10T07:53:11.269070+0000","last_peered":"2024-09-10T07:53:11.269070+0000","last_clean":"2024-09-10T07:53:11.269070+0000","last_became_active":"2024-09-10T07:53:11.266829+0000","last_became_peered":"2024-09-10T07:53:11.266829+0000","last_unstale":"2024-09-10T07:53:11.269070+0000","last_undegraded":"2024-09-10T07:53:11.269070+0000","last_fullsized":"2024-09-10T07:53:11.269070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.218726+0000","last_change":"2024-09-10T07:53:10.218726+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.218726+0000","last_undegraded":"2024-09-10T07:53:10.218726+0000","last_fullsized":"2024-09-10T07:53:10.218726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212879+0000","last_change":"2024-09-10T07:53:10.212879+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212879+0000","last_undegraded":"2024-09-10T07:53:10.212879+0000","last_fullsized":"2024-09-10T07:53:10.212879+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211714+0000","last_change":"2024-09-10T07:53:10.211714+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211714+0000","last_undegraded":"2024-09-10T07:53:10.211714+0000","last_fullsized":"2024-09-10T07:53:10.211714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260661+0000","last_change":"2024-09-10T07:53:11.254673+0000","last_active":"2024-09-10T07:53:11.260661+0000","last_peered":"2024-09-10T07:53:11.260661+0000","last_clean":"2024-09-10T07:53:11.260661+0000","last_became_active":"2024-09-10T07:53:11.251614+0000","last_became_peered":"2024-09-10T07:53:11.251614+0000","last_unstale":"2024-09-10T07:53:11.260661+0000","last_undegraded":"2024-09-10T07:53:11.260661+0000","last_fullsized":"2024-09-10T07:53:11.260661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:50.934355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212721+0000","last_change":"2024-09-10T07:53:10.212721+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212721+0000","last_undegraded":"2024-09-10T07:53:10.212721+0000","last_fullsized":"2024-09-10T07:53:10.212721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214734+0000","last_change":"2024-09-10T07:53:10.214734+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214734+0000","last_undegraded":"2024-09-10T07:53:10.214734+0000","last_fullsized":"2024-09-10T07:53:10.214734+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214070+0000","last_change":"2024-09-10T07:53:10.214070+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214070+0000","last_undegraded":"2024-09-10T07:53:10.214070+0000","last_fullsized":"2024-09-10T07:53:10.214070+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239700+0000","last_change":"2024-09-10T07:53:11.234958+0000","last_active":"2024-09-10T07:53:11.239700+0000","last_peered":"2024-09-10T07:53:11.239700+0000","last_clean":"2024-09-10T07:53:11.239700+0000","last_became_active":"2024-09-10T07:53:11.234678+0000","last_became_peered":"2024-09-10T07:53:11.234678+0000","last_unstale":"2024-09-10T07:53:11.239700+0000","last_undegraded":"2024-09-10T07:53:11.239700+0000","last_fullsized":"2024-09-10T07:53:11.239700+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:47.186154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233477+0000","last_change":"2024-09-10T07:53:11.248147+0000","last_active":"2024-09-10T07:53:12.233477+0000","last_peered":"2024-09-10T07:53:12.233477+0000","last_clean":"2024-09-10T07:53:12.233477+0000","last_became_active":"2024-09-10T07:53:11.247829+0000","last_became_peered":"2024-09-10T07:53:11.247829+0000","last_unstale":"2024-09-10T07:53:12.233477+0000","last_undegraded":"2024-09-10T07:53:12.233477+0000","last_fullsized":"2024-09-10T07:53:12.233477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:54.042192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232852+0000","last_change":"2024-09-10T07:53:11.249206+0000","last_active":"2024-09-10T07:53:12.232852+0000","last_peered":"2024-09-10T07:53:12.232852+0000","last_clean":"2024-09-10T07:53:12.232852+0000","last_became_active":"2024-09-10T07:53:11.248315+0000","last_became_peered":"2024-09-10T07:53:11.248315+0000","last_unstale":"2024-09-10T07:53:12.232852+0000","last_undegraded":"2024-09-10T07:53:12.232852+0000","last_fullsized":"2024-09-10T07:53:12.232852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:31.145698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211200+0000","last_change":"2024-09-10T07:53:10.211200+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211200+0000","last_undegraded":"2024-09-10T07:53:10.211200+0000","last_fullsized":"2024-09-10T07:53:10.211200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.263806+0000","last_change":"2024-09-10T07:53:11.262775+0000","last_active":"2024-09-10T07:53:11.263806+0000","last_peered":"2024-09-10T07:53:11.263806+0000","last_clean":"2024-09-10T07:53:11.263806+0000","last_became_active":"2024-09-10T07:53:11.262438+0000","last_became_peered":"2024-09-10T07:53:11.262438+0000","last_unstale":"2024-09-10T07:53:11.263806+0000","last_undegraded":"2024-09-10T07:53:11.263806+0000","last_fullsized":"2024-09-10T07:53:11.263806+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:12.859790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.257372+0000","last_change":"2024-09-10T07:53:11.255844+0000","last_active":"2024-09-10T07:53:11.257372+0000","last_peered":"2024-09-10T07:53:11.257372+0000","last_clean":"2024-09-10T07:53:11.257372+0000","last_became_active":"2024-09-10T07:53:11.255548+0000","last_became_peered":"2024-09-10T07:53:11.255548+0000","last_unstale":"2024-09-10T07:53:11.257372+0000","last_undegraded":"2024-09-10T07:53:11.257372+0000","last_fullsized":"2024-09-10T07:53:11.257372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:06.407725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214175+0000","last_change":"2024-09-10T07:53:10.214175+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214175+0000","last_undegraded":"2024-09-10T07:53:10.214175+0000","last_fullsized":"2024-09-10T07:53:10.214175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248744+0000","last_change":"2024-09-10T07:53:11.246590+0000","last_active":"2024-09-10T07:53:11.248744+0000","last_peered":"2024-09-10T07:53:11.248744+0000","last_clean":"2024-09-10T07:53:11.248744+0000","last_became_active":"2024-09-10T07:53:11.245943+0000","last_became_peered":"2024-09-10T07:53:11.245943+0000","last_unstale":"2024-09-10T07:53:11.248744+0000","last_undegraded":"2024-09-10T07:53:11.248744+0000","last_fullsized":"2024-09-10T07:53:11.248744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258576+0000","last_change":"2024-09-10T07:53:11.256724+0000","last_active":"2024-09-10T07:53:11.258576+0000","last_peered":"2024-09-10T07:53:11.258576+0000","last_clean":"2024-09-10T07:53:11.258576+0000","last_became_active":"2024-09-10T07:53:11.256427+0000","last_became_peered":"2024-09-10T07:53:11.256427+0000","last_unstale":"2024-09-10T07:53:11.258576+0000","last_undegraded":"2024-09-10T07:53:11.258576+0000","last_fullsized":"2024-09-10T07:53:11.258576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:19.541855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214213+0000","last_change":"2024-09-10T07:53:10.214213+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214213+0000","last_undegraded":"2024-09-10T07:53:10.214213+0000","last_fullsized":"2024-09-10T07:53:10.214213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.224154+0000","last_change":"2024-09-10T07:53:10.224154+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.224154+0000","last_undegraded":"2024-09-10T07:53:10.224154+0000","last_fullsized":"2024-09-10T07:53:10.224154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234095+0000","last_change":"2024-09-10T07:53:11.256491+0000","last_active":"2024-09-10T07:53:12.234095+0000","last_peered":"2024-09-10T07:53:12.234095+0000","last_clean":"2024-09-10T07:53:12.234095+0000","last_became_active":"2024-09-10T07:53:11.255961+0000","last_became_peered":"2024-09-10T07:53:11.255961+0000","last_unstale":"2024-09-10T07:53:12.234095+0000","last_undegraded":"2024-09-10T07:53:12.234095+0000","last_fullsized":"2024-09-10T07:53:12.234095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:11.297862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214870+0000","last_change":"2024-09-10T07:53:10.214870+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214870+0000","last_undegraded":"2024-09-10T07:53:10.214870+0000","last_fullsized":"2024-09-10T07:53:10.214870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217871+0000","last_change":"2024-09-10T07:53:10.217871+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217871+0000","last_undegraded":"2024-09-10T07:53:10.217871+0000","last_fullsized":"2024-09-10T07:53:10.217871+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236893+0000","last_change":"2024-09-10T07:53:10.235925+0000","last_active":"2024-09-10T07:53:10.236893+0000","last_peered":"2024-09-10T07:53:10.236893+0000","last_clean":"2024-09-10T07:53:10.236893+0000","last_became_active":"2024-09-10T07:53:10.235350+0000","last_became_peered":"2024-09-10T07:53:10.235350+0000","last_unstale":"2024-09-10T07:53:10.236893+0000","last_undegraded":"2024-09-10T07:53:10.236893+0000","last_fullsized":"2024-09-10T07:53:10.236893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215028+0000","last_change":"2024-09-10T07:53:10.215028+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215028+0000","last_undegraded":"2024-09-10T07:53:10.215028+0000","last_fullsized":"2024-09-10T07:53:10.215028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267919+0000","last_change":"2024-09-10T07:53:10.232379+0000","last_active":"2024-09-10T07:53:11.267919+0000","last_peered":"2024-09-10T07:53:11.267919+0000","last_clean":"2024-09-10T07:53:11.267919+0000","last_became_active":"2024-09-10T07:53:10.231931+0000","last_became_peered":"2024-09-10T07:53:10.231931+0000","last_unstale":"2024-09-10T07:53:11.267919+0000","last_undegraded":"2024-09-10T07:53:11.267919+0000","last_fullsized":"2024-09-10T07:53:11.267919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:22:58.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.215453+0000","last_change":"2024-09-10T07:53:10.215453+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.215453+0000","last_undegraded":"2024-09-10T07:53:10.215453+0000","last_fullsized":"2024-09-10T07:53:10.215453+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233603+0000","last_change":"2024-09-10T07:53:10.227323+0000","last_active":"2024-09-10T07:53:12.233603+0000","last_peered":"2024-09-10T07:53:12.233603+0000","last_clean":"2024-09-10T07:53:12.233603+0000","last_became_active":"2024-09-10T07:53:10.227012+0000","last_became_peered":"2024-09-10T07:53:10.227012+0000","last_unstale":"2024-09-10T07:53:12.233603+0000","last_undegraded":"2024-09-10T07:53:12.233603+0000","last_fullsized":"2024-09-10T07:53:12.233603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:56.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.249667+0000","last_change":"2024-09-10T07:53:11.246551+0000","last_active":"2024-09-10T07:53:11.249667+0000","last_peered":"2024-09-10T07:53:11.249667+0000","last_clean":"2024-09-10T07:53:11.249667+0000","last_became_active":"2024-09-10T07:53:11.246300+0000","last_became_peered":"2024-09-10T07:53:11.246300+0000","last_unstale":"2024-09-10T07:53:11.249667+0000","last_undegraded":"2024-09-10T07:53:11.249667+0000","last_fullsized":"2024-09-10T07:53:11.249667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234150+0000","last_change":"2024-09-10T07:53:10.231193+0000","last_active":"2024-09-10T07:53:10.234150+0000","last_peered":"2024-09-10T07:53:10.234150+0000","last_clean":"2024-09-10T07:53:10.234150+0000","last_became_active":"2024-09-10T07:53:10.226288+0000","last_became_peered":"2024-09-10T07:53:10.226288+0000","last_unstale":"2024-09-10T07:53:10.234150+0000","last_undegraded":"2024-09-10T07:53:10.234150+0000","last_fullsized":"2024-09-10T07:53:10.234150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:31.899936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.241212+0000","last_change":"2024-09-10T07:53:11.235832+0000","last_active":"2024-09-10T07:53:11.241212+0000","last_peered":"2024-09-10T07:53:11.241212+0000","last_clean":"2024-09-10T07:53:11.241212+0000","last_became_active":"2024-09-10T07:53:11.235506+0000","last_became_peered":"2024-09-10T07:53:11.235506+0000","last_unstale":"2024-09-10T07:53:11.241212+0000","last_undegraded":"2024-09-10T07:53:11.241212+0000","last_fullsized":"2024-09-10T07:53:11.241212+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:05.491743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233697+0000","last_change":"2024-09-10T07:53:10.228445+0000","last_active":"2024-09-10T07:53:10.233697+0000","last_peered":"2024-09-10T07:53:10.233697+0000","last_clean":"2024-09-10T07:53:10.233697+0000","last_became_active":"2024-09-10T07:53:10.227806+0000","last_became_peered":"2024-09-10T07:53:10.227806+0000","last_unstale":"2024-09-10T07:53:10.233697+0000","last_undegraded":"2024-09-10T07:53:10.233697+0000","last_fullsized":"2024-09-10T07:53:10.233697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:25:41.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.238929+0000","last_change":"2024-09-10T07:53:11.235298+0000","last_active":"2024-09-10T07:53:11.238929+0000","last_peered":"2024-09-10T07:53:11.238929+0000","last_clean":"2024-09-10T07:53:11.238929+0000","last_became_active":"2024-09-10T07:53:11.234953+0000","last_became_peered":"2024-09-10T07:53:11.234953+0000","last_unstale":"2024-09-10T07:53:11.238929+0000","last_undegraded":"2024-09-10T07:53:11.238929+0000","last_fullsized":"2024-09-10T07:53:11.238929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:14.411009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232971+0000","last_change":"2024-09-10T07:53:10.229226+0000","last_active":"2024-09-10T07:53:12.232971+0000","last_peered":"2024-09-10T07:53:12.232971+0000","last_clean":"2024-09-10T07:53:12.232971+0000","last_became_active":"2024-09-10T07:53:10.228904+0000","last_became_peered":"2024-09-10T07:53:10.228904+0000","last_unstale":"2024-09-10T07:53:12.232971+0000","last_undegraded":"2024-09-10T07:53:12.232971+0000","last_fullsized":"2024-09-10T07:53:12.232971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:29.688893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.214033+0000","last_change":"2024-09-10T07:53:10.214033+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.214033+0000","last_undegraded":"2024-09-10T07:53:10.214033+0000","last_fullsized":"2024-09-10T07:53:10.214033+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.229770+0000","last_change":"2024-09-10T07:53:10.228005+0000","last_active":"2024-09-10T07:53:11.229770+0000","last_peered":"2024-09-10T07:53:11.229770+0000","last_clean":"2024-09-10T07:53:11.229770+0000","last_became_active":"2024-09-10T07:53:10.227661+0000","last_became_peered":"2024-09-10T07:53:10.227661+0000","last_unstale":"2024-09-10T07:53:11.229770+0000","last_undegraded":"2024-09-10T07:53:11.229770+0000","last_fullsized":"2024-09-10T07:53:11.229770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:45.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.240934+0000","last_change":"2024-09-10T07:53:11.235959+0000","last_active":"2024-09-10T07:53:11.240934+0000","last_peered":"2024-09-10T07:53:11.240934+0000","last_clean":"2024-09-10T07:53:11.240934+0000","last_became_active":"2024-09-10T07:53:11.235687+0000","last_became_peered":"2024-09-10T07:53:11.235687+0000","last_unstale":"2024-09-10T07:53:11.240934+0000","last_undegraded":"2024-09-10T07:53:11.240934+0000","last_fullsized":"2024-09-10T07:53:11.240934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:00.667191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235952+0000","last_change":"2024-09-10T07:53:10.233196+0000","last_active":"2024-09-10T07:53:10.235952+0000","last_peered":"2024-09-10T07:53:10.235952+0000","last_clean":"2024-09-10T07:53:10.235952+0000","last_became_active":"2024-09-10T07:53:10.232510+0000","last_became_peered":"2024-09-10T07:53:10.232510+0000","last_unstale":"2024-09-10T07:53:10.235952+0000","last_undegraded":"2024-09-10T07:53:10.235952+0000","last_fullsized":"2024-09-10T07:53:10.235952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:11.857423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.213811+0000","last_change":"2024-09-10T07:53:10.213811+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.213811+0000","last_undegraded":"2024-09-10T07:53:10.213811+0000","last_fullsized":"2024-09-10T07:53:10.213811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234051+0000","last_change":"2024-09-10T07:53:10.213607+0000","last_active":"2024-09-10T07:53:12.234051+0000","last_peered":"2024-09-10T07:53:12.234051+0000","last_clean":"2024-09-10T07:53:12.234051+0000","last_became_active":"2024-09-10T07:53:10.213317+0000","last_became_peered":"2024-09-10T07:53:10.213317+0000","last_unstale":"2024-09-10T07:53:12.234051+0000","last_undegraded":"2024-09-10T07:53:12.234051+0000","last_fullsized":"2024-09-10T07:53:12.234051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:19.086909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.251040+0000","last_change":"2024-09-10T07:53:11.250003+0000","last_active":"2024-09-10T07:53:11.251040+0000","last_peered":"2024-09-10T07:53:11.251040+0000","last_clean":"2024-09-10T07:53:11.251040+0000","last_became_active":"2024-09-10T07:53:11.249735+0000","last_became_peered":"2024-09-10T07:53:11.249735+0000","last_unstale":"2024-09-10T07:53:11.251040+0000","last_undegraded":"2024-09-10T07:53:11.251040+0000","last_fullsized":"2024-09-10T07:53:11.251040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:53.440934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233009+0000","last_change":"2024-09-10T07:53:10.226296+0000","last_active":"2024-09-10T07:53:10.233009+0000","last_peered":"2024-09-10T07:53:10.233009+0000","last_clean":"2024-09-10T07:53:10.233009+0000","last_became_active":"2024-09-10T07:53:10.225937+0000","last_became_peered":"2024-09-10T07:53:10.225937+0000","last_unstale":"2024-09-10T07:53:10.233009+0000","last_undegraded":"2024-09-10T07:53:10.233009+0000","last_fullsized":"2024-09-10T07:53:10.233009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269185+0000","last_change":"2024-09-10T07:53:11.266667+0000","last_active":"2024-09-10T07:53:11.269185+0000","last_peered":"2024-09-10T07:53:11.269185+0000","last_clean":"2024-09-10T07:53:11.269185+0000","last_became_active":"2024-09-10T07:53:11.266400+0000","last_became_peered":"2024-09-10T07:53:11.266400+0000","last_unstale":"2024-09-10T07:53:11.269185+0000","last_undegraded":"2024-09-10T07:53:11.269185+0000","last_fullsized":"2024-09-10T07:53:11.269185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:39.890020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.269567+0000","last_change":"2024-09-10T07:53:10.223127+0000","last_active":"2024-09-10T07:53:11.269567+0000","last_peered":"2024-09-10T07:53:11.269567+0000","last_clean":"2024-09-10T07:53:11.269567+0000","last_became_active":"2024-09-10T07:53:10.222729+0000","last_became_peered":"2024-09-10T07:53:10.222729+0000","last_unstale":"2024-09-10T07:53:11.269567+0000","last_undegraded":"2024-09-10T07:53:11.269567+0000","last_fullsized":"2024-09-10T07:53:11.269567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.995404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261874+0000","last_change":"2024-09-10T07:53:11.260871+0000","last_active":"2024-09-10T07:53:11.261874+0000","last_peered":"2024-09-10T07:53:11.261874+0000","last_clean":"2024-09-10T07:53:11.261874+0000","last_became_active":"2024-09-10T07:53:11.260613+0000","last_became_peered":"2024-09-10T07:53:11.260613+0000","last_unstale":"2024-09-10T07:53:11.261874+0000","last_undegraded":"2024-09-10T07:53:11.261874+0000","last_fullsized":"2024-09-10T07:53:11.261874+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:15.810230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.277289+0000","last_change":"2024-09-10T07:53:10.235054+0000","last_active":"2024-09-10T07:53:11.277289+0000","last_peered":"2024-09-10T07:53:11.277289+0000","last_clean":"2024-09-10T07:53:11.277289+0000","last_became_active":"2024-09-10T07:53:10.234764+0000","last_became_peered":"2024-09-10T07:53:10.234764+0000","last_unstale":"2024-09-10T07:53:11.277289+0000","last_undegraded":"2024-09-10T07:53:11.277289+0000","last_fullsized":"2024-09-10T07:53:11.277289+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212467+0000","last_change":"2024-09-10T07:53:10.212467+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212467+0000","last_undegraded":"2024-09-10T07:53:10.212467+0000","last_fullsized":"2024-09-10T07:53:10.212467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.234989+0000","last_change":"2024-09-10T07:53:10.222685+0000","last_active":"2024-09-10T07:53:10.234989+0000","last_peered":"2024-09-10T07:53:10.234989+0000","last_clean":"2024-09-10T07:53:10.234989+0000","last_became_active":"2024-09-10T07:53:10.222344+0000","last_became_peered":"2024-09-10T07:53:10.222344+0000","last_unstale":"2024-09-10T07:53:10.234989+0000","last_undegraded":"2024-09-10T07:53:10.234989+0000","last_fullsized":"2024-09-10T07:53:10.234989+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:18.539489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233639+0000","last_change":"2024-09-10T07:53:11.257850+0000","last_active":"2024-09-10T07:53:12.233639+0000","last_peered":"2024-09-10T07:53:12.233639+0000","last_clean":"2024-09-10T07:53:12.233639+0000","last_became_active":"2024-09-10T07:53:11.255958+0000","last_became_peered":"2024-09-10T07:53:11.255958+0000","last_unstale":"2024-09-10T07:53:12.233639+0000","last_undegraded":"2024-09-10T07:53:12.233639+0000","last_fullsized":"2024-09-10T07:53:12.233639+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:05.576611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233362+0000","last_change":"2024-09-10T07:53:10.229874+0000","last_active":"2024-09-10T07:53:10.233362+0000","last_peered":"2024-09-10T07:53:10.233362+0000","last_clean":"2024-09-10T07:53:10.233362+0000","last_became_active":"2024-09-10T07:53:10.229597+0000","last_became_peered":"2024-09-10T07:53:10.229597+0000","last_unstale":"2024-09-10T07:53:10.233362+0000","last_undegraded":"2024-09-10T07:53:10.233362+0000","last_fullsized":"2024-09-10T07:53:10.233362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239396+0000","last_change":"2024-09-10T07:53:11.236857+0000","last_active":"2024-09-10T07:53:11.239396+0000","last_peered":"2024-09-10T07:53:11.239396+0000","last_clean":"2024-09-10T07:53:11.239396+0000","last_became_active":"2024-09-10T07:53:11.236085+0000","last_became_peered":"2024-09-10T07:53:11.236085+0000","last_unstale":"2024-09-10T07:53:11.239396+0000","last_undegraded":"2024-09-10T07:53:11.239396+0000","last_fullsized":"2024-09-10T07:53:11.239396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:25.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231957+0000","last_change":"2024-09-10T07:53:10.223641+0000","last_active":"2024-09-10T07:53:11.231957+0000","last_peered":"2024-09-10T07:53:11.231957+0000","last_clean":"2024-09-10T07:53:11.231957+0000","last_became_active":"2024-09-10T07:53:10.223373+0000","last_became_peered":"2024-09-10T07:53:10.223373+0000","last_unstale":"2024-09-10T07:53:11.231957+0000","last_undegraded":"2024-09-10T07:53:11.231957+0000","last_fullsized":"2024-09-10T07:53:11.231957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233338+0000","last_change":"2024-09-10T07:53:11.256226+0000","last_active":"2024-09-10T07:53:12.233338+0000","last_peered":"2024-09-10T07:53:12.233338+0000","last_clean":"2024-09-10T07:53:12.233338+0000","last_became_active":"2024-09-10T07:53:11.247430+0000","last_became_peered":"2024-09-10T07:53:11.247430+0000","last_unstale":"2024-09-10T07:53:12.233338+0000","last_undegraded":"2024-09-10T07:53:12.233338+0000","last_fullsized":"2024-09-10T07:53:12.233338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:22.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.232748+0000","last_change":"2024-09-10T07:53:10.223678+0000","last_active":"2024-09-10T07:53:10.232748+0000","last_peered":"2024-09-10T07:53:10.232748+0000","last_clean":"2024-09-10T07:53:10.232748+0000","last_became_active":"2024-09-10T07:53:10.223390+0000","last_became_peered":"2024-09-10T07:53:10.223390+0000","last_unstale":"2024-09-10T07:53:10.232748+0000","last_undegraded":"2024-09-10T07:53:10.232748+0000","last_fullsized":"2024-09-10T07:53:10.232748+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:18:35.321389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.252550+0000","last_change":"2024-09-10T07:53:11.251829+0000","last_active":"2024-09-10T07:53:11.252550+0000","last_peered":"2024-09-10T07:53:11.252550+0000","last_clean":"2024-09-10T07:53:11.252550+0000","last_became_active":"2024-09-10T07:53:11.251547+0000","last_became_peered":"2024-09-10T07:53:11.251547+0000","last_unstale":"2024-09-10T07:53:11.252550+0000","last_undegraded":"2024-09-10T07:53:11.252550+0000","last_fullsized":"2024-09-10T07:53:11.252550+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:43.617740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271292+0000","last_change":"2024-09-10T07:53:10.232882+0000","last_active":"2024-09-10T07:53:11.271292+0000","last_peered":"2024-09-10T07:53:11.271292+0000","last_clean":"2024-09-10T07:53:11.271292+0000","last_became_active":"2024-09-10T07:53:10.232094+0000","last_became_peered":"2024-09-10T07:53:10.232094+0000","last_unstale":"2024-09-10T07:53:11.271292+0000","last_undegraded":"2024-09-10T07:53:11.271292+0000","last_fullsized":"2024-09-10T07:53:11.271292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.250788+0000","last_change":"2024-09-10T07:53:11.248994+0000","last_active":"2024-09-10T07:53:11.250788+0000","last_peered":"2024-09-10T07:53:11.250788+0000","last_clean":"2024-09-10T07:53:11.250788+0000","last_became_active":"2024-09-10T07:53:11.248544+0000","last_became_peered":"2024-09-10T07:53:11.248544+0000","last_unstale":"2024-09-10T07:53:11.250788+0000","last_undegraded":"2024-09-10T07:53:11.250788+0000","last_fullsized":"2024-09-10T07:53:11.250788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:07.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268745+0000","last_change":"2024-09-10T07:53:10.213371+0000","last_active":"2024-09-10T07:53:11.268745+0000","last_peered":"2024-09-10T07:53:11.268745+0000","last_clean":"2024-09-10T07:53:11.268745+0000","last_became_active":"2024-09-10T07:53:10.211436+0000","last_became_peered":"2024-09-10T07:53:10.211436+0000","last_unstale":"2024-09-10T07:53:11.268745+0000","last_undegraded":"2024-09-10T07:53:11.268745+0000","last_fullsized":"2024-09-10T07:53:11.268745+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:54.890133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.239996+0000","last_change":"2024-09-10T07:53:11.235023+0000","last_active":"2024-09-10T07:53:11.239996+0000","last_peered":"2024-09-10T07:53:11.239996+0000","last_clean":"2024-09-10T07:53:11.239996+0000","last_became_active":"2024-09-10T07:53:11.234748+0000","last_became_peered":"2024-09-10T07:53:11.234748+0000","last_unstale":"2024-09-10T07:53:11.239996+0000","last_undegraded":"2024-09-10T07:53:11.239996+0000","last_fullsized":"2024-09-10T07:53:11.239996+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.136947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235891+0000","last_change":"2024-09-10T07:53:10.232440+0000","last_active":"2024-09-10T07:53:10.235891+0000","last_peered":"2024-09-10T07:53:10.235891+0000","last_clean":"2024-09-10T07:53:10.235891+0000","last_became_active":"2024-09-10T07:53:10.229715+0000","last_became_peered":"2024-09-10T07:53:10.229715+0000","last_unstale":"2024-09-10T07:53:10.235891+0000","last_undegraded":"2024-09-10T07:53:10.235891+0000","last_fullsized":"2024-09-10T07:53:10.235891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:05.012224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212060+0000","last_change":"2024-09-10T07:53:10.212060+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212060+0000","last_undegraded":"2024-09-10T07:53:10.212060+0000","last_fullsized":"2024-09-10T07:53:10.212060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.233676+0000","last_change":"2024-09-10T07:53:10.227747+0000","last_active":"2024-09-10T07:53:10.233676+0000","last_peered":"2024-09-10T07:53:10.233676+0000","last_clean":"2024-09-10T07:53:10.233676+0000","last_became_active":"2024-09-10T07:53:10.225865+0000","last_became_peered":"2024-09-10T07:53:10.225865+0000","last_unstale":"2024-09-10T07:53:10.233676+0000","last_undegraded":"2024-09-10T07:53:10.233676+0000","last_fullsized":"2024-09-10T07:53:10.233676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:21.770452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211732+0000","last_change":"2024-09-10T07:53:10.211732+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211732+0000","last_undegraded":"2024-09-10T07:53:10.211732+0000","last_fullsized":"2024-09-10T07:53:10.211732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.247861+0000","last_change":"2024-09-10T07:53:10.228787+0000","last_active":"2024-09-10T07:53:11.247861+0000","last_peered":"2024-09-10T07:53:11.247861+0000","last_clean":"2024-09-10T07:53:11.247861+0000","last_became_active":"2024-09-10T07:53:10.228445+0000","last_became_peered":"2024-09-10T07:53:10.228445+0000","last_unstale":"2024-09-10T07:53:11.247861+0000","last_undegraded":"2024-09-10T07:53:11.247861+0000","last_fullsized":"2024-09-10T07:53:11.247861+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:55.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211105+0000","last_change":"2024-09-10T07:53:10.211105+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211105+0000","last_undegraded":"2024-09-10T07:53:10.211105+0000","last_fullsized":"2024-09-10T07:53:10.211105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.245589+0000","last_change":"2024-09-10T07:53:10.228240+0000","last_active":"2024-09-10T07:53:11.245589+0000","last_peered":"2024-09-10T07:53:11.245589+0000","last_clean":"2024-09-10T07:53:11.245589+0000","last_became_active":"2024-09-10T07:53:10.227508+0000","last_became_peered":"2024-09-10T07:53:10.227508+0000","last_unstale":"2024-09-10T07:53:11.245589+0000","last_undegraded":"2024-09-10T07:53:11.245589+0000","last_fullsized":"2024-09-10T07:53:11.245589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:39.308304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.261353+0000","last_change":"2024-09-10T07:53:11.259531+0000","last_active":"2024-09-10T07:53:11.261353+0000","last_peered":"2024-09-10T07:53:11.261353+0000","last_clean":"2024-09-10T07:53:11.261353+0000","last_became_active":"2024-09-10T07:53:11.258000+0000","last_became_peered":"2024-09-10T07:53:11.258000+0000","last_unstale":"2024-09-10T07:53:11.261353+0000","last_undegraded":"2024-09-10T07:53:11.261353+0000","last_fullsized":"2024-09-10T07:53:11.261353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:06.014591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232682+0000","last_change":"2024-09-10T07:53:10.231034+0000","last_active":"2024-09-10T07:53:12.232682+0000","last_peered":"2024-09-10T07:53:12.232682+0000","last_clean":"2024-09-10T07:53:12.232682+0000","last_became_active":"2024-09-10T07:53:10.230111+0000","last_became_peered":"2024-09-10T07:53:10.230111+0000","last_unstale":"2024-09-10T07:53:12.232682+0000","last_undegraded":"2024-09-10T07:53:12.232682+0000","last_fullsized":"2024-09-10T07:53:12.232682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:58.029264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.217413+0000","last_change":"2024-09-10T07:53:10.217413+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.217413+0000","last_undegraded":"2024-09-10T07:53:10.217413+0000","last_fullsized":"2024-09-10T07:53:10.217413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.267444+0000","last_change":"2024-09-10T07:53:10.222067+0000","last_active":"2024-09-10T07:53:11.267444+0000","last_peered":"2024-09-10T07:53:11.267444+0000","last_clean":"2024-09-10T07:53:11.267444+0000","last_became_active":"2024-09-10T07:53:10.221728+0000","last_became_peered":"2024-09-10T07:53:10.221728+0000","last_unstale":"2024-09-10T07:53:11.267444+0000","last_undegraded":"2024-09-10T07:53:11.267444+0000","last_fullsized":"2024-09-10T07:53:11.267444+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212195+0000","last_change":"2024-09-10T07:53:10.212195+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212195+0000","last_undegraded":"2024-09-10T07:53:10.212195+0000","last_fullsized":"2024-09-10T07:53:10.212195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236035+0000","last_change":"2024-09-10T07:53:10.232887+0000","last_active":"2024-09-10T07:53:10.236035+0000","last_peered":"2024-09-10T07:53:10.236035+0000","last_clean":"2024-09-10T07:53:10.236035+0000","last_became_active":"2024-09-10T07:53:10.230235+0000","last_became_peered":"2024-09-10T07:53:10.230235+0000","last_unstale":"2024-09-10T07:53:10.236035+0000","last_undegraded":"2024-09-10T07:53:10.236035+0000","last_fullsized":"2024-09-10T07:53:10.236035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:14.807864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233395+0000","last_change":"2024-09-10T07:53:11.254067+0000","last_active":"2024-09-10T07:53:12.233395+0000","last_peered":"2024-09-10T07:53:12.233395+0000","last_clean":"2024-09-10T07:53:12.233395+0000","last_became_active":"2024-09-10T07:53:11.253708+0000","last_became_peered":"2024-09-10T07:53:11.253708+0000","last_unstale":"2024-09-10T07:53:12.233395+0000","last_undegraded":"2024-09-10T07:53:12.233395+0000","last_fullsized":"2024-09-10T07:53:12.233395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:01.851731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.235237+0000","last_change":"2024-09-10T07:53:10.223013+0000","last_active":"2024-09-10T07:53:10.235237+0000","last_peered":"2024-09-10T07:53:10.235237+0000","last_clean":"2024-09-10T07:53:10.235237+0000","last_became_active":"2024-09-10T07:53:10.222053+0000","last_became_peered":"2024-09-10T07:53:10.222053+0000","last_unstale":"2024-09-10T07:53:10.235237+0000","last_undegraded":"2024-09-10T07:53:10.235237+0000","last_fullsized":"2024-09-10T07:53:10.235237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:42.615374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.258362+0000","last_change":"2024-09-10T07:53:11.247894+0000","last_active":"2024-09-10T07:53:11.258362+0000","last_peered":"2024-09-10T07:53:11.258362+0000","last_clean":"2024-09-10T07:53:11.258362+0000","last_became_active":"2024-09-10T07:53:11.247569+0000","last_became_peered":"2024-09-10T07:53:11.247569+0000","last_unstale":"2024-09-10T07:53:11.258362+0000","last_undegraded":"2024-09-10T07:53:11.258362+0000","last_fullsized":"2024-09-10T07:53:11.258362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:37.700469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248875+0000","last_change":"2024-09-10T07:53:10.223877+0000","last_active":"2024-09-10T07:53:11.248875+0000","last_peered":"2024-09-10T07:53:11.248875+0000","last_clean":"2024-09-10T07:53:11.248875+0000","last_became_active":"2024-09-10T07:53:10.222951+0000","last_became_peered":"2024-09-10T07:53:10.222951+0000","last_unstale":"2024-09-10T07:53:11.248875+0000","last_undegraded":"2024-09-10T07:53:11.248875+0000","last_fullsized":"2024-09-10T07:53:11.248875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:16.490800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.211556+0000","last_change":"2024-09-10T07:53:10.211556+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.211556+0000","last_undegraded":"2024-09-10T07:53:10.211556+0000","last_fullsized":"2024-09-10T07:53:10.211556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233746+0000","last_change":"2024-09-10T07:53:10.227674+0000","last_active":"2024-09-10T07:53:12.233746+0000","last_peered":"2024-09-10T07:53:12.233746+0000","last_clean":"2024-09-10T07:53:12.233746+0000","last_became_active":"2024-09-10T07:53:10.227321+0000","last_became_peered":"2024-09-10T07:53:10.227321+0000","last_unstale":"2024-09-10T07:53:12.233746+0000","last_undegraded":"2024-09-10T07:53:12.233746+0000","last_fullsized":"2024-09-10T07:53:12.233746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:58.837406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.260406+0000","last_change":"2024-09-10T07:53:11.252691+0000","last_active":"2024-09-10T07:53:11.260406+0000","last_peered":"2024-09-10T07:53:11.260406+0000","last_clean":"2024-09-10T07:53:11.260406+0000","last_became_active":"2024-09-10T07:53:11.252471+0000","last_became_peered":"2024-09-10T07:53:11.252471+0000","last_unstale":"2024-09-10T07:53:11.260406+0000","last_undegraded":"2024-09-10T07:53:11.260406+0000","last_fullsized":"2024-09-10T07:53:11.260406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:38:41.229077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.248309+0000","last_change":"2024-09-10T07:53:10.230718+0000","last_active":"2024-09-10T07:53:11.248309+0000","last_peered":"2024-09-10T07:53:11.248309+0000","last_clean":"2024-09-10T07:53:11.248309+0000","last_became_active":"2024-09-10T07:53:10.230504+0000","last_became_peered":"2024-09-10T07:53:10.230504+0000","last_unstale":"2024-09-10T07:53:11.248309+0000","last_undegraded":"2024-09-10T07:53:11.248309+0000","last_fullsized":"2024-09-10T07:53:11.248309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:14.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-10T07:53:10.212681+0000","last_change":"2024-09-10T07:53:10.212681+0000","last_active":"2024-09-10T07:53:10.199182+0000","last_peered":"2024-09-10T07:53:10.199182+0000","last_clean":"2024-09-10T07:53:10.199182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T07:53:10.212681+0000","last_undegraded":"2024-09-10T07:53:10.212681+0000","last_fullsized":"2024-09-10T07:53:10.212681+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-10T07:53:10.236579+0000","last_change":"2024-09-10T07:53:10.234562+0000","last_active":"2024-09-10T07:53:10.236579+0000","last_peered":"2024-09-10T07:53:10.236579+0000","last_clean":"2024-09-10T07:53:10.236579+0000","last_became_active":"2024-09-10T07:53:10.234276+0000","last_became_peered":"2024-09-10T07:53:10.234276+0000","last_unstale":"2024-09-10T07:53:10.236579+0000","last_undegraded":"2024-09-10T07:53:10.236579+0000","last_fullsized":"2024-09-10T07:53:10.236579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:21.272222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.268617+0000","last_change":"2024-09-10T07:53:11.264027+0000","last_active":"2024-09-10T07:53:11.268617+0000","last_peered":"2024-09-10T07:53:11.268617+0000","last_clean":"2024-09-10T07:53:11.268617+0000","last_became_active":"2024-09-10T07:53:11.263741+0000","last_became_peered":"2024-09-10T07:53:11.263741+0000","last_unstale":"2024-09-10T07:53:11.268617+0000","last_undegraded":"2024-09-10T07:53:11.268617+0000","last_fullsized":"2024-09-10T07:53:11.268617+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:04.728438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232795+0000","last_change":"2024-09-10T07:53:10.224089+0000","last_active":"2024-09-10T07:53:12.232795+0000","last_peered":"2024-09-10T07:53:12.232795+0000","last_clean":"2024-09-10T07:53:12.232795+0000","last_became_active":"2024-09-10T07:53:10.222933+0000","last_became_peered":"2024-09-10T07:53:10.222933+0000","last_unstale":"2024-09-10T07:53:12.232795+0000","last_undegraded":"2024-09-10T07:53:12.232795+0000","last_fullsized":"2024-09-10T07:53:12.232795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:01.604342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234021+0000","last_change":"2024-09-10T07:53:11.266735+0000","last_active":"2024-09-10T07:53:12.234021+0000","last_peered":"2024-09-10T07:53:12.234021+0000","last_clean":"2024-09-10T07:53:12.234021+0000","last_became_active":"2024-09-10T07:53:11.266398+0000","last_became_peered":"2024-09-10T07:53:11.266398+0000","last_unstale":"2024-09-10T07:53:12.234021+0000","last_undegraded":"2024-09-10T07:53:12.234021+0000","last_fullsized":"2024-09-10T07:53:12.234021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:05.878447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.271008+0000","last_change":"2024-09-10T07:53:10.235291+0000","last_active":"2024-09-10T07:53:11.271008+0000","last_peered":"2024-09-10T07:53:11.271008+0000","last_clean":"2024-09-10T07:53:11.271008+0000","last_became_active":"2024-09-10T07:53:10.235015+0000","last_became_peered":"2024-09-10T07:53:10.235015+0000","last_unstale":"2024-09-10T07:53:11.271008+0000","last_undegraded":"2024-09-10T07:53:11.271008+0000","last_fullsized":"2024-09-10T07:53:11.271008+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":314,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T07:53:11.231633+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:53:11.231633+0000","last_peered":"2024-09-10T07:53:11.231633+0000","last_clean":"2024-09-10T07:53:11.231633+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:53:11.231633+0000","last_undegraded":"2024-09-10T07:53:11.231633+0000","last_fullsized":"2024-09-10T07:53:11.231633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476822,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476822,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":1127976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":529109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1212416,"data_stored":522834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1113202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1241088,"data_stored":536466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:53:20.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:53:20.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.324+0000 7fbaebe00640 1 Processor -- start 2024-09-10T07:53:20.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.324+0000 7fbaebe00640 1 -- start start 2024-09-10T07:53:20.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec0d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaebe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0d3060 con 0x7fbaec161de0 2024-09-10T07:53:20.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaebe00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0d3340 con 0x7fbaec161a00 2024-09-10T07:53:20.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaebe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0d3620 con 0x7fbaec0d1800 2024-09-10T07:53:20.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaea000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbaec161de0 0x7fbaec0d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56728/0 (socket says 172.21.15.3:56728) 2024-09-10T07:53:20.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.325+0000 7fbaea000640 1 -- 172.21.15.3:0/2587299511 learned_addr learned my addr 172.21.15.3:0/2587299511 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbaeb400640 1 --2- 172.21.15.3:0/2587299511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec0d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93883788 0 0) 0x7fbaec0d3060 con 0x7fbaec161de0 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbad0003520 con 0x7fbaec161de0 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178496298 0 0) 0x7fbaec0d3340 con 0x7fbaec161a00 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaec0d3060 con 0x7fbaec161a00 2024-09-10T07:53:20.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbaeb400640 1 -- 172.21.15.3:0/2587299511 >> v1:172.21.15.70:6789/0 conn(0x7fbaec161a00 legacy=0x7fbaec0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382615789 0 0) 0x7fbad0003520 con 0x7fbaec161de0 2024-09-10T07:53:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbaeb400640 1 -- 172.21.15.3:0/2587299511 >> v1:172.21.15.3:6789/0 conn(0x7fbaec161de0 legacy=0x7fbaec0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.326+0000 7fbaeb400640 1 -- 172.21.15.3:0/2587299511 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0d3900 con 0x7fbaec0d1800 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.327+0000 7fbaeb400640 1 --2- 172.21.15.3:0/2587299511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec0d2a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009dc0 tx=0x7fbae402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3625c4cafac98ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8001670 con 0x7fbaec0d1800 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4007540 con 0x7fbaec0d1800 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbae9600640 1 -- 172.21.15.3:0/2587299511 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40078f0 con 0x7fbaec0d1800 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 -- 172.21.15.3:0/2587299511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 msgr2=0x7fbaec0d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 --2- 172.21.15.3:0/2587299511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec0d2a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009dc0 tx=0x7fbae402a660 comp rx=0 tx=0).stop 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 -- 172.21.15.3:0/2587299511 shutdown_connections 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 --2- 172.21.15.3:0/2587299511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec0d2a80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 -- 172.21.15.3:0/2587299511 >> 172.21.15.3:0/2587299511 conn(0x7fbaec072770 msgr2=0x7fbaec06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:20.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 -- 172.21.15.3:0/2587299511 shutdown_connections 2024-09-10T07:53:20.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.328+0000 7fbaebe00640 1 -- 172.21.15.3:0/2587299511 wait complete. 2024-09-10T07:53:20.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.330+0000 7fbaebe00640 1 Processor -- start 2024-09-10T07:53:20.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.330+0000 7fbaebe00640 1 -- start start 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 0x7fbaec15d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b8b80 con 0x7fbaec161a00 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b8a00 con 0x7fbaec161de0 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaebe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0b8d00 con 0x7fbaec0d1800 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaea000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35938/0 (socket says 172.21.15.3:35938) 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaea000640 1 -- 172.21.15.3:0/146358947 learned_addr learned my addr 172.21.15.3:0/146358947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaeb400640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 0x7fbaec15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.331+0000 7fbaeaa00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 -- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 msgr2=0x7fbaec14c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec14c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 -- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 msgr2=0x7fbaec15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 0x7fbaec15d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 -- 172.21.15.3:0/146358947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0b5420 con 0x7fbaec161a00 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaeb400640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 0x7fbaec15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaea000640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fbad8001eb0 tx=0x7fbad800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8005ad0 con 0x7fbaec161a00 2024-09-10T07:53:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8005c70 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8005e60 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec0b56a0 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.332+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec0b5c60 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.333+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec0d2a80 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.334+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbad80072b0 con 0x7fbaec161a00 2024-09-10T07:53:20.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.335+0000 7fbae2a00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.335+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbad808ac50 con 0x7fbaec161a00 2024-09-10T07:53:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.335+0000 7fbaeaa00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.336+0000 7fbaeaa00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbadc007fd0 tx=0x7fbadc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.339+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbaec0b5420 con 0x7fbaec161a00 2024-09-10T07:53:20.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.520+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbaec160f40 con 0x7fbacc06be30 2024-09-10T07:53:20.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.526+0000 7fbae2a00640 1 -- 172.21.15.3:0/146358947 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252408 (secure 0 0 0) 0x7fbaec160f40 con 0x7fbacc06be30 2024-09-10T07:53:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:20.530 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:53:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 msgr2=0x7fbacc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbadc007fd0 tx=0x7fbadc009040 comp rx=0 tx=0).stop 2024-09-10T07:53:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 msgr2=0x7fbaec1f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fbad8001eb0 tx=0x7fbad800cbe0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 shutdown_connections 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbacc06be30 0x7fbacc06e2f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaec161de0 0x7fbaec15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaec161a00 0x7fbaec1f50d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 --2- 172.21.15.3:0/146358947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbaec0d1800 0x7fbaec14c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 >> 172.21.15.3:0/146358947 conn(0x7fbaec072770 msgr2=0x7fbaec0b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 shutdown_connections 2024-09-10T07:53:20.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.531+0000 7fbaebe00640 1 -- 172.21.15.3:0/146358947 wait complete. 2024-09-10T07:53:20.547 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-10T07:53:19.581465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264676,"kb_used_data":10984,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710044,"statfs":{"total":773094113280,"available":772823085056,"internally_reserved":0,"allocated":11247616,"data_stored":5523028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.353323"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234059+0000","last_change":"2024-09-10T07:53:11.265143+0000","last_active":"2024-09-10T07:53:12.234059+0000","last_peered":"2024-09-10T07:53:12.234059+0000","last_clean":"2024-09-10T07:53:12.234059+0000","last_became_active":"2024-09-10T07:53:11.264823+0000","last_became_peered":"2024-09-10T07:53:11.264823+0000","last_unstale":"2024-09-10T07:53:12.234059+0000","last_undegraded":"2024-09-10T07:53:12.234059+0000","last_fullsized":"2024-09-10T07:53:12.234059+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:07.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234529+0000","last_change":"2024-09-10T07:53:11.265559+0000","last_active":"2024-09-10T07:53:12.234529+0000","last_peered":"2024-09-10T07:53:12.234529+0000","last_clean":"2024-09-10T07:53:12.234529+0000","last_became_active":"2024-09-10T07:53:11.260913+0000","last_became_peered":"2024-09-10T07:53:11.260913+0000","last_unstale":"2024-09-10T07:53:12.234529+0000","last_undegraded":"2024-09-10T07:53:12.234529+0000","last_fullsized":"2024-09-10T07:53:12.234529+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.717712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233484+0000","last_change":"2024-09-10T07:53:11.252106+0000","last_active":"2024-09-10T07:53:12.233484+0000","last_peered":"2024-09-10T07:53:12.233484+0000","last_clean":"2024-09-10T07:53:12.233484+0000","last_became_active":"2024-09-10T07:53:11.251807+0000","last_became_peered":"2024-09-10T07:53:11.251807+0000","last_unstale":"2024-09-10T07:53:12.233484+0000","last_undegraded":"2024-09-10T07:53:12.233484+0000","last_fullsized":"2024-09-10T07:53:12.233484+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:01.780830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234117+0000","last_change":"2024-09-10T07:53:11.265463+0000","last_active":"2024-09-10T07:53:12.234117+0000","last_peered":"2024-09-10T07:53:12.234117+0000","last_clean":"2024-09-10T07:53:12.234117+0000","last_became_active":"2024-09-10T07:53:11.265175+0000","last_became_peered":"2024-09-10T07:53:11.265175+0000","last_unstale":"2024-09-10T07:53:12.234117+0000","last_undegraded":"2024-09-10T07:53:12.234117+0000","last_fullsized":"2024-09-10T07:53:12.234117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:53.440934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440988+0000","last_change":"2024-09-10T07:53:11.234549+0000","last_active":"2024-09-10T07:53:12.440988+0000","last_peered":"2024-09-10T07:53:12.440988+0000","last_clean":"2024-09-10T07:53:12.440988+0000","last_became_active":"2024-09-10T07:53:11.234201+0000","last_became_peered":"2024-09-10T07:53:11.234201+0000","last_unstale":"2024-09-10T07:53:12.440988+0000","last_undegraded":"2024-09-10T07:53:12.440988+0000","last_fullsized":"2024-09-10T07:53:12.440988+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:34.377642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440929+0000","last_change":"2024-09-10T07:53:11.236870+0000","last_active":"2024-09-10T07:53:12.440929+0000","last_peered":"2024-09-10T07:53:12.440929+0000","last_clean":"2024-09-10T07:53:12.440929+0000","last_became_active":"2024-09-10T07:53:11.235843+0000","last_became_peered":"2024-09-10T07:53:11.235843+0000","last_unstale":"2024-09-10T07:53:12.440929+0000","last_undegraded":"2024-09-10T07:53:12.440929+0000","last_fullsized":"2024-09-10T07:53:12.440929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:27.651486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441046+0000","last_change":"2024-09-10T07:53:11.255394+0000","last_active":"2024-09-10T07:53:12.441046+0000","last_peered":"2024-09-10T07:53:12.441046+0000","last_clean":"2024-09-10T07:53:12.441046+0000","last_became_active":"2024-09-10T07:53:11.254136+0000","last_became_peered":"2024-09-10T07:53:11.254136+0000","last_unstale":"2024-09-10T07:53:12.441046+0000","last_undegraded":"2024-09-10T07:53:12.441046+0000","last_fullsized":"2024-09-10T07:53:12.441046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:11.116097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233314+0000","last_change":"2024-09-10T07:53:11.250032+0000","last_active":"2024-09-10T07:53:12.233314+0000","last_peered":"2024-09-10T07:53:12.233314+0000","last_clean":"2024-09-10T07:53:12.233314+0000","last_became_active":"2024-09-10T07:53:11.249626+0000","last_became_peered":"2024-09-10T07:53:11.249626+0000","last_unstale":"2024-09-10T07:53:12.233314+0000","last_undegraded":"2024-09-10T07:53:12.233314+0000","last_fullsized":"2024-09-10T07:53:12.233314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:17.283338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441430+0000","last_change":"2024-09-10T07:53:11.243615+0000","last_active":"2024-09-10T07:53:12.441430+0000","last_peered":"2024-09-10T07:53:12.441430+0000","last_clean":"2024-09-10T07:53:12.441430+0000","last_became_active":"2024-09-10T07:53:11.243351+0000","last_became_peered":"2024-09-10T07:53:11.243351+0000","last_unstale":"2024-09-10T07:53:12.441430+0000","last_undegraded":"2024-09-10T07:53:12.441430+0000","last_fullsized":"2024-09-10T07:53:12.441430+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:19.591113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.231466+0000","last_change":"2024-09-10T07:53:11.267004+0000","last_active":"2024-09-10T07:53:12.231466+0000","last_peered":"2024-09-10T07:53:12.231466+0000","last_clean":"2024-09-10T07:53:12.231466+0000","last_became_active":"2024-09-10T07:53:11.266699+0000","last_became_peered":"2024-09-10T07:53:11.266699+0000","last_unstale":"2024-09-10T07:53:12.231466+0000","last_undegraded":"2024-09-10T07:53:12.231466+0000","last_fullsized":"2024-09-10T07:53:12.231466+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:48.470207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440704+0000","last_change":"2024-09-10T07:53:11.236475+0000","last_active":"2024-09-10T07:53:12.440704+0000","last_peered":"2024-09-10T07:53:12.440704+0000","last_clean":"2024-09-10T07:53:12.440704+0000","last_became_active":"2024-09-10T07:53:11.235672+0000","last_became_peered":"2024-09-10T07:53:11.235672+0000","last_unstale":"2024-09-10T07:53:12.440704+0000","last_undegraded":"2024-09-10T07:53:12.440704+0000","last_fullsized":"2024-09-10T07:53:12.440704+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:38.397740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440985+0000","last_change":"2024-09-10T07:53:11.267139+0000","last_active":"2024-09-10T07:53:12.440985+0000","last_peered":"2024-09-10T07:53:12.440985+0000","last_clean":"2024-09-10T07:53:12.440985+0000","last_became_active":"2024-09-10T07:53:11.266829+0000","last_became_peered":"2024-09-10T07:53:11.266829+0000","last_unstale":"2024-09-10T07:53:12.440985+0000","last_undegraded":"2024-09-10T07:53:12.440985+0000","last_fullsized":"2024-09-10T07:53:12.440985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:08.715930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440328+0000","last_change":"2024-09-10T07:53:11.246352+0000","last_active":"2024-09-10T07:53:12.440328+0000","last_peered":"2024-09-10T07:53:12.440328+0000","last_clean":"2024-09-10T07:53:12.440328+0000","last_became_active":"2024-09-10T07:53:11.246037+0000","last_became_peered":"2024-09-10T07:53:11.246037+0000","last_unstale":"2024-09-10T07:53:12.440328+0000","last_undegraded":"2024-09-10T07:53:12.440328+0000","last_fullsized":"2024-09-10T07:53:12.440328+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.136947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234521+0000","last_change":"2024-09-10T07:53:11.271061+0000","last_active":"2024-09-10T07:53:12.234521+0000","last_peered":"2024-09-10T07:53:12.234521+0000","last_clean":"2024-09-10T07:53:12.234521+0000","last_became_active":"2024-09-10T07:53:11.270340+0000","last_became_peered":"2024-09-10T07:53:11.270340+0000","last_unstale":"2024-09-10T07:53:12.234521+0000","last_undegraded":"2024-09-10T07:53:12.234521+0000","last_fullsized":"2024-09-10T07:53:12.234521+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:15.810230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233663+0000","last_change":"2024-09-10T07:53:11.261668+0000","last_active":"2024-09-10T07:53:12.233663+0000","last_peered":"2024-09-10T07:53:12.233663+0000","last_clean":"2024-09-10T07:53:12.233663+0000","last_became_active":"2024-09-10T07:53:11.261082+0000","last_became_peered":"2024-09-10T07:53:11.261082+0000","last_unstale":"2024-09-10T07:53:12.233663+0000","last_undegraded":"2024-09-10T07:53:12.233663+0000","last_fullsized":"2024-09-10T07:53:12.233663+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:15.547987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440315+0000","last_change":"2024-09-10T07:53:11.254673+0000","last_active":"2024-09-10T07:53:12.440315+0000","last_peered":"2024-09-10T07:53:12.440315+0000","last_clean":"2024-09-10T07:53:12.440315+0000","last_became_active":"2024-09-10T07:53:11.251614+0000","last_became_peered":"2024-09-10T07:53:11.251614+0000","last_unstale":"2024-09-10T07:53:12.440315+0000","last_undegraded":"2024-09-10T07:53:12.440315+0000","last_fullsized":"2024-09-10T07:53:12.440315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:27.773781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255300+0000","last_change":"2024-09-10T07:53:11.247909+0000","last_active":"2024-09-10T07:53:12.255300+0000","last_peered":"2024-09-10T07:53:12.255300+0000","last_clean":"2024-09-10T07:53:12.255300+0000","last_became_active":"2024-09-10T07:53:11.247595+0000","last_became_peered":"2024-09-10T07:53:11.247595+0000","last_unstale":"2024-09-10T07:53:12.255300+0000","last_undegraded":"2024-09-10T07:53:12.255300+0000","last_fullsized":"2024-09-10T07:53:12.255300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:50:54.214802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234371+0000","last_change":"2024-09-10T07:53:11.261199+0000","last_active":"2024-09-10T07:53:12.234371+0000","last_peered":"2024-09-10T07:53:12.234371+0000","last_clean":"2024-09-10T07:53:12.234371+0000","last_became_active":"2024-09-10T07:53:11.260502+0000","last_became_peered":"2024-09-10T07:53:11.260502+0000","last_unstale":"2024-09-10T07:53:12.234371+0000","last_undegraded":"2024-09-10T07:53:12.234371+0000","last_fullsized":"2024-09-10T07:53:12.234371+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:19.541855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255433+0000","last_change":"2024-09-10T07:53:11.264200+0000","last_active":"2024-09-10T07:53:12.255433+0000","last_peered":"2024-09-10T07:53:12.255433+0000","last_clean":"2024-09-10T07:53:12.255433+0000","last_became_active":"2024-09-10T07:53:11.263834+0000","last_became_peered":"2024-09-10T07:53:11.263834+0000","last_unstale":"2024-09-10T07:53:12.255433+0000","last_undegraded":"2024-09-10T07:53:12.255433+0000","last_fullsized":"2024-09-10T07:53:12.255433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:05:40.028081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441135+0000","last_change":"2024-09-10T07:53:11.234958+0000","last_active":"2024-09-10T07:53:12.441135+0000","last_peered":"2024-09-10T07:53:12.441135+0000","last_clean":"2024-09-10T07:53:12.441135+0000","last_became_active":"2024-09-10T07:53:11.234678+0000","last_became_peered":"2024-09-10T07:53:11.234678+0000","last_unstale":"2024-09-10T07:53:12.441135+0000","last_undegraded":"2024-09-10T07:53:12.441135+0000","last_fullsized":"2024-09-10T07:53:12.441135+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:48.470207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233477+0000","last_change":"2024-09-10T07:53:11.248147+0000","last_active":"2024-09-10T07:53:12.233477+0000","last_peered":"2024-09-10T07:53:12.233477+0000","last_clean":"2024-09-10T07:53:12.233477+0000","last_became_active":"2024-09-10T07:53:11.247829+0000","last_became_peered":"2024-09-10T07:53:11.247829+0000","last_unstale":"2024-09-10T07:53:12.233477+0000","last_undegraded":"2024-09-10T07:53:12.233477+0000","last_fullsized":"2024-09-10T07:53:12.233477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:54.042192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232852+0000","last_change":"2024-09-10T07:53:11.249206+0000","last_active":"2024-09-10T07:53:12.232852+0000","last_peered":"2024-09-10T07:53:12.232852+0000","last_clean":"2024-09-10T07:53:12.232852+0000","last_became_active":"2024-09-10T07:53:11.248315+0000","last_became_peered":"2024-09-10T07:53:11.248315+0000","last_unstale":"2024-09-10T07:53:12.232852+0000","last_undegraded":"2024-09-10T07:53:12.232852+0000","last_fullsized":"2024-09-10T07:53:12.232852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:31.145698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233925+0000","last_change":"2024-09-10T07:53:11.266141+0000","last_active":"2024-09-10T07:53:12.233925+0000","last_peered":"2024-09-10T07:53:12.233925+0000","last_clean":"2024-09-10T07:53:12.233925+0000","last_became_active":"2024-09-10T07:53:11.261434+0000","last_became_peered":"2024-09-10T07:53:11.261434+0000","last_unstale":"2024-09-10T07:53:12.233925+0000","last_undegraded":"2024-09-10T07:53:12.233925+0000","last_fullsized":"2024-09-10T07:53:12.233925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:17.493167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440228+0000","last_change":"2024-09-10T07:53:11.262775+0000","last_active":"2024-09-10T07:53:12.440228+0000","last_peered":"2024-09-10T07:53:12.440228+0000","last_clean":"2024-09-10T07:53:12.440228+0000","last_became_active":"2024-09-10T07:53:11.262438+0000","last_became_peered":"2024-09-10T07:53:11.262438+0000","last_unstale":"2024-09-10T07:53:12.440228+0000","last_undegraded":"2024-09-10T07:53:12.440228+0000","last_fullsized":"2024-09-10T07:53:12.440228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:38.674643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.231591+0000","last_change":"2024-09-10T07:53:11.255844+0000","last_active":"2024-09-10T07:53:12.231591+0000","last_peered":"2024-09-10T07:53:12.231591+0000","last_clean":"2024-09-10T07:53:12.231591+0000","last_became_active":"2024-09-10T07:53:11.255548+0000","last_became_peered":"2024-09-10T07:53:11.255548+0000","last_unstale":"2024-09-10T07:53:12.231591+0000","last_undegraded":"2024-09-10T07:53:12.231591+0000","last_fullsized":"2024-09-10T07:53:12.231591+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440468+0000","last_change":"2024-09-10T07:53:11.253087+0000","last_active":"2024-09-10T07:53:12.440468+0000","last_peered":"2024-09-10T07:53:12.440468+0000","last_clean":"2024-09-10T07:53:12.440468+0000","last_became_active":"2024-09-10T07:53:11.252826+0000","last_became_peered":"2024-09-10T07:53:11.252826+0000","last_unstale":"2024-09-10T07:53:12.440468+0000","last_undegraded":"2024-09-10T07:53:12.440468+0000","last_fullsized":"2024-09-10T07:53:12.440468+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:04.728438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441831+0000","last_change":"2024-09-10T07:53:11.246590+0000","last_active":"2024-09-10T07:53:12.441831+0000","last_peered":"2024-09-10T07:53:12.441831+0000","last_clean":"2024-09-10T07:53:12.441831+0000","last_became_active":"2024-09-10T07:53:11.245943+0000","last_became_peered":"2024-09-10T07:53:11.245943+0000","last_unstale":"2024-09-10T07:53:12.441831+0000","last_undegraded":"2024-09-10T07:53:12.441831+0000","last_fullsized":"2024-09-10T07:53:12.441831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:22.033454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440555+0000","last_change":"2024-09-10T07:53:11.256724+0000","last_active":"2024-09-10T07:53:12.440555+0000","last_peered":"2024-09-10T07:53:12.440555+0000","last_clean":"2024-09-10T07:53:12.440555+0000","last_became_active":"2024-09-10T07:53:11.256427+0000","last_became_peered":"2024-09-10T07:53:11.256427+0000","last_unstale":"2024-09-10T07:53:12.440555+0000","last_undegraded":"2024-09-10T07:53:12.440555+0000","last_fullsized":"2024-09-10T07:53:12.440555+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:46.766730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440426+0000","last_change":"2024-09-10T07:53:11.249779+0000","last_active":"2024-09-10T07:53:12.440426+0000","last_peered":"2024-09-10T07:53:12.440426+0000","last_clean":"2024-09-10T07:53:12.440426+0000","last_became_active":"2024-09-10T07:53:11.249501+0000","last_became_peered":"2024-09-10T07:53:11.249501+0000","last_unstale":"2024-09-10T07:53:12.440426+0000","last_undegraded":"2024-09-10T07:53:12.440426+0000","last_fullsized":"2024-09-10T07:53:12.440426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:05.491743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440198+0000","last_change":"2024-09-10T07:53:11.254494+0000","last_active":"2024-09-10T07:53:12.440198+0000","last_peered":"2024-09-10T07:53:12.440198+0000","last_clean":"2024-09-10T07:53:12.440198+0000","last_became_active":"2024-09-10T07:53:11.254202+0000","last_became_peered":"2024-09-10T07:53:11.254202+0000","last_unstale":"2024-09-10T07:53:12.440198+0000","last_undegraded":"2024-09-10T07:53:12.440198+0000","last_fullsized":"2024-09-10T07:53:12.440198+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:00.667191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234095+0000","last_change":"2024-09-10T07:53:11.256491+0000","last_active":"2024-09-10T07:53:12.234095+0000","last_peered":"2024-09-10T07:53:12.234095+0000","last_clean":"2024-09-10T07:53:12.234095+0000","last_became_active":"2024-09-10T07:53:11.255961+0000","last_became_peered":"2024-09-10T07:53:11.255961+0000","last_unstale":"2024-09-10T07:53:12.234095+0000","last_undegraded":"2024-09-10T07:53:12.234095+0000","last_fullsized":"2024-09-10T07:53:12.234095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:11.297862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233342+0000","last_change":"2024-09-10T07:53:11.259132+0000","last_active":"2024-09-10T07:53:12.233342+0000","last_peered":"2024-09-10T07:53:12.233342+0000","last_clean":"2024-09-10T07:53:12.233342+0000","last_became_active":"2024-09-10T07:53:11.258833+0000","last_became_peered":"2024-09-10T07:53:11.258833+0000","last_unstale":"2024-09-10T07:53:12.233342+0000","last_undegraded":"2024-09-10T07:53:12.233342+0000","last_fullsized":"2024-09-10T07:53:12.233342+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:57.744666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255464+0000","last_change":"2024-09-10T07:53:11.251365+0000","last_active":"2024-09-10T07:53:12.255464+0000","last_peered":"2024-09-10T07:53:12.255464+0000","last_clean":"2024-09-10T07:53:12.255464+0000","last_became_active":"2024-09-10T07:53:11.251063+0000","last_became_peered":"2024-09-10T07:53:11.251063+0000","last_unstale":"2024-09-10T07:53:12.255464+0000","last_undegraded":"2024-09-10T07:53:12.255464+0000","last_fullsized":"2024-09-10T07:53:12.255464+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:16.835338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440537+0000","last_change":"2024-09-10T07:53:10.235925+0000","last_active":"2024-09-10T07:53:12.440537+0000","last_peered":"2024-09-10T07:53:12.440537+0000","last_clean":"2024-09-10T07:53:12.440537+0000","last_became_active":"2024-09-10T07:53:10.235350+0000","last_became_peered":"2024-09-10T07:53:10.235350+0000","last_unstale":"2024-09-10T07:53:12.440537+0000","last_undegraded":"2024-09-10T07:53:12.440537+0000","last_fullsized":"2024-09-10T07:53:12.440537+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:31.899936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440574+0000","last_change":"2024-09-10T07:53:11.249776+0000","last_active":"2024-09-10T07:53:12.440574+0000","last_peered":"2024-09-10T07:53:12.440574+0000","last_clean":"2024-09-10T07:53:12.440574+0000","last_became_active":"2024-09-10T07:53:11.249456+0000","last_became_peered":"2024-09-10T07:53:11.249456+0000","last_unstale":"2024-09-10T07:53:12.440574+0000","last_undegraded":"2024-09-10T07:53:12.440574+0000","last_fullsized":"2024-09-10T07:53:12.440574+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440622+0000","last_change":"2024-09-10T07:53:10.232379+0000","last_active":"2024-09-10T07:53:12.440622+0000","last_peered":"2024-09-10T07:53:12.440622+0000","last_clean":"2024-09-10T07:53:12.440622+0000","last_became_active":"2024-09-10T07:53:10.231931+0000","last_became_peered":"2024-09-10T07:53:10.231931+0000","last_unstale":"2024-09-10T07:53:12.440622+0000","last_undegraded":"2024-09-10T07:53:12.440622+0000","last_fullsized":"2024-09-10T07:53:12.440622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:22:58.384356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234370+0000","last_change":"2024-09-10T07:53:11.265997+0000","last_active":"2024-09-10T07:53:12.234370+0000","last_peered":"2024-09-10T07:53:12.234370+0000","last_clean":"2024-09-10T07:53:12.234370+0000","last_became_active":"2024-09-10T07:53:11.264896+0000","last_became_peered":"2024-09-10T07:53:11.264896+0000","last_unstale":"2024-09-10T07:53:12.234370+0000","last_undegraded":"2024-09-10T07:53:12.234370+0000","last_fullsized":"2024-09-10T07:53:12.234370+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:43.617740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233603+0000","last_change":"2024-09-10T07:53:10.227323+0000","last_active":"2024-09-10T07:53:12.233603+0000","last_peered":"2024-09-10T07:53:12.233603+0000","last_clean":"2024-09-10T07:53:12.233603+0000","last_became_active":"2024-09-10T07:53:10.227012+0000","last_became_peered":"2024-09-10T07:53:10.227012+0000","last_unstale":"2024-09-10T07:53:12.233603+0000","last_undegraded":"2024-09-10T07:53:12.233603+0000","last_fullsized":"2024-09-10T07:53:12.233603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:56.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.231481+0000","last_change":"2024-09-10T07:53:11.246551+0000","last_active":"2024-09-10T07:53:12.231481+0000","last_peered":"2024-09-10T07:53:12.231481+0000","last_clean":"2024-09-10T07:53:12.231481+0000","last_became_active":"2024-09-10T07:53:11.246300+0000","last_became_peered":"2024-09-10T07:53:11.246300+0000","last_unstale":"2024-09-10T07:53:12.231481+0000","last_undegraded":"2024-09-10T07:53:12.231481+0000","last_fullsized":"2024-09-10T07:53:12.231481+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.506631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234555+0000","last_change":"2024-09-10T07:53:10.231193+0000","last_active":"2024-09-10T07:53:12.234555+0000","last_peered":"2024-09-10T07:53:12.234555+0000","last_clean":"2024-09-10T07:53:12.234555+0000","last_became_active":"2024-09-10T07:53:10.226288+0000","last_became_peered":"2024-09-10T07:53:10.226288+0000","last_unstale":"2024-09-10T07:53:12.234555+0000","last_undegraded":"2024-09-10T07:53:12.234555+0000","last_fullsized":"2024-09-10T07:53:12.234555+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440382+0000","last_change":"2024-09-10T07:53:11.235832+0000","last_active":"2024-09-10T07:53:12.440382+0000","last_peered":"2024-09-10T07:53:12.440382+0000","last_clean":"2024-09-10T07:53:12.440382+0000","last_became_active":"2024-09-10T07:53:11.235506+0000","last_became_peered":"2024-09-10T07:53:11.235506+0000","last_unstale":"2024-09-10T07:53:12.440382+0000","last_undegraded":"2024-09-10T07:53:12.440382+0000","last_fullsized":"2024-09-10T07:53:12.440382+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:06.407725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233713+0000","last_change":"2024-09-10T07:53:10.228445+0000","last_active":"2024-09-10T07:53:12.233713+0000","last_peered":"2024-09-10T07:53:12.233713+0000","last_clean":"2024-09-10T07:53:12.233713+0000","last_became_active":"2024-09-10T07:53:10.227806+0000","last_became_peered":"2024-09-10T07:53:10.227806+0000","last_unstale":"2024-09-10T07:53:12.233713+0000","last_undegraded":"2024-09-10T07:53:12.233713+0000","last_fullsized":"2024-09-10T07:53:12.233713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441091+0000","last_change":"2024-09-10T07:53:11.235298+0000","last_active":"2024-09-10T07:53:12.441091+0000","last_peered":"2024-09-10T07:53:12.441091+0000","last_clean":"2024-09-10T07:53:12.441091+0000","last_became_active":"2024-09-10T07:53:11.234953+0000","last_became_peered":"2024-09-10T07:53:11.234953+0000","last_unstale":"2024-09-10T07:53:12.441091+0000","last_undegraded":"2024-09-10T07:53:12.441091+0000","last_fullsized":"2024-09-10T07:53:12.441091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:11.667107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232971+0000","last_change":"2024-09-10T07:53:10.229226+0000","last_active":"2024-09-10T07:53:12.232971+0000","last_peered":"2024-09-10T07:53:12.232971+0000","last_clean":"2024-09-10T07:53:12.232971+0000","last_became_active":"2024-09-10T07:53:10.228904+0000","last_became_peered":"2024-09-10T07:53:10.228904+0000","last_unstale":"2024-09-10T07:53:12.232971+0000","last_undegraded":"2024-09-10T07:53:12.232971+0000","last_fullsized":"2024-09-10T07:53:12.232971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:29.688893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440193+0000","last_change":"2024-09-10T07:53:11.245000+0000","last_active":"2024-09-10T07:53:12.440193+0000","last_peered":"2024-09-10T07:53:12.440193+0000","last_clean":"2024-09-10T07:53:12.440193+0000","last_became_active":"2024-09-10T07:53:11.244727+0000","last_became_peered":"2024-09-10T07:53:11.244727+0000","last_unstale":"2024-09-10T07:53:12.440193+0000","last_undegraded":"2024-09-10T07:53:12.440193+0000","last_fullsized":"2024-09-10T07:53:12.440193+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:47.186154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441162+0000","last_change":"2024-09-10T07:53:10.228005+0000","last_active":"2024-09-10T07:53:12.441162+0000","last_peered":"2024-09-10T07:53:12.441162+0000","last_clean":"2024-09-10T07:53:12.441162+0000","last_became_active":"2024-09-10T07:53:10.227661+0000","last_became_peered":"2024-09-10T07:53:10.227661+0000","last_unstale":"2024-09-10T07:53:12.441162+0000","last_undegraded":"2024-09-10T07:53:12.441162+0000","last_fullsized":"2024-09-10T07:53:12.441162+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:45.635654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.439911+0000","last_change":"2024-09-10T07:53:11.235959+0000","last_active":"2024-09-10T07:53:12.439911+0000","last_peered":"2024-09-10T07:53:12.439911+0000","last_clean":"2024-09-10T07:53:12.439911+0000","last_became_active":"2024-09-10T07:53:11.235687+0000","last_became_peered":"2024-09-10T07:53:11.235687+0000","last_unstale":"2024-09-10T07:53:12.439911+0000","last_undegraded":"2024-09-10T07:53:12.439911+0000","last_fullsized":"2024-09-10T07:53:12.439911+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:40.310671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.254993+0000","last_change":"2024-09-10T07:53:10.233196+0000","last_active":"2024-09-10T07:53:12.254993+0000","last_peered":"2024-09-10T07:53:12.254993+0000","last_clean":"2024-09-10T07:53:12.254993+0000","last_became_active":"2024-09-10T07:53:10.232510+0000","last_became_peered":"2024-09-10T07:53:10.232510+0000","last_unstale":"2024-09-10T07:53:12.254993+0000","last_undegraded":"2024-09-10T07:53:12.254993+0000","last_fullsized":"2024-09-10T07:53:12.254993+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:37.672277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234564+0000","last_change":"2024-09-10T07:53:11.261840+0000","last_active":"2024-09-10T07:53:12.234564+0000","last_peered":"2024-09-10T07:53:12.234564+0000","last_clean":"2024-09-10T07:53:12.234564+0000","last_became_active":"2024-09-10T07:53:11.258874+0000","last_became_peered":"2024-09-10T07:53:11.258874+0000","last_unstale":"2024-09-10T07:53:12.234564+0000","last_undegraded":"2024-09-10T07:53:12.234564+0000","last_fullsized":"2024-09-10T07:53:12.234564+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:12.859790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234051+0000","last_change":"2024-09-10T07:53:10.213607+0000","last_active":"2024-09-10T07:53:12.234051+0000","last_peered":"2024-09-10T07:53:12.234051+0000","last_clean":"2024-09-10T07:53:12.234051+0000","last_became_active":"2024-09-10T07:53:10.213317+0000","last_became_peered":"2024-09-10T07:53:10.213317+0000","last_unstale":"2024-09-10T07:53:12.234051+0000","last_undegraded":"2024-09-10T07:53:12.234051+0000","last_fullsized":"2024-09-10T07:53:12.234051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:19.086909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441980+0000","last_change":"2024-09-10T07:53:11.250003+0000","last_active":"2024-09-10T07:53:12.441980+0000","last_peered":"2024-09-10T07:53:12.441980+0000","last_clean":"2024-09-10T07:53:12.441980+0000","last_became_active":"2024-09-10T07:53:11.249735+0000","last_became_peered":"2024-09-10T07:53:11.249735+0000","last_unstale":"2024-09-10T07:53:12.441980+0000","last_undegraded":"2024-09-10T07:53:12.441980+0000","last_fullsized":"2024-09-10T07:53:12.441980+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:07.423220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234644+0000","last_change":"2024-09-10T07:53:10.226296+0000","last_active":"2024-09-10T07:53:12.234644+0000","last_peered":"2024-09-10T07:53:12.234644+0000","last_clean":"2024-09-10T07:53:12.234644+0000","last_became_active":"2024-09-10T07:53:10.225937+0000","last_became_peered":"2024-09-10T07:53:10.225937+0000","last_unstale":"2024-09-10T07:53:12.234644+0000","last_undegraded":"2024-09-10T07:53:12.234644+0000","last_fullsized":"2024-09-10T07:53:12.234644+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440750+0000","last_change":"2024-09-10T07:53:11.266667+0000","last_active":"2024-09-10T07:53:12.440750+0000","last_peered":"2024-09-10T07:53:12.440750+0000","last_clean":"2024-09-10T07:53:12.440750+0000","last_became_active":"2024-09-10T07:53:11.266400+0000","last_became_peered":"2024-09-10T07:53:11.266400+0000","last_unstale":"2024-09-10T07:53:12.440750+0000","last_undegraded":"2024-09-10T07:53:12.440750+0000","last_fullsized":"2024-09-10T07:53:12.440750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:50.186564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440801+0000","last_change":"2024-09-10T07:53:10.223127+0000","last_active":"2024-09-10T07:53:12.440801+0000","last_peered":"2024-09-10T07:53:12.440801+0000","last_clean":"2024-09-10T07:53:12.440801+0000","last_became_active":"2024-09-10T07:53:10.222729+0000","last_became_peered":"2024-09-10T07:53:10.222729+0000","last_unstale":"2024-09-10T07:53:12.440801+0000","last_undegraded":"2024-09-10T07:53:12.440801+0000","last_fullsized":"2024-09-10T07:53:12.440801+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.995404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441809+0000","last_change":"2024-09-10T07:53:11.260871+0000","last_active":"2024-09-10T07:53:12.441809+0000","last_peered":"2024-09-10T07:53:12.441809+0000","last_clean":"2024-09-10T07:53:12.441809+0000","last_became_active":"2024-09-10T07:53:11.260613+0000","last_became_peered":"2024-09-10T07:53:11.260613+0000","last_unstale":"2024-09-10T07:53:12.441809+0000","last_undegraded":"2024-09-10T07:53:12.441809+0000","last_fullsized":"2024-09-10T07:53:12.441809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:30.436267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441275+0000","last_change":"2024-09-10T07:53:10.235054+0000","last_active":"2024-09-10T07:53:12.441275+0000","last_peered":"2024-09-10T07:53:12.441275+0000","last_clean":"2024-09-10T07:53:12.441275+0000","last_became_active":"2024-09-10T07:53:10.234764+0000","last_became_peered":"2024-09-10T07:53:10.234764+0000","last_unstale":"2024-09-10T07:53:12.441275+0000","last_undegraded":"2024-09-10T07:53:12.441275+0000","last_fullsized":"2024-09-10T07:53:12.441275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:49.115509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234381+0000","last_change":"2024-09-10T07:53:11.265713+0000","last_active":"2024-09-10T07:53:12.234381+0000","last_peered":"2024-09-10T07:53:12.234381+0000","last_clean":"2024-09-10T07:53:12.234381+0000","last_became_active":"2024-09-10T07:53:11.265414+0000","last_became_peered":"2024-09-10T07:53:11.265414+0000","last_unstale":"2024-09-10T07:53:12.234381+0000","last_undegraded":"2024-09-10T07:53:12.234381+0000","last_fullsized":"2024-09-10T07:53:12.234381+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:06.014591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255183+0000","last_change":"2024-09-10T07:53:10.222685+0000","last_active":"2024-09-10T07:53:12.255183+0000","last_peered":"2024-09-10T07:53:12.255183+0000","last_clean":"2024-09-10T07:53:12.255183+0000","last_became_active":"2024-09-10T07:53:10.222344+0000","last_became_peered":"2024-09-10T07:53:10.222344+0000","last_unstale":"2024-09-10T07:53:12.255183+0000","last_undegraded":"2024-09-10T07:53:12.255183+0000","last_fullsized":"2024-09-10T07:53:12.255183+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:27.715345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233639+0000","last_change":"2024-09-10T07:53:11.257850+0000","last_active":"2024-09-10T07:53:12.233639+0000","last_peered":"2024-09-10T07:53:12.233639+0000","last_clean":"2024-09-10T07:53:12.233639+0000","last_became_active":"2024-09-10T07:53:11.255958+0000","last_became_peered":"2024-09-10T07:53:11.255958+0000","last_unstale":"2024-09-10T07:53:12.233639+0000","last_undegraded":"2024-09-10T07:53:12.233639+0000","last_fullsized":"2024-09-10T07:53:12.233639+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:05.576611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234530+0000","last_change":"2024-09-10T07:53:10.229874+0000","last_active":"2024-09-10T07:53:12.234530+0000","last_peered":"2024-09-10T07:53:12.234530+0000","last_clean":"2024-09-10T07:53:12.234530+0000","last_became_active":"2024-09-10T07:53:10.229597+0000","last_became_peered":"2024-09-10T07:53:10.229597+0000","last_unstale":"2024-09-10T07:53:12.234530+0000","last_undegraded":"2024-09-10T07:53:12.234530+0000","last_fullsized":"2024-09-10T07:53:12.234530+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440502+0000","last_change":"2024-09-10T07:53:11.236857+0000","last_active":"2024-09-10T07:53:12.440502+0000","last_peered":"2024-09-10T07:53:12.440502+0000","last_clean":"2024-09-10T07:53:12.440502+0000","last_became_active":"2024-09-10T07:53:11.236085+0000","last_became_peered":"2024-09-10T07:53:11.236085+0000","last_unstale":"2024-09-10T07:53:12.440502+0000","last_undegraded":"2024-09-10T07:53:12.440502+0000","last_fullsized":"2024-09-10T07:53:12.440502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:21.906911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440164+0000","last_change":"2024-09-10T07:53:10.223641+0000","last_active":"2024-09-10T07:53:12.440164+0000","last_peered":"2024-09-10T07:53:12.440164+0000","last_clean":"2024-09-10T07:53:12.440164+0000","last_became_active":"2024-09-10T07:53:10.223373+0000","last_became_peered":"2024-09-10T07:53:10.223373+0000","last_unstale":"2024-09-10T07:53:12.440164+0000","last_undegraded":"2024-09-10T07:53:12.440164+0000","last_fullsized":"2024-09-10T07:53:12.440164+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:32:00.958420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233338+0000","last_change":"2024-09-10T07:53:11.256226+0000","last_active":"2024-09-10T07:53:12.233338+0000","last_peered":"2024-09-10T07:53:12.233338+0000","last_clean":"2024-09-10T07:53:12.233338+0000","last_became_active":"2024-09-10T07:53:11.247430+0000","last_became_peered":"2024-09-10T07:53:11.247430+0000","last_unstale":"2024-09-10T07:53:12.233338+0000","last_undegraded":"2024-09-10T07:53:12.233338+0000","last_fullsized":"2024-09-10T07:53:12.233338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:22.801105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":22,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233934+0000","last_change":"2024-09-10T07:53:10.223678+0000","last_active":"2024-09-10T07:53:12.233934+0000","last_peered":"2024-09-10T07:53:12.233934+0000","last_clean":"2024-09-10T07:53:12.233934+0000","last_became_active":"2024-09-10T07:53:10.223390+0000","last_became_peered":"2024-09-10T07:53:10.223390+0000","last_unstale":"2024-09-10T07:53:12.233934+0000","last_undegraded":"2024-09-10T07:53:12.233934+0000","last_fullsized":"2024-09-10T07:53:12.233934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:32.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441944+0000","last_change":"2024-09-10T07:53:11.251829+0000","last_active":"2024-09-10T07:53:12.441944+0000","last_peered":"2024-09-10T07:53:12.441944+0000","last_clean":"2024-09-10T07:53:12.441944+0000","last_became_active":"2024-09-10T07:53:11.251547+0000","last_became_peered":"2024-09-10T07:53:11.251547+0000","last_unstale":"2024-09-10T07:53:12.441944+0000","last_undegraded":"2024-09-10T07:53:12.441944+0000","last_fullsized":"2024-09-10T07:53:12.441944+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:13.567582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441533+0000","last_change":"2024-09-10T07:53:10.232882+0000","last_active":"2024-09-10T07:53:12.441533+0000","last_peered":"2024-09-10T07:53:12.441533+0000","last_clean":"2024-09-10T07:53:12.441533+0000","last_became_active":"2024-09-10T07:53:10.232094+0000","last_became_peered":"2024-09-10T07:53:10.232094+0000","last_unstale":"2024-09-10T07:53:12.441533+0000","last_undegraded":"2024-09-10T07:53:12.441533+0000","last_fullsized":"2024-09-10T07:53:12.441533+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.103755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441720+0000","last_change":"2024-09-10T07:53:11.248994+0000","last_active":"2024-09-10T07:53:12.441720+0000","last_peered":"2024-09-10T07:53:12.441720+0000","last_clean":"2024-09-10T07:53:12.441720+0000","last_became_active":"2024-09-10T07:53:11.248544+0000","last_became_peered":"2024-09-10T07:53:11.248544+0000","last_unstale":"2024-09-10T07:53:12.441720+0000","last_undegraded":"2024-09-10T07:53:12.441720+0000","last_fullsized":"2024-09-10T07:53:12.441720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:33.648889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440910+0000","last_change":"2024-09-10T07:53:10.213371+0000","last_active":"2024-09-10T07:53:12.440910+0000","last_peered":"2024-09-10T07:53:12.440910+0000","last_clean":"2024-09-10T07:53:12.440910+0000","last_became_active":"2024-09-10T07:53:10.211436+0000","last_became_peered":"2024-09-10T07:53:10.211436+0000","last_unstale":"2024-09-10T07:53:12.440910+0000","last_undegraded":"2024-09-10T07:53:12.440910+0000","last_fullsized":"2024-09-10T07:53:12.440910+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:54.890133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441343+0000","last_change":"2024-09-10T07:53:11.235023+0000","last_active":"2024-09-10T07:53:12.441343+0000","last_peered":"2024-09-10T07:53:12.441343+0000","last_clean":"2024-09-10T07:53:12.441343+0000","last_became_active":"2024-09-10T07:53:11.234748+0000","last_became_peered":"2024-09-10T07:53:11.234748+0000","last_unstale":"2024-09-10T07:53:12.441343+0000","last_undegraded":"2024-09-10T07:53:12.441343+0000","last_fullsized":"2024-09-10T07:53:12.441343+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.506631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255084+0000","last_change":"2024-09-10T07:53:10.232440+0000","last_active":"2024-09-10T07:53:12.255084+0000","last_peered":"2024-09-10T07:53:12.255084+0000","last_clean":"2024-09-10T07:53:12.255084+0000","last_became_active":"2024-09-10T07:53:10.229715+0000","last_became_peered":"2024-09-10T07:53:10.229715+0000","last_unstale":"2024-09-10T07:53:12.255084+0000","last_undegraded":"2024-09-10T07:53:12.255084+0000","last_fullsized":"2024-09-10T07:53:12.255084+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:45.764363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255040+0000","last_change":"2024-09-10T07:53:11.257232+0000","last_active":"2024-09-10T07:53:12.255040+0000","last_peered":"2024-09-10T07:53:12.255040+0000","last_clean":"2024-09-10T07:53:12.255040+0000","last_became_active":"2024-09-10T07:53:11.256546+0000","last_became_peered":"2024-09-10T07:53:11.256546+0000","last_unstale":"2024-09-10T07:53:12.255040+0000","last_undegraded":"2024-09-10T07:53:12.255040+0000","last_fullsized":"2024-09-10T07:53:12.255040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:15.943777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233855+0000","last_change":"2024-09-10T07:53:10.227747+0000","last_active":"2024-09-10T07:53:12.233855+0000","last_peered":"2024-09-10T07:53:12.233855+0000","last_clean":"2024-09-10T07:53:12.233855+0000","last_became_active":"2024-09-10T07:53:10.225865+0000","last_became_peered":"2024-09-10T07:53:10.225865+0000","last_unstale":"2024-09-10T07:53:12.233855+0000","last_undegraded":"2024-09-10T07:53:12.233855+0000","last_fullsized":"2024-09-10T07:53:12.233855+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255311+0000","last_change":"2024-09-10T07:53:11.256292+0000","last_active":"2024-09-10T07:53:12.255311+0000","last_peered":"2024-09-10T07:53:12.255311+0000","last_clean":"2024-09-10T07:53:12.255311+0000","last_became_active":"2024-09-10T07:53:11.255952+0000","last_became_peered":"2024-09-10T07:53:11.255952+0000","last_unstale":"2024-09-10T07:53:12.255311+0000","last_undegraded":"2024-09-10T07:53:12.255311+0000","last_fullsized":"2024-09-10T07:53:12.255311+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:31.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.439979+0000","last_change":"2024-09-10T07:53:10.228787+0000","last_active":"2024-09-10T07:53:12.439979+0000","last_peered":"2024-09-10T07:53:12.439979+0000","last_clean":"2024-09-10T07:53:12.439979+0000","last_became_active":"2024-09-10T07:53:10.228445+0000","last_became_peered":"2024-09-10T07:53:10.228445+0000","last_unstale":"2024-09-10T07:53:12.439979+0000","last_undegraded":"2024-09-10T07:53:12.439979+0000","last_fullsized":"2024-09-10T07:53:12.439979+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:55.010094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233541+0000","last_change":"2024-09-10T07:53:11.256460+0000","last_active":"2024-09-10T07:53:12.233541+0000","last_peered":"2024-09-10T07:53:12.233541+0000","last_clean":"2024-09-10T07:53:12.233541+0000","last_became_active":"2024-09-10T07:53:11.256159+0000","last_became_peered":"2024-09-10T07:53:11.256159+0000","last_unstale":"2024-09-10T07:53:12.233541+0000","last_undegraded":"2024-09-10T07:53:12.233541+0000","last_fullsized":"2024-09-10T07:53:12.233541+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:56.012461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.231320+0000","last_change":"2024-09-10T07:53:10.228240+0000","last_active":"2024-09-10T07:53:12.231320+0000","last_peered":"2024-09-10T07:53:12.231320+0000","last_clean":"2024-09-10T07:53:12.231320+0000","last_became_active":"2024-09-10T07:53:10.227508+0000","last_became_peered":"2024-09-10T07:53:10.227508+0000","last_unstale":"2024-09-10T07:53:12.231320+0000","last_undegraded":"2024-09-10T07:53:12.231320+0000","last_fullsized":"2024-09-10T07:53:12.231320+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:39.308304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440067+0000","last_change":"2024-09-10T07:53:11.259531+0000","last_active":"2024-09-10T07:53:12.440067+0000","last_peered":"2024-09-10T07:53:12.440067+0000","last_clean":"2024-09-10T07:53:12.440067+0000","last_became_active":"2024-09-10T07:53:11.258000+0000","last_became_peered":"2024-09-10T07:53:11.258000+0000","last_unstale":"2024-09-10T07:53:12.440067+0000","last_undegraded":"2024-09-10T07:53:12.440067+0000","last_fullsized":"2024-09-10T07:53:12.440067+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.717712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232682+0000","last_change":"2024-09-10T07:53:10.231034+0000","last_active":"2024-09-10T07:53:12.232682+0000","last_peered":"2024-09-10T07:53:12.232682+0000","last_clean":"2024-09-10T07:53:12.232682+0000","last_became_active":"2024-09-10T07:53:10.230111+0000","last_became_peered":"2024-09-10T07:53:10.230111+0000","last_unstale":"2024-09-10T07:53:12.232682+0000","last_undegraded":"2024-09-10T07:53:12.232682+0000","last_fullsized":"2024-09-10T07:53:12.232682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:58.029264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255555+0000","last_change":"2024-09-10T07:53:11.256458+0000","last_active":"2024-09-10T07:53:12.255555+0000","last_peered":"2024-09-10T07:53:12.255555+0000","last_clean":"2024-09-10T07:53:12.255555+0000","last_became_active":"2024-09-10T07:53:11.256045+0000","last_became_peered":"2024-09-10T07:53:11.256045+0000","last_unstale":"2024-09-10T07:53:12.255555+0000","last_undegraded":"2024-09-10T07:53:12.255555+0000","last_fullsized":"2024-09-10T07:53:12.255555+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:25.968183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441816+0000","last_change":"2024-09-10T07:53:10.222067+0000","last_active":"2024-09-10T07:53:12.441816+0000","last_peered":"2024-09-10T07:53:12.441816+0000","last_clean":"2024-09-10T07:53:12.441816+0000","last_became_active":"2024-09-10T07:53:10.221728+0000","last_became_peered":"2024-09-10T07:53:10.221728+0000","last_unstale":"2024-09-10T07:53:12.441816+0000","last_undegraded":"2024-09-10T07:53:12.441816+0000","last_fullsized":"2024-09-10T07:53:12.441816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:44.463540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255400+0000","last_change":"2024-09-10T07:53:11.274363+0000","last_active":"2024-09-10T07:53:12.255400+0000","last_peered":"2024-09-10T07:53:12.255400+0000","last_clean":"2024-09-10T07:53:12.255400+0000","last_became_active":"2024-09-10T07:53:11.274030+0000","last_became_peered":"2024-09-10T07:53:11.274030+0000","last_unstale":"2024-09-10T07:53:12.255400+0000","last_undegraded":"2024-09-10T07:53:12.255400+0000","last_fullsized":"2024-09-10T07:53:12.255400+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:18.674198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255633+0000","last_change":"2024-09-10T07:53:10.232887+0000","last_active":"2024-09-10T07:53:12.255633+0000","last_peered":"2024-09-10T07:53:12.255633+0000","last_clean":"2024-09-10T07:53:12.255633+0000","last_became_active":"2024-09-10T07:53:10.230235+0000","last_became_peered":"2024-09-10T07:53:10.230235+0000","last_unstale":"2024-09-10T07:53:12.255633+0000","last_undegraded":"2024-09-10T07:53:12.255633+0000","last_fullsized":"2024-09-10T07:53:12.255633+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:29.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233395+0000","last_change":"2024-09-10T07:53:11.254067+0000","last_active":"2024-09-10T07:53:12.233395+0000","last_peered":"2024-09-10T07:53:12.233395+0000","last_clean":"2024-09-10T07:53:12.233395+0000","last_became_active":"2024-09-10T07:53:11.253708+0000","last_became_peered":"2024-09-10T07:53:11.253708+0000","last_unstale":"2024-09-10T07:53:12.233395+0000","last_undegraded":"2024-09-10T07:53:12.233395+0000","last_fullsized":"2024-09-10T07:53:12.233395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:01.851731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255420+0000","last_change":"2024-09-10T07:53:10.223013+0000","last_active":"2024-09-10T07:53:12.255420+0000","last_peered":"2024-09-10T07:53:12.255420+0000","last_clean":"2024-09-10T07:53:12.255420+0000","last_became_active":"2024-09-10T07:53:10.222053+0000","last_became_peered":"2024-09-10T07:53:10.222053+0000","last_unstale":"2024-09-10T07:53:12.255420+0000","last_undegraded":"2024-09-10T07:53:12.255420+0000","last_fullsized":"2024-09-10T07:53:12.255420+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:32.646523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441759+0000","last_change":"2024-09-10T07:53:11.247894+0000","last_active":"2024-09-10T07:53:12.441759+0000","last_peered":"2024-09-10T07:53:12.441759+0000","last_clean":"2024-09-10T07:53:12.441759+0000","last_became_active":"2024-09-10T07:53:11.247569+0000","last_became_peered":"2024-09-10T07:53:11.247569+0000","last_unstale":"2024-09-10T07:53:12.441759+0000","last_undegraded":"2024-09-10T07:53:12.441759+0000","last_fullsized":"2024-09-10T07:53:12.441759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:41.496006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441658+0000","last_change":"2024-09-10T07:53:10.223877+0000","last_active":"2024-09-10T07:53:12.441658+0000","last_peered":"2024-09-10T07:53:12.441658+0000","last_clean":"2024-09-10T07:53:12.441658+0000","last_became_active":"2024-09-10T07:53:10.222951+0000","last_became_peered":"2024-09-10T07:53:10.222951+0000","last_unstale":"2024-09-10T07:53:12.441658+0000","last_undegraded":"2024-09-10T07:53:12.441658+0000","last_fullsized":"2024-09-10T07:53:12.441658+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:16.490800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255222+0000","last_change":"2024-09-10T07:53:11.256940+0000","last_active":"2024-09-10T07:53:12.255222+0000","last_peered":"2024-09-10T07:53:12.255222+0000","last_clean":"2024-09-10T07:53:12.255222+0000","last_became_active":"2024-09-10T07:53:11.255982+0000","last_became_peered":"2024-09-10T07:53:11.255982+0000","last_unstale":"2024-09-10T07:53:12.255222+0000","last_undegraded":"2024-09-10T07:53:12.255222+0000","last_fullsized":"2024-09-10T07:53:12.255222+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:06.179103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.233746+0000","last_change":"2024-09-10T07:53:10.227674+0000","last_active":"2024-09-10T07:53:12.233746+0000","last_peered":"2024-09-10T07:53:12.233746+0000","last_clean":"2024-09-10T07:53:12.233746+0000","last_became_active":"2024-09-10T07:53:10.227321+0000","last_became_peered":"2024-09-10T07:53:10.227321+0000","last_unstale":"2024-09-10T07:53:12.233746+0000","last_undegraded":"2024-09-10T07:53:12.233746+0000","last_fullsized":"2024-09-10T07:53:12.233746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:57:58.837406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441231+0000","last_change":"2024-09-10T07:53:11.252691+0000","last_active":"2024-09-10T07:53:12.441231+0000","last_peered":"2024-09-10T07:53:12.441231+0000","last_clean":"2024-09-10T07:53:12.441231+0000","last_became_active":"2024-09-10T07:53:11.252471+0000","last_became_peered":"2024-09-10T07:53:11.252471+0000","last_unstale":"2024-09-10T07:53:12.441231+0000","last_undegraded":"2024-09-10T07:53:12.441231+0000","last_fullsized":"2024-09-10T07:53:12.441231+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:35.664993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440072+0000","last_change":"2024-09-10T07:53:10.230718+0000","last_active":"2024-09-10T07:53:12.440072+0000","last_peered":"2024-09-10T07:53:12.440072+0000","last_clean":"2024-09-10T07:53:12.440072+0000","last_became_active":"2024-09-10T07:53:10.230504+0000","last_became_peered":"2024-09-10T07:53:10.230504+0000","last_unstale":"2024-09-10T07:53:12.440072+0000","last_undegraded":"2024-09-10T07:53:12.440072+0000","last_fullsized":"2024-09-10T07:53:12.440072+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:14.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.255237+0000","last_change":"2024-09-10T07:53:11.248370+0000","last_active":"2024-09-10T07:53:12.255237+0000","last_peered":"2024-09-10T07:53:12.255237+0000","last_clean":"2024-09-10T07:53:12.255237+0000","last_became_active":"2024-09-10T07:53:11.248094+0000","last_became_peered":"2024-09-10T07:53:11.248094+0000","last_unstale":"2024-09-10T07:53:12.255237+0000","last_undegraded":"2024-09-10T07:53:12.255237+0000","last_fullsized":"2024-09-10T07:53:12.255237+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:11.486532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440185+0000","last_change":"2024-09-10T07:53:10.234562+0000","last_active":"2024-09-10T07:53:12.440185+0000","last_peered":"2024-09-10T07:53:12.440185+0000","last_clean":"2024-09-10T07:53:12.440185+0000","last_became_active":"2024-09-10T07:53:10.234276+0000","last_became_peered":"2024-09-10T07:53:10.234276+0000","last_unstale":"2024-09-10T07:53:12.440185+0000","last_undegraded":"2024-09-10T07:53:12.440185+0000","last_fullsized":"2024-09-10T07:53:12.440185+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440063+0000","last_change":"2024-09-10T07:53:11.264027+0000","last_active":"2024-09-10T07:53:12.440063+0000","last_peered":"2024-09-10T07:53:12.440063+0000","last_clean":"2024-09-10T07:53:12.440063+0000","last_became_active":"2024-09-10T07:53:11.263741+0000","last_became_peered":"2024-09-10T07:53:11.263741+0000","last_unstale":"2024-09-10T07:53:12.440063+0000","last_undegraded":"2024-09-10T07:53:12.440063+0000","last_fullsized":"2024-09-10T07:53:12.440063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.232795+0000","last_change":"2024-09-10T07:53:10.224089+0000","last_active":"2024-09-10T07:53:12.232795+0000","last_peered":"2024-09-10T07:53:12.232795+0000","last_clean":"2024-09-10T07:53:12.232795+0000","last_became_active":"2024-09-10T07:53:10.222933+0000","last_became_peered":"2024-09-10T07:53:10.222933+0000","last_unstale":"2024-09-10T07:53:12.232795+0000","last_undegraded":"2024-09-10T07:53:12.232795+0000","last_fullsized":"2024-09-10T07:53:12.232795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:01.604342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.234021+0000","last_change":"2024-09-10T07:53:11.266735+0000","last_active":"2024-09-10T07:53:12.234021+0000","last_peered":"2024-09-10T07:53:12.234021+0000","last_clean":"2024-09-10T07:53:12.234021+0000","last_became_active":"2024-09-10T07:53:11.266398+0000","last_became_peered":"2024-09-10T07:53:11.266398+0000","last_unstale":"2024-09-10T07:53:12.234021+0000","last_undegraded":"2024-09-10T07:53:12.234021+0000","last_fullsized":"2024-09-10T07:53:12.234021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:10.199182+0000","last_clean_scrub_stamp":"2024-09-10T07:53:10.199182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:05.878447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.441717+0000","last_change":"2024-09-10T07:53:10.235291+0000","last_active":"2024-09-10T07:53:12.441717+0000","last_peered":"2024-09-10T07:53:12.441717+0000","last_clean":"2024-09-10T07:53:12.441717+0000","last_became_active":"2024-09-10T07:53:10.235015+0000","last_became_peered":"2024-09-10T07:53:10.235015+0000","last_unstale":"2024-09-10T07:53:12.441717+0000","last_undegraded":"2024-09-10T07:53:12.441717+0000","last_fullsized":"2024-09-10T07:53:12.441717+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:53:09.196815+0000","last_clean_scrub_stamp":"2024-09-10T07:53:09.196815+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:14.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":316,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T07:53:12.440047+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:53:12.440047+0000","last_peered":"2024-09-10T07:53:12.440047+0000","last_clean":"2024-09-10T07:53:12.440047+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:53:12.440047+0000","last_undegraded":"2024-09-10T07:53:12.440047+0000","last_fullsized":"2024-09-10T07:53:12.440047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":542378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1253376,"data_stored":542254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1134276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476822,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":542184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1253376,"data_stored":542766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":542184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1863680,"data_stored":1134220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1253376,"data_stored":542766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:53:20.551 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:53:20.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T07:53:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.752+0000 7fd033600640 1 Processor -- start 2024-09-10T07:53:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.752+0000 7fd033600640 1 -- start start 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd033600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd02c0b4c90 0x7fd02c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd033600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c072c50 0x7fd02c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd033600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd02c15eff0 con 0x7fd02c15b230 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd033600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd02c15fbe0 con 0x7fd02c0b4c90 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd033600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd02c15c8d0 con 0x7fd02c072c50 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd030c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd02c15b230 0x7fd02c0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56734/0 (socket says 172.21.15.3:56734) 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd030c00640 1 -- 172.21.15.3:0/3485561975 learned_addr learned my addr 172.21.15.3:0/3485561975 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.753+0000 7fd023e00640 1 --2- 172.21.15.3:0/3485561975 >> v2:172.21.15.3:3300/0 conn(0x7fd02c0b4c90 0x7fd02c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd031600640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c072c50 0x7fd02c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd023e00640 1 -- 172.21.15.3:0/3485561975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c072c50 msgr2=0x7fd02c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd023e00640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c072c50 0x7fd02c071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd023e00640 1 -- 172.21.15.3:0/3485561975 >> v1:172.21.15.3:6789/0 conn(0x7fd02c15b230 legacy=0x7fd02c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd023e00640 1 -- 172.21.15.3:0/3485561975 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c15f2b0 con 0x7fd02c0b4c90 2024-09-10T07:53:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.754+0000 7fd02be00640 1 -- 172.21.15.3:0/3485561975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607465488 0 0) 0x7fd02c15eff0 con 0x7fd02c15b230 2024-09-10T07:53:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.755+0000 7fd023e00640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c15c390 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd01402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5974b9cd96d34042 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.755+0000 7fd02be00640 1 -- 172.21.15.3:0/3485561975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014037070 con 0x7fd02c0b4c90 2024-09-10T07:53:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.755+0000 7fd02be00640 1 -- 172.21.15.3:0/3485561975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd014032440 con 0x7fd02c0b4c90 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.755+0000 7fd02be00640 1 -- 172.21.15.3:0/3485561975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014032810 con 0x7fd02c0b4c90 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 -- 172.21.15.3:0/3485561975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 msgr2=0x7fd02c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c15c390 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd01402a180 comp rx=0 tx=0).stop 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 -- 172.21.15.3:0/3485561975 shutdown_connections 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c072c50 0x7fd02c071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 --2- 172.21.15.3:0/3485561975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c15c390 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 -- 172.21.15.3:0/3485561975 >> 172.21.15.3:0/3485561975 conn(0x7fd02c0b5cb0 msgr2=0x7fd02c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 -- 172.21.15.3:0/3485561975 shutdown_connections 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.756+0000 7fd033600640 1 -- 172.21.15.3:0/3485561975 wait complete. 2024-09-10T07:53:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.757+0000 7fd033600640 1 Processor -- start 2024-09-10T07:53:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.757+0000 7fd033600640 1 -- start start 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 0x7fd02c1f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 0x7fd02c14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c15b610 con 0x7fd02c0b4c90 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c15f870 con 0x7fd02c072c50 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.758+0000 7fd033600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd02c15b790 con 0x7fd02c15b230 2024-09-10T07:53:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35958/0 (socket says 172.21.15.3:35958) 2024-09-10T07:53:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 -- 172.21.15.3:0/1801680389 learned_addr learned my addr 172.21.15.3:0/1801680389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd030c00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 0x7fd02c1f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd031600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 0x7fd02c14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 -- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 msgr2=0x7fd02c14f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 0x7fd02c14f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 -- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 msgr2=0x7fd02c1f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 0x7fd02c1f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 -- 172.21.15.3:0/1801680389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c0b2d60 con 0x7fd02c0b4c90 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd031600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 0x7fd02c14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd030c00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 0x7fd02c1f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.759+0000 7fd023e00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd0140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.760+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014041070 con 0x7fd02c0b4c90 2024-09-10T07:53:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.760+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01403c930 con 0x7fd02c0b4c90 2024-09-10T07:53:20.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.760+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014037040 con 0x7fd02c0b4c90 2024-09-10T07:53:20.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.760+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c0affc0 con 0x7fd02c0b4c90 2024-09-10T07:53:20.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.760+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c0b0580 con 0x7fd02c0b4c90 2024-09-10T07:53:20.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.761+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02c06e1e0 con 0x7fd02c0b4c90 2024-09-10T07:53:20.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.762+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd014044070 con 0x7fd02c0b4c90 2024-09-10T07:53:20.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.763+0000 7fd029600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 0x7fd00806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:20.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.763+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd0140ab990 con 0x7fd02c0b4c90 2024-09-10T07:53:20.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.763+0000 7fd030c00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 0x7fd00806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:20.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.763+0000 7fd030c00640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 0x7fd00806e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000f60 tx=0x7fd01c007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:20.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:20.767+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd01407a330 con 0x7fd02c0b4c90 2024-09-10T07:53:21.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.019+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd02c151370 con 0x7fd02c0b4c90 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.021+0000 7fd029600640 1 -- 172.21.15.3:0/1801680389 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fd014079c30 con 0x7fd02c0b4c90 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAU+d9m4iqgJBAAc918kD/aiS+TlZeTU8IqCQ== 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-10T07:53:21.021 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T07:53:21.022 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 msgr2=0x7fd00806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 0x7fd00806e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000f60 tx=0x7fd01c007c10 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 msgr2=0x7fd02c0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd0140046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 shutdown_connections 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd00806be30 0x7fd00806e2f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd02c15b230 0x7fd02c14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02c0b4c90 0x7fd02c0b1860 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 --2- 172.21.15.3:0/1801680389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c072c50 0x7fd02c1f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 >> 172.21.15.3:0/1801680389 conn(0x7fd02c0b5cb0 msgr2=0x7fd02c06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 shutdown_connections 2024-09-10T07:53:21.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.024+0000 7fd033600640 1 -- 172.21.15.3:0/1801680389 wait complete. 2024-09-10T07:53:21.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.195+0000 7f3482800640 1 Processor -- start 2024-09-10T07:53:21.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.196+0000 7f3482800640 1 -- start start 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3482800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 0x7f347c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3482800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3482800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0d0a80 con 0x7f347c160710 2024-09-10T07:53:21.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3482800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0cfdd0 con 0x7f347c160af0 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3482800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f347c15cc90 con 0x7f347c0d05f0 2024-09-10T07:53:21.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f347be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.197+0000 7f3473e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 0x7f347c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:39006/0 (socket says 172.21.15.3:39006) 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 -- 172.21.15.3:0/2251514797 learned_addr learned my addr 172.21.15.3:0/2251514797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347b400640 1 --1- 172.21.15.3:0/2251514797 >> v1:172.21.15.70:6789/0 conn(0x7f347c160710 0x7f347c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:36252/0 (socket says 172.21.15.3:36252) 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 -- 172.21.15.3:0/2251514797 >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 msgr2=0x7f347c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 --2- 172.21.15.3:0/2251514797 >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 0x7f347c0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 -- 172.21.15.3:0/2251514797 >> v1:172.21.15.70:6789/0 conn(0x7f347c160710 legacy=0x7f347c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f347be00640 1 -- 172.21.15.3:0/2251514797 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c15cf70 con 0x7f347c0d05f0 2024-09-10T07:53:21.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.198+0000 7f3473e00640 1 --2- 172.21.15.3:0/2251514797 >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 0x7f347c0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.199+0000 7f347be00640 1 --2- 172.21.15.3:0/2251514797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3474009e50 tx=0x7f347402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b3b36248c0a88a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.199+0000 7f347aa00640 1 -- 172.21.15.3:0/2251514797 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474036070 con 0x7f347c0d05f0 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.199+0000 7f347aa00640 1 -- 172.21.15.3:0/2251514797 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3474031600 con 0x7f347c0d05f0 2024-09-10T07:53:21.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.199+0000 7f347aa00640 1 -- 172.21.15.3:0/2251514797 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740319d0 con 0x7f347c0d05f0 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 -- 172.21.15.3:0/2251514797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 msgr2=0x7f347c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 --2- 172.21.15.3:0/2251514797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3474009e50 tx=0x7f347402a730 comp rx=0 tx=0).stop 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 -- 172.21.15.3:0/2251514797 shutdown_connections 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 --2- 172.21.15.3:0/2251514797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c0d05f0 0x7f347c072070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 --2- 172.21.15.3:0/2251514797 >> v2:172.21.15.70:3300/0 conn(0x7f347c160af0 0x7f347c0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 -- 172.21.15.3:0/2251514797 >> 172.21.15.3:0/2251514797 conn(0x7f347c06d000 msgr2=0x7f347c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 -- 172.21.15.3:0/2251514797 shutdown_connections 2024-09-10T07:53:21.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.200+0000 7f3482800640 1 -- 172.21.15.3:0/2251514797 wait complete. 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.202+0000 7f3482800640 1 Processor -- start 2024-09-10T07:53:21.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.202+0000 7f3482800640 1 -- start start 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f347c0d05f0 0x7f347c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c160af0 0x7f347c1f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f347c15d770 con 0x7f347c160710 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f347c15c960 con 0x7f347c0d05f0 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3482800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f347c156980 con 0x7f347c160af0 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35966/0 (socket says 172.21.15.3:35966) 2024-09-10T07:53:21.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 -- 172.21.15.3:0/2272073378 learned_addr learned my addr 172.21.15.3:0/2272073378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f347be00640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c160af0 0x7f347c1f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f347b400640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f347c0d05f0 0x7f347c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 -- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c160af0 msgr2=0x7f347c1f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c160af0 0x7f347c1f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 -- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f347c0d05f0 msgr2=0x7f347c1431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f347c0d05f0 0x7f347c1431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.203+0000 7f3473e00640 1 -- 172.21.15.3:0/2272073378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c1539c0 con 0x7f347c160710 2024-09-10T07:53:21.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3473e00640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f346400e960 tx=0x7f346400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464017070 con 0x7f347c160710 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346400ca00 con 0x7f347c160710 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400cdd0 con 0x7f347c160710 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c153ca0 con 0x7f347c160710 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.204+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c153f80 con 0x7f347c160710 2024-09-10T07:53:21.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.205+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c076090 con 0x7f347c160710 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.206+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f346401a070 con 0x7f347c160710 2024-09-10T07:53:21.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.207+0000 7f3473400640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 0x7f345806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.207+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f346408b0e0 con 0x7f347c160710 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.207+0000 7f347b400640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 0x7f345806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:21.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.208+0000 7f347b400640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 0x7f345806e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f346c007fd0 tx=0x7f346c007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:21.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.212+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f347c076090 con 0x7f347c160710 2024-09-10T07:53:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.433+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347c0cfaf0 con 0x7f347c160710 2024-09-10T07:53:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.433+0000 7f3473400640 1 -- 172.21.15.3:0/2272073378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5222 (secure 0 0 0) 0x7f3464059320 con 0x7f347c160710 2024-09-10T07:53:21.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:21.433 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":103,"btime":"2024-09-10T07:53:14:241064+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:13.662468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[7543]},"id":7}]} 2024-09-10T07:53:21.433 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 103 2024-09-10T07:53:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.435+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 msgr2=0x7f345806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.435+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 0x7f345806e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f346c007fd0 tx=0x7f346c007e80 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.435+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 msgr2=0x7f347c1f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.435+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f346400e960 tx=0x7f346400ee20 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 shutdown_connections 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f345806be60 0x7f345806e320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f347c160af0 0x7f347c1f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f347c160710 0x7f347c1f4480 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 --2- 172.21.15.3:0/2272073378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f347c0d05f0 0x7f347c1431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 >> 172.21.15.3:0/2272073378 conn(0x7f347c06d000 msgr2=0x7f347c0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 shutdown_connections 2024-09-10T07:53:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:21.436+0000 7f3482800640 1 -- 172.21.15.3:0/2272073378 wait complete. 2024-09-10T07:53:21.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:53:10.636980+0000', 'modified': '2024-09-10T07:53:13.662468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7543}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7543': {'gid': 7543, 'name': 'c', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/3773991953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 3773991953}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 3773991953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7543, 'qdb_cluster': [7543]} 2024-09-10T07:53:21.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T07:53:21.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:53:22.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.014+0000 7f97ae600640 1 Processor -- start 2024-09-10T07:53:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.014+0000 7f97ae600640 1 -- start start 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.015+0000 7f97ae600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 0x7f97a80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.015+0000 7f97ae600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a80cf930 con 0x7f97a80cd7d0 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.015+0000 7f97ae600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a80cfc10 con 0x7f97a80b40d0 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.015+0000 7f97ae600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f97a80b38d0 con 0x7f97a8162ac0 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a6a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f97a80cd7d0 0x7f97a8070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56740/0 (socket says 172.21.15.3:56740) 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a6a00640 1 -- 172.21.15.3:0/3715213354 learned_addr learned my addr 172.21.15.3:0/3715213354 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a7400640 1 --2- 172.21.15.3:0/3715213354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 0x7f97a80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559563274 0 0) 0x7f97a80cf930 con 0x7f97a80cd7d0 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f978c003520 con 0x7f97a80cd7d0 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a7400640 1 -- 172.21.15.3:0/3715213354 >> v1:172.21.15.70:6789/0 conn(0x7f97a80b40d0 legacy=0x7f97a80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007626620 0 0) 0x7f97a80cfc10 con 0x7f97a80b40d0 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a7400640 1 -- 172.21.15.3:0/3715213354 >> v1:172.21.15.3:6789/0 conn(0x7f97a80cd7d0 legacy=0x7f97a8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.016+0000 7f97a7400640 1 -- 172.21.15.3:0/3715213354 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80b3bb0 con 0x7f97a8162ac0 2024-09-10T07:53:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.017+0000 7f97a7400640 1 --2- 172.21.15.3:0/3715213354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 0x7f97a80cf250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97980097d0 tx=0x7f979802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85c65ced2d1c942 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798037070 con 0x7f97a8162ac0 2024-09-10T07:53:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798032440 con 0x7f97a8162ac0 2024-09-10T07:53:22.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97a6000640 1 -- 172.21.15.3:0/3715213354 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980327f0 con 0x7f97a8162ac0 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 -- 172.21.15.3:0/3715213354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 msgr2=0x7f97a80cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 --2- 172.21.15.3:0/3715213354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 0x7f97a80cf250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97980097d0 tx=0x7f979802a210 comp rx=0 tx=0).stop 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 -- 172.21.15.3:0/3715213354 shutdown_connections 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 --2- 172.21.15.3:0/3715213354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a8162ac0 0x7f97a80cf250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 -- 172.21.15.3:0/3715213354 >> 172.21.15.3:0/3715213354 conn(0x7f97a806d000 msgr2=0x7f97a806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 -- 172.21.15.3:0/3715213354 shutdown_connections 2024-09-10T07:53:22.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.018+0000 7f97ae600640 1 -- 172.21.15.3:0/3715213354 wait complete. 2024-09-10T07:53:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.020+0000 7f97ae600640 1 Processor -- start 2024-09-10T07:53:22.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.020+0000 7f97ae600640 1 -- start start 2024-09-10T07:53:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 0x7f97a80b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a80cd7d0 0x7f97a80af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8162ea0 con 0x7f97a8162ac0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8071590 con 0x7f97a80b40d0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97ae600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f97a8163020 con 0x7f97a80cd7d0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35970/0 (socket says 172.21.15.3:35970) 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.021+0000 7f97a7e00640 1 -- 172.21.15.3:0/2326203219 learned_addr learned my addr 172.21.15.3:0/2326203219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a80cd7d0 0x7f97a80af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7400640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 0x7f97a80b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 -- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a80cd7d0 msgr2=0x7f97a80af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a80cd7d0 0x7f97a80af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 -- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 msgr2=0x7f97a80b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 0x7f97a80b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 -- 172.21.15.3:0/2326203219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80b0e90 con 0x7f97a8162ac0 2024-09-10T07:53:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7400640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 0x7f97a80b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c760 tx=0x7f97a000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f979f400640 1 -- 172.21.15.3:0/2326203219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0003da0 con 0x7f97a8162ac0 2024-09-10T07:53:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f979f400640 1 -- 172.21.15.3:0/2326203219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a0005070 con 0x7f97a8162ac0 2024-09-10T07:53:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f979f400640 1 -- 172.21.15.3:0/2326203219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0005460 con 0x7f97a8162ac0 2024-09-10T07:53:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.022+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a80b1170 con 0x7f97a8162ac0 2024-09-10T07:53:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.023+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a80b1a10 con 0x7f97a8162ac0 2024-09-10T07:53:22.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.025+0000 7f979f400640 1 -- 172.21.15.3:0/2326203219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97a0005600 con 0x7f97a8162ac0 2024-09-10T07:53:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.026+0000 7f979f400640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 0x7f978806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.026+0000 7f979f400640 1 -- 172.21.15.3:0/2326203219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f97a008ad70 con 0x7f97a8162ac0 2024-09-10T07:53:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.027+0000 7f97a7400640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 0x7f978806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.027+0000 7f97b13f7740 1 Processor -- start 2024-09-10T07:53:22.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.027+0000 7f97b13f7740 1 -- start start 2024-09-10T07:53:22.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.027+0000 7f97a7400640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 0x7f978806e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9798034000 tx=0x7f979802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d515b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 0x557d515720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557d514d03b0 con 0x557d51573480 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557d514d5410 con 0x557d5156f4b0 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557d5147d620 con 0x557d51571cf0 2024-09-10T07:53:22.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35978/0 (socket says 172.21.15.3:35978) 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 -- 172.21.15.3:0/2465054037 learned_addr learned my addr 172.21.15.3:0/2465054037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7400640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d515b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 0x557d515720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 -- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 msgr2=0x557d515b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d515b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 -- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 msgr2=0x557d515720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 0x557d515720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7e00640 1 -- 172.21.15.3:0/2465054037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d511e82d0 con 0x557d51573480 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a7400640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d515b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.028+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 0x557d515720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:22.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.029+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f97a00892a0 tx=0x7f97a0058a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0c83ab6eff2e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.029+0000 7f9793e00640 1 -- 172.21.15.3:0/2465054037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0087070 con 0x557d51573480 2024-09-10T07:53:22.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.029+0000 7f9793e00640 1 -- 172.21.15.3:0/2465054037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a005d460 con 0x557d51573480 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.029+0000 7f9793e00640 1 -- 172.21.15.3:0/2465054037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0071030 con 0x557d51573480 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 -- 172.21.15.3:0/2465054037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 msgr2=0x557d5156c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f97a00892a0 tx=0x7f97a0058a10 comp rx=0 tx=0).stop 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 -- 172.21.15.3:0/2465054037 shutdown_connections 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d5156c570 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5156f4b0 0x557d515720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2465054037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d515b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 -- 172.21.15.3:0/2465054037 >> 172.21.15.3:0/2465054037 conn(0x557d5156e740 msgr2=0x557d515b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 -- 172.21.15.3:0/2465054037 shutdown_connections 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.030+0000 7f97b13f7740 1 -- 172.21.15.3:0/2465054037 wait complete. 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.031+0000 7f97b13f7740 1 -- start start 2024-09-10T07:53:22.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.031+0000 7f97b13f7740 1 Processor -- start 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d51592650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d5159b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557d51490780 con 0x557d5159be20 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557d514d13d0 con 0x557d51573480 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557d5156dbe0 con 0x557d51571cf0 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97b13f7740 10 client.0 _ll_register_callbacks cb 0x557d514ce5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35986/0 (socket says 172.21.15.3:35986) 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 -- 172.21.15.3:0/3315486729 learned_addr learned my addr 172.21.15.3:0/3315486729 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7400640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d51592650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a6a00640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d5159b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 -- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 msgr2=0x557d51592650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d51592650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 -- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 msgr2=0x557d5159b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.032+0000 7f97a7e00640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d5159b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f97a7e00640 1 -- 172.21.15.3:0/3315486729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d512b26b0 con 0x557d5159be20 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f97a7e00640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f97a0059040 tx=0x7f97a0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada3784457ce909a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0078060 con 0x557d5159be20 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f97a0059310 con 0x557d5159be20 2024-09-10T07:53:22.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a005d460 con 0x557d5159be20 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f97a007e050 con 0x557d5159be20 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f97b13f7740 10 client.8181 fetch_fsmap learned FSMap version 103 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x557d51593650 con 0x557d5159be20 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a005d460 con 0x557d5159be20 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.033+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 103) ==== 3502+0+0 (secure 0 0 0) 0x7f97a0081020 con 0x557d5159be20 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 10 client.8181 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 10 client.8181 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 20 client.8181 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515997f0 0x557d51599bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 4 client.8181 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d514e2b40 con 0x557d515997f0 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97a7400640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515997f0 0x557d51599bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f97a7400640 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515997f0 0x557d51599bd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93de203368ef14f server_cookie=7c2e22a101102d29 in_seq=0 out_seq=0 2024-09-10T07:53:22.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.034+0000 7f9790c00640 10 client.8181 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.035+0000 7f9790c00640 1 -- 172.21.15.3:0/3315486729 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d514e2b40 con 0x557d515997f0 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.035+0000 7f9790c00640 10 client.8181 handle_command_reply: tid=0 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 client.8181 shutdown 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 msgr2=0x557d5156cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f97a0059040 tx=0x7f97a0001250 comp rx=0 tx=0).stop 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 shutdown_connections 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d5159be20 0x557d5156cb40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d5159b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515997f0 0x557d51599bd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3315486729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d51592650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 >> 172.21.15.3:0/3315486729 conn(0x557d5156f4b0 msgr2=0x557d515b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f9790c00640 0 client.8181 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 shutdown_connections 2024-09-10T07:53:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 1 -- 172.21.15.3:0/3315486729 wait complete. 2024-09-10T07:53:22.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.036+0000 7f97b13f7740 20 client.8181 trim_cache size 0 max 16384 2024-09-10T07:53:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.055+0000 7f97b13f7740 1 Processor -- start 2024-09-10T07:53:22.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.055+0000 7f97b13f7740 1 -- start start 2024-09-10T07:53:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d515c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 0x557d515c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8203b70 con 0x557d51571cf0 2024-09-10T07:53:22.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81646a0 con 0x557d51573480 2024-09-10T07:53:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f97a8164cf0 con 0x557d5159be20 2024-09-10T07:53:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36002/0 (socket says 172.21.15.3:36002) 2024-09-10T07:53:22.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 -- 172.21.15.3:0/3839149454 learned_addr learned my addr 172.21.15.3:0/3839149454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7e00640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 0x557d515c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 -- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 msgr2=0x557d515c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a6a00640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d515c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 0x557d515c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 -- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 msgr2=0x557d515c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d515c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7400640 1 -- 172.21.15.3:0/3839149454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5155e350 con 0x557d51571cf0 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a7e00640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 0x557d515c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.056+0000 7f97a6a00640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d515c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97a7400640 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f979802a800 tx=0x7f979802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199925f7c4b33d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f9793400640 1 -- 172.21.15.3:0/3839149454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798037040 con 0x557d51571cf0 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f9793400640 1 -- 172.21.15.3:0/3839149454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798003820 con 0x557d51571cf0 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f9793400640 1 -- 172.21.15.3:0/3839149454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798003bd0 con 0x557d51571cf0 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 -- 172.21.15.3:0/3839149454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 msgr2=0x557d5162e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f979802a800 tx=0x7f979802c000 comp rx=0 tx=0).stop 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 -- 172.21.15.3:0/3839149454 shutdown_connections 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d5159be20 0x557d515c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d51573480 0x557d515c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 --2- 172.21.15.3:0/3839149454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51571cf0 0x557d5162e840 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.057+0000 7f97b13f7740 1 -- 172.21.15.3:0/3839149454 >> 172.21.15.3:0/3839149454 conn(0x557d5156f4b0 msgr2=0x557d515da810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.058+0000 7f97b13f7740 1 -- 172.21.15.3:0/3839149454 shutdown_connections 2024-09-10T07:53:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.058+0000 7f97b13f7740 1 -- 172.21.15.3:0/3839149454 wait complete. 2024-09-10T07:53:22.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.058+0000 7f97b13f7740 1 -- start start 2024-09-10T07:53:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.058+0000 7f97b13f7740 1 Processor -- start 2024-09-10T07:53:22.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d5162a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5159be20 0x557d515c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557d51490780 con 0x557d51573480 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557d5156dbe0 con 0x557d5159be20 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557d514d13d0 con 0x557d51571cf0 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.059+0000 7f97b13f7740 10 client.0 _ll_register_callbacks cb 0x7f97a0085070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36016/0 (socket says 172.21.15.3:36016) 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 -- 172.21.15.3:0/2265317672 learned_addr learned my addr 172.21.15.3:0/2265317672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a7400640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d5162a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a7e00640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5159be20 0x557d515c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 -- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 msgr2=0x557d5162a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d5162a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 -- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5159be20 msgr2=0x557d515c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5159be20 0x557d515c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 -- 172.21.15.3:0/2265317672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d51598580 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f97a6a00640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f9794005880 tx=0x7f9794001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428bb5d5ba701b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.060+0000 7f9790c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794005c40 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9794005de0 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9798003820 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f9794014950 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 10 client.8187 fetch_fsmap learned FSMap version 103 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x557d515d5300 con 0x557d51573480 2024-09-10T07:53:22.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798003820 con 0x557d51573480 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 103) ==== 3502+0+0 (secure 0 0 0) 0x7f9794027020 con 0x557d51573480 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 10 client.8187 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 10 client.8187 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 20 client.8187 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515d4ea0 0x557d5156b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 4 client.8187 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.061+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "status"}) -- 0x557d514dce90 con 0x557d515d4ea0 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f97a7400640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515d4ea0 0x557d5156b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f97a7400640 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515d4ea0 0x557d5156b530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404f3da22372ff64 server_cookie=731682354924c243 in_seq=0 out_seq=0 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f9790c00640 10 client.8187 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f9790c00640 1 -- 172.21.15.3:0/2265317672 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x557d514dce90 con 0x557d515d4ea0 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f9790c00640 10 client.8187 handle_command_reply: tid=0 2024-09-10T07:53:22.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f97b13f7740 1 client.8187 shutdown 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 msgr2=0x557d515c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.062+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f9794005880 tx=0x7f9794001cb0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 shutdown_connections 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557d5159be20 0x557d515c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557d51573480 0x557d515c5ed0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x557d515d4ea0 0x557d5156b530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 --2- 172.21.15.3:0/2265317672 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557d51571cf0 0x557d5162a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f9790c00640 0 client.8187 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 >> 172.21.15.3:0/2265317672 conn(0x557d5156f4b0 msgr2=0x557d515da810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 shutdown_connections 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 1 -- 172.21.15.3:0/2265317672 wait complete. 2024-09-10T07:53:22.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97b13f7740 20 client.8187 trim_cache size 0 max 16384 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7543, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.819544799000001, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 103, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 102, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.528918576000001 2024-09-10T07:53:22.067 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 msgr2=0x7f978806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 0x7f978806e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9798034000 tx=0x7f979802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 msgr2=0x7f97a81f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.063+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c760 tx=0x7f97a000cc20 comp rx=0 tx=0).stop 2024-09-10T07:53:22.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 shutdown_connections 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f978806be30 0x7f978806e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a8162ac0 0x7f97a81f7b10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f97a80cd7d0 0x7f97a80af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 --2- 172.21.15.3:0/2326203219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a80b40d0 0x7f97a80b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 >> 172.21.15.3:0/2326203219 conn(0x7f97a806d000 msgr2=0x7f97a80cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 shutdown_connections 2024-09-10T07:53:22.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.064+0000 7f97ae600640 1 -- 172.21.15.3:0/2326203219 wait complete. 2024-09-10T07:53:22.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T07:53:22.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.234+0000 7f7c9ca00640 1 Processor -- start 2024-09-10T07:53:22.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.235+0000 7f7c9ca00640 1 -- start start 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c9ca00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7c98160710 0x7f7c980cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c9ca00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7c980d07c0 0x7f7c98072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c9ca00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c980cfdd0 con 0x7f7c98160af0 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c9ca00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9815cc90 con 0x7f7c98160710 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c9ca00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9815cf70 con 0x7f7c980d07c0 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c95e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7c98160710 0x7f7c980cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c95e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7c98160710 0x7f7c980cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36018/0 (socket says 172.21.15.3:36018) 2024-09-10T07:53:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c95e00640 1 -- 172.21.15.3:0/1666078319 learned_addr learned my addr 172.21.15.3:0/1666078319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.236+0000 7f7c96800640 1 --2- 172.21.15.3:0/1666078319 >> v2:172.21.15.70:3300/0 conn(0x7f7c980d07c0 0x7f7c98072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.237+0000 7f7c95e00640 1 -- 172.21.15.3:0/1666078319 >> v2:172.21.15.70:3300/0 conn(0x7f7c980d07c0 msgr2=0x7f7c98072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.237+0000 7f7c95e00640 1 --2- 172.21.15.3:0/1666078319 >> v2:172.21.15.70:3300/0 conn(0x7f7c980d07c0 0x7f7c98072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.237+0000 7f7c95e00640 1 -- 172.21.15.3:0/1666078319 >> v1:172.21.15.70:6789/0 conn(0x7f7c98160af0 legacy=0x7f7c980d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.237+0000 7f7c95e00640 1 -- 172.21.15.3:0/1666078319 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9815d250 con 0x7f7c98160710 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c95e00640 1 --2- 172.21.15.3:0/1666078319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160710 0x7f7c980cf2c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7c980718d0 tx=0x7f7c8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b1842343eeb8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c95400640 1 -- 172.21.15.3:0/1666078319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88037070 con 0x7f7c98160710 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c95400640 1 -- 172.21.15.3:0/1666078319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c88032440 con 0x7f7c98160710 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c95400640 1 -- 172.21.15.3:0/1666078319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88032810 con 0x7f7c98160710 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 -- 172.21.15.3:0/1666078319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160710 msgr2=0x7f7c980cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/1666078319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160710 0x7f7c980cf2c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7c980718d0 tx=0x7f7c8802a230 comp rx=0 tx=0).stop 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 -- 172.21.15.3:0/1666078319 shutdown_connections 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/1666078319 >> v2:172.21.15.70:3300/0 conn(0x7f7c980d07c0 0x7f7c98072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/1666078319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160710 0x7f7c980cf2c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.238+0000 7f7c9ca00640 1 -- 172.21.15.3:0/1666078319 >> 172.21.15.3:0/1666078319 conn(0x7f7c9806d000 msgr2=0x7f7c9806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.239+0000 7f7c9ca00640 1 -- 172.21.15.3:0/1666078319 shutdown_connections 2024-09-10T07:53:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.239+0000 7f7c9ca00640 1 -- 172.21.15.3:0/1666078319 wait complete. 2024-09-10T07:53:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.240+0000 7f7c9ca00640 1 Processor -- start 2024-09-10T07:53:22.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.240+0000 7f7c9ca00640 1 -- start start 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 0x7f7c9814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c98160710 0x7f7c981e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9815d770 con 0x7f7c98160af0 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980cf800 con 0x7f7c980d07c0 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c9ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7c981e7720 con 0x7f7c98160710 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c96800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c96800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36020/0 (socket says 172.21.15.3:36020) 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.241+0000 7f7c96800640 1 -- 172.21.15.3:0/493290850 learned_addr learned my addr 172.21.15.3:0/493290850 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c95e00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 0x7f7c9814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c8fe00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c98160710 0x7f7c981e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 -- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c98160710 msgr2=0x7f7c981e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c98160710 0x7f7c981e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 -- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 msgr2=0x7f7c9814a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 0x7f7c9814a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 -- 172.21.15.3:0/493290850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c981e6e80 con 0x7f7c98160af0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c95e00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 0x7f7c9814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c96800640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007e40 tx=0x7f7c9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.242+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014620 con 0x7f7c98160af0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.243+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90019070 con 0x7f7c98160af0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.243+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014a20 con 0x7f7c98160af0 2024-09-10T07:53:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.243+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981e7160 con 0x7f7c98160af0 2024-09-10T07:53:22.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.243+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c98155160 con 0x7f7c98160af0 2024-09-10T07:53:22.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.244+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c98155360 con 0x7f7c98160af0 2024-09-10T07:53:22.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.245+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c90014bc0 con 0x7f7c98160af0 2024-09-10T07:53:22.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.245+0000 7f7c8e000640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 0x7f7c7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:22.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.246+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f7c9008b810 con 0x7f7c98160af0 2024-09-10T07:53:22.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.246+0000 7f7c95e00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 0x7f7c7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:22.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.246+0000 7f7c95e00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 0x7f7c7406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c88034000 tx=0x7f7c8802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:22.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.250+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7c98155360 con 0x7f7c98160af0 2024-09-10T07:53:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.473+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c980cc400 con 0x7f7c98160af0 2024-09-10T07:53:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.474+0000 7f7c8e000640 1 -- 172.21.15.3:0/493290850 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7f7c9005a120 con 0x7f7c98160af0 2024-09-10T07:53:22.474 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T07:53:22.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 msgr2=0x7f7c7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 0x7f7c7406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7c88034000 tx=0x7f7c8802e000 comp rx=0 tx=0).stop 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 msgr2=0x7f7c981e80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007e40 tx=0x7f7c9000e4b0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 shutdown_connections 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7c7406be30 0x7f7c7406e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c98160af0 0x7f7c981e80a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c98160710 0x7f7c981e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 --2- 172.21.15.3:0/493290850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c980d07c0 0x7f7c9814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.476+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 >> 172.21.15.3:0/493290850 conn(0x7f7c9806d000 msgr2=0x7f7c980b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.477+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 shutdown_connections 2024-09-10T07:53:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22.477+0000 7f7c9ca00640 1 -- 172.21.15.3:0/493290850 wait complete. 2024-09-10T07:53:22.491 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T07:53:22.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:53:22.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:22.492 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-10T07:53:22.528 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T07:53:22.528 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T07:53:22.529 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42293sec preferred_lft 42293sec 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f049:8dff:fedb:b4fb/64 scope link 2024-09-10T07:53:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 6a:3f:57:0d:62:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::683f:57ff:fe0d:6229/64 scope link 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:c2:31:7f:5f:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acc2:31ff:fe7f:5f44/64 scope link 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-10T07:53:22.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:22.531 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T07:53:22.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:53:22.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:53:22.623 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-10T07:53:22.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:22.626 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-10T07:53:22.702 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-10T07:53:22.702 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T07:53:22.702 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T07:53:22.703 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T07:53:22.703 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T07:53:22.704 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-10T07:53:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T07:53:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:22.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:22.902 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:53:22.937 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:53:22.937 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:53:22.938 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:53:22.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:22.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-10T07:53:23.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:53:23.034 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T07:53:23.069 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T07:53:23.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T07:53:23.070 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:53:23.144 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T07:53:23.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:23.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:53:23.273 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:53:23.274 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:53:23.274 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:53:23.289 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:53:23.289 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:53:23.289 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:53:23.289 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:53:23.289 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.70:3300,172.21.15.70:3301" 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790" 2024-09-10T07:53:23.312 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.70:6789,172.21.15.70:6790:/ 2024-09-10T07:53:23.313 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46339922-75e0-4945-8bb6-1acaef196937" will pass to kernel 2024-09-10T07:53:23.315 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T07:53:23.315 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-10T07:53:23.315 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T07:53:23.315 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T07:53:23.315 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T07:53:23.318 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T07:53:23.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:23.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T07:53:23.375 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T07:53:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T07:53:23.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:23.473 INFO:teuthology.orchestra.run.smithi003.stdout:client7669 2024-09-10T07:53:23.473 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:53:23.473 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:53:23.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-10T07:53:23.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.710+0000 7fdeff000640 1 Processor -- start 2024-09-10T07:53:23.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.711+0000 7fdeff000640 1 -- start start 2024-09-10T07:53:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdeff000640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdef80aefb0 0x7fdef8072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdeff000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef815faf0 con 0x7fdef8147ce0 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdeff000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef815ed30 con 0x7fdef80728d0 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdeff000640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdef815eaf0 con 0x7fdef80aefb0 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdef7e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdef8147ce0 0x7fdef80ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56750/0 (socket says 172.21.15.3:56750) 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdef7e00640 1 -- 172.21.15.3:0/3539761442 learned_addr learned my addr 172.21.15.3:0/3539761442 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.712+0000 7fdeefe00640 1 --2- 172.21.15.3:0/3539761442 >> v2:172.21.15.70:3300/0 conn(0x7fdef80aefb0 0x7fdef8072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125632755 0 0) 0x7fdef815faf0 con 0x7fdef8147ce0 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdedc003520 con 0x7fdef8147ce0 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1117043423 0 0) 0x7fdedc003520 con 0x7fdef8147ce0 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdeefe00640 1 -- 172.21.15.3:0/3539761442 >> v1:172.21.15.70:6789/0 conn(0x7fdef80728d0 legacy=0x7fdef806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdeefe00640 1 -- 172.21.15.3:0/3539761442 >> v1:172.21.15.3:6789/0 conn(0x7fdef8147ce0 legacy=0x7fdef80ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:23.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.713+0000 7fdeefe00640 1 -- 172.21.15.3:0/3539761442 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef815f010 con 0x7fdef80aefb0 2024-09-10T07:53:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.714+0000 7fdeefe00640 1 --2- 172.21.15.3:0/3539761442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80aefb0 0x7fdef8072390 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdee00097a0 tx=0x7fdee002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80350632b181f126 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.714+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0037070 con 0x7fdef80aefb0 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.714+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee002ab60 con 0x7fdef80aefb0 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.714+0000 7fdef7400640 1 -- 172.21.15.3:0/3539761442 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8001670 con 0x7fdef80aefb0 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 -- 172.21.15.3:0/3539761442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80aefb0 msgr2=0x7fdef8072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 --2- 172.21.15.3:0/3539761442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80aefb0 0x7fdef8072390 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdee00097a0 tx=0x7fdee002a0d0 comp rx=0 tx=0).stop 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 -- 172.21.15.3:0/3539761442 shutdown_connections 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 --2- 172.21.15.3:0/3539761442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80aefb0 0x7fdef8072390 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 -- 172.21.15.3:0/3539761442 >> 172.21.15.3:0/3539761442 conn(0x7fdef80b5cb0 msgr2=0x7fdef806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 -- 172.21.15.3:0/3539761442 shutdown_connections 2024-09-10T07:53:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.715+0000 7fdeff000640 1 -- 172.21.15.3:0/3539761442 wait complete. 2024-09-10T07:53:23.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.717+0000 7fdeff000640 1 Processor -- start 2024-09-10T07:53:23.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.717+0000 7fdeff000640 1 -- start start 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80728d0 0x7fdef80cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdef80aefb0 0x7fdef813c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdef815f660 con 0x7fdef8147ce0 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdef80730c0 con 0x7fdef80728d0 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeff000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdef8072d60 con 0x7fdef80aefb0 2024-09-10T07:53:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36032/0 (socket says 172.21.15.3:36032) 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 -- 172.21.15.3:0/3455622146 learned_addr learned my addr 172.21.15.3:0/3455622146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdef7e00640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80728d0 0x7fdef80cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdeefe00640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdef80aefb0 0x7fdef813c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 -- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdef80aefb0 msgr2=0x7fdef813c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdef80aefb0 0x7fdef813c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 -- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80728d0 msgr2=0x7fdef80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80728d0 0x7fdef80cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.718+0000 7fdefd000640 1 -- 172.21.15.3:0/3455622146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef813c9f0 con 0x7fdef8147ce0 2024-09-10T07:53:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdefd000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdef000c8f0 tx=0x7fdef000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0003da0 con 0x7fdef8147ce0 2024-09-10T07:53:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef0005070 con 0x7fdef8147ce0 2024-09-10T07:53:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0005460 con 0x7fdef8147ce0 2024-09-10T07:53:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef81f8160 con 0x7fdef8147ce0 2024-09-10T07:53:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.719+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef81f8720 con 0x7fdef8147ce0 2024-09-10T07:53:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.720+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef8148150 con 0x7fdef8147ce0 2024-09-10T07:53:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.721+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdef0005600 con 0x7fdef8147ce0 2024-09-10T07:53:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.722+0000 7fdef4c00640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 0x7fded406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.723+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fdef008add0 con 0x7fdef8147ce0 2024-09-10T07:53:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.723+0000 7fdef7e00640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 0x7fded406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.723+0000 7fdef7e00640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 0x7fded406e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdee8001eb0 tx=0x7fdee8001490 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.727+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdef0059090 con 0x7fdef8147ce0 2024-09-10T07:53:23.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:23.947+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fdef8147500 con 0x7fdef8147ce0 2024-09-10T07:53:24.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.651+0000 7fdef4c00640 1 -- 172.21.15.3:0/3455622146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v104) ==== 105+0+0 (secure 0 0 0) 0x7fdef005d390 con 0x7fdef8147ce0 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 msgr2=0x7fded406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 0x7fded406e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdee8001eb0 tx=0x7fdee8001490 comp rx=0 tx=0).stop 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 msgr2=0x7fdef813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdef000c8f0 tx=0x7fdef000cdb0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 shutdown_connections 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fded406be30 0x7fded406e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef8147ce0 0x7fdef813e370 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdef80aefb0 0x7fdef813c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 --2- 172.21.15.3:0/3455622146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdef80728d0 0x7fdef80cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.655+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 >> 172.21.15.3:0/3455622146 conn(0x7fdef80b5cb0 msgr2=0x7fdef8140df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.656+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 shutdown_connections 2024-09-10T07:53:24.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.656+0000 7fdeff000640 1 -- 172.21.15.3:0/3455622146 wait complete. 2024-09-10T07:53:24.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.839+0000 7f1f4aa00640 1 Processor -- start 2024-09-10T07:53:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.840+0000 7f1f4aa00640 1 -- start start 2024-09-10T07:53:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4aa00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f440b5b20 0x7f1f4406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4aa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4406d480 con 0x7f1f4406d070 2024-09-10T07:53:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4aa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f440b6290 con 0x7f1f44148000 2024-09-10T07:53:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4aa00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4406e4c0 con 0x7f1f440b5b20 2024-09-10T07:53:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4a000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1f4406d070 0x7f1f4406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53148/0 (socket says 172.21.15.3:53148) 2024-09-10T07:53:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f4a000640 1 -- 172.21.15.3:0/4156120411 learned_addr learned my addr 172.21.15.3:0/4156120411 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.841+0000 7f1f48c00640 1 --2- 172.21.15.3:0/4156120411 >> v2:172.21.15.70:3300/0 conn(0x7f1f440b5b20 0x7f1f4406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.842+0000 7f1f43e00640 1 -- 172.21.15.3:0/4156120411 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990306526 0 0) 0x7f1f3c001490 con 0x7f1f4406d070 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.842+0000 7f1f43e00640 1 -- 172.21.15.3:0/4156120411 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f2c003520 con 0x7f1f4406d070 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.842+0000 7f1f48c00640 1 -- 172.21.15.3:0/4156120411 >> v1:172.21.15.70:6789/0 conn(0x7f1f44148000 legacy=0x7f1f440afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.842+0000 7f1f48c00640 1 -- 172.21.15.3:0/4156120411 >> v1:172.21.15.3:6789/0 conn(0x7f1f4406d070 legacy=0x7f1f4406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.842+0000 7f1f48c00640 1 -- 172.21.15.3:0/4156120411 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f440b6570 con 0x7f1f440b5b20 2024-09-10T07:53:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.843+0000 7f1f48c00640 1 --2- 172.21.15.3:0/4156120411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f440b5b20 0x7f1f4406cb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f3002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c27f4f355e7307c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.843+0000 7f1f43e00640 1 -- 172.21.15.3:0/4156120411 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30037070 con 0x7f1f440b5b20 2024-09-10T07:53:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.843+0000 7f1f43e00640 1 -- 172.21.15.3:0/4156120411 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f30032440 con 0x7f1f440b5b20 2024-09-10T07:53:24.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.843+0000 7f1f43e00640 1 -- 172.21.15.3:0/4156120411 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f300327f0 con 0x7f1f440b5b20 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 -- 172.21.15.3:0/4156120411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f440b5b20 msgr2=0x7f1f4406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/4156120411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f440b5b20 0x7f1f4406cb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097a0 tx=0x7f1f3002a230 comp rx=0 tx=0).stop 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 -- 172.21.15.3:0/4156120411 shutdown_connections 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/4156120411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f440b5b20 0x7f1f4406cb30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 -- 172.21.15.3:0/4156120411 >> 172.21.15.3:0/4156120411 conn(0x7f1f44071fb0 msgr2=0x7f1f440723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 -- 172.21.15.3:0/4156120411 shutdown_connections 2024-09-10T07:53:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.844+0000 7f1f4aa00640 1 -- 172.21.15.3:0/4156120411 wait complete. 2024-09-10T07:53:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.846+0000 7f1f4aa00640 1 Processor -- start 2024-09-10T07:53:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.846+0000 7f1f4aa00640 1 -- start start 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f4406d070 0x7f1f4413b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f440b5b20 0x7f1f44146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4406e6f0 con 0x7f1f44148000 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1f440b6b30 con 0x7f1f4406d070 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1f440b5f90 con 0x7f1f440b5b20 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60410/0 (socket says 172.21.15.3:60410) 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 -- 172.21.15.3:0/2031029194 learned_addr learned my addr 172.21.15.3:0/2031029194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f48c00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f440b5b20 0x7f1f44146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f49600640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f4406d070 0x7f1f4413b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 -- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f440b5b20 msgr2=0x7f1f44146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f440b5b20 0x7f1f44146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 -- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f4406d070 msgr2=0x7f1f4413b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f4406d070 0x7f1f4413b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.847+0000 7f1f4a000640 1 -- 172.21.15.3:0/2031029194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44143240 con 0x7f1f44148000 2024-09-10T07:53:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f4a000640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00cb50 tx=0x7f1f3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f44148000 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00ad40 con 0x7f1f44148000 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c012620 con 0x7f1f44148000 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f44143520 con 0x7f1f44148000 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.848+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4413bf30 con 0x7f1f44148000 2024-09-10T07:53:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.849+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f440b4a00 con 0x7f1f44148000 2024-09-10T07:53:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.850+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f3c01a070 con 0x7f1f44148000 2024-09-10T07:53:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.851+0000 7f1f41600640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 0x7f1f2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.851+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f1f3c08ac80 con 0x7f1f44148000 2024-09-10T07:53:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.851+0000 7f1f49600640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 0x7f1f2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.852+0000 7f1f49600640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 0x7f1f2406e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1f380016e0 tx=0x7f1f38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:24.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:24.855+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f440b4a00 con 0x7f1f44148000 2024-09-10T07:53:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.079+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f441526d0 con 0x7f1f44148000 2024-09-10T07:53:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.080+0000 7f1f41600640 1 -- 172.21.15.3:0/2031029194 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5239 (secure 0 0 0) 0x7f1f4406d480 con 0x7f1f44148000 2024-09-10T07:53:25.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:25.079 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":105,"btime":"2024-09-10T07:53:24:656261+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:24.656237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7543,"mds_1":6875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[7543]},"id":7}]} 2024-09-10T07:53:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 105 2024-09-10T07:53:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 msgr2=0x7f1f2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:25.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 0x7f1f2406e2f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1f380016e0 tx=0x7f1f38009040 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 msgr2=0x7f1f441470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00cb50 tx=0x7f1f3c00a430 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 shutdown_connections 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1f2406be30 0x7f1f2406e2f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f44148000 0x7f1f441470e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f440b5b20 0x7f1f44146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 --2- 172.21.15.3:0/2031029194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f4406d070 0x7f1f4413b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 >> 172.21.15.3:0/2031029194 conn(0x7f1f44071fb0 msgr2=0x7f1f44072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 shutdown_connections 2024-09-10T07:53:25.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:25.082+0000 7f1f4aa00640 1 -- 172.21.15.3:0/2031029194 wait complete. 2024-09-10T07:53:25.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:53:10.636980+0000', 'modified': '2024-09-10T07:53:24.656237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7543, 'mds_1': 6875}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6875': {'gid': 6875, 'name': 'a', 'rank': 1, 'incarnation': 105, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6836/3004970849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 3004970849}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 3004970849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7543': {'gid': 7543, 'name': 'c', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/3773991953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 3773991953}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 3773991953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7543, 'qdb_cluster': [7543]} 2024-09-10T07:53:25.500 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6875:up:creating 2024-09-10T07:53:26.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.671+0000 7fe641a00640 1 Processor -- start 2024-09-10T07:53:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.671+0000 7fe641a00640 1 -- start start 2024-09-10T07:53:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe641a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe63c15e8e0 0x7fe63c162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe641a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe63c0cf1f0 0x7fe63c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe641a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe63c0b7920 con 0x7fe63c0d19d0 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe641a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0cf940 con 0x7fe63c15e8e0 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe641a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0cfc20 con 0x7fe63c0cf1f0 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe63aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe63c15e8e0 0x7fe63c162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe63b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe63c0d19d0 0x7fe63c0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53154/0 (socket says 172.21.15.3:53154) 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe63b400640 1 -- 172.21.15.3:0/565592539 learned_addr learned my addr 172.21.15.3:0/565592539 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.673+0000 7fe63a000640 1 --2- 172.21.15.3:0/565592539 >> v2:172.21.15.70:3300/0 conn(0x7fe63c0cf1f0 0x7fe63c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe639600640 1 -- 172.21.15.3:0/565592539 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598650896 0 0) 0x7fe634001490 con 0x7fe63c0d19d0 2024-09-10T07:53:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe639600640 1 -- 172.21.15.3:0/565592539 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe620003520 con 0x7fe63c0d19d0 2024-09-10T07:53:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe63aa00640 1 -- 172.21.15.3:0/565592539 >> v2:172.21.15.70:3300/0 conn(0x7fe63c0cf1f0 msgr2=0x7fe63c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe63aa00640 1 --2- 172.21.15.3:0/565592539 >> v2:172.21.15.70:3300/0 conn(0x7fe63c0cf1f0 0x7fe63c0d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe63aa00640 1 -- 172.21.15.3:0/565592539 >> v1:172.21.15.3:6789/0 conn(0x7fe63c0d19d0 legacy=0x7fe63c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.674+0000 7fe63aa00640 1 -- 172.21.15.3:0/565592539 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c0d2ae0 con 0x7fe63c15e8e0 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.675+0000 7fe63aa00640 1 --2- 172.21.15.3:0/565592539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c162cf0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe63c08d720 tx=0x7fe62c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6141b4aebca0ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.675+0000 7fe639600640 1 -- 172.21.15.3:0/565592539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c037070 con 0x7fe63c15e8e0 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.675+0000 7fe639600640 1 -- 172.21.15.3:0/565592539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c032440 con 0x7fe63c15e8e0 2024-09-10T07:53:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.675+0000 7fe639600640 1 -- 172.21.15.3:0/565592539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c02ae60 con 0x7fe63c15e8e0 2024-09-10T07:53:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 -- 172.21.15.3:0/565592539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 msgr2=0x7fe63c162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 --2- 172.21.15.3:0/565592539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c162cf0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe63c08d720 tx=0x7fe62c02a110 comp rx=0 tx=0).stop 2024-09-10T07:53:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 -- 172.21.15.3:0/565592539 shutdown_connections 2024-09-10T07:53:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 --2- 172.21.15.3:0/565592539 >> v2:172.21.15.70:3300/0 conn(0x7fe63c0cf1f0 0x7fe63c0d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 --2- 172.21.15.3:0/565592539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c162cf0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 -- 172.21.15.3:0/565592539 >> 172.21.15.3:0/565592539 conn(0x7fe63c06bd00 msgr2=0x7fe63c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 -- 172.21.15.3:0/565592539 shutdown_connections 2024-09-10T07:53:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.676+0000 7fe641a00640 1 -- 172.21.15.3:0/565592539 wait complete. 2024-09-10T07:53:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.677+0000 7fe641a00640 1 Processor -- start 2024-09-10T07:53:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 -- start start 2024-09-10T07:53:26.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe63c0cf1f0 0x7fe63c160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe63c0d19d0 0x7fe63c15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0d42f0 con 0x7fe63c15e8e0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0cf630 con 0x7fe63c0d19d0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.678+0000 7fe641a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c0d4470 con 0x7fe63c0cf1f0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60426/0 (socket says 172.21.15.3:60426) 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 -- 172.21.15.3:0/3949936919 learned_addr learned my addr 172.21.15.3:0/3949936919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63aa00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe63c0cf1f0 0x7fe63c160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63a000640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe63c0d19d0 0x7fe63c15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 -- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe63c0cf1f0 msgr2=0x7fe63c160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe63c0cf1f0 0x7fe63c160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 -- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe63c0d19d0 msgr2=0x7fe63c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe63c0d19d0 0x7fe63c15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 -- 172.21.15.3:0/3949936919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c0ce6f0 con 0x7fe63c15e8e0 2024-09-10T07:53:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.679+0000 7fe63b400640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe63400cb40 tx=0x7fe63400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.680+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634017070 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.680+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63400ac40 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.680+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634012400 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.680+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c0cc1d0 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.680+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c0cc470 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.681+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c0d1490 con 0x7fe63c15e8e0 2024-09-10T07:53:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.682+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe63401a070 con 0x7fe63c15e8e0 2024-09-10T07:53:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.683+0000 7fe632a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.683+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe63408acf0 con 0x7fe63c15e8e0 2024-09-10T07:53:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.683+0000 7fe63aa00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:26.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.684+0000 7fe63aa00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 0x7fe61c06e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe62c034000 tx=0x7fe62c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:26.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.687+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe63c0b7920 con 0x7fe63c15e8e0 2024-09-10T07:53:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.907+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe63c0d4f10 con 0x7fe63c15e8e0 2024-09-10T07:53:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.908+0000 7fe632a00640 1 -- 172.21.15.3:0/3949936919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5262 (secure 0 0 0) 0x7fe634059600 con 0x7fe63c15e8e0 2024-09-10T07:53:26.907 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:26.907 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"btime":"2024-09-10T07:53:26:667916+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:26.667912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:26.908 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 108 2024-09-10T07:53:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 msgr2=0x7fe61c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 0x7fe61c06e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe62c034000 tx=0x7fe62c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:26.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 msgr2=0x7fe63c161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe63400cb40 tx=0x7fe63400a060 comp rx=0 tx=0).stop 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 shutdown_connections 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe63c15e8e0 0x7fe63c161280 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe63c0d19d0 0x7fe63c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 --2- 172.21.15.3:0/3949936919 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe63c0cf1f0 0x7fe63c160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 >> 172.21.15.3:0/3949936919 conn(0x7fe63c06bd00 msgr2=0x7fe63c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 shutdown_connections 2024-09-10T07:53:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:26.910+0000 7fe641a00640 1 -- 172.21.15.3:0/3949936919 wait complete. 2024-09-10T07:53:27.199 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:53:27.200+0000 7f2cd0200640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T07:53:27.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T07:53:10.636980+0000', 'modified': '2024-09-10T07:53:26.667912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7543, 'mds_1': 6875, 'mds_2': 6872}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6872': {'gid': 6872, 'name': 'd', 'rank': 2, 'incarnation': 107, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.70:6835/3268462742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 3268462742}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3268462742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6875': {'gid': 6875, 'name': 'a', 'rank': 1, 'incarnation': 105, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.3:6836/3004970849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 3004970849}, {'type': 'v1', 'addr': '172.21.15.3:6836', 'nonce': 3004970849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7543': {'gid': 7543, 'name': 'c', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6837/3773991953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6835', 'nonce': 3773991953}, {'type': 'v1', 'addr': '172.21.15.3:6837', 'nonce': 3773991953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7543, 'qdb_cluster': [6872, 6875, 7543]} 2024-09-10T07:53:27.405 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T07:53:27.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T07:53:27.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.566+0000 7fb29ae00640 1 Processor -- start 2024-09-10T07:53:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.566+0000 7fb29ae00640 1 -- start start 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29ae00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 0x7fb2940cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29ae00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 0x7fb2940d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29ae00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2940cee40 con 0x7fb2940d4150 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29ae00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb2940d3780 con 0x7fb29415e860 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29ae00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2940cf120 con 0x7fb2940cfbf0 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29a400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb2940d4150 0x7fb2940b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53164/0 (socket says 172.21.15.3:53164) 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb29a400640 1 -- 172.21.15.3:0/3281895942 learned_addr learned my addr 172.21.15.3:0/3281895942 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299000640 1 --2- 172.21.15.3:0/3281895942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 0x7fb2940d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299a00640 1 --2- 172.21.15.3:0/3281895942 >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 0x7fb2940cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201959293 0 0) 0x7fb2940cee40 con 0x7fb2940d4150 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb27c003520 con 0x7fb2940d4150 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299000640 1 -- 172.21.15.3:0/3281895942 >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 msgr2=0x7fb2940cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299000640 1 --2- 172.21.15.3:0/3281895942 >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 0x7fb2940cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299000640 1 -- 172.21.15.3:0/3281895942 >> v1:172.21.15.3:6789/0 conn(0x7fb2940d4150 legacy=0x7fb2940b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 204188265 0 0) 0x7fb27c003520 con 0x7fb2940d4150 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299000640 1 -- 172.21.15.3:0/3281895942 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2940d3410 con 0x7fb2940cfbf0 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.567+0000 7fb299a00640 1 --2- 172.21.15.3:0/3281895942 >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 0x7fb2940cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:27.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.568+0000 7fb299000640 1 --2- 172.21.15.3:0/3281895942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 0x7fb2940d3c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7acf9ab23c2dd551 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280037070 con 0x7fb2940cfbf0 2024-09-10T07:53:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28002aba0 con 0x7fb2940cfbf0 2024-09-10T07:53:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb293e00640 1 -- 172.21.15.3:0/3281895942 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0019f0 con 0x7fb2940cfbf0 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 -- 172.21.15.3:0/3281895942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 msgr2=0x7fb2940d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 --2- 172.21.15.3:0/3281895942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 0x7fb2940d3c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a110 comp rx=0 tx=0).stop 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 -- 172.21.15.3:0/3281895942 shutdown_connections 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 --2- 172.21.15.3:0/3281895942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb2940cfbf0 0x7fb2940d3c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 --2- 172.21.15.3:0/3281895942 >> v2:172.21.15.70:3300/0 conn(0x7fb29415e860 0x7fb2940cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 -- 172.21.15.3:0/3281895942 >> 172.21.15.3:0/3281895942 conn(0x7fb2940d4650 msgr2=0x7fb2940d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 -- 172.21.15.3:0/3281895942 shutdown_connections 2024-09-10T07:53:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.569+0000 7fb29ae00640 1 -- 172.21.15.3:0/3281895942 wait complete. 2024-09-10T07:53:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.571+0000 7fb29ae00640 1 Processor -- start 2024-09-10T07:53:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.571+0000 7fb29ae00640 1 -- start start 2024-09-10T07:53:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 0x7fb294159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 0x7fb2940b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2940b7f90 con 0x7fb2940d4150 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2940d2460 con 0x7fb2940cfbf0 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb29ae00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2940b8110 con 0x7fb29415e860 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb299000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb299000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60428/0 (socket says 172.21.15.3:60428) 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.572+0000 7fb299000640 1 -- 172.21.15.3:0/4070012019 learned_addr learned my addr 172.21.15.3:0/4070012019 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb29a400640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 0x7fb2940b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299a00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 0x7fb294159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 -- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 msgr2=0x7fb2940b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 0x7fb2940b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 -- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 msgr2=0x7fb294159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 0x7fb294159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 -- 172.21.15.3:0/4070012019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb294159790 con 0x7fb2940d4150 2024-09-10T07:53:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb29a400640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 0x7fb2940b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299a00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 0x7fb294159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:27.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb299000640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098c0 tx=0x7fb2800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.573+0000 7fb291600640 1 -- 172.21.15.3:0/4070012019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280037050 con 0x7fb2940d4150 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.574+0000 7fb291600640 1 -- 172.21.15.3:0/4070012019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb280043070 con 0x7fb2940d4150 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.574+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2941f5060 con 0x7fb2940d4150 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.574+0000 7fb291600640 1 -- 172.21.15.3:0/4070012019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280003e00 con 0x7fb2940d4150 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.574+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2941f4580 con 0x7fb2940d4150 2024-09-10T07:53:27.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.576+0000 7fb291600640 1 -- 172.21.15.3:0/4070012019 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb280048460 con 0x7fb2940d4150 2024-09-10T07:53:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.576+0000 7fb291600640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 0x7fb27406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.577+0000 7fb291600640 1 -- 172.21.15.3:0/4070012019 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb2800ab3d0 con 0x7fb2940d4150 2024-09-10T07:53:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.577+0000 7fb299a00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 0x7fb27406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.577+0000 7fb29f2da740 1 Processor -- start 2024-09-10T07:53:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.577+0000 7fb29f2da740 1 -- start start 2024-09-10T07:53:27.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.577+0000 7fb299a00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 0x7fb27406e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb288009ac0 tx=0x7fb288008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5ed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a5660 0x5645ce5a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5645ce5063b0 con 0x5645ce5a9630 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5645ce50b410 con 0x5645ce5a5660 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.578+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5645ce4b3620 con 0x5645ce5a7ea0 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60442/0 (socket says 172.21.15.3:60442) 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 -- 172.21.15.3:0/1401904133 learned_addr learned my addr 172.21.15.3:0/1401904133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb299a00640 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb299000640 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a5660 0x5645ce5a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 -- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 msgr2=0x5645ce5ed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5ed640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 -- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a5660 msgr2=0x5645ce5a8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a5660 0x5645ce5a8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 -- 172.21.15.3:0/1401904133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ce21e2d0 con 0x5645ce5a9630 2024-09-10T07:53:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb29a400640 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00d230 tx=0x7fb28c00d6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffa36e4e86cc84d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.579+0000 7fb286000640 1 -- 172.21.15.3:0/1401904133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c018070 con 0x5645ce5a9630 2024-09-10T07:53:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb286000640 1 -- 172.21.15.3:0/1401904133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c0074e0 con 0x5645ce5a9630 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb286000640 1 -- 172.21.15.3:0/1401904133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007890 con 0x5645ce5a9630 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 -- 172.21.15.3:0/1401904133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 msgr2=0x5645ce5a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00d230 tx=0x7fb28c00d6f0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 -- 172.21.15.3:0/1401904133 shutdown_connections 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5a2720 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a5660 0x5645ce5a8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1401904133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5ed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 -- 172.21.15.3:0/1401904133 >> 172.21.15.3:0/1401904133 conn(0x5645ce5a48f0 msgr2=0x5645ce5ecb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 -- 172.21.15.3:0/1401904133 shutdown_connections 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.580+0000 7fb29f2da740 1 -- 172.21.15.3:0/1401904133 wait complete. 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.581+0000 7fb29f2da740 1 -- start start 2024-09-10T07:53:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.581+0000 7fb29f2da740 1 Processor -- start 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 0x5645ce5d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5645ce4c6780 con 0x5645ce5d0bc0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5645ce5073d0 con 0x5645ce5a9630 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5645ce5a3d90 con 0x5645ce5a7ea0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.582+0000 7fb29f2da740 10 client.0 _ll_register_callbacks cb 0x5645ce5045d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60452/0 (socket says 172.21.15.3:60452) 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 -- 172.21.15.3:0/3796609101 learned_addr learned my addr 172.21.15.3:0/3796609101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb299a00640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb299000640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 0x5645ce5d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 -- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 msgr2=0x5645ce5c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 -- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 msgr2=0x5645ce5d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 0x5645ce5d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 -- 172.21.15.3:0/3796609101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ce2e86b0 con 0x5645ce5d0bc0 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb299000640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 0x5645ce5d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb299a00640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb29a400640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005c00 tx=0x7fb28c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144a9878899612cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.583+0000 7fb27aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:27.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c023020 con 0x5645ce5d0bc0 2024-09-10T07:53:27.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb28c013470 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c0074e0 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fb28c02d050 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 10 client.8226 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x5645ce5d2a20 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c0074e0 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7fb28c030020 con 0x5645ce5d0bc0 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 10 client.8226 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 10 client.8226 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 20 client.8226 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce609010 0x5645ce60b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 4 client.8226 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.584+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ce518b40 con 0x5645ce609010 2024-09-10T07:53:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.585+0000 7fb299a00640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce609010 0x5645ce60b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.587+0000 7fb299a00640 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce609010 0x5645ce60b400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6e7c15136ee02fd server_cookie=e66bd8044aba3ba5 in_seq=0 out_seq=0 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.587+0000 7fb27aa00640 10 client.8226 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb27aa00640 1 -- 172.21.15.3:0/3796609101 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645ce518b40 con 0x5645ce609010 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb27aa00640 10 client.8226 handle_command_reply: tid=0 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 client.8226 shutdown 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 msgr2=0x5645ce5a2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005c00 tx=0x7fb28c005b70 comp rx=0 tx=0).stop 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 shutdown_connections 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce609010 0x5645ce60b400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5d0bc0 0x5645ce5a2820 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5a9630 0x5645ce5d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 --2- 172.21.15.3:0/3796609101 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb27aa00640 0 client.8226 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 >> 172.21.15.3:0/3796609101 conn(0x5645ce5a5660 msgr2=0x5645ce5eab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 shutdown_connections 2024-09-10T07:53:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 1 -- 172.21.15.3:0/3796609101 wait complete. 2024-09-10T07:53:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.588+0000 7fb29f2da740 20 client.8226 trim_cache size 0 max 16384 2024-09-10T07:53:27.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.606+0000 7fb29f2da740 1 Processor -- start 2024-09-10T07:53:27.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.606+0000 7fb29f2da740 1 -- start start 2024-09-10T07:53:27.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941ffad0 con 0x5645ce5a9630 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2942001a0 con 0x5645ce5d0bc0 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.607+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb2941614d0 con 0x5645ce5a7ea0 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60458/0 (socket says 172.21.15.3:60458) 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 -- 172.21.15.3:0/2120165850 learned_addr learned my addr 172.21.15.3:0/2120165850 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 -- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 msgr2=0x5645ce5eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 -- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 msgr2=0x5645ce5eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 -- 172.21.15.3:0/2120165850 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ce507360 con 0x5645ce5a9630 2024-09-10T07:53:27.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb299000640 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec67ac3d63426a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb285600640 1 -- 172.21.15.3:0/2120165850 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280079670 con 0x5645ce5a9630 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb285600640 1 -- 172.21.15.3:0/2120165850 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb280038040 con 0x5645ce5a9630 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.608+0000 7fb285600640 1 -- 172.21.15.3:0/2120165850 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28007da20 con 0x5645ce5a9630 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 -- 172.21.15.3:0/2120165850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 msgr2=0x5645ce5ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280009190 comp rx=0 tx=0).stop 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 -- 172.21.15.3:0/2120165850 shutdown_connections 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a9630 0x5645ce5ee670 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 --2- 172.21.15.3:0/2120165850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a7ea0 0x5645ce5eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 -- 172.21.15.3:0/2120165850 >> 172.21.15.3:0/2120165850 conn(0x5645ce5a5660 msgr2=0x5645ce5ec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 -- 172.21.15.3:0/2120165850 shutdown_connections 2024-09-10T07:53:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.609+0000 7fb29f2da740 1 -- 172.21.15.3:0/2120165850 wait complete. 2024-09-10T07:53:27.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.610+0000 7fb29f2da740 1 -- start start 2024-09-10T07:53:27.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.610+0000 7fb29f2da740 1 Processor -- start 2024-09-10T07:53:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a9630 0x5645ce5cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5645ce5cfd50 con 0x5645ce5a7ea0 2024-09-10T07:53:27.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5645ce5fd800 con 0x5645ce5d0bc0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5645ce5d03a0 con 0x5645ce5a9630 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29f2da740 10 client.0 _ll_register_callbacks cb 0x7fb28c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb299a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb299a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60464/0 (socket says 172.21.15.3:60464) 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb299a00640 1 -- 172.21.15.3:0/1470910000 learned_addr learned my addr 172.21.15.3:0/1470910000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb299000640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a9630 0x5645ce5cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.611+0000 7fb29a400640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 -- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a9630 msgr2=0x5645ce5cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a9630 0x5645ce5cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 -- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 msgr2=0x5645ce5cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 -- 172.21.15.3:0/1470910000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ce5ed260 con 0x5645ce5a7ea0 2024-09-10T07:53:27.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb299a00640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb288010c10 tx=0x7fb288009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8356e5f372432a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb27aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28801c070 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb280038040 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb288025860 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fb288036050 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb29f2da740 10 client.8232 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.612+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x5645ce605840 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb280038040 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7fb288039020 con 0x5645ce5a7ea0 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 10 client.8232 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 10 client.8232 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 20 client.8232 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce605190 0x5645ce5fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 4 client.8232 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:53:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "status"}) -- 0x5645ce512e90 con 0x5645ce605190 2024-09-10T07:53:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.613+0000 7fb299000640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce605190 0x5645ce5fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb299000640 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce605190 0x5645ce5fb1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e18ddf07cc27082a server_cookie=f376896508963210 in_seq=0 out_seq=0 2024-09-10T07:53:27.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb27aa00640 10 client.8232 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb27aa00640 1 -- 172.21.15.3:0/1470910000 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5645ce512e90 con 0x5645ce605190 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb27aa00640 10 client.8232 handle_command_reply: tid=0 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 client.8232 shutdown 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 msgr2=0x5645ce66a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb288010c10 tx=0x7fb288009190 comp rx=0 tx=0).stop 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 shutdown_connections 2024-09-10T07:53:27.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x5645ce605190 0x5645ce5fb1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5645ce5d0bc0 0x5645ce5cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5645ce5a9630 0x5645ce5cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 --2- 172.21.15.3:0/1470910000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645ce5a7ea0 0x5645ce66a880 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb27aa00640 0 client.8232 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.614+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 >> 172.21.15.3:0/1470910000 conn(0x5645ce5a5660 msgr2=0x5645ce5d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 shutdown_connections 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29f2da740 1 -- 172.21.15.3:0/1470910000 wait complete. 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29f2da740 20 client.8232 trim_cache size 0 max 16384 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:53:27.618 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6872, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 1.947032973, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 108, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 102, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.083243757000002 2024-09-10T07:53:27.619 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 msgr2=0x7fb27406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 0x7fb27406e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb288009ac0 tx=0x7fb288008040 comp rx=0 tx=0).stop 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 msgr2=0x7fb2940b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098c0 tx=0x7fb2800046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 shutdown_connections 2024-09-10T07:53:27.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb27406be60 0x7fb27406e320 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb29415e860 0x7fb2940b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2940d4150 0x7fb2940b5480 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 --2- 172.21.15.3:0/4070012019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2940cfbf0 0x7fb294159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.615+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 >> 172.21.15.3:0/4070012019 conn(0x7fb2940d4650 msgr2=0x7fb294072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.616+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 shutdown_connections 2024-09-10T07:53:27.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.616+0000 7fb29ae00640 1 -- 172.21.15.3:0/4070012019 wait complete. 2024-09-10T07:53:27.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T07:53:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.790+0000 7f42a1e00640 1 Processor -- start 2024-09-10T07:53:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.790+0000 7f42a1e00640 1 -- start start 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.791+0000 7f42a1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 0x7f429c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.791+0000 7f42a1e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429c0ba570 con 0x7f429c0d2510 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.791+0000 7f42a1e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429c0cff80 con 0x7f429c0d1580 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.791+0000 7f42a1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f429c0d2ce0 con 0x7f429c15e820 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f429c0d2510 0x7f429c0d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53170/0 (socket says 172.21.15.3:53170) 2024-09-10T07:53:27.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429be00640 1 -- 172.21.15.3:0/3453964211 learned_addr learned my addr 172.21.15.3:0/3453964211 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f42a0a00640 1 --2- 172.21.15.3:0/3453964211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 0x7f429c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422462683 0 0) 0x7f4288001110 con 0x7f429c0d2510 2024-09-10T07:53:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4284003520 con 0x7f429c0d2510 2024-09-10T07:53:27.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762333801 0 0) 0x7f429c0cff80 con 0x7f429c0d1580 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4288001110 con 0x7f429c0d1580 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f42a0a00640 1 -- 172.21.15.3:0/3453964211 >> v1:172.21.15.70:6789/0 conn(0x7f429c0d1580 legacy=0x7f429c0b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f42a0a00640 1 -- 172.21.15.3:0/3453964211 >> v1:172.21.15.3:6789/0 conn(0x7f429c0d2510 legacy=0x7f429c0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.792+0000 7f42a0a00640 1 -- 172.21.15.3:0/3453964211 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c0d2fc0 con 0x7f429c15e820 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.794+0000 7f42a0a00640 1 --2- 172.21.15.3:0/3453964211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 0x7f429c0cf880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42900097d0 tx=0x7f429002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b8a254e0743cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.794+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290037070 con 0x7f429c15e820 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.794+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290032440 con 0x7f429c15e820 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.794+0000 7f429b400640 1 -- 172.21.15.3:0/3453964211 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290032810 con 0x7f429c15e820 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 -- 172.21.15.3:0/3453964211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 msgr2=0x7f429c0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 --2- 172.21.15.3:0/3453964211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 0x7f429c0cf880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42900097d0 tx=0x7f429002a200 comp rx=0 tx=0).stop 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 -- 172.21.15.3:0/3453964211 shutdown_connections 2024-09-10T07:53:27.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 --2- 172.21.15.3:0/3453964211 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c15e820 0x7f429c0cf880 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 -- 172.21.15.3:0/3453964211 >> 172.21.15.3:0/3453964211 conn(0x7f429c072560 msgr2=0x7f429c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 -- 172.21.15.3:0/3453964211 shutdown_connections 2024-09-10T07:53:27.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.795+0000 7f42a1e00640 1 -- 172.21.15.3:0/3453964211 wait complete. 2024-09-10T07:53:27.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.796+0000 7f42a1e00640 1 Processor -- start 2024-09-10T07:53:27.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.797+0000 7f42a1e00640 1 -- start start 2024-09-10T07:53:27.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.797+0000 7f42a1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c0d1580 0x7f429c0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.797+0000 7f42a1e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f429c0d2510 0x7f429c1f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.797+0000 7f42a1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f429c15ec00 con 0x7f429c15e820 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0d3580 con 0x7f429c0d2510 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f429c0d41b0 con 0x7f429c0d1580 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60474/0 (socket says 172.21.15.3:60474) 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 -- 172.21.15.3:0/410406010 learned_addr learned my addr 172.21.15.3:0/410406010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f429be00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f429c0d2510 0x7f429c1f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a0a00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c0d1580 0x7f429c0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 -- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c0d1580 msgr2=0x7f429c0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c0d1580 0x7f429c0cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 -- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f429c0d2510 msgr2=0x7f429c1f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f429c0d2510 0x7f429c1f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.798+0000 7f42a1400640 1 -- 172.21.15.3:0/410406010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c0ce240 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f42a1400640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f429400c6e0 tx=0x7f429400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f4298c00640 1 -- 172.21.15.3:0/410406010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294003b70 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f4298c00640 1 -- 172.21.15.3:0/410406010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294003d10 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c1f4b50 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f4298c00640 1 -- 172.21.15.3:0/410406010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42940052d0 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.799+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c1f50b0 con 0x7f429c15e820 2024-09-10T07:53:27.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.801+0000 7f4298c00640 1 -- 172.21.15.3:0/410406010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4294005470 con 0x7f429c15e820 2024-09-10T07:53:27.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.802+0000 7f4298c00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.802+0000 7f4298c00640 1 -- 172.21.15.3:0/410406010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f429408ad10 con 0x7f429c15e820 2024-09-10T07:53:27.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.802+0000 7f42a0a00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.803+0000 7f42a61c9740 1 Processor -- start 2024-09-10T07:53:27.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.803+0000 7f42a61c9740 1 -- start start 2024-09-10T07:53:27.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.803+0000 7f42a0a00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 0x7f427c06e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4290034000 tx=0x7f429002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94e7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 0x564eb94a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564eb94003b0 con 0x564eb94a3630 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x564eb9405410 con 0x564eb949f660 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564eb93ad620 con 0x564eb94a1ea0 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60488/0 (socket says 172.21.15.3:60488) 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 -- 172.21.15.3:0/3855897005 learned_addr learned my addr 172.21.15.3:0/3855897005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 -- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 msgr2=0x564eb94e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a0a00640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f429be00640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 0x564eb94a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94e7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 -- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 msgr2=0x564eb94a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 0x564eb94a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a1400640 1 -- 172.21.15.3:0/3855897005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb91182d0 con 0x564eb94a3630 2024-09-10T07:53:27.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f42a0a00640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:27.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.804+0000 7f429be00640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 0x564eb94a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:27.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f42a1400640 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4294058ed0 tx=0x7f4294011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcea9acca90f12f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f428d200640 1 -- 172.21.15.3:0/3855897005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294061b00 con 0x564eb94a3630 2024-09-10T07:53:27.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f428d200640 1 -- 172.21.15.3:0/3855897005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294072020 con 0x564eb94a3630 2024-09-10T07:53:27.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f428d200640 1 -- 172.21.15.3:0/3855897005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294005760 con 0x564eb94a3630 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f42a61c9740 1 -- 172.21.15.3:0/3855897005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 msgr2=0x564eb949c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.805+0000 7f42a61c9740 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4294058ed0 tx=0x7f4294011000 comp rx=0 tx=0).stop 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 -- 172.21.15.3:0/3855897005 shutdown_connections 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a3630 0x564eb949c720 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb949f660 0x564eb94a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 --2- 172.21.15.3:0/3855897005 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 -- 172.21.15.3:0/3855897005 >> 172.21.15.3:0/3855897005 conn(0x564eb949e8f0 msgr2=0x564eb94e6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 -- 172.21.15.3:0/3855897005 shutdown_connections 2024-09-10T07:53:27.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 -- 172.21.15.3:0/3855897005 wait complete. 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.806+0000 7f42a61c9740 1 -- start start 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 Processor -- start 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564eb93c0780 con 0x564eb94cba60 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x564eb94013d0 con 0x564eb94a3630 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564eb949dd90 con 0x564eb94a1ea0 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.807+0000 7f42a61c9740 10 client.0 _ll_register_callbacks cb 0x564eb93fe5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60492/0 (socket says 172.21.15.3:60492) 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 -- 172.21.15.3:0/4227618561 learned_addr learned my addr 172.21.15.3:0/4227618561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a0a00640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f429be00640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 -- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 msgr2=0x564eb94f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 -- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 msgr2=0x564eb94c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 -- 172.21.15.3:0/4227618561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb91e26b0 con 0x564eb94cba60 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f42a1400640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4294058c40 tx=0x7f4294001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7382a2eb3f86b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.808+0000 7f4281600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294057240 con 0x564eb94cba60 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f42940555b0 con 0x564eb94cba60 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4294072020 con 0x564eb94cba60 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f4294058390 con 0x564eb94cba60 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 10 client.8244 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:27.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x564eb94cc7d0 con 0x564eb94cba60 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294072020 con 0x564eb94cba60 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7f4294058530 con 0x564eb94cba60 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 10 client.8244 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 10 client.8244 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 20 client.8244 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c99a0 0x564eb94c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 4 client.8244 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.809+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eb9412b40 con 0x564eb94c99a0 2024-09-10T07:53:27.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.810+0000 7f42a0a00640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c99a0 0x564eb94c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.810+0000 7f4281600640 10 client.8244 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.810+0000 7f42a0a00640 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c99a0 0x564eb94c9d80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fb3df19952ae8d1 server_cookie=681b413d8a45074f in_seq=0 out_seq=0 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f4281600640 1 -- 172.21.15.3:0/4227618561 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564eb9412b40 con 0x564eb94c99a0 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f4281600640 10 client.8244 handle_command_reply: tid=0 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f42a61c9740 1 client.8244 shutdown 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 msgr2=0x564eb94cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4294058c40 tx=0x7f4294001250 comp rx=0 tx=0).stop 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 shutdown_connections 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.811+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c99a0 0x564eb94c9d80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94cbec0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 --2- 172.21.15.3:0/4227618561 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb94f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 >> 172.21.15.3:0/4227618561 conn(0x564eb949f660 msgr2=0x564eb94e6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f4281600640 0 client.8244 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:27.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 shutdown_connections 2024-09-10T07:53:27.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 1 -- 172.21.15.3:0/4227618561 wait complete. 2024-09-10T07:53:27.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.812+0000 7f42a61c9740 20 client.8244 trim_cache size 0 max 16384 2024-09-10T07:53:27.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.830+0000 7f42a61c9740 1 Processor -- start 2024-09-10T07:53:27.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 -- start start 2024-09-10T07:53:27.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb950a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb9498040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1ffbf0 con 0x564eb94cba60 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f429c2002c0 con 0x564eb94a3630 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.831+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f429c1fecf0 con 0x564eb94a1ea0 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60496/0 (socket says 172.21.15.3:60496) 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 -- 172.21.15.3:0/1840962914 learned_addr learned my addr 172.21.15.3:0/1840962914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a0a00640 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb950a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f429be00640 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb9498040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 -- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 msgr2=0x564eb950a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb950a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 -- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 msgr2=0x564eb9498040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb9498040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 -- 172.21.15.3:0/1840962914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb94e3760 con 0x564eb94cba60 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.832+0000 7f42a1400640 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4294001560 tx=0x7f4294073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ef97661dbec1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f4283e00640 1 -- 172.21.15.3:0/1840962914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429406d120 con 0x564eb94cba60 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f4283e00640 1 -- 172.21.15.3:0/1840962914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294072040 con 0x564eb94cba60 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f4283e00640 1 -- 172.21.15.3:0/1840962914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294058530 con 0x564eb94cba60 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 -- 172.21.15.3:0/1840962914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 msgr2=0x564eb94e4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4294001560 tx=0x7f4294073000 comp rx=0 tx=0).stop 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 -- 172.21.15.3:0/1840962914 shutdown_connections 2024-09-10T07:53:27.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94cba60 0x564eb94e4d80 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb9498040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1840962914 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94a1ea0 0x564eb950a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 -- 172.21.15.3:0/1840962914 >> 172.21.15.3:0/1840962914 conn(0x564eb949f660 msgr2=0x564eb94c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 -- 172.21.15.3:0/1840962914 shutdown_connections 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.833+0000 7f42a61c9740 1 -- 172.21.15.3:0/1840962914 wait complete. 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.834+0000 7f42a61c9740 1 -- start start 2024-09-10T07:53:27.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.834+0000 7f42a61c9740 1 Processor -- start 2024-09-10T07:53:27.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94cba60 0x564eb94c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564eb93c0780 con 0x564eb94a1ea0 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x564eb949dd90 con 0x564eb94a3630 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x564eb94013d0 con 0x564eb94cba60 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a61c9740 10 client.0 _ll_register_callbacks cb 0x7f4294058930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60506/0 (socket says 172.21.15.3:60506) 2024-09-10T07:53:27.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.835+0000 7f42a0a00640 1 -- 172.21.15.3:0/1010216484 learned_addr learned my addr 172.21.15.3:0/1010216484 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 -- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94cba60 msgr2=0x564eb94c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f429be00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94cba60 0x564eb94c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 -- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 msgr2=0x564eb94ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 -- 172.21.15.3:0/1010216484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb94ea560 con 0x564eb94a1ea0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f429be00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a0a00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f42900098f0 tx=0x7f4290004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aa0b82510bd698 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f4281600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429005a060 con 0x564eb94a1ea0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4290047040 con 0x564eb94a1ea0 2024-09-10T07:53:27.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.836+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4294072040 con 0x564eb94a1ea0 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f4290060050 con 0x564eb94a1ea0 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 10 client.8250 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x564eb94ea880 con 0x564eb94a1ea0 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294072040 con 0x564eb94a1ea0 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7f4290063020 con 0x564eb94a1ea0 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 10 client.8250 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 10 client.8250 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 20 client.8250 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c8290 0x564eb949fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 4 client.8250 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "status"}) -- 0x564eb940ce90 con 0x564eb94c8290 2024-09-10T07:53:27.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.837+0000 7f429be00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c8290 0x564eb949fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f429be00640 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c8290 0x564eb949fc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db5a523c20342c7b server_cookie=25b23bcd1a20e0e in_seq=0 out_seq=0 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f4281600640 10 client.8250 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f4281600640 1 -- 172.21.15.3:0/1010216484 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x564eb940ce90 con 0x564eb94c8290 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f4281600640 10 client.8250 handle_command_reply: tid=0 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 client.8250 shutdown 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 msgr2=0x564eb9564120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f42900098f0 tx=0x7f4290004730 comp rx=0 tx=0).stop 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 shutdown_connections 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x564eb94c8290 0x564eb949fc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x564eb94cba60 0x564eb94c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564eb94a3630 0x564eb94ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 --2- 172.21.15.3:0/1010216484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eb94a1ea0 0x564eb9564120 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f4281600640 0 client.8250 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.838+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 >> 172.21.15.3:0/1010216484 conn(0x564eb949f660 msgr2=0x564eb94e8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 shutdown_connections 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a61c9740 1 -- 172.21.15.3:0/1010216484 wait complete. 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a61c9740 20 client.8250 trim_cache size 0 max 16384 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-10T07:53:27.843 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6875, 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 3.1804636980000001, 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 108, 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 102, 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.305428064000001 2024-09-10T07:53:27.844 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 msgr2=0x7f427c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 0x7f427c06e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4290034000 tx=0x7f429002e000 comp rx=0 tx=0).stop 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 msgr2=0x7f429c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.839+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f429400c6e0 tx=0x7f429400cba0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 shutdown_connections 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f429c15e820 0x7f429c0b3270 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f429c0d2510 0x7f429c1f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 --2- 172.21.15.3:0/410406010 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f429c0d1580 0x7f429c0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:27.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 >> 172.21.15.3:0/410406010 conn(0x7f429c072560 msgr2=0x7f429c0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:27.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 shutdown_connections 2024-09-10T07:53:27.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:27.840+0000 7f42a1e00640 1 -- 172.21.15.3:0/410406010 wait complete. 2024-09-10T07:53:27.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T07:53:28.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.014+0000 7fec9fe00640 1 Processor -- start 2024-09-10T07:53:28.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.014+0000 7fec9fe00640 1 -- start start 2024-09-10T07:53:28.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.014+0000 7fec9fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 0x7feca0073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feca00d4050 0x7feca0073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca00cfdc0 con 0x7feca015e860 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9fe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7feca00d3780 con 0x7feca00d4050 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feca00cee40 con 0x7feca00d3c10 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feca015e860 0x7feca00cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53176/0 (socket says 172.21.15.3:53176) 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9ea00640 1 -- 172.21.15.3:0/1631247862 learned_addr learned my addr 172.21.15.3:0/1631247862 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec97e00640 1 --2- 172.21.15.3:0/1631247862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 0x7feca0073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9f400640 1 --2- 172.21.15.3:0/1631247862 >> v2:172.21.15.70:3300/0 conn(0x7feca00d4050 0x7feca0073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093177119 0 0) 0x7feca00cfdc0 con 0x7feca015e860 2024-09-10T07:53:28.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec84003520 con 0x7feca015e860 2024-09-10T07:53:28.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec97e00640 1 -- 172.21.15.3:0/1631247862 >> v2:172.21.15.70:3300/0 conn(0x7feca00d4050 msgr2=0x7feca0073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec97e00640 1 --2- 172.21.15.3:0/1631247862 >> v2:172.21.15.70:3300/0 conn(0x7feca00d4050 0x7feca0073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.016+0000 7fec97e00640 1 -- 172.21.15.3:0/1631247862 >> v1:172.21.15.3:6789/0 conn(0x7feca015e860 legacy=0x7feca00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.015+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1494477044 0 0) 0x7fec84003520 con 0x7feca015e860 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.016+0000 7fec97e00640 1 -- 172.21.15.3:0/1631247862 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca00cf120 con 0x7feca00d3c10 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec97e00640 1 --2- 172.21.15.3:0/1631247862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 0x7feca0073150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec880097a0 tx=0x7fec8802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4016806e8a3af3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88037070 con 0x7feca00d3c10 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8802aba0 con 0x7feca00d3c10 2024-09-10T07:53:28.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec9e000640 1 -- 172.21.15.3:0/1631247862 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90001670 con 0x7feca00d3c10 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec9fe00640 1 -- 172.21.15.3:0/1631247862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 msgr2=0x7feca0073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.017+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1631247862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 0x7feca0073150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec880097a0 tx=0x7fec8802a110 comp rx=0 tx=0).stop 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 -- 172.21.15.3:0/1631247862 shutdown_connections 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1631247862 >> v2:172.21.15.70:3300/0 conn(0x7feca00d4050 0x7feca0073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1631247862 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d3c10 0x7feca0073150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 -- 172.21.15.3:0/1631247862 >> 172.21.15.3:0/1631247862 conn(0x7feca00d4650 msgr2=0x7feca00d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 -- 172.21.15.3:0/1631247862 shutdown_connections 2024-09-10T07:53:28.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.018+0000 7fec9fe00640 1 -- 172.21.15.3:0/1631247862 wait complete. 2024-09-10T07:53:28.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.019+0000 7fec9fe00640 1 Processor -- start 2024-09-10T07:53:28.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.019+0000 7fec9fe00640 1 -- start start 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d4050 0x7feca00ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 0x7feca00abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feca00d2460 con 0x7feca00d3c10 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feca00cf400 con 0x7feca015e860 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feca00b7880 con 0x7feca00d4050 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60508/0 (socket says 172.21.15.3:60508) 2024-09-10T07:53:28.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.020+0000 7fec9ea00640 1 -- 172.21.15.3:0/1745554535 learned_addr learned my addr 172.21.15.3:0/1745554535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 -- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d4050 msgr2=0x7feca00ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d4050 0x7feca00ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9f400640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 0x7feca00abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 -- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 msgr2=0x7feca00abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 0x7feca00abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 -- 172.21.15.3:0/1745554535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca00cd680 con 0x7feca00d3c10 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9f400640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 0x7feca00abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9ea00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c730 tx=0x7fec9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec96a00640 1 -- 172.21.15.3:0/1745554535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90017070 con 0x7feca00d3c10 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec96a00640 1 -- 172.21.15.3:0/1745554535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90005ad0 con 0x7feca00d3c10 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca00cd960 con 0x7feca00d3c10 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.021+0000 7fec96a00640 1 -- 172.21.15.3:0/1745554535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90007070 con 0x7feca00d3c10 2024-09-10T07:53:28.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.022+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca00cdf20 con 0x7feca00d3c10 2024-09-10T07:53:28.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.024+0000 7fec96a00640 1 -- 172.21.15.3:0/1745554535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec9001a070 con 0x7feca00d3c10 2024-09-10T07:53:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7fec96a00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7fec96a00640 1 -- 172.21.15.3:0/1745554535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fec9008a2b0 con 0x7feca00d3c10 2024-09-10T07:53:28.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7fec97e00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7feca8a0b740 1 Processor -- start 2024-09-10T07:53:28.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7feca8a0b740 1 -- start start 2024-09-10T07:53:28.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.025+0000 7fec97e00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 0x7fec7c06e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fec88034000 tx=0x7fec8802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c5ea0 0x55600060b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005c0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5560005243b0 con 0x5560005c3660 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556000529410 con 0x5560005c5ea0 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5560004d1620 con 0x5560005c7630 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60524/0 (socket says 172.21.15.3:60524) 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7fec97e00640 1 -- 172.21.15.3:0/757244424 learned_addr learned my addr 172.21.15.3:0/757244424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.026+0000 7fec9ea00640 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c5ea0 0x55600060b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec9f400640 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 -- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 msgr2=0x5560005c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005c0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 -- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c5ea0 msgr2=0x55600060b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c5ea0 0x55600060b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 -- 172.21.15.3:0/757244424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600023c2d0 con 0x5560005c3660 2024-09-10T07:53:28.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec97e00640 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fec8802ac70 tx=0x7fec88008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f6e39e7be3f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec8f400640 1 -- 172.21.15.3:0/757244424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8804c070 con 0x5560005c3660 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec8f400640 1 -- 172.21.15.3:0/757244424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88032b40 con 0x5560005c3660 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.027+0000 7fec8f400640 1 -- 172.21.15.3:0/757244424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88037040 con 0x5560005c3660 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 -- 172.21.15.3:0/757244424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 msgr2=0x5560005c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fec8802ac70 tx=0x7fec88008030 comp rx=0 tx=0).stop 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 -- 172.21.15.3:0/757244424 shutdown_connections 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c3660 0x5560005c6280 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 --2- 172.21.15.3:0/757244424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c5ea0 0x55600060b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 -- 172.21.15.3:0/757244424 >> 172.21.15.3:0/757244424 conn(0x5560005c28f0 msgr2=0x55600060ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 -- 172.21.15.3:0/757244424 shutdown_connections 2024-09-10T07:53:28.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.028+0000 7feca8a0b740 1 -- 172.21.15.3:0/757244424 wait complete. 2024-09-10T07:53:28.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.029+0000 7feca8a0b740 1 -- start start 2024-09-10T07:53:28.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.029+0000 7feca8a0b740 1 Processor -- start 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x55600061b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x5560005eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5560004e4780 con 0x5560005c5ea0 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5560005253d0 con 0x5560005c45f0 2024-09-10T07:53:28.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5560005c1d90 con 0x5560005c7630 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7feca8a0b740 10 client.0 _ll_register_callbacks cb 0x5560005225d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60538/0 (socket says 172.21.15.3:60538) 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.030+0000 7fec9ea00640 1 -- 172.21.15.3:0/2526438187 learned_addr learned my addr 172.21.15.3:0/2526438187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9f400640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x5560005eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec97e00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x55600061b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 -- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 msgr2=0x55600061b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x55600061b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 -- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 msgr2=0x5560005eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x5560005eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 -- 172.21.15.3:0/2526438187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560003066b0 con 0x5560005c5ea0 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9f400640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x5560005eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:28.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec97e00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x55600061b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec9ea00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fec900549d0 tx=0x7fec900611e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d61e364d5a92c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec83e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90088b60 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fec90088d00 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec88032b40 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.031+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fec90088170 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 10 client.8262 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x5560005e6b40 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88032b40 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7fec900579c0 con 0x5560005c5ea0 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 10 client.8262 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 10 client.8262 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 20 client.8262 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:28.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5560005e4290 0x55600061e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 4 client.8262 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:28.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556000536b40 con 0x5560005e4290 2024-09-10T07:53:28.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.032+0000 7fec97e00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5560005e4290 0x55600061e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.033+0000 7fec97e00640 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5560005e4290 0x55600061e900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9475e2813cfe21c3 server_cookie=9a9008b56219cf0d in_seq=0 out_seq=0 2024-09-10T07:53:28.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.033+0000 7fec83e00640 10 client.8262 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:28.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.033+0000 7fec83e00640 1 -- 172.21.15.3:0/2526438187 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556000536b40 con 0x5560005e4290 2024-09-10T07:53:28.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.033+0000 7fec83e00640 10 client.8262 handle_command_reply: tid=0 2024-09-10T07:53:28.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 client.8262 shutdown 2024-09-10T07:53:28.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 msgr2=0x5560005e5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fec900549d0 tx=0x7fec900611e0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 shutdown_connections 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x5560005eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5560005e4290 0x55600061e900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x55600061b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 --2- 172.21.15.3:0/2526438187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x5560005e5d50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 >> 172.21.15.3:0/2526438187 conn(0x5560005c3660 msgr2=0x556000608b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7fec83e00640 0 client.8262 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 shutdown_connections 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 1 -- 172.21.15.3:0/2526438187 wait complete. 2024-09-10T07:53:28.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.034+0000 7feca8a0b740 20 client.8262 trim_cache size 0 max 16384 2024-09-10T07:53:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.053+0000 7feca8a0b740 1 Processor -- start 2024-09-10T07:53:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.053+0000 7feca8a0b740 1 -- start start 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c45f0 0x5560005c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 0x5560006090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feca01ffab0 con 0x5560005c5ea0 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feca0200180 con 0x5560005c7630 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feca01f8270 con 0x5560005c45f0 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60542/0 (socket says 172.21.15.3:60542) 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 -- 172.21.15.3:0/4208833390 learned_addr learned my addr 172.21.15.3:0/4208833390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 -- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c45f0 msgr2=0x5560005c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c45f0 0x5560005c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 -- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 msgr2=0x5560006090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec9f400640 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 0x5560006090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 0x5560006090d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec97e00640 1 -- 172.21.15.3:0/4208833390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560005eddb0 con 0x5560005c5ea0 2024-09-10T07:53:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.054+0000 7fec9f400640 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 0x5560006090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.055+0000 7fec97e00640 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fec88035040 tx=0x7fec8803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f70a9d2bd96f61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.055+0000 7fec8ea00640 1 -- 172.21.15.3:0/4208833390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88056030 con 0x5560005c5ea0 2024-09-10T07:53:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.055+0000 7fec8ea00640 1 -- 172.21.15.3:0/4208833390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88032ae0 con 0x5560005c5ea0 2024-09-10T07:53:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.055+0000 7fec8ea00640 1 -- 172.21.15.3:0/4208833390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8804c040 con 0x5560005c5ea0 2024-09-10T07:53:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- 172.21.15.3:0/4208833390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 msgr2=0x556000608b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fec88035040 tx=0x7fec8803d000 comp rx=0 tx=0).stop 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- 172.21.15.3:0/4208833390 shutdown_connections 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c7630 0x5560006090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x556000608b90 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 --2- 172.21.15.3:0/4208833390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c45f0 0x5560005c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- 172.21.15.3:0/4208833390 >> 172.21.15.3:0/4208833390 conn(0x5560005c3660 msgr2=0x5560005c2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- 172.21.15.3:0/4208833390 shutdown_connections 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- 172.21.15.3:0/4208833390 wait complete. 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.056+0000 7feca8a0b740 1 -- start start 2024-09-10T07:53:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.057+0000 7feca8a0b740 1 Processor -- start 2024-09-10T07:53:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x556000688e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5560005e4110 con 0x5560005c5ea0 2024-09-10T07:53:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5560005e71f0 con 0x5560005c45f0 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5560005e4760 con 0x5560005c7630 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7feca8a0b740 10 client.0 _ll_register_callbacks cb 0x7fec90058570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60558/0 (socket says 172.21.15.3:60558) 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 -- 172.21.15.3:0/3112451072 learned_addr learned my addr 172.21.15.3:0/3112451072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec9ea00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x556000688e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec9f400640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 -- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 msgr2=0x5560005e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 -- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 msgr2=0x556000688e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x556000688e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec97e00640 1 -- 172.21.15.3:0/3112451072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560006244a0 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.058+0000 7fec9ea00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x556000688e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec97e00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fec88000c00 tx=0x7fec88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c668635c45082899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88067060 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec88032ae0 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fec88004340 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fec8806d050 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 10 client.8268 fetch_fsmap learned FSMap version 108 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x55600068a080 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88032ae0 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 108) ==== 3542+0+0 (secure 0 0 0) 0x7fec88070020 con 0x5560005c5ea0 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 10 client.8268 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 10 client.8268 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T07:53:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 20 client.8268 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.059+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556000608cd0 0x5560006090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7feca8a0b740 4 client.8268 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "status"}) -- 0x556000530e90 con 0x556000608cd0 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7fec9ea00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556000608cd0 0x5560006090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7fec9ea00640 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556000608cd0 0x5560006090b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47d3c0ca1f17d9f server_cookie=5afc81fe0383f159 in_seq=0 out_seq=0 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7fec83e00640 10 client.8268 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7fec83e00640 1 -- 172.21.15.3:0/3112451072 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x556000530e90 con 0x556000608cd0 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7fec83e00640 10 client.8268 handle_command_reply: tid=0 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.060+0000 7feca8a0b740 1 client.8268 shutdown 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 msgr2=0x55600062d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fec88000c00 tx=0x7fec88004730 comp rx=0 tx=0).stop 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 shutdown_connections 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556000608cd0 0x5560006090b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5560005c7630 0x5560005e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5560005c5ea0 0x55600062d710 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7fec83e00640 0 client.8268 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 --2- 172.21.15.3:0/3112451072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5560005c45f0 0x556000688e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 >> 172.21.15.3:0/3112451072 conn(0x5560005c3660 msgr2=0x5560005c2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 shutdown_connections 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 1 -- 172.21.15.3:0/3112451072 wait complete. 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.061+0000 7feca8a0b740 20 client.8268 trim_cache size 0 max 16384 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "46339922-75e0-4945-8bb6-1acaef196937", 2024-09-10T07:53:28.065 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7543, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 16.817702672999999, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 108, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 102, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.527072665999999 2024-09-10T07:53:28.066 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:28.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 msgr2=0x7fec7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 0x7fec7c06e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fec88034000 tx=0x7fec8802c000 comp rx=0 tx=0).stop 2024-09-10T07:53:28.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 msgr2=0x7feca01f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c730 tx=0x7fec9000cbf0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 shutdown_connections 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feca015e860 0x7feca00abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feca00d4050 0x7feca00ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 --2- 172.21.15.3:0/1745554535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca00d3c10 0x7feca01f15b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 >> 172.21.15.3:0/1745554535 conn(0x7feca00d4650 msgr2=0x7feca00cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 shutdown_connections 2024-09-10T07:53:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.062+0000 7fec9fe00640 1 -- 172.21.15.3:0/1745554535 wait complete. 2024-09-10T07:53:28.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:28.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.235+0000 7f5999a00640 1 Processor -- start 2024-09-10T07:53:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.235+0000 7f5999a00640 1 -- start start 2024-09-10T07:53:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5994073150 0x7f5994073600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940cee40 con 0x7f59940cfbf0 2024-09-10T07:53:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f59940d3780 con 0x7f5994073150 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f59940cf120 con 0x7f599415e860 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f598b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59940cfbf0 0x7f59940d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53188/0 (socket says 172.21.15.3:53188) 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f598b400640 1 -- 172.21.15.3:0/1586494631 learned_addr learned my addr 172.21.15.3:0/1586494631 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5994073150 0x7f5994073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5993e00640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999000640 1 -- 172.21.15.3:0/1586494631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 msgr2=0x7f59940cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999000640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999000640 1 -- 172.21.15.3:0/1586494631 >> v1:172.21.15.3:6789/0 conn(0x7f59940cfbf0 legacy=0x7f59940d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5999000640 1 -- 172.21.15.3:0/1586494631 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940d3410 con 0x7f5994073150 2024-09-10T07:53:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.236+0000 7f5993400640 1 -- 172.21.15.3:0/1586494631 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141283969 0 0) 0x7f59940cee40 con 0x7f59940cfbf0 2024-09-10T07:53:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.237+0000 7f5999000640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5994073150 0x7f5994073600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f598c0097a0 tx=0x7f598c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc41698723ef1c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.237+0000 7f5993400640 1 -- 172.21.15.3:0/1586494631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c037070 con 0x7f5994073150 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.237+0000 7f5993400640 1 -- 172.21.15.3:0/1586494631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598c032440 con 0x7f5994073150 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.237+0000 7f5993400640 1 -- 172.21.15.3:0/1586494631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c032810 con 0x7f5994073150 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 -- 172.21.15.3:0/1586494631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5994073150 msgr2=0x7f5994073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5994073150 0x7f5994073600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f598c0097a0 tx=0x7f598c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 -- 172.21.15.3:0/1586494631 shutdown_connections 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5994073150 0x7f5994073600 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 --2- 172.21.15.3:0/1586494631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 -- 172.21.15.3:0/1586494631 >> 172.21.15.3:0/1586494631 conn(0x7f59940d4650 msgr2=0x7f59940d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 -- 172.21.15.3:0/1586494631 shutdown_connections 2024-09-10T07:53:28.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.238+0000 7f5999a00640 1 -- 172.21.15.3:0/1586494631 wait complete. 2024-09-10T07:53:28.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.240+0000 7f5999a00640 1 Processor -- start 2024-09-10T07:53:28.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.240+0000 7f5999a00640 1 -- start start 2024-09-10T07:53:28.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5994073150 0x7f59940acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59940d2460 con 0x7f59940cfbf0 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f59940cf400 con 0x7f5994073150 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f59940b7880 con 0x7f599415e860 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60576/0 (socket says 172.21.15.3:60576) 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 -- 172.21.15.3:0/513185982 learned_addr learned my addr 172.21.15.3:0/513185982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5993e00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5994073150 0x7f59940acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f5999000640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 -- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 msgr2=0x7f59940aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 -- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5994073150 msgr2=0x7f59940acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.241+0000 7f598b400640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5994073150 0x7f59940acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f598b400640 1 -- 172.21.15.3:0/513185982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940ac020 con 0x7f59940cfbf0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f598b400640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c480 tx=0x7f597c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c018070 con 0x7f59940cfbf0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c0074e0 con 0x7f59940cfbf0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c007690 con 0x7f59940cfbf0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59940ac300 con 0x7f59940cfbf0 2024-09-10T07:53:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.242+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59940ac8c0 con 0x7f59940cfbf0 2024-09-10T07:53:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.243+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59940cf6b0 con 0x7f59940cfbf0 2024-09-10T07:53:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.244+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f597c0049e0 con 0x7f59940cfbf0 2024-09-10T07:53:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.245+0000 7f5990c00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.245+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f597c08a0f0 con 0x7f59940cfbf0 2024-09-10T07:53:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.245+0000 7f5993e00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.246+0000 7f5993e00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5984011340 tx=0x7f598400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.250+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f597c058450 con 0x7f59940cfbf0 2024-09-10T07:53:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.473+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59940adae0 con 0x7f59940cfbf0 2024-09-10T07:53:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.473+0000 7f5990c00640 1 -- 172.21.15.3:0/513185982 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5262 (secure 0 0 0) 0x7f597c05c750 con 0x7f59940cfbf0 2024-09-10T07:53:28.473 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:28.473 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":108,"btime":"2024-09-10T07:53:26:667916+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:26.667912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 108 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 msgr2=0x7f597006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 0x7f597006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5984011340 tx=0x7f598400b040 comp rx=0 tx=0).stop 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 msgr2=0x7f59940ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c480 tx=0x7f597c00c940 comp rx=0 tx=0).stop 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 shutdown_connections 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f597006be30 0x7f597006e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f599415e860 0x7f59940aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59940cfbf0 0x7f59940ad480 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 --2- 172.21.15.3:0/513185982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5994073150 0x7f59940acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 >> 172.21.15.3:0/513185982 conn(0x7f59940d4650 msgr2=0x7f59940b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 shutdown_connections 2024-09-10T07:53:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:28.476+0000 7f5999a00640 1 -- 172.21.15.3:0/513185982 wait complete. 2024-09-10T07:53:28.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:28.493 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:53:28.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:28.539 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-10T07:53:28.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:28.581 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-10T07:53:28.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:28.658 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-10T07:53:28.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:28.734 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-10T07:53:28.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:53:29.024 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:53:29.025 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:29.025 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:53:31.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:53:31.212 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:53:31.213 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:31.213 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:53:33.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:53:33.401 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:53:33.402 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:33.402 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:53:35.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:53:35.588 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-10T07:53:35.588 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:35.589 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:53:37.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-10T07:53:37.777 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-10T07:53:37.778 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:53:37.778 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-10T07:53:37.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:53:37.952 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:53:37.953 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:37.953 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:53:39.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:53:40.141 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:53:40.142 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:40.142 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:53:42.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-10T07:53:42.330 INFO:teuthology.orchestra.run.smithi003.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-10T07:53:42.331 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-10T07:53:42.331 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:53:42.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-10T07:53:42.513 INFO:teuthology.orchestra.run.smithi070.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-10T07:53:42.514 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-10T07:53:42.514 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-10T07:53:42.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2024-09-10T07:53:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.673+0000 7fb289e00640 1 Processor -- start 2024-09-10T07:53:42.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.673+0000 7fb289e00640 1 -- start start 2024-09-10T07:53:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.674+0000 7fb289e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb28406c230 0x7fb28415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.674+0000 7fb289e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28415f0f0 0x7fb2841608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.674+0000 7fb289e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28415f560 con 0x7fb284148290 2024-09-10T07:53:42.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.674+0000 7fb289e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb2840c9850 con 0x7fb28406c230 2024-09-10T07:53:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.674+0000 7fb289e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb28415f7e0 con 0x7fb28415f0f0 2024-09-10T07:53:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb288a00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb284148290 0x7fb28406bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:41372/0 (socket says 172.21.15.3:41372) 2024-09-10T07:53:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb288a00640 1 -- 172.21.15.3:0/372460740 learned_addr learned my addr 172.21.15.3:0/372460740 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb28406c230 0x7fb28415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb289400640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28415f0f0 0x7fb2841608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283e00640 1 -- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28415f0f0 msgr2=0x7fb2841608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283e00640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28415f0f0 0x7fb2841608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283e00640 1 -- 172.21.15.3:0/372460740 >> v1:172.21.15.70:6789/0 conn(0x7fb284148290 legacy=0x7fb28406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283400640 1 -- 172.21.15.3:0/372460740 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953325758 0 0) 0x7fb278001110 con 0x7fb284148290 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.675+0000 7fb283e00640 1 -- 172.21.15.3:0/372460740 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28415fe10 con 0x7fb28406c230 2024-09-10T07:53:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.676+0000 7fb283e00640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb28406c230 0x7fb28415ebb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb2700097a0 tx=0x7fb27002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7053d1d35c9ca3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb283400640 1 -- 172.21.15.3:0/372460740 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270037070 con 0x7fb28406c230 2024-09-10T07:53:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb283400640 1 -- 172.21.15.3:0/372460740 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27002ae40 con 0x7fb28406c230 2024-09-10T07:53:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb283400640 1 -- 172.21.15.3:0/372460740 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270032660 con 0x7fb28406c230 2024-09-10T07:53:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 -- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb28406c230 msgr2=0x7fb28415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb28406c230 0x7fb28415ebb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb2700097a0 tx=0x7fb27002a0f0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 -- 172.21.15.3:0/372460740 shutdown_connections 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28415f0f0 0x7fb2841608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 --2- 172.21.15.3:0/372460740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb28406c230 0x7fb28415ebb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.677+0000 7fb289e00640 1 -- 172.21.15.3:0/372460740 >> 172.21.15.3:0/372460740 conn(0x7fb2840d4580 msgr2=0x7fb2840d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.678+0000 7fb289e00640 1 -- 172.21.15.3:0/372460740 shutdown_connections 2024-09-10T07:53:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.678+0000 7fb289e00640 1 -- 172.21.15.3:0/372460740 wait complete. 2024-09-10T07:53:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.679+0000 7fb289e00640 1 Processor -- start 2024-09-10T07:53:42.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.679+0000 7fb289e00640 1 -- start start 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 0x7fb2840b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb284148290 0x7fb2840b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb28415fac0 con 0x7fb28415f0f0 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841603d0 con 0x7fb284148290 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb284160e30 con 0x7fb28406c230 2024-09-10T07:53:42.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb289400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb288a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 0x7fb2840b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.680+0000 7fb283e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb284148290 0x7fb2840b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44292/0 (socket says 172.21.15.3:44292) 2024-09-10T07:53:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 -- 172.21.15.3:0/3596334802 learned_addr learned my addr 172.21.15.3:0/3596334802 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 -- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 msgr2=0x7fb2840b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 0x7fb2840b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 -- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb284148290 msgr2=0x7fb2840b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb284148290 0x7fb2840b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 -- 172.21.15.3:0/3596334802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2840b7fc0 con 0x7fb28415f0f0 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb288a00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 0x7fb2840b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb289400640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009d30 tx=0x7fb27c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.681+0000 7fb280c00640 1 -- 172.21.15.3:0/3596334802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c018020 con 0x7fb28415f0f0 2024-09-10T07:53:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.682+0000 7fb280c00640 1 -- 172.21.15.3:0/3596334802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c014480 con 0x7fb28415f0f0 2024-09-10T07:53:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.682+0000 7fb280c00640 1 -- 172.21.15.3:0/3596334802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c014870 con 0x7fb28415f0f0 2024-09-10T07:53:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.682+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb284159d40 con 0x7fb28415f0f0 2024-09-10T07:53:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.682+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28415a300 con 0x7fb28415f0f0 2024-09-10T07:53:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.684+0000 7fb280c00640 1 -- 172.21.15.3:0/3596334802 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb27c005d10 con 0x7fb28415f0f0 2024-09-10T07:53:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.685+0000 7fb280c00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 0x7fb26406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.685+0000 7fb280c00640 1 -- 172.21.15.3:0/3596334802 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb27c08b820 con 0x7fb28415f0f0 2024-09-10T07:53:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.685+0000 7fb288a00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 0x7fb26406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.686+0000 7fb28e2b2740 1 Processor -- start 2024-09-10T07:53:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.686+0000 7fb288a00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 0x7fb26406e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb278000f60 tx=0x7fb278007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.686+0000 7fb28e2b2740 1 -- start start 2024-09-10T07:53:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.686+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.686+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f519e0 0x55c1d4f50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c1d4eb9660 con 0x55c1d4f9bea0 2024-09-10T07:53:42.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c1d4e61680 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c1d4eb5620 con 0x55c1d4f519e0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb289400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46034/0 (socket says 172.21.15.3:46034) 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 -- 172.21.15.3:0/1504683281 learned_addr learned my addr 172.21.15.3:0/1504683281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb283e00640 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f519e0 0x55c1d4f50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 -- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f519e0 msgr2=0x55c1d4f50240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f519e0 0x55c1d4f50240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 -- 172.21.15.3:0/1504683281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 msgr2=0x55c1d4f9c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f9c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 -- 172.21.15.3:0/1504683281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d4c967d0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.687+0000 7fb288a00640 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb278004e00 tx=0x7fb278013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc17d9ac9356066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb275200640 1 -- 172.21.15.3:0/1504683281 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278026070 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb275200640 1 -- 172.21.15.3:0/1504683281 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278012350 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb275200640 1 -- 172.21.15.3:0/1504683281 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278012720 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 -- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 msgr2=0x55c1d4f6b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb278004e00 tx=0x7fb278013000 comp rx=0 tx=0).stop 2024-09-10T07:53:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 -- 172.21.15.3:0/1504683281 shutdown_connections 2024-09-10T07:53:42.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f9c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f519e0 0x55c1d4f50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/1504683281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f6b8d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.688+0000 7fb28e2b2740 1 -- 172.21.15.3:0/1504683281 >> 172.21.15.3:0/1504683281 conn(0x55c1d4f52410 msgr2=0x55c1d4f9aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.689+0000 7fb28e2b2740 1 -- 172.21.15.3:0/1504683281 shutdown_connections 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.689+0000 7fb28e2b2740 1 -- 172.21.15.3:0/1504683281 wait complete. 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.689+0000 7fb28e2b2740 1 -- start start 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.689+0000 7fb28e2b2740 1 Processor -- start 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f95ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4f7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c1d4f50940 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c1d4e424c0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c1d4f50c20 con 0x55c1d4f80e50 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.690+0000 7fb28e2b2740 10 client.0 _ll_register_callbacks cb 0x55c1d4eb1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44304/0 (socket says 172.21.15.3:44304) 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 -- 172.21.15.3:0/3880803969 learned_addr learned my addr 172.21.15.3:0/3880803969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb283e00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f95ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 -- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 msgr2=0x55c1d4f7fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb289400640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4f7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4f7fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 -- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 msgr2=0x55c1d4f95ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f95ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 -- 172.21.15.3:0/3880803969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d4bcc3f0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb283e00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f95ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:42.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb289400640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4f7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb288a00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb278004d70 tx=0x7fb278000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de808eb01a5949 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb269600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278026070 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb278012df0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.691+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb278012350 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fb278035050 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 10 client.8286 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55c1d4b8c0f0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278012350 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7fb278038020 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 10 client.8286 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 10 client.8286 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:42.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 20 client.8286 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d4f93250 0x55c1d4f93680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 4 client.8286 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d4ec6d90 con 0x55c1d4f93250 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.692+0000 7fb283e00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d4f93250 0x55c1d4f93680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.693+0000 7fb283e00640 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d4f93250 0x55c1d4f93680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46cbeeeed269669 server_cookie=7a22235c45f853c3 in_seq=0 out_seq=0 2024-09-10T07:53:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.693+0000 7fb269600640 10 client.8286 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb269600640 1 -- 172.21.15.3:0/3880803969 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1d4ec6d90 con 0x55c1d4f93250 2024-09-10T07:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb269600640 10 client.8286 handle_command_reply: tid=0 2024-09-10T07:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 client.8286 shutdown 2024-09-10T07:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 msgr2=0x55c1d4fab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb278004d70 tx=0x7fb278000ed0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 shutdown_connections 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d4f93250 0x55c1d4f93680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4f7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f95ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3880803969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fab730 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb269600640 0 client.8286 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 >> 172.21.15.3:0/3880803969 conn(0x55c1d4f519e0 msgr2=0x55c1d4f4f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 shutdown_connections 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3880803969 wait complete. 2024-09-10T07:53:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.694+0000 7fb28e2b2740 20 client.8286 trim_cache size 0 max 16384 2024-09-10T07:53:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.710+0000 7fb28e2b2740 1 Processor -- start 2024-09-10T07:53:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.710+0000 7fb28e2b2740 1 -- start start 2024-09-10T07:53:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.710+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f80830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.710+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4fbed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.710+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb284200bb0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841ffd80 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb284200480 con 0x55c1d4f80e50 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44310/0 (socket says 172.21.15.3:44310) 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 -- 172.21.15.3:0/2898216993 learned_addr learned my addr 172.21.15.3:0/2898216993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb288a00640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f80830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb283e00640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4fbed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 -- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 msgr2=0x55c1d4fbed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4fbed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 -- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 msgr2=0x55c1d4f80830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f80830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 -- 172.21.15.3:0/2898216993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d4f96af0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb288a00640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f80830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.711+0000 7fb289400640 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb27c089c20 tx=0x7fb27c060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b415b04b28f81b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb26be00640 1 -- 172.21.15.3:0/2898216993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c088070 con 0x55c1d4f9bea0 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb26be00640 1 -- 172.21.15.3:0/2898216993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c057de0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb26be00640 1 -- 172.21.15.3:0/2898216993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0600b0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 -- 172.21.15.3:0/2898216993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 msgr2=0x55c1d4f91430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb27c089c20 tx=0x7fb27c060040 comp rx=0 tx=0).stop 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 -- 172.21.15.3:0/2898216993 shutdown_connections 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f9bea0 0x55c1d4f91430 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f80e50 0x55c1d4fbed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/2898216993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4f80830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 -- 172.21.15.3:0/2898216993 >> 172.21.15.3:0/2898216993 conn(0x55c1d4f519e0 msgr2=0x55c1d4f6b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 -- 172.21.15.3:0/2898216993 shutdown_connections 2024-09-10T07:53:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.712+0000 7fb28e2b2740 1 -- 172.21.15.3:0/2898216993 wait complete. 2024-09-10T07:53:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.713+0000 7fb28e2b2740 1 -- start start 2024-09-10T07:53:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.713+0000 7fb28e2b2740 1 Processor -- start 2024-09-10T07:53:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f80e50 0x55c1d4f9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f9bea0 0x55c1d5015fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb28415b130 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c1d501a8c0 con 0x55c1d4f80e50 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c1d4f55af0 con 0x55c1d4f9bea0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb28e2b2740 10 client.0 _ll_register_callbacks cb 0x7fb27c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb288a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb288a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44322/0 (socket says 172.21.15.3:44322) 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb288a00640 1 -- 172.21.15.3:0/3168025477 learned_addr learned my addr 172.21.15.3:0/3168025477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.714+0000 7fb283e00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f80e50 0x55c1d4f9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb289400640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f9bea0 0x55c1d5015fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 -- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f9bea0 msgr2=0x55c1d5015fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f9bea0 0x55c1d5015fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 -- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f80e50 msgr2=0x55c1d4f9cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f80e50 0x55c1d4f9cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 -- 172.21.15.3:0/3168025477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d4f55dd0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb288a00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb278004fd0 tx=0x7fb278009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd4ebc10a33be44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb269600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278012350 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb278012df0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.715+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27c057de0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fb278049050 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 10 client.8292 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55c1d4f564d0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c057de0 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7fb27804c020 con 0x55c1d4f6b4f0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 10 client.8292 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 10 client.8292 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 20 client.8292 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d5019c90 0x55c1d501a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 4 client.8292 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1d4f82420 con 0x55c1d5019c90 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.716+0000 7fb283e00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d5019c90 0x55c1d501a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.717+0000 7fb283e00640 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d5019c90 0x55c1d501a090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b34288b8dd585a server_cookie=3be5abc8d74c43d7 in_seq=0 out_seq=0 2024-09-10T07:53:42.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.717+0000 7fb269600640 10 client.8292 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb269600640 1 -- 172.21.15.3:0/3168025477 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1d4f82420 con 0x55c1d5019c90 2024-09-10T07:53:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb269600640 10 client.8292 handle_command_reply: tid=0 2024-09-10T07:53:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 client.8292 shutdown 2024-09-10T07:53:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 msgr2=0x55c1d4fbd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb278004fd0 tx=0x7fb278009030 comp rx=0 tx=0).stop 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 shutdown_connections 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55c1d5019c90 0x55c1d501a090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c1d4f9bea0 0x55c1d5015fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c1d4f80e50 0x55c1d4f9cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 --2- 172.21.15.3:0/3168025477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c1d4f6b4f0 0x55c1d4fbd6a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb269600640 0 client.8292 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 >> 172.21.15.3:0/3168025477 conn(0x55c1d4f519e0 msgr2=0x55c1d4f80650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 shutdown_connections 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.728+0000 7fb28e2b2740 1 -- 172.21.15.3:0/3168025477 wait complete. 2024-09-10T07:53:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb28e2b2740 20 client.8292 trim_cache size 0 max 16384 2024-09-10T07:53:42.730 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:42.730 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:53:42.730 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:42.730 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:42.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 msgr2=0x7fb26406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 0x7fb26406e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb278000f60 tx=0x7fb278007b50 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 msgr2=0x7fb284144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009d30 tx=0x7fb27c00c490 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 shutdown_connections 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb26406be60 0x7fb26406e320 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb28415f0f0 0x7fb284144c80 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb284148290 0x7fb2840b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 --2- 172.21.15.3:0/3596334802 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb28406c230 0x7fb2840b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.729+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 >> 172.21.15.3:0/3596334802 conn(0x7fb2840d4580 msgr2=0x7fb284145960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.730+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 shutdown_connections 2024-09-10T07:53:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.730+0000 7fb289e00640 1 -- 172.21.15.3:0/3596334802 wait complete. 2024-09-10T07:53:42.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2024-09-10T07:53:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.903+0000 7f9ee2800640 1 Processor -- start 2024-09-10T07:53:42.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.903+0000 7f9ee2800640 1 -- start start 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.904+0000 7f9ee2800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9edc0b4c90 0x7f9edc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.904+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9edc071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.904+0000 7f9ee2800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9edc15eff0 con 0x7f9edc15b230 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.904+0000 7f9ee2800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9edc15fbe0 con 0x7f9edc0b4c90 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.904+0000 7f9ee2800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc15c8d0 con 0x7f9edc072c50 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9edb400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9edc15b230 0x7f9edc0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55664/0 (socket says 172.21.15.3:55664) 2024-09-10T07:53:42.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9edb400640 1 -- 172.21.15.3:0/350255287 learned_addr learned my addr 172.21.15.3:0/350255287 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9edbe00640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9edc071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9edaa00640 1 --2- 172.21.15.3:0/350255287 >> v2:172.21.15.70:3300/0 conn(0x7f9edc0b4c90 0x7f9edc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444166806 0 0) 0x7f9edc15eff0 con 0x7f9edc15b230 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ec0003520 con 0x7f9edc15b230 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2164828252 0 0) 0x7f9ec0003520 con 0x7f9edc15b230 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 msgr2=0x7f9edc071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9edc071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 >> v2:172.21.15.70:3300/0 conn(0x7f9edc0b4c90 msgr2=0x7f9edc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 --2- 172.21.15.3:0/350255287 >> v2:172.21.15.70:3300/0 conn(0x7f9edc0b4c90 0x7f9edc15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc15f2b0 con 0x7f9edc15b230 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f9ecc001670 con 0x7f9edc15b230 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 >> v1:172.21.15.3:6789/0 conn(0x7f9edc15b230 legacy=0x7f9edc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec00039b0 0x7f9ec0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9ec0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc15fbe0 con 0x7f9ec00039b0 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc15c8d0 con 0x7f9ec0004d40 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.905+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec000b9c0 con 0x7f9edc072c50 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec00039b0 0x7f9ec0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edbe00640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edaa00640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9ec0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 -- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 msgr2=0x7f9ec0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc072c50 0x7f9ec0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 -- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec0004d40 msgr2=0x7f9ec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.906+0000 7f9edb400640 1 -- 172.21.15.3:0/350255287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec0003520 con 0x7f9ec00039b0 2024-09-10T07:53:42.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.907+0000 7f9edb400640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec00039b0 0x7f9ec0003dd0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c190 tx=0x7f9ec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4769df1bd1ef232 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.907+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc007c80 con 0x7f9ec00039b0 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.907+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ecc005ad0 con 0x7f9ec00039b0 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.907+0000 7f9eda000640 1 -- 172.21.15.3:0/350255287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc007070 con 0x7f9ec00039b0 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 -- 172.21.15.3:0/350255287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec00039b0 msgr2=0x7f9ec0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 --2- 172.21.15.3:0/350255287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec00039b0 0x7f9ec0003dd0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c190 tx=0x7f9ec0005830 comp rx=0 tx=0).stop 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9edbe00640 1 -- 172.21.15.3:0/350255287 reap_dead start 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 -- 172.21.15.3:0/350255287 shutdown_connections 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 -- 172.21.15.3:0/350255287 >> 172.21.15.3:0/350255287 conn(0x7f9edc0b5cb0 msgr2=0x7f9edc06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 -- 172.21.15.3:0/350255287 shutdown_connections 2024-09-10T07:53:42.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.908+0000 7f9ee2800640 1 -- 172.21.15.3:0/350255287 wait complete. 2024-09-10T07:53:42.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.910+0000 7f9ee2800640 1 Processor -- start 2024-09-10T07:53:42.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.910+0000 7f9ee2800640 1 -- start start 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec00039b0 0x7f9edc0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc0aede0 0x7f9edc1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc15b610 con 0x7f9edc1f8200 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc15f870 con 0x7f9ec00039b0 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9ee2800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc15b790 con 0x7f9edc0aede0 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44338/0 (socket says 172.21.15.3:44338) 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9edbe00640 1 -- 172.21.15.3:0/3066811987 learned_addr learned my addr 172.21.15.3:0/3066811987 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9edb400640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec00039b0 0x7f9edc0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.911+0000 7f9edaa00640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc0aede0 0x7f9edc1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 -- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc0aede0 msgr2=0x7f9edc1f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc0aede0 0x7f9edc1f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 -- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec00039b0 msgr2=0x7f9edc0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec00039b0 0x7f9edc0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 -- 172.21.15.3:0/3066811987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc1f4890 con 0x7f9edc1f8200 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9edbe00640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06f460 tx=0x7f9ed400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9ed3400640 1 -- 172.21.15.3:0/3066811987 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4003e60 con 0x7f9edc1f8200 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9ed3400640 1 -- 172.21.15.3:0/3066811987 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed401c070 con 0x7f9edc1f8200 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9ed3400640 1 -- 172.21.15.3:0/3066811987 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400fb70 con 0x7f9edc1f8200 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc1f4b70 con 0x7f9edc1f8200 2024-09-10T07:53:42.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.912+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc1f5130 con 0x7f9edc1f8200 2024-09-10T07:53:42.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.915+0000 7f9ed3400640 1 -- 172.21.15.3:0/3066811987 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ed400fd10 con 0x7f9edc1f8200 2024-09-10T07:53:42.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.915+0000 7f9ed3400640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 0x7f9ebc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.915+0000 7f9ed3400640 1 -- 172.21.15.3:0/3066811987 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed408b700 con 0x7f9edc1f8200 2024-09-10T07:53:42.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.916+0000 7f9edb400640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 0x7f9ebc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.916+0000 7f9ee5636740 1 Processor -- start 2024-09-10T07:53:42.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.916+0000 7f9ee5636740 1 -- start start 2024-09-10T07:53:42.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.916+0000 7f9edb400640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 0x7f9ebc06e320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc032000 tx=0x7f9ecc034040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x559355946200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593558fb480 0x559355946740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559355863660 con 0x559355915420 2024-09-10T07:53:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55935580b680 con 0x559355915800 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55935585f620 con 0x5593558fb480 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9edb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9edb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44344/0 (socket says 172.21.15.3:44344) 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.917+0000 7f9edb400640 1 -- 172.21.15.3:0/2891879791 learned_addr learned my addr 172.21.15.3:0/2891879791 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 -- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593558fb480 msgr2=0x559355946740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593558fb480 0x559355946740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 -- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 msgr2=0x559355946200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x559355946200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 -- 172.21.15.3:0/2891879791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593556407d0 con 0x559355915420 2024-09-10T07:53:42.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9edb400640 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c280 tx=0x7f9ecc007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acc11c8cc18733f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9ec7e00640 1 -- 172.21.15.3:0/2891879791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc050070 con 0x559355915420 2024-09-10T07:53:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9ec7e00640 1 -- 172.21.15.3:0/2891879791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ecc038040 con 0x559355915420 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.918+0000 7f9ec7e00640 1 -- 172.21.15.3:0/2891879791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc037030 con 0x559355915420 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 -- 172.21.15.3:0/2891879791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 msgr2=0x559355945c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c280 tx=0x7f9ecc007000 comp rx=0 tx=0).stop 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 -- 172.21.15.3:0/2891879791 shutdown_connections 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5593558fb480 0x559355946740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x559355946200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 --2- 172.21.15.3:0/2891879791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355915420 0x559355945c40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 -- 172.21.15.3:0/2891879791 >> 172.21.15.3:0/2891879791 conn(0x5593558fc220 msgr2=0x559355944840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 -- 172.21.15.3:0/2891879791 shutdown_connections 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.919+0000 7f9ee5636740 1 -- 172.21.15.3:0/2891879791 wait complete. 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.920+0000 7f9ee5636740 1 -- start start 2024-09-10T07:53:42.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.920+0000 7f9ee5636740 1 Processor -- start 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915800 0x5593559584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5593558fa3e0 con 0x559355958a00 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5593557ec4c0 con 0x559355915420 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5593558fa6c0 con 0x559355915800 2024-09-10T07:53:42.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9ee5636740 10 client.0 _ll_register_callbacks cb 0x55935585be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edb400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915800 0x5593559584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edb400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46100/0 (socket says 172.21.15.3:46100) 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44348/0 (socket says 172.21.15.3:44348) 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.921+0000 7f9edb400640 1 -- 172.21.15.3:0/3638327836 learned_addr learned my addr 172.21.15.3:0/3638327836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 -- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915800 msgr2=0x5593559584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915800 0x5593559584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 -- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 msgr2=0x559355943520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 -- 172.21.15.3:0/3638327836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593555763f0 con 0x559355915420 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edbe00640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:42.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9edb400640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c2b0 tx=0x7f9ecc0019e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb92090a9312039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ec4c00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc037070 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mon.1 v2:172.21.15.70:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9ecc056670 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ecc038040 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f9ecc050040 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ee5636740 10 client.7756 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.922+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5593555360f0 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mon.1 v2:172.21.15.70:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ecc038040 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mon.1 v2:172.21.15.70:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f9ecc03d040 con 0x559355915420 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 10 client.7756 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 10 client.7756 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 20 client.7756 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:42.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593ee70 0x55935593f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 4 client.7756 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:42.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559355870d90 con 0x55935593ee70 2024-09-10T07:53:42.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.923+0000 7f9edaa00640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593ee70 0x55935593f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.924+0000 7f9edaa00640 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593ee70 0x55935593f280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=323f0f3c4b0713d4 server_cookie=fe173de7576abc2 in_seq=0 out_seq=0 2024-09-10T07:53:42.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.924+0000 7f9ec4c00640 10 client.7756 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.924+0000 7f9ec4c00640 1 -- 172.21.15.3:0/3638327836 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559355870d90 con 0x55935593ee70 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.924+0000 7f9ec4c00640 10 client.7756 handle_command_reply: tid=0 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.924+0000 7f9ee5636740 1 client.7756 shutdown 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 msgr2=0x559355914130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c2b0 tx=0x7f9ecc0019e0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 shutdown_connections 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593ee70 0x55935593f280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355943520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915800 0x5593559584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ec4c00640 0 client.7756 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 --2- 172.21.15.3:0/3638327836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915420 0x559355914130 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 >> 172.21.15.3:0/3638327836 conn(0x5593558fb480 msgr2=0x559355945c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 shutdown_connections 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 1 -- 172.21.15.3:0/3638327836 wait complete. 2024-09-10T07:53:42.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.925+0000 7f9ee5636740 20 client.7756 trim_cache size 0 max 16384 2024-09-10T07:53:42.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.940+0000 7f9ee5636740 1 Processor -- start 2024-09-10T07:53:42.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.940+0000 7f9ee5636740 1 -- start start 2024-09-10T07:53:42.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x55935593e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc201450 con 0x559355958a00 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc200620 con 0x559355915800 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.941+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc200d20 con 0x559355915420 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44352/0 (socket says 172.21.15.3:44352) 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 -- 172.21.15.3:0/990505535 learned_addr learned my addr 172.21.15.3:0/990505535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edb400640 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edaa00640 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x55935593e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 -- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 msgr2=0x5593559543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 -- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 msgr2=0x55935593e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x55935593e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 -- 172.21.15.3:0/990505535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559355899b80 con 0x559355958a00 2024-09-10T07:53:42.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9edbe00640 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009fd0 tx=0x7f9ed4009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7550748d14e79856 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9ec7400640 1 -- 172.21.15.3:0/990505535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed405dcc0 con 0x559355958a00 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9ec7400640 1 -- 172.21.15.3:0/990505535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed405ee50 con 0x559355958a00 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.942+0000 7f9ec7400640 1 -- 172.21.15.3:0/990505535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40624f0 con 0x559355958a00 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 -- 172.21.15.3:0/990505535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 msgr2=0x559355948820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009fd0 tx=0x7f9ed4009400 comp rx=0 tx=0).stop 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 -- 172.21.15.3:0/990505535 shutdown_connections 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x559355948820 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x55935593e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 --2- 172.21.15.3:0/990505535 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 -- 172.21.15.3:0/990505535 >> 172.21.15.3:0/990505535 conn(0x5593558fb480 msgr2=0x55935593dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 -- 172.21.15.3:0/990505535 shutdown_connections 2024-09-10T07:53:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.943+0000 7f9ee5636740 1 -- 172.21.15.3:0/990505535 wait complete. 2024-09-10T07:53:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.944+0000 7f9ee5636740 1 -- start start 2024-09-10T07:53:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.944+0000 7f9ee5636740 1 Processor -- start 2024-09-10T07:53:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x5593558f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1583e0 con 0x559355958a00 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x559355948400 con 0x559355915800 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559355954ad0 con 0x559355915420 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9ee5636740 10 client.0 _ll_register_callbacks cb 0x7f9ecc04e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44364/0 (socket says 172.21.15.3:44364) 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.945+0000 7f9edbe00640 1 -- 172.21.15.3:0/855617874 learned_addr learned my addr 172.21.15.3:0/855617874 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edaa00640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x5593558f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edb400640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 -- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 msgr2=0x5593559c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 -- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 msgr2=0x5593558f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x5593558f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 -- 172.21.15.3:0/855617874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559355953d10 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edaa00640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x5593558f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9edbe00640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4059ad0 tx=0x7f9ed40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312a24e3c7100ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9ec4c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4089070 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed4078960 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed405ee50 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.946+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4057cd0 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 10 client.8310 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5593559c2960 con 0x559355958a00 2024-09-10T07:53:42.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed405ee50 con 0x559355958a00 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f9ed4058d60 con 0x559355958a00 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 10 client.8310 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 10 client.8310 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 20 client.8310 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593cde0 0x55935593d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 4 client.8310 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55935592a4a0 con 0x55935593cde0 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.947+0000 7f9edb400640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593cde0 0x55935593d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.948+0000 7f9edb400640 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593cde0 0x55935593d1f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e9efee49666f97 server_cookie=705decb08b467806 in_seq=0 out_seq=0 2024-09-10T07:53:42.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.948+0000 7f9ec4c00640 10 client.8310 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ec4c00640 1 -- 172.21.15.3:0/855617874 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55935592a4a0 con 0x55935593cde0 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ec4c00640 10 client.8310 handle_command_reply: tid=0 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 client.8310 shutdown 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 msgr2=0x5593558f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4059ad0 tx=0x7f9ed40021f0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 shutdown_connections 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55935593cde0 0x55935593d1f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559355958a00 0x5593558f9790 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559355915800 0x5593558f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 --2- 172.21.15.3:0/855617874 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x559355915420 0x5593559c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ec4c00640 0 client.8310 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 >> 172.21.15.3:0/855617874 conn(0x5593558fb480 msgr2=0x559355942c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 shutdown_connections 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 1 -- 172.21.15.3:0/855617874 wait complete. 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.960+0000 7f9ee5636740 20 client.8310 trim_cache size 0 max 16384 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:42.961 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 msgr2=0x7f9ebc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 0x7f9ebc06e320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc032000 tx=0x7f9ecc034040 comp rx=0 tx=0).stop 2024-09-10T07:53:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 msgr2=0x7f9edc0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06f460 tx=0x7f9ed400f1b0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 shutdown_connections 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f9ebc06be60 0x7f9ebc06e320 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9edc1f8200 0x7f9edc0ce0a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9edc0aede0 0x7f9edc1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 --2- 172.21.15.3:0/3066811987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec00039b0 0x7f9edc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 >> 172.21.15.3:0/3066811987 conn(0x7f9edc0b5cb0 msgr2=0x7f9edc0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 shutdown_connections 2024-09-10T07:53:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:42.961+0000 7f9ee2800640 1 -- 172.21.15.3:0/3066811987 wait complete. 2024-09-10T07:53:42.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2024-09-10T07:53:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.134+0000 7f74ad600640 1 Processor -- start 2024-09-10T07:53:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.135+0000 7f74ad600640 1 -- start start 2024-09-10T07:53:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f74ad600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 0x7f74a815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f74ad600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a815eff0 con 0x7f74a8072c50 2024-09-10T07:53:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f74ad600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a815f2b0 con 0x7f74a80b4c90 2024-09-10T07:53:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f74ad600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f74a815fbe0 con 0x7f74a815b230 2024-09-10T07:53:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f749e000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f74a8072c50 0x7f74a80b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55676/0 (socket says 172.21.15.3:55676) 2024-09-10T07:53:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f749e000640 1 -- 172.21.15.3:0/2709026416 learned_addr learned my addr 172.21.15.3:0/2709026416 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.136+0000 7f74a6a00640 1 --2- 172.21.15.3:0/2709026416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 0x7f74a815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357865665 0 0) 0x7f74a815eff0 con 0x7f74a8072c50 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f748c005730 con 0x7f74a8072c50 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329087436 0 0) 0x7f74a815f2b0 con 0x7f74a80b4c90 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a815eff0 con 0x7f74a80b4c90 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6a00640 1 -- 172.21.15.3:0/2709026416 >> v1:172.21.15.70:6789/0 conn(0x7f74a80b4c90 legacy=0x7f74a8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6a00640 1 -- 172.21.15.3:0/2709026416 >> v1:172.21.15.3:6789/0 conn(0x7f74a8072c50 legacy=0x7f74a80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2994548606 0 0) 0x7f748c005730 con 0x7f74a8072c50 2024-09-10T07:53:43.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.137+0000 7f74a6a00640 1 -- 172.21.15.3:0/2709026416 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a815f590 con 0x7f74a815b230 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.138+0000 7f74a6a00640 1 --2- 172.21.15.3:0/2709026416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 0x7f74a815c390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7498009800 tx=0x7f749802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d316a90a1d1063e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.138+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498037070 con 0x7f74a815b230 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.138+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749802ac50 con 0x7f74a815b230 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.138+0000 7f74a6000640 1 -- 172.21.15.3:0/2709026416 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490001670 con 0x7f74a815b230 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 -- 172.21.15.3:0/2709026416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 msgr2=0x7f74a815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 --2- 172.21.15.3:0/2709026416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 0x7f74a815c390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7498009800 tx=0x7f749802a1c0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 -- 172.21.15.3:0/2709026416 shutdown_connections 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 --2- 172.21.15.3:0/2709026416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a815b230 0x7f74a815c390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 -- 172.21.15.3:0/2709026416 >> 172.21.15.3:0/2709026416 conn(0x7f74a80b5cb0 msgr2=0x7f74a806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 -- 172.21.15.3:0/2709026416 shutdown_connections 2024-09-10T07:53:43.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.139+0000 7f74ad600640 1 -- 172.21.15.3:0/2709026416 wait complete. 2024-09-10T07:53:43.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.141+0000 7f74ad600640 1 Processor -- start 2024-09-10T07:53:43.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.141+0000 7f74ad600640 1 -- start start 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a80b4c90 0x7f74a8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74a815b230 0x7f74a813e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74a815b610 con 0x7f74a815b230 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f74a815c8d0 con 0x7f74a80b4c90 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74ad600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f74a815b790 con 0x7f74a8072c50 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74a815b230 0x7f74a813e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:45512/0 (socket says 172.21.15.3:45512) 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 -- 172.21.15.3:0/4024630949 learned_addr learned my addr 172.21.15.3:0/4024630949 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f749e000640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a80b4c90 0x7f74a8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 -- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a80b4c90 msgr2=0x7f74a8141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a80b4c90 0x7f74a8141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 -- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74a815b230 msgr2=0x7f74a813e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74a815b230 0x7f74a813e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.142+0000 7f74a6a00640 1 -- 172.21.15.3:0/4024630949 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a80b1f60 con 0x7f74a8072c50 2024-09-10T07:53:43.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f74a6a00640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7498009720 tx=0x7f74980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f749f400640 1 -- 172.21.15.3:0/4024630949 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498042020 con 0x7f74a8072c50 2024-09-10T07:53:43.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f749f400640 1 -- 172.21.15.3:0/4024630949 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749803d470 con 0x7f74a8072c50 2024-09-10T07:53:43.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a81f4f50 con 0x7f74a8072c50 2024-09-10T07:53:43.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f749f400640 1 -- 172.21.15.3:0/4024630949 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498037040 con 0x7f74a8072c50 2024-09-10T07:53:43.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.143+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81f5510 con 0x7f74a8072c50 2024-09-10T07:53:43.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.145+0000 7f749f400640 1 -- 172.21.15.3:0/4024630949 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7498045020 con 0x7f74a8072c50 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.146+0000 7f749f400640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 0x7f748406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.146+0000 7f749f400640 1 -- 172.21.15.3:0/4024630949 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f74980ab590 con 0x7f74a8072c50 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.146+0000 7f749e000640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 0x7f748406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.147+0000 7f74b0426740 1 Processor -- start 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.147+0000 7f749e000640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 0x7f748406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f74a81f24f0 tx=0x7f7490001c70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.147+0000 7f74b0426740 1 -- start start 2024-09-10T07:53:43.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433800 0x55daea464200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea419480 0x55daea464740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55daea381660 con 0x55daea419480 2024-09-10T07:53:43.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55daea329680 con 0x55daea433420 2024-09-10T07:53:43.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55daea37d620 con 0x55daea433800 2024-09-10T07:53:43.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea419480 0x55daea464740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f749e000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433800 0x55daea464200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46146/0 (socket says 172.21.15.3:46146) 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.148+0000 7f74a6a00640 1 -- 172.21.15.3:0/1715767007 learned_addr learned my addr 172.21.15.3:0/1715767007 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 -- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433800 msgr2=0x55daea464200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433800 0x55daea464200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 -- 172.21.15.3:0/1715767007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea419480 msgr2=0x55daea464740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea419480 0x55daea464740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 -- 172.21.15.3:0/1715767007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daea15e7d0 con 0x55daea433420 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7498075d80 tx=0x7f7498079530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81133926a5cf8892 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f7497000640 1 -- 172.21.15.3:0/1715767007 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749802ac50 con 0x55daea433420 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f7497000640 1 -- 172.21.15.3:0/1715767007 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7498079970 con 0x55daea433420 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.149+0000 7f7497000640 1 -- 172.21.15.3:0/1715767007 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749807dc70 con 0x55daea433420 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 -- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 msgr2=0x55daea463c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7498075d80 tx=0x7f7498079530 comp rx=0 tx=0).stop 2024-09-10T07:53:43.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 -- 172.21.15.3:0/1715767007 shutdown_connections 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea419480 0x55daea464740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433800 0x55daea464200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 --2- 172.21.15.3:0/1715767007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea463c40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 -- 172.21.15.3:0/1715767007 >> 172.21.15.3:0/1715767007 conn(0x55daea41a220 msgr2=0x55daea462840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 -- 172.21.15.3:0/1715767007 shutdown_connections 2024-09-10T07:53:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.150+0000 7f74b0426740 1 -- 172.21.15.3:0/1715767007 wait complete. 2024-09-10T07:53:43.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.151+0000 7f74b0426740 1 -- start start 2024-09-10T07:53:43.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.151+0000 7f74b0426740 1 Processor -- start 2024-09-10T07:53:43.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea448ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55daea329680 con 0x55daea433420 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55daea30a4c0 con 0x55daea433800 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55daea4183e0 con 0x55daea44bb50 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74b0426740 10 client.0 _ll_register_callbacks cb 0x55daea379e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44408/0 (socket says 172.21.15.3:44408) 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 -- 172.21.15.3:0/1341387415 learned_addr learned my addr 172.21.15.3:0/1341387415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f749e000640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a7400640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea448ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 -- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 msgr2=0x55daea448ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea448ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 -- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 msgr2=0x55daea44b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.152+0000 7f74a6a00640 1 -- 172.21.15.3:0/1341387415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daea0943f0 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f74980ab010 tx=0x7f7498000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f18ad05fdde686 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74980824a0 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f74980800b0 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7498079970 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f749807dc70 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 10 client.8322 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55daea0540f0 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7498079970 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f7498082010 con 0x55daea433420 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 10 client.8322 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:43.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 10 client.8322 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 20 client.8322 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea47e590 0x55daea480980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 4 client.8322 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.153+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daea38ed90 con 0x55daea47e590 2024-09-10T07:53:43.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.154+0000 7f749e000640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea47e590 0x55daea480980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.154+0000 7f749e000640 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea47e590 0x55daea480980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d9ce796b9de4b36 server_cookie=18c2e0eae16feee in_seq=0 out_seq=0 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.154+0000 7f748b400640 10 client.8322 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.155+0000 7f748b400640 1 -- 172.21.15.3:0/1341387415 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55daea38ed90 con 0x55daea47e590 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.155+0000 7f748b400640 10 client.8322 handle_command_reply: tid=0 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.155+0000 7f74b0426740 1 client.8322 shutdown 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.155+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 msgr2=0x55daea45d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.155+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f74980ab010 tx=0x7f7498000bc0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 shutdown_connections 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea47e590 0x55daea480980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea448ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f748b400640 0 client.8322 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 --2- 172.21.15.3:0/1341387415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433420 0x55daea45d610 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 >> 172.21.15.3:0/1341387415 conn(0x55daea419480 msgr2=0x55daea45c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 shutdown_connections 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 1 -- 172.21.15.3:0/1341387415 wait complete. 2024-09-10T07:53:43.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.156+0000 7f74b0426740 20 client.8322 trim_cache size 0 max 16384 2024-09-10T07:53:43.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.171+0000 7f74b0426740 1 Processor -- start 2024-09-10T07:53:43.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.171+0000 7f74b0426740 1 -- start start 2024-09-10T07:53:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433420 0x55daea462970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8200be0 con 0x55daea44bb50 2024-09-10T07:53:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f74a80af570 con 0x55daea433800 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f74a82004b0 con 0x55daea433420 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74a7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44414/0 (socket says 172.21.15.3:44414) 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74a7400640 1 -- 172.21.15.3:0/2884896301 learned_addr learned my addr 172.21.15.3:0/2884896301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f749e000640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.172+0000 7f74a6a00640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433420 0x55daea462970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 -- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433420 msgr2=0x55daea462970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433420 0x55daea462970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 -- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 msgr2=0x55daea44a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 -- 172.21.15.3:0/2884896301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daea462eb0 con 0x55daea44bb50 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f749e000640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f74a7400640 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001f70 tx=0x7f74a00016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b764baef0ec91f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f7496600640 1 -- 172.21.15.3:0/2884896301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0006b50 con 0x55daea44bb50 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f7496600640 1 -- 172.21.15.3:0/2884896301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a0006cf0 con 0x55daea44bb50 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.173+0000 7f7496600640 1 -- 172.21.15.3:0/2884896301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0003b90 con 0x55daea44bb50 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- 172.21.15.3:0/2884896301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 msgr2=0x55daea448660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001f70 tx=0x7f74a00016d0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- 172.21.15.3:0/2884896301 shutdown_connections 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea44bb50 0x55daea448660 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433800 0x55daea44a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 --2- 172.21.15.3:0/2884896301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea433420 0x55daea462970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- 172.21.15.3:0/2884896301 >> 172.21.15.3:0/2884896301 conn(0x55daea419480 msgr2=0x55daea44b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- 172.21.15.3:0/2884896301 shutdown_connections 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- 172.21.15.3:0/2884896301 wait complete. 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.174+0000 7f74b0426740 1 -- start start 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.175+0000 7f74b0426740 1 Processor -- start 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.175+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea4e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.175+0000 7f74b0426740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.175+0000 7f74b0426740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea4486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.175+0000 7f74b0426740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55daea45d6d0 con 0x55daea433800 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f74a806ec90 con 0x55daea433420 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f74b0426740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55daea4e10b0 con 0x55daea44bb50 2024-09-10T07:53:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f74b0426740 10 client.0 _ll_register_callbacks cb 0x7f74a0001920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44420/0 (socket says 172.21.15.3:44420) 2024-09-10T07:53:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea4e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 -- 172.21.15.3:0/1945736336 learned_addr learned my addr 172.21.15.3:0/1945736336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f74a7400640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea4486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 -- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 msgr2=0x55daea4486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea4486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 -- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 msgr2=0x55daea4e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea4e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 -- 172.21.15.3:0/1945736336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daea4481a0 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f749e000640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f74a80d0290 tx=0x7f7490001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5e46cd917d574e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.176+0000 7f748b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74900066c0 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f74900223f0 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a0006cf0 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f7490035050 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 10 client.8328 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55daea462830 con 0x55daea433800 2024-09-10T07:53:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a0006cf0 con 0x55daea433800 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f7490038020 con 0x55daea433800 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 10 client.8328 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 10 client.8328 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 20 client.8328 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea462260 0x55daea448be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 4 client.8328 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.177+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55daea3890e0 con 0x55daea462260 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.178+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea462260 0x55daea448be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.178+0000 7f74a6a00640 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea462260 0x55daea448be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e42b03daf76bea7c server_cookie=cca979bc624c0869 in_seq=0 out_seq=0 2024-09-10T07:53:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.178+0000 7f748b400640 10 client.8328 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f748b400640 1 -- 172.21.15.3:0/1945736336 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55daea3890e0 con 0x55daea462260 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f748b400640 10 client.8328 handle_command_reply: tid=0 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 client.8328 shutdown 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 msgr2=0x55daea44a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f74a80d0290 tx=0x7f7490001710 comp rx=0 tx=0).stop 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 shutdown_connections 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55daea462260 0x55daea448be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55daea44bb50 0x55daea4486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55daea433800 0x55daea44a1e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 --2- 172.21.15.3:0/1945736336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55daea433420 0x55daea4e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f748b400640 0 client.8328 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 >> 172.21.15.3:0/1945736336 conn(0x55daea419480 msgr2=0x55daea44b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 shutdown_connections 2024-09-10T07:53:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 1 -- 172.21.15.3:0/1945736336 wait complete. 2024-09-10T07:53:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.187+0000 7f74b0426740 20 client.8328 trim_cache size 0 max 16384 2024-09-10T07:53:43.188 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:43.188 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:53:43.188 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:43.188 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 msgr2=0x7f748406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 0x7f748406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f74a81f24f0 tx=0x7f7490001c70 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 msgr2=0x7f74a81f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7498009720 tx=0x7f74980046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 shutdown_connections 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f748406be30 0x7f748406e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74a815b230 0x7f74a813e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a80b4c90 0x7f74a8141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 --2- 172.21.15.3:0/4024630949 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f74a8072c50 0x7f74a81f4900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 >> 172.21.15.3:0/4024630949 conn(0x7f74a80b5cb0 msgr2=0x7f74a80ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 shutdown_connections 2024-09-10T07:53:43.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43.188+0000 7f74ad600640 1 -- 172.21.15.3:0/4024630949 wait complete. 2024-09-10T07:53:43.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:43.203 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/ 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-10T07:53:43.247 INFO:teuthology.orchestra.run.smithi003.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-10T07:53:43.248 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:43.248 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:43.249 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:43.249 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:43.249 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-10T07:53:43.249 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:43.389 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627778 2024-09-10T07:53:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:43.400 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:43.400 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:43.400 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:43.400 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:43.400 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-10T07:53:43.401 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:43.540 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627779 2024-09-10T07:53:43.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-10T07:53:43.552 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:43.694 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627780 2024-09-10T07:53:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-10T07:53:43.705 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:43.846 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627781 2024-09-10T07:53:43.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-10T07:53:43.857 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:43.997 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627782 2024-09-10T07:53:44.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> stdin-killer -- python3 -c ' 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> import stat 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-10T07:53:44.009 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-10T07:53:44.150 INFO:teuthology.orchestra.run.smithi003.stdout:1099511627783 2024-09-10T07:53:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T07:53:44.161 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-10T07:53:44.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.279+0000 7f1aba246f40 1 Processor -- start 2024-09-10T07:53:44.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.279+0000 7f1aba246f40 1 -- start start 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 0x556329582870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563294a7f30 0x5563294a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55632944fb30 con 0x556329578440 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556329453b70 con 0x5563294a7f30 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1aba246f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556329580900 con 0x7f1a9c003ab0 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.280+0000 7f1ab6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35798/0 (socket says 172.21.15.3:35798) 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 -- 172.21.15.3:0/460407047 learned_addr learned my addr 172.21.15.3:0/460407047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab7e00640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563294a7f30 0x5563294a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab7400640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 0x556329582870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 -- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 msgr2=0x556329582870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 0x556329582870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 -- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563294a7f30 msgr2=0x5563294a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563294a7f30 0x5563294a83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 -- 172.21.15.3:0/460407047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563295790f0 con 0x556329578440 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab7400640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 0x556329582870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009940 tx=0x7f1aa4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415a9d25429ca554 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab5600640 1 -- 172.21.15.3:0/460407047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4019030 con 0x556329578440 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab5600640 1 -- 172.21.15.3:0/460407047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4008790 con 0x556329578440 2024-09-10T07:53:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.281+0000 7f1ab5600640 1 -- 172.21.15.3:0/460407047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4008b60 con 0x556329578440 2024-09-10T07:53:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 -- 172.21.15.3:0/460407047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 msgr2=0x556329578840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009940 tx=0x7f1aa4005a00 comp rx=0 tx=0).stop 2024-09-10T07:53:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 -- 172.21.15.3:0/460407047 shutdown_connections 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5563294a7f30 0x5563294a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329578840 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 --2- 172.21.15.3:0/460407047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a9c003ab0 0x556329582870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 -- 172.21.15.3:0/460407047 >> 172.21.15.3:0/460407047 conn(0x5563294d7d40 msgr2=0x5563295810d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 -- 172.21.15.3:0/460407047 shutdown_connections 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.282+0000 7f1aba246f40 1 -- 172.21.15.3:0/460407047 wait complete. 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.283+0000 7f1aba246f40 1 Processor -- start 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.283+0000 7f1aba246f40 1 -- start start 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5563294a7f30 0x55632956df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329576670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c003ab0 0x556329576bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5563295747a0 con 0x556329578440 2024-09-10T07:53:44.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5563294a92d0 con 0x7f1a9c003ab0 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1aba246f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5563294d7aa0 con 0x5563294a7f30 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329576670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329576670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35810/0 (socket says 172.21.15.3:35810) 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 -- 172.21.15.3:0/1669969760 learned_addr learned my addr 172.21.15.3:0/1669969760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 -- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5563294a7f30 msgr2=0x55632956df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5563294a7f30 0x55632956df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 -- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c003ab0 msgr2=0x556329576bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a9c003ab0 0x556329576bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.284+0000 7f1ab6a00640 1 -- 172.21.15.3:0/1669969760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563295811f0 con 0x556329578440 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1ab6a00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329576670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009ec0 tx=0x7f1aa4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1aafe00640 1 -- 172.21.15.3:0/1669969760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4015980 con 0x556329578440 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1aafe00640 1 -- 172.21.15.3:0/1669969760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4022070 con 0x556329578440 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1aafe00640 1 -- 172.21.15.3:0/1669969760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4015d80 con 0x556329578440 2024-09-10T07:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563295774c0 con 0x556329578440 2024-09-10T07:53:44.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.285+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556329577a80 con 0x556329578440 2024-09-10T07:53:44.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.287+0000 7f1aafe00640 1 -- 172.21.15.3:0/1669969760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1aa401c070 con 0x556329578440 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1aafe00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a9806a810 0x7f1a9806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1aafe00640 1 -- 172.21.15.3:0/1669969760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f1aa408bb60 con 0x556329578440 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1ab7400640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a9806a810 0x7f1a9806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1aba246f40 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x556329616d30 0x5563296171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 14.12 14:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5563296176f0 con 0x556329616d30 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1ab7e00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x556329616d30 0x5563296171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.288+0000 7f1ab7400640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a9806a810 0x7f1a9806ccd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8001cc0 tx=0x7f1aa8001c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.289+0000 7f1ab7e00640 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x556329616d30 0x5563296171b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1ab7e00640 1 -- 172.21.15.3:0/1669969760 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v102'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1ab0003030 con 0x556329616d30 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x556329616d30 msgr2=0x5563296171b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x556329616d30 0x5563296171b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a9806a810 msgr2=0x7f1a9806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f1a9806a810 0x7f1a9806ccd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8001cc0 tx=0x7f1aa8001c10 comp rx=0 tx=0).stop 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 msgr2=0x556329576670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 --2- 172.21.15.3:0/1669969760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556329578440 0x556329576670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009ec0 tx=0x7f1aa4002fb0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1ab7e00640 1 -- 172.21.15.3:0/1669969760 reap_dead start 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 shutdown_connections 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 >> 172.21.15.3:0/1669969760 conn(0x5563294d7d40 msgr2=0x5563294da3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 shutdown_connections 2024-09-10T07:53:44.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.291+0000 7f1aba246f40 1 -- 172.21.15.3:0/1669969760 wait complete. 2024-09-10T07:53:44.295 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-10T07:53:44.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.373+0000 7fc56c198f40 1 Processor -- start 2024-09-10T07:53:44.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.373+0000 7fc56c198f40 1 -- start start 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 0x55721515e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 0x55721515d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55721502bb30 con 0x5572150ceda0 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55721502fb70 con 0x7fc5500030f0 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc56c198f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55721515c900 con 0x557215155230 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc569e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc569e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35828/0 (socket says 172.21.15.3:35828) 2024-09-10T07:53:44.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.374+0000 7fc569e00640 1 -- 172.21.15.3:0/1185604270 learned_addr learned my addr 172.21.15.3:0/1185604270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569400640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 0x55721515e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc568a00640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 0x55721515d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 -- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 msgr2=0x55721515d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 0x55721515d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 -- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 msgr2=0x55721515e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 0x55721515e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 -- 172.21.15.3:0/1185604270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721515ea90 con 0x5572150ceda0 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569400640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 0x55721515e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc568a00640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 0x55721515d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc569e00640 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x5572150d1b70 tx=0x7fc564003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279048efc5084f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc563400640 1 -- 172.21.15.3:0/1185604270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564018070 con 0x5572150ceda0 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc563400640 1 -- 172.21.15.3:0/1185604270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc564003110 con 0x5572150ceda0 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.375+0000 7fc563400640 1 -- 172.21.15.3:0/1185604270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564003500 con 0x5572150ceda0 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 -- 172.21.15.3:0/1185604270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 msgr2=0x557215154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x5572150d1b70 tx=0x7fc564003aa0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 -- 172.21.15.3:0/1185604270 shutdown_connections 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572150ceda0 0x557215154090 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557215155230 0x55721515d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 --2- 172.21.15.3:0/1185604270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5500030f0 0x55721515e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 -- 172.21.15.3:0/1185604270 >> 172.21.15.3:0/1185604270 conn(0x5572150b3d40 msgr2=0x5572150ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 -- 172.21.15.3:0/1185604270 shutdown_connections 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.376+0000 7fc56c198f40 1 -- 172.21.15.3:0/1185604270 wait complete. 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.377+0000 7fc56c198f40 1 Processor -- start 2024-09-10T07:53:44.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.377+0000 7fc56c198f40 1 -- start start 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5572150ceda0 0x55721514e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557215155230 0x557215152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 0x557215152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557215155990 con 0x7fc5500030f0 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557215154e60 con 0x557215155230 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc56c198f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5572150cf610 con 0x5572150ceda0 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 0x557215152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 0x557215152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35844/0 (socket says 172.21.15.3:35844) 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 -- 172.21.15.3:0/286109797 learned_addr learned my addr 172.21.15.3:0/286109797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc568a00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557215155230 0x557215152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569400640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5572150ceda0 0x55721514e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 -- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5572150ceda0 msgr2=0x55721514e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5572150ceda0 0x55721514e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 -- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557215155230 msgr2=0x557215152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557215155230 0x557215152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc569e00640 1 -- 172.21.15.3:0/286109797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557215158910 con 0x7fc5500030f0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.378+0000 7fc568a00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557215155230 0x557215152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc569e00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 0x557215152d60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x557215083a10 tx=0x7fc5640015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc562000640 1 -- 172.21.15.3:0/286109797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564018030 con 0x7fc5500030f0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc562000640 1 -- 172.21.15.3:0/286109797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc564006760 con 0x7fc5500030f0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557215156720 con 0x7fc5500030f0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc562000640 1 -- 172.21.15.3:0/286109797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564006020 con 0x7fc5500030f0 2024-09-10T07:53:44.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.379+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721514d280 con 0x7fc5500030f0 2024-09-10T07:53:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.381+0000 7fc562000640 1 -- 172.21.15.3:0/286109797 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc564027020 con 0x7fc5500030f0 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.382+0000 7fc562000640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc54c06a810 0x7fc54c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.382+0000 7fc562000640 1 -- 172.21.15.3:0/286109797 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fc56408a800 con 0x7fc5500030f0 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.382+0000 7fc569400640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc54c06a810 0x7fc54c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.382+0000 7fc56c198f40 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55721514e750 0x5572151feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.382+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- osd_op(unknown.0.0:1 14.12 14:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55721515d8f0 con 0x55721514e750 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.383+0000 7fc569400640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc54c06a810 0x7fc54c06ccd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0010e0 tx=0x7fc55c001430 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.383+0000 7fc568a00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55721514e750 0x5572151feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.384+0000 7fc568a00640 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55721514e750 0x5572151feb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc568a00640 1 -- 172.21.15.3:0/286109797 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v102'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc55805d040 con 0x55721514e750 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55721514e750 msgr2=0x5572151feb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x55721514e750 0x5572151feb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc54c06a810 msgr2=0x7fc54c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc54c06a810 0x7fc54c06ccd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0010e0 tx=0x7fc55c001430 comp rx=0 tx=0).stop 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 msgr2=0x557215152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 --2- 172.21.15.3:0/286109797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5500030f0 0x557215152d60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x557215083a10 tx=0x7fc5640015b0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc569e00640 1 -- 172.21.15.3:0/286109797 reap_dead start 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 shutdown_connections 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 >> 172.21.15.3:0/286109797 conn(0x5572150b3d40 msgr2=0x5572150ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 shutdown_connections 2024-09-10T07:53:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.386+0000 7fc56c198f40 1 -- 172.21.15.3:0/286109797 wait complete. 2024-09-10T07:53:44.390 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-10T07:53:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.467+0000 7fb46a955f40 1 Processor -- start 2024-09-10T07:53:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.467+0000 7fb46a955f40 1 -- start start 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a2909d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a2908ce0 0x55f7a29a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f7a29b3620 con 0x55f7a29a5690 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f7a2882b30 con 0x55f7a29a52b0 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.468+0000 7fb46a955f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f7a29b3900 con 0x55f7a2908ce0 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35868/0 (socket says 172.21.15.3:35868) 2024-09-10T07:53:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 -- 172.21.15.3:0/1974570369 learned_addr learned my addr 172.21.15.3:0/1974570369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467c00640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a2909d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb468600640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a2908ce0 0x55f7a29a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 -- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a2908ce0 msgr2=0x55f7a29a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a2908ce0 0x55f7a29a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 -- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 msgr2=0x55f7a2909d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a2909d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 -- 172.21.15.3:0/1974570369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a29092f0 con 0x55f7a29a5690 2024-09-10T07:53:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467c00640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a2909d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb467200640 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097e0 tx=0x7fb454005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f836a1e9b82b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.469+0000 7fb465e00640 1 -- 172.21.15.3:0/1974570369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454008890 con 0x55f7a29a5690 2024-09-10T07:53:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb465e00640 1 -- 172.21.15.3:0/1974570369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45401a070 con 0x55f7a29a5690 2024-09-10T07:53:44.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb465e00640 1 -- 172.21.15.3:0/1974570369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454008c30 con 0x55f7a29a5690 2024-09-10T07:53:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 -- 172.21.15.3:0/1974570369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 msgr2=0x55f7a290a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097e0 tx=0x7fb454005a00 comp rx=0 tx=0).stop 2024-09-10T07:53:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 -- 172.21.15.3:0/1974570369 shutdown_connections 2024-09-10T07:53:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a2908ce0 0x55f7a29a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a29a5690 0x55f7a290a2c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 --2- 172.21.15.3:0/1974570369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a2909d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 -- 172.21.15.3:0/1974570369 >> 172.21.15.3:0/1974570369 conn(0x55f7a290ad40 msgr2=0x55f7a28dbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 -- 172.21.15.3:0/1974570369 shutdown_connections 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.470+0000 7fb46a955f40 1 -- 172.21.15.3:0/1974570369 wait complete. 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.471+0000 7fb46a955f40 1 Processor -- start 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.471+0000 7fb46a955f40 1 -- start start 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 0x55f7a28dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a29a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a29a5690 0x55f7a29a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f7a29095d0 con 0x55f7a2908ce0 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55f7a29a63a0 con 0x55f7a29a52b0 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb46a955f40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55f7a29098b0 con 0x55f7a29a5690 2024-09-10T07:53:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb467c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 0x55f7a28dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb467c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 0x55f7a28dc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35876/0 (socket says 172.21.15.3:35876) 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb467c00640 1 -- 172.21.15.3:0/2810964685 learned_addr learned my addr 172.21.15.3:0/2810964685 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb467200640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a29a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.472+0000 7fb468600640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a29a5690 0x55f7a29a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 -- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a29a5690 msgr2=0x55f7a29a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a29a5690 0x55f7a29a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 -- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 msgr2=0x55f7a29a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a29a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 -- 172.21.15.3:0/2810964685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a29b1270 con 0x55f7a2908ce0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb468600640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55f7a29a5690 0x55f7a29a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467200640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f7a29a52b0 0x55f7a29a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb467c00640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 0x55f7a28dc0b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb458026440 tx=0x7fb458001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb464a00640 1 -- 172.21.15.3:0/2810964685 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45806a070 con 0x55f7a2908ce0 2024-09-10T07:53:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb464a00640 1 -- 172.21.15.3:0/2810964685 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458055610 con 0x55f7a2908ce0 2024-09-10T07:53:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb464a00640 1 -- 172.21.15.3:0/2810964685 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458009780 con 0x55f7a2908ce0 2024-09-10T07:53:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7a29b1550 con 0x55f7a2908ce0 2024-09-10T07:53:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.473+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a29b1b10 con 0x55f7a2908ce0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.475+0000 7fb464a00640 1 -- 172.21.15.3:0/2810964685 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb458053030 con 0x55f7a2908ce0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.476+0000 7fb464a00640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb44806a810 0x7fb44806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.476+0000 7fb464a00640 1 -- 172.21.15.3:0/2810964685 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb4580cd510 con 0x55f7a2908ce0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.476+0000 7fb467200640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb44806a810 0x7fb44806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.476+0000 7fb46a955f40 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x55f7a29a2ea0 0x55f7a29b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.476+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- osd_op(unknown.0.0:1 14.5 14:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55f7a29b07d0 con 0x55f7a29a2ea0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.477+0000 7fb467200640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb44806a810 0x7fb44806ccd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55f7a2928890 tx=0x7fb454008000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.477+0000 7fb468600640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x55f7a29a2ea0 0x55f7a29b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.477+0000 7fb468600640 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x55f7a29a2ea0 0x55f7a29b0290 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb468600640 1 -- 172.21.15.3:0/2810964685 <== osd.5 v2:172.21.15.70:6824/2669649466 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v102'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb46000a8a0 con 0x55f7a29a2ea0 2024-09-10T07:53:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x55f7a29a2ea0 msgr2=0x55f7a29b0290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x55f7a29a2ea0 0x55f7a29b0290 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb44806a810 msgr2=0x7fb44806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb44806a810 0x7fb44806ccd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55f7a2928890 tx=0x7fb454008000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 msgr2=0x55f7a28dc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 --2- 172.21.15.3:0/2810964685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f7a2908ce0 0x55f7a28dc0b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb458026440 tx=0x7fb458001a60 comp rx=0 tx=0).stop 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb468600640 1 -- 172.21.15.3:0/2810964685 reap_dead start 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 shutdown_connections 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.479+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 >> 172.21.15.3:0/2810964685 conn(0x55f7a290ad40 msgr2=0x55f7a29aaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.480+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 shutdown_connections 2024-09-10T07:53:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.480+0000 7fb46a955f40 1 -- 172.21.15.3:0/2810964685 wait complete. 2024-09-10T07:53:44.483 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-10T07:53:44.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.560+0000 7f01e934cf40 1 Processor -- start 2024-09-10T07:53:44.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.560+0000 7f01e934cf40 1 -- start start 2024-09-10T07:53:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec3317a0 0x5630ec3ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec333220 0x5630ec466790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5630ec332150 con 0x5630ec332e40 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5630ec2a7b30 con 0x5630ec3317a0 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e934cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5630ec3d8620 con 0x5630ec333220 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35900/0 (socket says 172.21.15.3:35900) 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 -- 172.21.15.3:0/1429683465 learned_addr learned my addr 172.21.15.3:0/1429683465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e6600640 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec3317a0 0x5630ec3ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01e7000640 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec333220 0x5630ec466790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 -- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec333220 msgr2=0x5630ec466790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec333220 0x5630ec466790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 -- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec3317a0 msgr2=0x5630ec3ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec3317a0 0x5630ec3ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.561+0000 7f01dfe00640 1 -- 172.21.15.3:0/1429683465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec34cda0 con 0x5630ec332e40 2024-09-10T07:53:44.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.562+0000 7f01dfe00640 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f01d000a430 tx=0x7f01d000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f4863b0560acd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.562+0000 7f01e5200640 1 -- 172.21.15.3:0/1429683465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000b920 con 0x5630ec332e40 2024-09-10T07:53:44.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.562+0000 7f01e5200640 1 -- 172.21.15.3:0/1429683465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d000f030 con 0x5630ec332e40 2024-09-10T07:53:44.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.562+0000 7f01e5200640 1 -- 172.21.15.3:0/1429683465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000bd20 con 0x5630ec332e40 2024-09-10T07:53:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 -- 172.21.15.3:0/1429683465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 msgr2=0x5630ec466250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f01d000a430 tx=0x7f01d000a8f0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 -- 172.21.15.3:0/1429683465 shutdown_connections 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec333220 0x5630ec466790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec466250 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 --2- 172.21.15.3:0/1429683465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec3317a0 0x5630ec3ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 -- 172.21.15.3:0/1429683465 >> 172.21.15.3:0/1429683465 conn(0x5630ec32fd40 msgr2=0x5630ec34c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 -- 172.21.15.3:0/1429683465 shutdown_connections 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 -- 172.21.15.3:0/1429683465 wait complete. 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.563+0000 7f01e934cf40 1 Processor -- start 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 -- start start 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec3317a0 0x5630ec3cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec3d4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec333220 0x5630ec3d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5630ec3d04c0 con 0x5630ec332e40 2024-09-10T07:53:44.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5630ec3d0270 con 0x5630ec333220 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.564+0000 7f01e934cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5630ec3d07a0 con 0x5630ec3317a0 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec3d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec3d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35904/0 (socket says 172.21.15.3:35904) 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 -- 172.21.15.3:0/3644335301 learned_addr learned my addr 172.21.15.3:0/3644335301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01e7000640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec333220 0x5630ec3d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01e6600640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec3317a0 0x5630ec3cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 -- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec3317a0 msgr2=0x5630ec3cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec3317a0 0x5630ec3cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 -- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec333220 msgr2=0x5630ec3d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5630ec333220 0x5630ec3d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 -- 172.21.15.3:0/3644335301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec46ddd0 con 0x5630ec332e40 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01e6600640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5630ec3317a0 0x5630ec3cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.565+0000 7f01dfe00640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec3d4090 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f01d000f000 tx=0x7f01d0002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.566+0000 7f01dea00640 1 -- 172.21.15.3:0/3644335301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0024070 con 0x5630ec332e40 2024-09-10T07:53:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.566+0000 7f01dea00640 1 -- 172.21.15.3:0/3644335301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0014620 con 0x5630ec332e40 2024-09-10T07:53:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.566+0000 7f01dea00640 1 -- 172.21.15.3:0/3644335301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0014a10 con 0x5630ec332e40 2024-09-10T07:53:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.566+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630ec3d4ab0 con 0x5630ec332e40 2024-09-10T07:53:44.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.566+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ec3d5070 con 0x5630ec332e40 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.568+0000 7f01dea00640 1 -- 172.21.15.3:0/3644335301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01d0027070 con 0x5630ec332e40 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.568+0000 7f01dea00640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01c406a810 0x7f01c406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.568+0000 7f01dea00640 1 -- 172.21.15.3:0/3644335301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f01d008a930 con 0x5630ec332e40 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.569+0000 7f01e6600640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01c406a810 0x7f01c406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.569+0000 7f01e934cf40 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5630ec3d45d0 0x5630ec46e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.569+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- osd_op(unknown.0.0:1 14.9 14:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5630ec46ed20 con 0x5630ec3d45d0 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.569+0000 7f01e7000640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5630ec3d45d0 0x5630ec46e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.569+0000 7f01e6600640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01c406a810 0x7f01c406ccd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01d8057000 tx=0x7f01d8052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.570+0000 7f01e7000640 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5630ec3d45d0 0x5630ec46e7e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.571+0000 7f01e7000640 1 -- 172.21.15.3:0/3644335301 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v102'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f01e0002a80 con 0x5630ec3d45d0 2024-09-10T07:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.571+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5630ec3d45d0 msgr2=0x5630ec46e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.571+0000 7f01e934cf40 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x5630ec3d45d0 0x5630ec46e7e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01c406a810 msgr2=0x7f01c406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f01c406a810 0x7f01c406ccd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01d8057000 tx=0x7f01d8052000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 msgr2=0x5630ec3d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 --2- 172.21.15.3:0/3644335301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5630ec332e40 0x5630ec3d4090 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f01d000f000 tx=0x7f01d0002f70 comp rx=0 tx=0).stop 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e7000640 1 -- 172.21.15.3:0/3644335301 reap_dead start 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 shutdown_connections 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 >> 172.21.15.3:0/3644335301 conn(0x5630ec32fd40 msgr2=0x5630ec330e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 shutdown_connections 2024-09-10T07:53:44.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.572+0000 7f01e934cf40 1 -- 172.21.15.3:0/3644335301 wait complete. 2024-09-10T07:53:44.575 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-10T07:53:44.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.653+0000 7f401371cf40 1 Processor -- start 2024-09-10T07:53:44.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.653+0000 7f401371cf40 1 -- start start 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940f0e40 0x5574941090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 0x5574940ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557494065b30 con 0x5574940efeb0 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x557494069b70 con 0x5574940f1220 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f401371cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557494196900 con 0x5574940f0e40 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f400fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f400fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35920/0 (socket says 172.21.15.3:35920) 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f400fe00640 1 -- 172.21.15.3:0/3694485568 learned_addr learned my addr 172.21.15.3:0/3694485568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f4010800640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 0x5574940ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.654+0000 7f400f400640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940f0e40 0x5574941090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 -- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940f0e40 msgr2=0x5574941090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940f0e40 0x5574941090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 -- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 msgr2=0x5574940ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 0x5574940ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 -- 172.21.15.3:0/3694485568 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574940ef3a0 con 0x5574940efeb0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f4010800640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 0x5574940ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400fe00640 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4000001de0 tx=0x7f400000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed642a11607ee333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400e000640 1 -- 172.21.15.3:0/3694485568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000bde0 con 0x5574940efeb0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400e000640 1 -- 172.21.15.3:0/3694485568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400000f400 con 0x5574940efeb0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.655+0000 7f400e000640 1 -- 172.21.15.3:0/3694485568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000f7f0 con 0x5574940efeb0 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 -- 172.21.15.3:0/3694485568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 msgr2=0x55749418ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4000001de0 tx=0x7f400000b5b0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 -- 172.21.15.3:0/3694485568 shutdown_connections 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f1220 0x5574940ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940f0e40 0x5574941090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 --2- 172.21.15.3:0/3694485568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940efeb0 0x55749418ad80 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 -- 172.21.15.3:0/3694485568 >> 172.21.15.3:0/3694485568 conn(0x5574940edd40 msgr2=0x5574940ee120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 -- 172.21.15.3:0/3694485568 shutdown_connections 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 -- 172.21.15.3:0/3694485568 wait complete. 2024-09-10T07:53:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.656+0000 7f401371cf40 1 Processor -- start 2024-09-10T07:53:44.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 -- start start 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940efeb0 0x557494108880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f0e40 0x55749422f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 0x55749422fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557494198740 con 0x5574940f1220 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55749418b360 con 0x5574940f0e40 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.657+0000 7f401371cf40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557494198a20 con 0x5574940efeb0 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 0x55749422fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f400fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940efeb0 0x557494108880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 0x55749422fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35928/0 (socket says 172.21.15.3:35928) 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 -- 172.21.15.3:0/1378430665 learned_addr learned my addr 172.21.15.3:0/1378430665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f400f400640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f0e40 0x55749422f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 -- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940efeb0 msgr2=0x557494108880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574940efeb0 0x557494108880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 -- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f0e40 msgr2=0x55749422f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574940f0e40 0x55749422f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.658+0000 7f4010800640 1 -- 172.21.15.3:0/1378430665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557494190c70 con 0x5574940f1220 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f4010800640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 0x55749422fd80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4008025a60 tx=0x7f4008003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f400cc00640 1 -- 172.21.15.3:0/1378430665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400804fb70 con 0x5574940f1220 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f400cc00640 1 -- 172.21.15.3:0/1378430665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4008067070 con 0x5574940f1220 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5574940ec880 con 0x5574940f1220 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f400cc00640 1 -- 172.21.15.3:0/1378430665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008062470 con 0x5574940f1220 2024-09-10T07:53:44.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.659+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557494184970 con 0x5574940f1220 2024-09-10T07:53:44.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.661+0000 7f400cc00640 1 -- 172.21.15.3:0/1378430665 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4008062610 con 0x5574940f1220 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f400cc00640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3ff006a810 0x7f3ff006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f400cc00640 1 -- 172.21.15.3:0/1378430665 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f40080cd830 con 0x5574940f1220 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f400fe00640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3ff006a810 0x7f3ff006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f401371cf40 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x557494184d20 0x55749418dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- osd_op(unknown.0.0:1 14.5 14:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x557494185270 con 0x557494184d20 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.662+0000 7f400fe00640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3ff006a810 0x7f3ff006ccd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f400000b5f0 tx=0x7f4000009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.663+0000 7f400f400640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x557494184d20 0x55749418dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.663+0000 7f400f400640 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x557494184d20 0x55749418dfd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f400f400640 1 -- 172.21.15.3:0/1378430665 <== osd.5 v2:172.21.15.70:6824/2669649466 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v102'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3ffc00e4f0 con 0x557494184d20 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x557494184d20 msgr2=0x55749418dfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x557494184d20 0x55749418dfd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3ff006a810 msgr2=0x7f3ff006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3ff006a810 0x7f3ff006ccd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f400000b5f0 tx=0x7f4000009000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 msgr2=0x55749422fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 --2- 172.21.15.3:0/1378430665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5574940f1220 0x55749422fd80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4008025a60 tx=0x7f4008003330 comp rx=0 tx=0).stop 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f4010800640 1 -- 172.21.15.3:0/1378430665 reap_dead start 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 shutdown_connections 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 >> 172.21.15.3:0/1378430665 conn(0x5574940edd40 msgr2=0x5574940ee120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 shutdown_connections 2024-09-10T07:53:44.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.665+0000 7f401371cf40 1 -- 172.21.15.3:0/1378430665 wait complete. 2024-09-10T07:53:44.669 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-10T07:53:44.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.747+0000 7f04a2b1ff40 1 Processor -- start 2024-09-10T07:53:44.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.747+0000 7f04a2b1ff40 1 -- start start 2024-09-10T07:53:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7bb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1f7b2fb30 con 0x55b1f7bbae40 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b1f7b33b70 con 0x55b1f7bbb220 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b1f7c60900 con 0x55b1f7bb97a0 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f049e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.748+0000 7f049e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35934/0 (socket says 172.21.15.3:35934) 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 -- 172.21.15.3:0/3717199649 learned_addr learned my addr 172.21.15.3:0/3717199649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049f200640 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049fc00640 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7bb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 -- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 msgr2=0x55b1f7c524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 -- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 msgr2=0x55b1f7bb5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7bb5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 -- 172.21.15.3:0/3717199649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f7c59eb0 con 0x55b1f7bbae40 2024-09-10T07:53:44.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049e800640 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f048c001eb0 tx=0x7f048c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b77311eee987461 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049d400640 1 -- 172.21.15.3:0/3717199649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c012020 con 0x55b1f7bbae40 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049d400640 1 -- 172.21.15.3:0/3717199649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c00e670 con 0x55b1f7bbae40 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.749+0000 7f049d400640 1 -- 172.21.15.3:0/3717199649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c00ea80 con 0x55b1f7bbae40 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/3717199649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 msgr2=0x55b1f7bd4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f048c001eb0 tx=0x7f048c00b780 comp rx=0 tx=0).stop 2024-09-10T07:53:44.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/3717199649 shutdown_connections 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7bb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7bd4d20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/3717199649 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/3717199649 >> 172.21.15.3:0/3717199649 conn(0x55b1f7bb7d40 msgr2=0x55b1f7b87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/3717199649 shutdown_connections 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.750+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/3717199649 wait complete. 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.751+0000 7f04a2b1ff40 1 Processor -- start 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.751+0000 7f04a2b1ff40 1 -- start start 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c3f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7cea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7c621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b1f7c52a10 con 0x55b1f7bbae40 2024-09-10T07:53:44.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b1f7b88640 con 0x55b1f7bbb220 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f04a2b1ff40 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b1f7b88890 con 0x55b1f7bb97a0 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f049e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7cea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f049e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7cea5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35942/0 (socket says 172.21.15.3:35942) 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f049e800640 1 -- 172.21.15.3:0/100449595 learned_addr learned my addr 172.21.15.3:0/100449595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f049f200640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c3f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.752+0000 7f049fc00640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7c621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 -- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 msgr2=0x55b1f7c3f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b1f7bb97a0 0x55b1f7c3f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 -- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 msgr2=0x55b1f7c621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b1f7bbb220 0x55b1f7c621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 -- 172.21.15.3:0/100449595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f7c57f40 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f049e800640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7cea5c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f0497e00640 1 -- 172.21.15.3:0/100449595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c027070 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f0497e00640 1 -- 172.21.15.3:0/100449595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c00e470 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b1f7ceaf80 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f0497e00640 1 -- 172.21.15.3:0/100449595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c019a50 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.753+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f7ceb540 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.755+0000 7f0497e00640 1 -- 172.21.15.3:0/100449595 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f048c02a070 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.756+0000 7f0497e00640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f048006a810 0x7f048006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.756+0000 7f0497e00640 1 -- 172.21.15.3:0/100449595 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f048c08b180 con 0x55b1f7bbae40 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.756+0000 7f049f200640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f048006a810 0x7f048006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.756+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55b1f7c51c90 0x55b1f7c58700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.756+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- osd_op(unknown.0.0:1 14.1d 14:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55b1f7c58c40 con 0x55b1f7c51c90 2024-09-10T07:53:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.757+0000 7f049fc00640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55b1f7c51c90 0x55b1f7c58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.757+0000 7f049f200640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f048006a810 0x7f048006ccd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0490057000 tx=0x7f0490052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.758+0000 7f049fc00640 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55b1f7c51c90 0x55b1f7c58700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.759+0000 7f049fc00640 1 -- 172.21.15.3:0/100449595 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v102'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0498002680 con 0x55b1f7c51c90 2024-09-10T07:53:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55b1f7c51c90 msgr2=0x55b1f7c58700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x55b1f7c51c90 0x55b1f7c58700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f048006a810 msgr2=0x7f048006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f048006a810 0x7f048006ccd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0490057000 tx=0x7f0490052000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 msgr2=0x55b1f7cea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 --2- 172.21.15.3:0/100449595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b1f7bbae40 0x55b1f7cea5c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c000ed0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f049fc00640 1 -- 172.21.15.3:0/100449595 reap_dead start 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 shutdown_connections 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 >> 172.21.15.3:0/100449595 conn(0x55b1f7bb7d40 msgr2=0x55b1f7b87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 shutdown_connections 2024-09-10T07:53:44.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.760+0000 7f04a2b1ff40 1 -- 172.21.15.3:0/100449595 wait complete. 2024-09-10T07:53:44.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:44.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2024-09-10T07:53:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.925+0000 7f9987200640 1 Processor -- start 2024-09-10T07:53:44.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.925+0000 7f9987200640 1 -- start start 2024-09-10T07:53:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.926+0000 7f9987200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f99880b4bc0 0x7f99881690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9987200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880b42a0 con 0x7f9988148090 2024-09-10T07:53:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9987200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880ce390 con 0x7f99880b3e30 2024-09-10T07:53:44.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9987200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9988148470 con 0x7f99880b4bc0 2024-09-10T07:53:44.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9985e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9988148090 0x7f99880cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49670/0 (socket says 172.21.15.3:49670) 2024-09-10T07:53:44.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9985e00640 1 -- 172.21.15.3:0/1769264548 learned_addr learned my addr 172.21.15.3:0/1769264548 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:44.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9986800640 1 --2- 172.21.15.3:0/1769264548 >> v2:172.21.15.70:3300/0 conn(0x7f99880b4bc0 0x7f99881690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121417578 0 0) 0x7f9978001110 con 0x7f9988148090 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f996c003520 con 0x7f9988148090 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747076006 0 0) 0x7f99880ce390 con 0x7f99880b3e30 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9986800640 1 -- 172.21.15.3:0/1769264548 >> v1:172.21.15.70:6789/0 conn(0x7f99880b3e30 legacy=0x7f99880b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9986800640 1 -- 172.21.15.3:0/1769264548 >> v1:172.21.15.3:6789/0 conn(0x7f9988148090 legacy=0x7f99880cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.927+0000 7f9986800640 1 -- 172.21.15.3:0/1769264548 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880ce670 con 0x7f99880b4bc0 2024-09-10T07:53:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.928+0000 7f9986800640 1 --2- 172.21.15.3:0/1769264548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f99881690e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9980009dc0 tx=0x7f998002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ab9272858cb9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007740 con 0x7f99880b4bc0 2024-09-10T07:53:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800078e0 con 0x7f99880b4bc0 2024-09-10T07:53:44.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9985400640 1 -- 172.21.15.3:0/1769264548 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007cb0 con 0x7f99880b4bc0 2024-09-10T07:53:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9987200640 1 -- 172.21.15.3:0/1769264548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 msgr2=0x7f99881690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9987200640 1 --2- 172.21.15.3:0/1769264548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f99881690e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9980009dc0 tx=0x7f998002a730 comp rx=0 tx=0).stop 2024-09-10T07:53:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9987200640 1 -- 172.21.15.3:0/1769264548 shutdown_connections 2024-09-10T07:53:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9987200640 1 --2- 172.21.15.3:0/1769264548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f99881690e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.929+0000 7f9987200640 1 -- 172.21.15.3:0/1769264548 >> 172.21.15.3:0/1769264548 conn(0x7f998806d000 msgr2=0x7f998806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.930+0000 7f9987200640 1 -- 172.21.15.3:0/1769264548 shutdown_connections 2024-09-10T07:53:44.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.930+0000 7f9987200640 1 -- 172.21.15.3:0/1769264548 wait complete. 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.931+0000 7f9987200640 1 Processor -- start 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.931+0000 7f9987200640 1 -- start start 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f9988161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9988148090 0x7f99880720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99880cef10 con 0x7f99880b3e30 2024-09-10T07:53:44.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99880cec30 con 0x7f99880b4bc0 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9987200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d0440 con 0x7f9988148090 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9985e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9985e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35948/0 (socket says 172.21.15.3:35948) 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.932+0000 7f9985e00640 1 -- 172.21.15.3:0/3329786302 learned_addr learned my addr 172.21.15.3:0/3329786302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9986800640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9988148090 0x7f99880720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f997fe00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f9988161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 -- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9988148090 msgr2=0x7f99880720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9988148090 0x7f99880720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 -- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 msgr2=0x7f9988161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f9988161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 -- 172.21.15.3:0/3329786302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9988072c90 con 0x7f99880b3e30 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f997fe00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f9988161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9985e00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f997800c3a0 tx=0x7f997800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f997e000640 1 -- 172.21.15.3:0/3329786302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978017070 con 0x7f99880b3e30 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f997e000640 1 -- 172.21.15.3:0/3329786302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9978005ce0 con 0x7f99880b3e30 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f997e000640 1 -- 172.21.15.3:0/3329786302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978007310 con 0x7f99880b3e30 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.933+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880729b0 con 0x7f99880b3e30 2024-09-10T07:53:44.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.934+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998813e260 con 0x7f99880b3e30 2024-09-10T07:53:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.936+0000 7f997e000640 1 -- 172.21.15.3:0/3329786302 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f997801a070 con 0x7f99880b3e30 2024-09-10T07:53:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.936+0000 7f997e000640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 0x7f996406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.936+0000 7f997e000640 1 -- 172.21.15.3:0/3329786302 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f997808a2d0 con 0x7f99880b3e30 2024-09-10T07:53:44.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.937+0000 7f997fe00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 0x7f996406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.937+0000 7f998ffba740 1 Processor -- start 2024-09-10T07:53:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.937+0000 7f998ffba740 1 -- start start 2024-09-10T07:53:44.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.937+0000 7f997fe00640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 0x7f996406e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f998813faf0 tx=0x7f9970009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 0x55e48a087360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e489fe99c0 con 0x55e48a088710 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e489f918f0 con 0x55e48a084740 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e489fe5980 con 0x55e48a086f80 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35964/0 (socket says 172.21.15.3:35964) 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.938+0000 7f9986800640 1 -- 172.21.15.3:0/3196139602 learned_addr learned my addr 172.21.15.3:0/3196139602 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9985e00640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f997fe00640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 0x55e48a087360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 -- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 msgr2=0x55e48a0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 -- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 msgr2=0x55e48a087360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 0x55e48a087360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 -- 172.21.15.3:0/3196139602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e489dc4a50 con 0x55e48a088710 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9985e00640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f997fe00640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 0x55e48a087360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9986800640 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b60 tx=0x7f998002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45274005931c9f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9976a00640 1 -- 172.21.15.3:0/3196139602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800078c0 con 0x55e48a088710 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9976a00640 1 -- 172.21.15.3:0/3196139602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980007a60 con 0x55e48a088710 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.939+0000 7f9976a00640 1 -- 172.21.15.3:0/3196139602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007e30 con 0x55e48a088710 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 -- 172.21.15.3:0/3196139602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 msgr2=0x55e48a081800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b60 tx=0x7f998002e000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 -- 172.21.15.3:0/3196139602 shutdown_connections 2024-09-10T07:53:44.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a088710 0x55e48a081800 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a084740 0x55e48a087360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 --2- 172.21.15.3:0/3196139602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 -- 172.21.15.3:0/3196139602 >> 172.21.15.3:0/3196139602 conn(0x55e48a0839d0 msgr2=0x55e48a0cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 -- 172.21.15.3:0/3196139602 shutdown_connections 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.940+0000 7f998ffba740 1 -- 172.21.15.3:0/3196139602 wait complete. 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.941+0000 7f998ffba740 1 -- start start 2024-09-10T07:53:44.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.941+0000 7f998ffba740 1 Processor -- start 2024-09-10T07:53:44.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a0ca4a0 0x55e48a0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e48a082e70 con 0x55e48a086f80 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e489fa4b40 con 0x55e48a0ca4a0 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e48a0ccf50 con 0x55e48a088710 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f998ffba740 10 client.0 _ll_register_callbacks cb 0x55e489fa6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f9985e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f9985e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35972/0 (socket says 172.21.15.3:35972) 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.942+0000 7f9985e00640 1 -- 172.21.15.3:0/3087773798 learned_addr learned my addr 172.21.15.3:0/3087773798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 -- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 msgr2=0x55e48a0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 -- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a0ca4a0 msgr2=0x55e48a0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a0ca4a0 0x55e48a0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 -- 172.21.15.3:0/3087773798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e489cbc3e0 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f9985e00640 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f997800cd60 tx=0x7f997805c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de42cfc86fbef74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f996b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978088d80 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f99780074b0 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9980007a60 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f997808a0f0 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f998ffba740 10 client.8394 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.943+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55e489e251e0 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980007a60 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f99780579c0 con 0x55e48a086f80 2024-09-10T07:53:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 10 client.8394 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 10 client.8394 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 20 client.8394 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0b1190 0x55e48a0df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 4 client.8394 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e489ff70f0 con 0x55e48a0b1190 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.944+0000 7f997fe00640 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0b1190 0x55e48a0df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.945+0000 7f997fe00640 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0b1190 0x55e48a0df970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8b39e348de8638 server_cookie=cfc28a491b02b9b3 in_seq=0 out_seq=0 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.945+0000 7f996b400640 10 client.8394 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.945+0000 7f996b400640 1 -- 172.21.15.3:0/3087773798 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e489ff70f0 con 0x55e48a0b1190 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.945+0000 7f996b400640 10 client.8394 handle_command_reply: tid=0 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.945+0000 7f998ffba740 1 client.8394 shutdown 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 msgr2=0x55e48a0aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f997800cd60 tx=0x7f997805c920 comp rx=0 tx=0).stop 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 shutdown_connections 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a0ca4a0 0x55e48a0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0b1190 0x55e48a0df970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 --2- 172.21.15.3:0/3087773798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a086f80 0x55e48a0aeb30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 >> 172.21.15.3:0/3087773798 conn(0x55e48a084740 msgr2=0x55e48a0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f996b400640 0 client.8394 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 shutdown_connections 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 1 -- 172.21.15.3:0/3087773798 wait complete. 2024-09-10T07:53:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.946+0000 7f998ffba740 20 client.8394 trim_cache size 0 max 16384 2024-09-10T07:53:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.964+0000 7f998ffba740 1 Processor -- start 2024-09-10T07:53:44.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.964+0000 7f998ffba740 1 -- start start 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 0x55e48a0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9988200960 con 0x55e48a0ca4a0 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99881ffb30 con 0x55e48a086f80 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9988200230 con 0x55e48a088710 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35982/0 (socket says 172.21.15.3:35982) 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.965+0000 7f9986800640 1 -- 172.21.15.3:0/2655053186 learned_addr learned my addr 172.21.15.3:0/2655053186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9985e00640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 0x55e48a0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f997fe00640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 -- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 msgr2=0x55e48a0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 -- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 msgr2=0x55e48a0aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 0x55e48a0aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 -- 172.21.15.3:0/2655053186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e489c954a0 con 0x55e48a0ca4a0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9985e00640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 0x55e48a0aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9986800640 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b90 tx=0x7f998002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5481b2e4cc0a427a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9976000640 1 -- 172.21.15.3:0/2655053186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800076c0 con 0x55e48a0ca4a0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9976000640 1 -- 172.21.15.3:0/2655053186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980007860 con 0x55e48a0ca4a0 2024-09-10T07:53:44.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.966+0000 7f9976000640 1 -- 172.21.15.3:0/2655053186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007c30 con 0x55e48a0ca4a0 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 -- 172.21.15.3:0/2655053186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 msgr2=0x55e48a0cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b90 tx=0x7f998002c000 comp rx=0 tx=0).stop 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 -- 172.21.15.3:0/2655053186 shutdown_connections 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a0cc060 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a088710 0x55e48a0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 --2- 172.21.15.3:0/2655053186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a086f80 0x55e48a0aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 -- 172.21.15.3:0/2655053186 >> 172.21.15.3:0/2655053186 conn(0x55e48a084740 msgr2=0x55e48a0c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 -- 172.21.15.3:0/2655053186 shutdown_connections 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.967+0000 7f998ffba740 1 -- 172.21.15.3:0/2655053186 wait complete. 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.968+0000 7f998ffba740 1 -- start start 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.968+0000 7f998ffba740 1 Processor -- start 2024-09-10T07:53:44.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a088710 0x55e48a14a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99881f7800 con 0x55e48a0ca4a0 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e48a086d90 con 0x55e48a088710 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e48a0ac610 con 0x55e48a086f80 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f998ffba740 10 client.0 _ll_register_callbacks cb 0x7f9978054810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35988/0 (socket says 172.21.15.3:35988) 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 -- 172.21.15.3:0/3690235155 learned_addr learned my addr 172.21.15.3:0/3690235155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f997fe00640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a088710 0x55e48a14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9985e00640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 -- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 msgr2=0x55e48a0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 -- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a088710 msgr2=0x55e48a14a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a088710 0x55e48a14a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.969+0000 7f9986800640 1 -- 172.21.15.3:0/3690235155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48a0a80c0 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f9986800640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9980049040 tx=0x7f9980004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fafb021501ee18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f996b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998005f060 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9980007c20 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9980007860 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f9980007e20 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f998ffba740 10 client.8400 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55e48a14aad0 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.970+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980007860 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f998004f050 con 0x55e48a0ca4a0 2024-09-10T07:53:44.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 10 client.8400 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 10 client.8400 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 20 client.8400 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0afe00 0x55e48a14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 4 client.8400 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55e489ff1440 con 0x55e48a0afe00 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f9985e00640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0afe00 0x55e48a14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f9985e00640 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0afe00 0x55e48a14b9e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ae8d0418a12757 server_cookie=b91ac886366067a in_seq=0 out_seq=0 2024-09-10T07:53:44.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.971+0000 7f996b400640 10 client.8400 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:44.975 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:53:44.973+0000 7f458f200640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x555e8ab87600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.975 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-09-10T07:53:44.973+0000 7f458f200640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/AsLsXsFsx@2},l=7669 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x555e8ab87b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f996b400640 1 -- 172.21.15.3:0/3690235155 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e489ff1440 con 0x55e48a0afe00 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f996b400640 10 client.8400 handle_command_reply: tid=0 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 client.8400 shutdown 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 msgr2=0x55e48a14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9980049040 tx=0x7f9980004ab0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 shutdown_connections 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e48a0ca4a0 0x55e48a14b4a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e48a088710 0x55e48a14a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e48a0afe00 0x55e48a14b9e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 --2- 172.21.15.3:0/3690235155 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e48a086f80 0x55e48a0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.972+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 >> 172.21.15.3:0/3690235155 conn(0x55e48a084740 msgr2=0x55e48a0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.973+0000 7f996b400640 0 client.8400 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:44.977 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:53:44.973+0000 7f2ccb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1) n(v3 rc2024-09-10T07:53:28.810097+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55c7157c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:53:28.810097+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.973+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 shutdown_connections 2024-09-10T07:53:44.978 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:53:44.973+0000 7fea99c00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v2 rc2024-09-10T07:53:28.810097+0000 4=0+4) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x562063eb1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:53:28.810097+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.973+0000 7f998ffba740 1 -- 172.21.15.3:0/3690235155 wait complete. 2024-09-10T07:53:44.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.973+0000 7f998ffba740 20 client.8400 trim_cache size 0 max 16384 2024-09-10T07:53:44.978 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:44.979 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-10T07:53:44.979 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:44.979 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-10T07:53:44.979 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:44.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 msgr2=0x7f996406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 0x7f996406e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f998813faf0 tx=0x7f9970009040 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 msgr2=0x7f99880c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f997800c3a0 tx=0x7f997800c860 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 shutdown_connections 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f996406be30 0x7f996406e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9988148090 0x7f99880720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99880b4bc0 0x7f9988161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 --2- 172.21.15.3:0/3329786302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99880b3e30 0x7f99880c5270 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 >> 172.21.15.3:0/3329786302 conn(0x7f998806d000 msgr2=0x7f99880b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:44.980 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f2cd3400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1) n(v4 rc2024-09-10T07:53:28.810097+0000 5=0+5) (inest sync->mix g=1,2) caps={7669=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55c7157c7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:53:28.810097+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 shutdown_connections 2024-09-10T07:53:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:44.974+0000 7f9987200640 1 -- 172.21.15.3:0/3329786302 wait complete. 2024-09-10T07:53:44.981 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-10T07:53:44.975+0000 7feaa1e00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v1 rc2024-09-10T07:53:28.810097+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7669=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x562063eb1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:53:28.810097+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:53:44.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:44.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-09-10T07:53:45.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.148+0000 7fda1c600640 1 Processor -- start 2024-09-10T07:53:45.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.148+0000 7fda1c600640 1 -- start start 2024-09-10T07:53:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1c600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda14072c50 0x7fda1415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1c600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1415eff0 con 0x7fda140b4c90 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1c600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1415f2b0 con 0x7fda1415b230 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1c600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fda1415fbe0 con 0x7fda14072c50 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda19200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fda140b4c90 0x7fda14071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49672/0 (socket says 172.21.15.3:49672) 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda19200640 1 -- 172.21.15.3:0/1831421710 learned_addr learned my addr 172.21.15.3:0/1831421710 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1a600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda14072c50 0x7fda1415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.150+0000 7fda1a600640 1 -- 172.21.15.3:0/1831421710 >> v1:172.21.15.70:6789/0 conn(0x7fda1415b230 legacy=0x7fda140b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.151+0000 7fda1a600640 1 -- 172.21.15.3:0/1831421710 >> v1:172.21.15.3:6789/0 conn(0x7fda140b4c90 legacy=0x7fda14071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.151+0000 7fda1a600640 1 -- 172.21.15.3:0/1831421710 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1415f590 con 0x7fda14072c50 2024-09-10T07:53:45.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.151+0000 7fda13e00640 1 -- 172.21.15.3:0/1831421710 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575491712 0 0) 0x7fda1415eff0 con 0x7fda140b4c90 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1a600640 1 --2- 172.21.15.3:0/1831421710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda14072c50 0x7fda1415c390 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009d90 tx=0x7fda0c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe16ef5f2f34da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda13e00640 1 -- 172.21.15.3:0/1831421710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c007740 con 0x7fda14072c50 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda13e00640 1 -- 172.21.15.3:0/1831421710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c0078e0 con 0x7fda14072c50 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda13e00640 1 -- 172.21.15.3:0/1831421710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c007cb0 con 0x7fda14072c50 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 -- 172.21.15.3:0/1831421710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda14072c50 msgr2=0x7fda1415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 --2- 172.21.15.3:0/1831421710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda14072c50 0x7fda1415c390 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009d90 tx=0x7fda0c02a6f0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 -- 172.21.15.3:0/1831421710 shutdown_connections 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 --2- 172.21.15.3:0/1831421710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda14072c50 0x7fda1415c390 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 -- 172.21.15.3:0/1831421710 >> 172.21.15.3:0/1831421710 conn(0x7fda140b5cb0 msgr2=0x7fda1406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.152+0000 7fda1c600640 1 -- 172.21.15.3:0/1831421710 shutdown_connections 2024-09-10T07:53:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.153+0000 7fda1c600640 1 -- 172.21.15.3:0/1831421710 wait complete. 2024-09-10T07:53:45.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.154+0000 7fda1c600640 1 Processor -- start 2024-09-10T07:53:45.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.154+0000 7fda1c600640 1 -- start start 2024-09-10T07:53:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 0x7fda141f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda1415b230 0x7fda141542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda1415b610 con 0x7fda140b4c90 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fda1415c8d0 con 0x7fda14072c50 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda1c600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fda1415b790 con 0x7fda1415b230 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.155+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36022/0 (socket says 172.21.15.3:36022) 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 -- 172.21.15.3:0/4183019328 learned_addr learned my addr 172.21.15.3:0/4183019328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19c00640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 0x7fda141f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda1a600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda1415b230 0x7fda141542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 -- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda1415b230 msgr2=0x7fda141542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda1415b230 0x7fda141542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 -- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 msgr2=0x7fda141f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 0x7fda141f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 -- 172.21.15.3:0/4183019328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda14153520 con 0x7fda140b4c90 2024-09-10T07:53:45.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19c00640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 0x7fda141f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda19200640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c4a0 tx=0x7fda0000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.156+0000 7fda11600640 1 -- 172.21.15.3:0/4183019328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00018070 con 0x7fda140b4c90 2024-09-10T07:53:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.157+0000 7fda11600640 1 -- 172.21.15.3:0/4183019328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda000074e0 con 0x7fda140b4c90 2024-09-10T07:53:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.157+0000 7fda11600640 1 -- 172.21.15.3:0/4183019328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00004a20 con 0x7fda140b4c90 2024-09-10T07:53:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.157+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141f4ab0 con 0x7fda140b4c90 2024-09-10T07:53:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.157+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141f5070 con 0x7fda140b4c90 2024-09-10T07:53:45.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.159+0000 7fda11600640 1 -- 172.21.15.3:0/4183019328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda00004bc0 con 0x7fda140b4c90 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.160+0000 7fda11600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 0x7fd9f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.160+0000 7fda11600640 1 -- 172.21.15.3:0/4183019328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fda0008a1a0 con 0x7fda140b4c90 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.160+0000 7fda19c00640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 0x7fd9f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.160+0000 7fda1f3eb740 1 Processor -- start 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 -- start start 2024-09-10T07:53:45.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda19c00640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 0x7fd9f406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fda08007fd0 tx=0x7fda08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 0x558cfd9b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558cfd980cf0 con 0x558cfd9bb210 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558cfd91c7f0 con 0x558cfd9b9a80 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.161+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558cfd8c4800 con 0x558cfd9b7240 2024-09-10T07:53:45.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36026/0 (socket says 172.21.15.3:36026) 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 -- 172.21.15.3:0/1956577546 learned_addr learned my addr 172.21.15.3:0/1956577546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda19200640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 0x558cfd9b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda19c00640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 -- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 msgr2=0x558cfd9b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 0x558cfd9b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 -- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 msgr2=0x558cfd9ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9ff220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 -- 172.21.15.3:0/1956577546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfd6f97d0 con 0x558cfd9bb210 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda19200640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 0x558cfd9b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.162+0000 7fda1a600640 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fda0c02ace0 tx=0x7fda0c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511339fe463d58d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda06000640 1 -- 172.21.15.3:0/1956577546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c007740 con 0x558cfd9bb210 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda06000640 1 -- 172.21.15.3:0/1956577546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c0078e0 con 0x558cfd9bb210 2024-09-10T07:53:45.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda06000640 1 -- 172.21.15.3:0/1956577546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c007c90 con 0x558cfd9bb210 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 -- 172.21.15.3:0/1956577546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 msgr2=0x558cfd9b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fda0c02ace0 tx=0x7fda0c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 -- 172.21.15.3:0/1956577546 shutdown_connections 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9b4070 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9b7240 0x558cfd9b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/1956577546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.163+0000 7fda1f3eb740 1 -- 172.21.15.3:0/1956577546 >> 172.21.15.3:0/1956577546 conn(0x558cfd9b6240 msgr2=0x558cfd9fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.164+0000 7fda1f3eb740 1 -- 172.21.15.3:0/1956577546 shutdown_connections 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.164+0000 7fda1f3eb740 1 -- 172.21.15.3:0/1956577546 wait complete. 2024-09-10T07:53:45.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.164+0000 7fda1f3eb740 1 -- start start 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.164+0000 7fda1f3eb740 1 Processor -- start 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558cfd8a5750 con 0x558cfd9bb210 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x558cfd9187b0 con 0x558cfd9b9a80 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558cfd9b56e0 con 0x558cfda0ebb0 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.165+0000 7fda1f3eb740 10 client.0 _ll_register_callbacks cb 0x558cfd62ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36040/0 (socket says 172.21.15.3:36040) 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 -- 172.21.15.3:0/3149119275 learned_addr learned my addr 172.21.15.3:0/3149119275 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19c00640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda1a600640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 -- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 msgr2=0x558cfda0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 -- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 msgr2=0x558cfd9db290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9db290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 -- 172.21.15.3:0/3149119275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfd5ef0f0 con 0x558cfd9bb210 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda1a600640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda19200640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ce60 tx=0x7fda00057dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd604de8c5b833eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fd9faa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00070370 con 0x558cfd9bb210 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fda000578b0 con 0x558cfd9bb210 2024-09-10T07:53:45.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.166+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c0078e0 con 0x558cfd9bb210 2024-09-10T07:53:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fda00088050 con 0x558cfd9bb210 2024-09-10T07:53:45.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 10 client.8415 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x558cfd759f60 con 0x558cfd9bb210 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c0078e0 con 0x558cfd9bb210 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7fda000541b0 con 0x558cfd9bb210 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 10 client.8415 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 10 client.8415 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 20 client.8415 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfda19350 0x558cfda1b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 4 client.8415 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfd929f20 con 0x558cfda19350 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.167+0000 7fda19c00640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfda19350 0x558cfda1b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.168+0000 7fda19c00640 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfda19350 0x558cfda1b740 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786feaef1593f9de server_cookie=ed14276457ff95df in_seq=0 out_seq=0 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.168+0000 7fd9faa00640 10 client.8415 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fd9faa00640 1 -- 172.21.15.3:0/3149119275 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cfd929f20 con 0x558cfda19350 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fd9faa00640 10 client.8415 handle_command_reply: tid=0 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 client.8415 shutdown 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 msgr2=0x558cfd9dd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ce60 tx=0x7fda00057dc0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 shutdown_connections 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfda19350 0x558cfda1b740 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfd9dd8e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/3149119275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfd9db290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fd9faa00640 0 client.8415 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 >> 172.21.15.3:0/3149119275 conn(0x558cfd9b7240 msgr2=0x558cfd9fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 shutdown_connections 2024-09-10T07:53:45.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 1 -- 172.21.15.3:0/3149119275 wait complete. 2024-09-10T07:53:45.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.169+0000 7fda1f3eb740 20 client.8415 trim_cache size 0 max 16384 2024-09-10T07:53:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.187+0000 7fda1f3eb740 1 Processor -- start 2024-09-10T07:53:45.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.187+0000 7fda1f3eb740 1 -- start start 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9bb210 0x558cfda00250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfda0ebb0 0x558cfda00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda14200b20 con 0x558cfd9b9a80 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fda141ffda0 con 0x558cfda0ebb0 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fda142003f0 con 0x558cfd9bb210 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda19c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda19c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36052/0 (socket says 172.21.15.3:36052) 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda19c00640 1 -- 172.21.15.3:0/847274978 learned_addr learned my addr 172.21.15.3:0/847274978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda19200640 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9bb210 0x558cfda00250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.188+0000 7fda1a600640 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfda0ebb0 0x558cfda00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 -- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9bb210 msgr2=0x558cfda00250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9bb210 0x558cfda00250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 -- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfda0ebb0 msgr2=0x558cfda00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfda0ebb0 0x558cfda00790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 -- 172.21.15.3:0/847274978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfd5c81b0 con 0x558cfd9b9a80 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda19c00640 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x558cfd9af6c0 tx=0x7fda08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b892229c21fcc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda05600640 1 -- 172.21.15.3:0/847274978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080054b0 con 0x558cfd9b9a80 2024-09-10T07:53:45.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda05600640 1 -- 172.21.15.3:0/847274978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda080035f0 con 0x558cfd9b9a80 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.189+0000 7fda05600640 1 -- 172.21.15.3:0/847274978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda080037a0 con 0x558cfd9b9a80 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 -- 172.21.15.3:0/847274978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 msgr2=0x558cfd9af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x558cfd9af6c0 tx=0x7fda08012000 comp rx=0 tx=0).stop 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 -- 172.21.15.3:0/847274978 shutdown_connections 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfda0ebb0 0x558cfda00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfd9bb210 0x558cfda00250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/847274978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9b9a80 0x558cfd9af6f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 -- 172.21.15.3:0/847274978 >> 172.21.15.3:0/847274978 conn(0x558cfd9b7240 msgr2=0x558cfd9fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 -- 172.21.15.3:0/847274978 shutdown_connections 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.190+0000 7fda1f3eb740 1 -- 172.21.15.3:0/847274978 wait complete. 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.191+0000 7fda1f3eb740 1 -- start start 2024-09-10T07:53:45.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.191+0000 7fda1f3eb740 1 Processor -- start 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfda00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558cfd9fbde0 con 0x558cfd9bb210 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fda141f8630 con 0x558cfd9b9a80 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x558cfd9fe300 con 0x558cfda0ebb0 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1f3eb740 10 client.0 _ll_register_callbacks cb 0x7fda00058230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36066/0 (socket says 172.21.15.3:36066) 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 -- 172.21.15.3:0/490204046 learned_addr learned my addr 172.21.15.3:0/490204046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19c00640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfda00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1a600640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 -- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 msgr2=0x558cfda658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 -- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 msgr2=0x558cfda00440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfda00440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19200640 1 -- 172.21.15.3:0/490204046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfd9b3700 con 0x558cfd9bb210 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda1a600640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.192+0000 7fda19c00640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfda00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fda19200640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fda00054de0 tx=0x7fda0005b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ce623761eb2edf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fd9faa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00060f90 con 0x558cfd9bb210 2024-09-10T07:53:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fda000578b0 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda080035f0 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fda00074030 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fda1f3eb740 10 client.8421 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x558cfd9e0d40 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.193+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda080035f0 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7fda00057320 con 0x558cfd9bb210 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 10 client.8421 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 10 client.8421 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 20 client.8421 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfd9b2f90 0x558cfda7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 4 client.8421 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x558cfd924270 con 0x558cfd9b2f90 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda19c00640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfd9b2f90 0x558cfda7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fda19c00640 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfd9b2f90 0x558cfda7f9b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779567f40564707 server_cookie=e079f15949c4071e in_seq=0 out_seq=0 2024-09-10T07:53:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.194+0000 7fd9faa00640 10 client.8421 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fd9faa00640 1 -- 172.21.15.3:0/490204046 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cfd924270 con 0x558cfd9b2f90 2024-09-10T07:53:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fd9faa00640 10 client.8421 handle_command_reply: tid=0 2024-09-10T07:53:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 client.8421 shutdown 2024-09-10T07:53:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 msgr2=0x558cfda65380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fda00054de0 tx=0x7fda0005b070 comp rx=0 tx=0).stop 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 shutdown_connections 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x558cfd9b2f90 0x558cfda7f9b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x558cfda0ebb0 0x558cfda658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558cfd9bb210 0x558cfda65380 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 --2- 172.21.15.3:0/490204046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558cfd9b9a80 0x558cfda00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fd9faa00640 0 client.8421 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 >> 172.21.15.3:0/490204046 conn(0x558cfd9b7240 msgr2=0x558cfda0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 shutdown_connections 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 1 -- 172.21.15.3:0/490204046 wait complete. 2024-09-10T07:53:45.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.195+0000 7fda1f3eb740 20 client.8421 trim_cache size 0 max 16384 2024-09-10T07:53:45.199 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:45.199 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:53:45.199 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:45.199 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 msgr2=0x7fd9f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 0x7fd9f406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fda08007fd0 tx=0x7fda08009040 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 msgr2=0x7fda14152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c4a0 tx=0x7fda0000c960 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 shutdown_connections 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd9f406be30 0x7fd9f406e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fda1415b230 0x7fda141542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda140b4c90 0x7fda14152fe0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 --2- 172.21.15.3:0/4183019328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda14072c50 0x7fda141f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 >> 172.21.15.3:0/4183019328 conn(0x7fda140b5cb0 msgr2=0x7fda140b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 shutdown_connections 2024-09-10T07:53:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.196+0000 7fda1c600640 1 -- 172.21.15.3:0/4183019328 wait complete. 2024-09-10T07:53:45.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:45.211 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-10T07:53:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.370+0000 7f9907e00640 1 Processor -- start 2024-09-10T07:53:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.370+0000 7f9907e00640 1 -- start start 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9908161b70 0x7f99080d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f99080d3860 0x7f99080d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99080d3de0 con 0x7f99080d4ae0 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f99080d4f50 con 0x7f99080d3860 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9908161f50 con 0x7f9908161b70 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9906000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f99080d3860 0x7f99080d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f99080d4ae0 0x7f99080d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49680/0 (socket says 172.21.15.3:49680) 2024-09-10T07:53:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.371+0000 7f9907400640 1 -- 172.21.15.3:0/4126081071 learned_addr learned my addr 172.21.15.3:0/4126081071 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.372+0000 7f9906a00640 1 --2- 172.21.15.3:0/4126081071 >> v2:172.21.15.70:3300/0 conn(0x7f9908161b70 0x7f99080d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.372+0000 7f9906000640 1 -- 172.21.15.3:0/4126081071 >> v2:172.21.15.70:3300/0 conn(0x7f9908161b70 msgr2=0x7f99080d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.372+0000 7f9906000640 1 --2- 172.21.15.3:0/4126081071 >> v2:172.21.15.70:3300/0 conn(0x7f9908161b70 0x7f99080d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.372+0000 7f9906000640 1 -- 172.21.15.3:0/4126081071 >> v1:172.21.15.3:6789/0 conn(0x7f99080d4ae0 legacy=0x7f99080d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.372+0000 7f9906000640 1 -- 172.21.15.3:0/4126081071 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99080d40c0 con 0x7f99080d3860 2024-09-10T07:53:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.373+0000 7f9906000640 1 --2- 172.21.15.3:0/4126081071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d3860 0x7f99080d45a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f99080d55c0 tx=0x7f98f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380a626e835fbc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.373+0000 7f9905600640 1 -- 172.21.15.3:0/4126081071 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4037070 con 0x7f99080d3860 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.373+0000 7f9905600640 1 -- 172.21.15.3:0/4126081071 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f4032440 con 0x7f99080d3860 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.373+0000 7f9905600640 1 -- 172.21.15.3:0/4126081071 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4032810 con 0x7f99080d3860 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 -- 172.21.15.3:0/4126081071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d3860 msgr2=0x7f99080d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 --2- 172.21.15.3:0/4126081071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d3860 0x7f99080d45a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f99080d55c0 tx=0x7f98f402a1e0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 -- 172.21.15.3:0/4126081071 shutdown_connections 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 --2- 172.21.15.3:0/4126081071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d3860 0x7f99080d45a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 --2- 172.21.15.3:0/4126081071 >> v2:172.21.15.70:3300/0 conn(0x7f9908161b70 0x7f99080d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 -- 172.21.15.3:0/4126081071 >> 172.21.15.3:0/4126081071 conn(0x7f9908062630 msgr2=0x7f9908062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 -- 172.21.15.3:0/4126081071 shutdown_connections 2024-09-10T07:53:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.374+0000 7f9907e00640 1 -- 172.21.15.3:0/4126081071 wait complete. 2024-09-10T07:53:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.375+0000 7f9907e00640 1 Processor -- start 2024-09-10T07:53:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.376+0000 7f9907e00640 1 -- start start 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.376+0000 7f9907e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 0x7f9908159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.376+0000 7f9907e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.376+0000 7f9907e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 0x7f9908157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9907e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99080c9fc0 con 0x7f99080d4ae0 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9907e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99080cf970 con 0x7f9908161b70 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9907e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f99080cf2d0 con 0x7f99080d3860 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36074/0 (socket says 172.21.15.3:36074) 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 -- 172.21.15.3:0/2219126882 learned_addr learned my addr 172.21.15.3:0/2219126882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9907400640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 0x7f9908157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906a00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 0x7f9908159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 -- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 msgr2=0x7f9908159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 0x7f9908159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 -- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 msgr2=0x7f9908157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 0x7f9908157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 -- 172.21.15.3:0/2219126882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9908159f60 con 0x7f99080d4ae0 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906a00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 0x7f9908159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9907400640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 0x7f9908157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.377+0000 7f9906000640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f98f4035040 tx=0x7f98f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.378+0000 7f98fea00640 1 -- 172.21.15.3:0/2219126882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4037040 con 0x7f99080d4ae0 2024-09-10T07:53:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.378+0000 7f98fea00640 1 -- 172.21.15.3:0/2219126882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f4030770 con 0x7f99080d4ae0 2024-09-10T07:53:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.378+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990815aea0 con 0x7f99080d4ae0 2024-09-10T07:53:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.378+0000 7f98fea00640 1 -- 172.21.15.3:0/2219126882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4030b80 con 0x7f99080d4ae0 2024-09-10T07:53:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.378+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990815b460 con 0x7f99080d4ae0 2024-09-10T07:53:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.380+0000 7f98fea00640 1 -- 172.21.15.3:0/2219126882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98f4044020 con 0x7f99080d4ae0 2024-09-10T07:53:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.381+0000 7f98fea00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 0x7f98e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.381+0000 7f98fea00640 1 -- 172.21.15.3:0/2219126882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f98f40ab460 con 0x7f99080d4ae0 2024-09-10T07:53:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.381+0000 7f9906a00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 0x7f98e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.382+0000 7f9910aa1740 1 Processor -- start 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.382+0000 7f9906a00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 0x7f98e806e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006fd0 tx=0x7f98f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.382+0000 7f9910aa1740 1 -- start start 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2d838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d699e0 0x5613b2d68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5613b2cd1660 con 0x5613b2db3ea0 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5613b2c79680 con 0x5613b2d834f0 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5613b2ccd620 con 0x5613b2d699e0 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36078/0 (socket says 172.21.15.3:36078) 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 -- 172.21.15.3:0/1208845997 learned_addr learned my addr 172.21.15.3:0/1208845997 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9906000640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d699e0 0x5613b2d68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9906a00640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2d838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 -- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d699e0 msgr2=0x5613b2d68240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d699e0 0x5613b2d68240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 -- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 msgr2=0x5613b2d838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2d838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9907400640 1 -- 172.21.15.3:0/1208845997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b2aae7d0 con 0x5613b2db3ea0 2024-09-10T07:53:45.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.383+0000 7f9906a00640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2d838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f9907400640 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9900002200 tx=0x7f990000d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259d889a7c71360b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f98f3400640 1 -- 172.21.15.3:0/1208845997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900018070 con 0x5613b2db3ea0 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f98f3400640 1 -- 172.21.15.3:0/1208845997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9900013670 con 0x5613b2db3ea0 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f98f3400640 1 -- 172.21.15.3:0/1208845997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900013a40 con 0x5613b2db3ea0 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f9910aa1740 1 -- 172.21.15.3:0/1208845997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 msgr2=0x5613b2db43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.384+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9900002200 tx=0x7f990000d5e0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 -- 172.21.15.3:0/1208845997 shutdown_connections 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db43f0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d699e0 0x5613b2d68240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1208845997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2d838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 -- 172.21.15.3:0/1208845997 >> 172.21.15.3:0/1208845997 conn(0x5613b2d6a410 msgr2=0x5613b2db2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 -- 172.21.15.3:0/1208845997 shutdown_connections 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 -- 172.21.15.3:0/1208845997 wait complete. 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.385+0000 7f9910aa1740 1 -- start start 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.386+0000 7f9910aa1740 1 Processor -- start 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2daf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2dac850 0x5613b2dc6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5613b2d68940 con 0x5613b2db3ea0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5613b2c5a4c0 con 0x5613b2d834f0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5613b2d68c20 con 0x5613b2dac850 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9910aa1740 10 client.0 _ll_register_callbacks cb 0x5613b2cc9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36094/0 (socket says 172.21.15.3:36094) 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 -- 172.21.15.3:0/1006926439 learned_addr learned my addr 172.21.15.3:0/1006926439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9907400640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2dac850 0x5613b2dc6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906a00640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2daf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 -- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2dac850 msgr2=0x5613b2dc6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2dac850 0x5613b2dc6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 -- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 msgr2=0x5613b2daf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2daf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.387+0000 7f9906000640 1 -- 172.21.15.3:0/1006926439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b29e43f0 con 0x5613b2db3ea0 2024-09-10T07:53:45.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9906000640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009560 tx=0x7f98f4079e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658009c55c04524d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4082250 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f98f4030d20 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9900013670 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f98f4075ce0 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9910aa1740 10 client.8436 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5613b29a40f0 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9900013670 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f98f403a030 con 0x5613b2db3ea0 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9910aa1740 10 client.8436 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.388+0000 7f9910aa1740 10 client.8436 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9910aa1740 20 client.8436 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2da9a30 0x5613b2da9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9910aa1740 4 client.8436 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613b2cded90 con 0x5613b2da9a30 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9906a00640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2da9a30 0x5613b2da9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f9906a00640 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2da9a30 0x5613b2da9ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ff21720fdd2f1bc server_cookie=c3829740d729132d in_seq=0 out_seq=0 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.389+0000 7f98e7e00640 10 client.8436 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f98e7e00640 1 -- 172.21.15.3:0/1006926439 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613b2cded90 con 0x5613b2da9a30 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f98e7e00640 10 client.8436 handle_command_reply: tid=0 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 client.8436 shutdown 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 msgr2=0x5613b2da8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009560 tx=0x7f98f4079e30 comp rx=0 tx=0).stop 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 shutdown_connections 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2da9a30 0x5613b2da9ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2dac850 0x5613b2dc6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2da8880 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f98e7e00640 0 client.8436 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 --2- 172.21.15.3:0/1006926439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2d834f0 0x5613b2daf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.390+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 >> 172.21.15.3:0/1006926439 conn(0x5613b2d699e0 msgr2=0x5613b2db0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.391+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 shutdown_connections 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.391+0000 7f9910aa1740 1 -- 172.21.15.3:0/1006926439 wait complete. 2024-09-10T07:53:45.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.391+0000 7f9910aa1740 20 client.8436 trim_cache size 0 max 16384 2024-09-10T07:53:45.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.409+0000 7f9910aa1740 1 Processor -- start 2024-09-10T07:53:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.409+0000 7f9910aa1740 1 -- start start 2024-09-10T07:53:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2dd7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9908200c00 con 0x5613b2db3ea0 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99081ffdd0 con 0x5613b2dac850 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f99082004d0 con 0x5613b2d834f0 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36110/0 (socket says 172.21.15.3:36110) 2024-09-10T07:53:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9907400640 1 -- 172.21.15.3:0/2118921485 learned_addr learned my addr 172.21.15.3:0/2118921485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.410+0000 7f9906a00640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2dd7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9906000640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 -- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 msgr2=0x5613b2dd7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2dd7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 -- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 msgr2=0x5613b2db2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 -- 172.21.15.3:0/2118921485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b2dac1a0 con 0x5613b2db3ea0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9906000640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f9907400640 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f990000e010 tx=0x7f990000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e29a3dd46eb662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f98f2a00640 1 -- 172.21.15.3:0/2118921485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900025070 con 0x5613b2db3ea0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f98f2a00640 1 -- 172.21.15.3:0/2118921485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9900021480 con 0x5613b2db3ea0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.411+0000 7f98f2a00640 1 -- 172.21.15.3:0/2118921485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900018030 con 0x5613b2db3ea0 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 -- 172.21.15.3:0/2118921485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 msgr2=0x5613b2dae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f990000e010 tx=0x7f990000b000 comp rx=0 tx=0).stop 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 -- 172.21.15.3:0/2118921485 shutdown_connections 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2dae1f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 --2- 172.21.15.3:0/2118921485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2dd7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 -- 172.21.15.3:0/2118921485 >> 172.21.15.3:0/2118921485 conn(0x5613b2d699e0 msgr2=0x5613b2d69110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 -- 172.21.15.3:0/2118921485 shutdown_connections 2024-09-10T07:53:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.412+0000 7f9910aa1740 1 -- 172.21.15.3:0/2118921485 wait complete. 2024-09-10T07:53:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.413+0000 7f9910aa1740 1 -- start start 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.413+0000 7f9910aa1740 1 Processor -- start 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2db4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5613b2da9e40 con 0x5613b2db3ea0 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99081f3890 con 0x5613b2dac850 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5613b2d69dc0 con 0x5613b2d834f0 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9910aa1740 10 client.0 _ll_register_callbacks cb 0x7f98f4079440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36124/0 (socket says 172.21.15.3:36124) 2024-09-10T07:53:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 -- 172.21.15.3:0/3963763764 learned_addr learned my addr 172.21.15.3:0/3963763764 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9906a00640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2db4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9906000640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 -- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 msgr2=0x5613b2db4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2db4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 -- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 msgr2=0x5613b2db4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.414+0000 7f9907400640 1 -- 172.21.15.3:0/3963763764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b2d82d00 con 0x5613b2db3ea0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f9907400640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9900002200 tx=0x7f9900001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4d88b2fb5fdaac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f98e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900025040 con 0x5613b2db3ea0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9900013670 con 0x5613b2db3ea0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9900021480 con 0x5613b2db3ea0 2024-09-10T07:53:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f9900013890 con 0x5613b2db3ea0 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f9910aa1740 10 client.8442 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.415+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5613b2d83ce0 con 0x5613b2db3ea0 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9900021480 con 0x5613b2db3ea0 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f990001e050 con 0x5613b2db3ea0 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 10 client.8442 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 10 client.8442 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 20 client.8442 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2dd7410 0x5613b2db2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 4 client.8442 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613b2cd90e0 con 0x5613b2dd7410 2024-09-10T07:53:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.416+0000 7f9906a00640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2dd7410 0x5613b2db2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9906a00640 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2dd7410 0x5613b2db2550 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4536d3f410506e20 server_cookie=3dbdea1487aeea0e in_seq=0 out_seq=0 2024-09-10T07:53:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f98e7e00640 10 client.8442 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f98e7e00640 1 -- 172.21.15.3:0/3963763764 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5613b2cd90e0 con 0x5613b2dd7410 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f98e7e00640 10 client.8442 handle_command_reply: tid=0 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 client.8442 shutdown 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 msgr2=0x5613b2db2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9900002200 tx=0x7f9900001250 comp rx=0 tx=0).stop 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 shutdown_connections 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x5613b2dd7410 0x5613b2db2550 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613b2db3ea0 0x5613b2db2010 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5613b2dac850 0x5613b2db4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 --2- 172.21.15.3:0/3963763764 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5613b2d834f0 0x5613b2db4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f98e7e00640 0 client.8442 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 >> 172.21.15.3:0/3963763764 conn(0x5613b2d699e0 msgr2=0x5613b2dafa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 shutdown_connections 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 1 -- 172.21.15.3:0/3963763764 wait complete. 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.417+0000 7f9910aa1740 20 client.8442 trim_cache size 0 max 16384 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:45.421 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 msgr2=0x7f98e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 0x7f98e806e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006fd0 tx=0x7f98f8008040 comp rx=0 tx=0).stop 2024-09-10T07:53:45.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 msgr2=0x7f9908156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f98f4035040 tx=0x7f98f40046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 shutdown_connections 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98e806be30 0x7f98e806e2f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908161b70 0x7f9908157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99080d4ae0 0x7f9908156c30 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 --2- 172.21.15.3:0/2219126882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99080d3860 0x7f9908159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 >> 172.21.15.3:0/2219126882 conn(0x7f9908062630 msgr2=0x7f99081602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 shutdown_connections 2024-09-10T07:53:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.418+0000 7f9907e00640 1 -- 172.21.15.3:0/2219126882 wait complete. 2024-09-10T07:53:45.433 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:45.433 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:45.433 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-10T07:53:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.592+0000 7f5d52800640 1 Processor -- start 2024-09-10T07:53:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.592+0000 7f5d52800640 1 -- start start 2024-09-10T07:53:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.593+0000 7f5d52800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d4c15b230 0x7f5d4c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.593+0000 7f5d52800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c072c50 0x7f5d4c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.593+0000 7f5d52800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c15eff0 con 0x7f5d4c0b4ac0 2024-09-10T07:53:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.593+0000 7f5d52800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4c15fbe0 con 0x7f5d4c15b230 2024-09-10T07:53:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.593+0000 7f5d52800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c15c8d0 con 0x7f5d4c072c50 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d4c0b4ac0 0x7f5d4c06cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49684/0 (socket says 172.21.15.3:49684) 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4be00640 1 -- 172.21.15.3:0/3696861050 learned_addr learned my addr 172.21.15.3:0/3696861050 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4b400640 1 --2- 172.21.15.3:0/3696861050 >> v2:172.21.15.3:3300/0 conn(0x7f5d4c15b230 0x7f5d4c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c072c50 0x7f5d4c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4b400640 1 -- 172.21.15.3:0/3696861050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c072c50 msgr2=0x7f5d4c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4b400640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c072c50 0x7f5d4c0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4b400640 1 -- 172.21.15.3:0/3696861050 >> v1:172.21.15.3:6789/0 conn(0x7f5d4c0b4ac0 legacy=0x7f5d4c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4b400640 1 -- 172.21.15.3:0/3696861050 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c15f2b0 con 0x7f5d4c15b230 2024-09-10T07:53:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.594+0000 7f5d4a000640 1 -- 172.21.15.3:0/3696861050 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 232995923 0 0) 0x7f5d4c15eff0 con 0x7f5d4c0b4ac0 2024-09-10T07:53:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.595+0000 7f5d4b400640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c15b230 0x7f5d4c15c390 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0097a0 tx=0x7f5d3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9493fa02abea335 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.595+0000 7f5d4a000640 1 -- 172.21.15.3:0/3696861050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c037070 con 0x7f5d4c15b230 2024-09-10T07:53:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.595+0000 7f5d4a000640 1 -- 172.21.15.3:0/3696861050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c032440 con 0x7f5d4c15b230 2024-09-10T07:53:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.595+0000 7f5d4a000640 1 -- 172.21.15.3:0/3696861050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c032810 con 0x7f5d4c15b230 2024-09-10T07:53:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 -- 172.21.15.3:0/3696861050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c15b230 msgr2=0x7f5d4c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c15b230 0x7f5d4c15c390 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0097a0 tx=0x7f5d3c02a180 comp rx=0 tx=0).stop 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 -- 172.21.15.3:0/3696861050 shutdown_connections 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c072c50 0x7f5d4c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 --2- 172.21.15.3:0/3696861050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c15b230 0x7f5d4c15c390 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 -- 172.21.15.3:0/3696861050 >> 172.21.15.3:0/3696861050 conn(0x7f5d4c0b5cb0 msgr2=0x7f5d4c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 -- 172.21.15.3:0/3696861050 shutdown_connections 2024-09-10T07:53:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.596+0000 7f5d52800640 1 -- 172.21.15.3:0/3696861050 wait complete. 2024-09-10T07:53:45.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.598+0000 7f5d52800640 1 Processor -- start 2024-09-10T07:53:45.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.598+0000 7f5d52800640 1 -- start start 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 0x7f5d4c1443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c15b230 0x7f5d4c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c15b610 con 0x7f5d4c072c50 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c15f870 con 0x7f5d4c0b4ac0 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d52800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c15b790 con 0x7f5d4c15b230 2024-09-10T07:53:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36136/0 (socket says 172.21.15.3:36136) 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 -- 172.21.15.3:0/1039082082 learned_addr learned my addr 172.21.15.3:0/1039082082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 -- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c15b230 msgr2=0x7f5d4c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c15b230 0x7f5d4c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4b400640 1 -- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 msgr2=0x7f5d4c1443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.599+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 0x7f5d4c1443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d4b400640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 0x7f5d4c1443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d4b400640 1 -- 172.21.15.3:0/1039082082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c159340 con 0x7f5d4c072c50 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 0x7f5d4c1443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d4b400640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c035040 tx=0x7f5d3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d43400640 1 -- 172.21.15.3:0/1039082082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c037040 con 0x7f5d4c072c50 2024-09-10T07:53:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d43400640 1 -- 172.21.15.3:0/1039082082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c030770 con 0x7f5d4c072c50 2024-09-10T07:53:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d43400640 1 -- 172.21.15.3:0/1039082082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c030b80 con 0x7f5d4c072c50 2024-09-10T07:53:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c144920 con 0x7f5d4c072c50 2024-09-10T07:53:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.600+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1598a0 con 0x7f5d4c072c50 2024-09-10T07:53:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.602+0000 7f5d43400640 1 -- 172.21.15.3:0/1039082082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d3c044020 con 0x7f5d4c072c50 2024-09-10T07:53:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.603+0000 7f5d43400640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 0x7f5d2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.603+0000 7f5d43400640 1 -- 172.21.15.3:0/1039082082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c0ab490 con 0x7f5d4c072c50 2024-09-10T07:53:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.603+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 0x7f5d2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.604+0000 7f5d556b8740 1 Processor -- start 2024-09-10T07:53:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.604+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 0x7f5d2c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af800 tx=0x7f5d38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.604+0000 7f5d556b8740 1 -- start start 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c420 0x563a818acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 0x563a818ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563a817ca660 con 0x563a81862480 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563a81772680 con 0x563a8187c800 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563a817c6620 con 0x563a8187c420 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36148/0 (socket says 172.21.15.3:36148) 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d4be00640 1 -- 172.21.15.3:0/3676791780 learned_addr learned my addr 172.21.15.3:0/3676791780 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d4b400640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c420 0x563a818acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.605+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 0x563a818ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 -- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c420 msgr2=0x563a818acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c420 0x563a818acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 -- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 msgr2=0x563a818ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 0x563a818ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 -- 172.21.15.3:0/3676791780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a815a77d0 con 0x563a81862480 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 0x563a818ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d4be00640 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001d60 tx=0x7f5d4400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a6f541bfc50468 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d37e00640 1 -- 172.21.15.3:0/3676791780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400c8b0 con 0x563a81862480 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d37e00640 1 -- 172.21.15.3:0/3676791780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4400ca50 con 0x563a81862480 2024-09-10T07:53:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.606+0000 7f5d37e00640 1 -- 172.21.15.3:0/3676791780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400ce00 con 0x563a81862480 2024-09-10T07:53:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 -- 172.21.15.3:0/3676791780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 msgr2=0x563a818ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5d44001d60 tx=0x7f5d4400c0e0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 -- 172.21.15.3:0/3676791780 shutdown_connections 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a81862480 0x563a818ad740 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c800 0x563a818ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 --2- 172.21.15.3:0/3676791780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c420 0x563a818acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 -- 172.21.15.3:0/3676791780 >> 172.21.15.3:0/3676791780 conn(0x563a81863220 msgr2=0x563a818ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 -- 172.21.15.3:0/3676791780 shutdown_connections 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.607+0000 7f5d556b8740 1 -- 172.21.15.3:0/3676791780 wait complete. 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.608+0000 7f5d556b8740 1 -- start start 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.608+0000 7f5d556b8740 1 Processor -- start 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a818825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563a818613e0 con 0x563a8187c420 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563a817534c0 con 0x563a818a8c10 2024-09-10T07:53:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563a818616c0 con 0x563a8187c800 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d556b8740 10 client.0 _ll_register_callbacks cb 0x563a817c2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36156/0 (socket says 172.21.15.3:36156) 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 -- 172.21.15.3:0/556823584 learned_addr learned my addr 172.21.15.3:0/556823584 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4be00640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a818825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 -- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 msgr2=0x563a818a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 -- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 msgr2=0x563a818825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a818825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4b400640 1 -- 172.21.15.3:0/556823584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a814dd3f0 con 0x563a8187c420 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.609+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d4b400640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a9890 tx=0x7f5d3c079da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50add038c3ccb4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c082280 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c030d20 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4400ca50 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f5d3c0ab0b0 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d556b8740 10 client.8457 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.610+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563a8149d0f0 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4400ca50 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f5d3c03a030 con 0x563a8187c420 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 10 client.8457 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 10 client.8457 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 20 client.8457 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81882b30 0x563a81882f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 4 client.8457 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.611+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a817d7d90 con 0x563a81882b30 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.612+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81882b30 0x563a81882f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.612+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81882b30 0x563a81882f10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f96f74dc23086f3c server_cookie=d6d8101eaaba001 in_seq=0 out_seq=0 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.612+0000 7f5d34c00640 10 client.8457 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.613+0000 7f5d34c00640 1 -- 172.21.15.3:0/556823584 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a817d7d90 con 0x563a81882b30 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.613+0000 7f5d34c00640 10 client.8457 handle_command_reply: tid=0 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.613+0000 7f5d556b8740 1 client.8457 shutdown 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.613+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 msgr2=0x563a818a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.613+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a9890 tx=0x7f5d3c079da0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 shutdown_connections 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81882b30 0x563a81882f10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a818825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 --2- 172.21.15.3:0/556823584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818a3f50 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 >> 172.21.15.3:0/556823584 conn(0x563a81862480 msgr2=0x563a818ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d34c00640 0 client.8457 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 shutdown_connections 2024-09-10T07:53:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 1 -- 172.21.15.3:0/556823584 wait complete. 2024-09-10T07:53:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.614+0000 7f5d556b8740 20 client.8457 trim_cache size 0 max 16384 2024-09-10T07:53:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.632+0000 7f5d556b8740 1 Processor -- start 2024-09-10T07:53:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.632+0000 7f5d556b8740 1 -- start start 2024-09-10T07:53:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 0x563a818a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c200b60 con 0x563a818a8c10 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1ffd30 con 0x563a8187c420 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c200430 con 0x563a8187c800 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36164/0 (socket says 172.21.15.3:36164) 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 -- 172.21.15.3:0/2229216724 learned_addr learned my addr 172.21.15.3:0/2229216724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 -- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 msgr2=0x563a818ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4b400640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 0x563a818a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 -- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 msgr2=0x563a818a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 0x563a818a9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4be00640 1 -- 172.21.15.3:0/2229216724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a81892f40 con 0x563a818a8c10 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.633+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.634+0000 7f5d4b400640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 0x563a818a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.634+0000 7f5d4be00640 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d440012e0 tx=0x7f5d44006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58db3b3ddeb13baa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.634+0000 7f5d37400640 1 -- 172.21.15.3:0/2229216724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44024070 con 0x563a818a8c10 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.634+0000 7f5d37400640 1 -- 172.21.15.3:0/2229216724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4400b030 con 0x563a818a8c10 2024-09-10T07:53:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.634+0000 7f5d37400640 1 -- 172.21.15.3:0/2229216724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44019040 con 0x563a818a8c10 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- 172.21.15.3:0/2229216724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 msgr2=0x563a818ab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d440012e0 tx=0x7f5d44006000 comp rx=0 tx=0).stop 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- 172.21.15.3:0/2229216724 shutdown_connections 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a818a8c10 0x563a818ab5a0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a818ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 --2- 172.21.15.3:0/2229216724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8187c420 0x563a818a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- 172.21.15.3:0/2229216724 >> 172.21.15.3:0/2229216724 conn(0x563a81862480 msgr2=0x563a818a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- 172.21.15.3:0/2229216724 shutdown_connections 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- 172.21.15.3:0/2229216724 wait complete. 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.635+0000 7f5d556b8740 1 -- start start 2024-09-10T07:53:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 Processor -- start 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a8190cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a81891c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c15cda0 con 0x563a8187c420 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563a81883240 con 0x563a818a8c10 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563a8187fd90 con 0x563a8187c800 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.636+0000 7f5d556b8740 10 client.0 _ll_register_callbacks cb 0x7f5d3c0791e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36172/0 (socket says 172.21.15.3:36172) 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 -- 172.21.15.3:0/4099794216 learned_addr learned my addr 172.21.15.3:0/4099794216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4be00640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a81891c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a8190cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 -- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 msgr2=0x563a8190cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a8190cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 -- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 msgr2=0x563a81891c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a81891c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 -- 172.21.15.3:0/4099794216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a81868700 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a8190cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d4b400640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009560 tx=0x7f5d3c0a97f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22943a99d911607 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c082280 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c082000 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.637+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4400b030 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f5d3c079900 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 10 client.8463 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563a8192dfc0 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4400b030 con 0x563a8187c420 2024-09-10T07:53:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f5d3c098060 con 0x563a8187c420 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 10 client.8463 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 10 client.8463 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 20 client.8463 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81892150 0x563a8192a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 4 client.8463 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a817d20e0 con 0x563a81892150 2024-09-10T07:53:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.638+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81892150 0x563a8192a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d4aa00640 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81892150 0x563a8192a1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c097169efc4650d server_cookie=bb7630742af86ca8 in_seq=0 out_seq=0 2024-09-10T07:53:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d34c00640 10 client.8463 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d34c00640 1 -- 172.21.15.3:0/4099794216 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563a817d20e0 con 0x563a81892150 2024-09-10T07:53:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d34c00640 10 client.8463 handle_command_reply: tid=0 2024-09-10T07:53:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d556b8740 1 client.8463 shutdown 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 msgr2=0x563a818b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.639+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009560 tx=0x7f5d3c0a97f0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 shutdown_connections 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x563a81892150 0x563a8192a1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a818a8c10 0x563a81891c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563a8187c800 0x563a8190cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 --2- 172.21.15.3:0/4099794216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563a8187c420 0x563a818b0d60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d34c00640 0 client.8463 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 >> 172.21.15.3:0/4099794216 conn(0x563a81862480 msgr2=0x563a818a1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 shutdown_connections 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 1 -- 172.21.15.3:0/4099794216 wait complete. 2024-09-10T07:53:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d556b8740 20 client.8463 trim_cache size 0 max 16384 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.644 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 msgr2=0x7f5d2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 0x7f5d2c06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af800 tx=0x7f5d38008040 comp rx=0 tx=0).stop 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 msgr2=0x7f5d4c15a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c035040 tx=0x7f5d3c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 shutdown_connections 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5d2c06be30 0x7f5d2c06e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5d4c15b230 0x7f5d4c0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0b4ac0 0x7f5d4c1443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 --2- 172.21.15.3:0/1039082082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d4c072c50 0x7f5d4c15a380 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.640+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 >> 172.21.15.3:0/1039082082 conn(0x7f5d4c0b5cb0 msgr2=0x7f5d4c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.641+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 shutdown_connections 2024-09-10T07:53:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.641+0000 7f5d52800640 1 -- 172.21.15.3:0/1039082082 wait complete. 2024-09-10T07:53:45.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:45.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:45.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T07:53:45.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.814+0000 7f216ac00640 1 Processor -- start 2024-09-10T07:53:45.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.815+0000 7f216ac00640 1 -- start start 2024-09-10T07:53:45.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f216ac00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f216415e820 0x7f21640d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f216ac00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21640cfac0 0x7f216406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f216ac00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21640d2ed0 con 0x7f216406bb20 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f216ac00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f216406c4b0 con 0x7f21640cfac0 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f216ac00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f21640d2c80 con 0x7f216415e820 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f215be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f216406bb20 0x7f21640cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49690/0 (socket says 172.21.15.3:49690) 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f215be00640 1 -- 172.21.15.3:0/2268194987 learned_addr learned my addr 172.21.15.3:0/2268194987 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f2168c00640 1 --2- 172.21.15.3:0/2268194987 >> v2:172.21.15.3:3300/0 conn(0x7f21640cfac0 0x7f216406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.816+0000 7f2163e00640 1 --2- 172.21.15.3:0/2268194987 >> v2:172.21.15.70:3300/0 conn(0x7f216415e820 0x7f21640d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2163400640 1 -- 172.21.15.3:0/2268194987 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375320782 0 0) 0x7f21640d2ed0 con 0x7f216406bb20 2024-09-10T07:53:45.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2163400640 1 -- 172.21.15.3:0/2268194987 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2148003520 con 0x7f216406bb20 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2168c00640 1 -- 172.21.15.3:0/2268194987 >> v2:172.21.15.70:3300/0 conn(0x7f216415e820 msgr2=0x7f21640d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2168c00640 1 --2- 172.21.15.3:0/2268194987 >> v2:172.21.15.70:3300/0 conn(0x7f216415e820 0x7f21640d3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2168c00640 1 -- 172.21.15.3:0/2268194987 >> v1:172.21.15.3:6789/0 conn(0x7f216406bb20 legacy=0x7f21640cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.817+0000 7f2168c00640 1 -- 172.21.15.3:0/2268194987 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21640d31b0 con 0x7f21640cfac0 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f2168c00640 1 --2- 172.21.15.3:0/2268194987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21640cfac0 0x7f216406bf70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f216408d700 tx=0x7f215c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98886af7e62e3bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f2163400640 1 -- 172.21.15.3:0/2268194987 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c007740 con 0x7f21640cfac0 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f2163400640 1 -- 172.21.15.3:0/2268194987 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c0078e0 con 0x7f21640cfac0 2024-09-10T07:53:45.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f2163400640 1 -- 172.21.15.3:0/2268194987 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c007cb0 con 0x7f21640cfac0 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f216ac00640 1 -- 172.21.15.3:0/2268194987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21640cfac0 msgr2=0x7f216406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.818+0000 7f216ac00640 1 --2- 172.21.15.3:0/2268194987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21640cfac0 0x7f216406bf70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f216408d700 tx=0x7f215c02a680 comp rx=0 tx=0).stop 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 -- 172.21.15.3:0/2268194987 shutdown_connections 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 --2- 172.21.15.3:0/2268194987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21640cfac0 0x7f216406bf70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 --2- 172.21.15.3:0/2268194987 >> v2:172.21.15.70:3300/0 conn(0x7f216415e820 0x7f21640d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 -- 172.21.15.3:0/2268194987 >> 172.21.15.3:0/2268194987 conn(0x7f2164072770 msgr2=0x7f21640d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 -- 172.21.15.3:0/2268194987 shutdown_connections 2024-09-10T07:53:45.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.819+0000 7f216ac00640 1 -- 172.21.15.3:0/2268194987 wait complete. 2024-09-10T07:53:45.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.820+0000 7f216ac00640 1 Processor -- start 2024-09-10T07:53:45.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.820+0000 7f216ac00640 1 -- start start 2024-09-10T07:53:45.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f216406bb20 0x7f21640cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21640cfac0 0x7f2164150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21640d3a50 con 0x7f216415e820 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f21640d3770 con 0x7f21640cfac0 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.821+0000 7f216ac00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f21640d44b0 con 0x7f216406bb20 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36186/0 (socket says 172.21.15.3:36186) 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 -- 172.21.15.3:0/1329752735 learned_addr learned my addr 172.21.15.3:0/1329752735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f215be00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21640cfac0 0x7f2164150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2163e00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f216406bb20 0x7f21640cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 -- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f216406bb20 msgr2=0x7f21640cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f216406bb20 0x7f21640cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 -- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21640cfac0 msgr2=0x7f2164150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21640cfac0 0x7f2164150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.822+0000 7f2168c00640 1 -- 172.21.15.3:0/1329752735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21641510d0 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f2168c00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f215c02ac70 tx=0x7f215c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f2160c00640 1 -- 172.21.15.3:0/1329752735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c040070 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f2160c00640 1 -- 172.21.15.3:0/1329752735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c031bc0 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21640c9fd0 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f2160c00640 1 -- 172.21.15.3:0/1329752735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c009450 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.823+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21640ca870 con 0x7f216415e820 2024-09-10T07:53:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.825+0000 7f2160c00640 1 -- 172.21.15.3:0/1329752735 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f215c039040 con 0x7f216415e820 2024-09-10T07:53:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.826+0000 7f2160c00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 0x7f214006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.826+0000 7f2160c00640 1 -- 172.21.15.3:0/1329752735 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f215c0ac5e0 con 0x7f216415e820 2024-09-10T07:53:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.826+0000 7f2163e00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 0x7f214006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.827+0000 7f216da37740 1 Processor -- start 2024-09-10T07:53:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.827+0000 7f216da37740 1 -- start start 2024-09-10T07:53:45.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.827+0000 7f2163e00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 0x7f214006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2154009b40 tx=0x7f2154009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96796c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d9674c480 0x563d96797740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d966b4660 con 0x563d96766800 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d9665c680 con 0x563d96766420 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d966b0620 con 0x563d9674c480 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36200/0 (socket says 172.21.15.3:36200) 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 -- 172.21.15.3:0/2173594737 learned_addr learned my addr 172.21.15.3:0/2173594737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f2163e00640 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96796c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f2168c00640 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d9674c480 0x563d96797740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 -- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d9674c480 msgr2=0x563d96797740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d9674c480 0x563d96797740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 -- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 msgr2=0x563d96796c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96796c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.828+0000 7f215be00640 1 -- 172.21.15.3:0/2173594737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d964917d0 con 0x563d96766800 2024-09-10T07:53:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f215be00640 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f214c0017d0 tx=0x7f214c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7192c1eaeda90bac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f2153e00640 1 -- 172.21.15.3:0/2173594737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c005ac0 con 0x563d96766800 2024-09-10T07:53:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f2153e00640 1 -- 172.21.15.3:0/2173594737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c018070 con 0x563d96766800 2024-09-10T07:53:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f2153e00640 1 -- 172.21.15.3:0/2173594737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0070d0 con 0x563d96766800 2024-09-10T07:53:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f216da37740 1 -- 172.21.15.3:0/2173594737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 msgr2=0x563d96797200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.829+0000 7f216da37740 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f214c0017d0 tx=0x7f214c000bc0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 -- 172.21.15.3:0/2173594737 shutdown_connections 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d9674c480 0x563d96797740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96766800 0x563d96797200 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 --2- 172.21.15.3:0/2173594737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96796c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 -- 172.21.15.3:0/2173594737 >> 172.21.15.3:0/2173594737 conn(0x563d9674d220 msgr2=0x563d96795840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 -- 172.21.15.3:0/2173594737 shutdown_connections 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 -- 172.21.15.3:0/2173594737 wait complete. 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.830+0000 7f216da37740 1 -- start start 2024-09-10T07:53:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.831+0000 7f216da37740 1 Processor -- start 2024-09-10T07:53:45.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96765860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d9674b3e0 con 0x563d96755140 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d9663d4c0 con 0x563d96766420 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d9674b6c0 con 0x563d96766800 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f216da37740 10 client.0 _ll_register_callbacks cb 0x563d966ace80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2163e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96765860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36202/0 (socket says 172.21.15.3:36202) 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 -- 172.21.15.3:0/656050989 learned_addr learned my addr 172.21.15.3:0/656050989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f215be00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 -- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 msgr2=0x563d967a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 -- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 msgr2=0x563d96765860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96765860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.832+0000 7f2168c00640 1 -- 172.21.15.3:0/656050989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d963c73f0 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2168c00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f215c02a6c0 tx=0x7f215c07aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2020ff912c51b72b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c083580 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f215c0810b0 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c018070 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f215c0830f0 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 10 client.8478 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563d963870f0 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c018070 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f215c04d030 con 0x563d96755140 2024-09-10T07:53:45.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 10 client.8478 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 10 client.8478 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 20 client.8478 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967913f0 0x563d967a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 4 client.8478 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.833+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d966c1d90 con 0x563d967913f0 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.834+0000 7f2163e00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967913f0 0x563d967a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.834+0000 7f2163e00640 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967913f0 0x563d967a9590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9f3bccdfa50899 server_cookie=2e25983793c3192d in_seq=0 out_seq=0 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.834+0000 7f2150c00640 10 client.8478 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f2150c00640 1 -- 172.21.15.3:0/656050989 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d966c1d90 con 0x563d967913f0 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f2150c00640 10 client.8478 handle_command_reply: tid=0 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 client.8478 shutdown 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 msgr2=0x563d967555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f215c02a6c0 tx=0x7f215c07aa50 comp rx=0 tx=0).stop 2024-09-10T07:53:45.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 shutdown_connections 2024-09-10T07:53:45.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967555c0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967913f0 0x563d967a9590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 --2- 172.21.15.3:0/656050989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d96765860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f2150c00640 0 client.8478 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.835+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 >> 172.21.15.3:0/656050989 conn(0x563d9674c480 msgr2=0x563d9678f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.836+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 shutdown_connections 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.836+0000 7f216da37740 1 -- 172.21.15.3:0/656050989 wait complete. 2024-09-10T07:53:45.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.836+0000 7f216da37740 20 client.8478 trim_cache size 0 max 16384 2024-09-10T07:53:45.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.854+0000 7f216da37740 1 Processor -- start 2024-09-10T07:53:45.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.854+0000 7f216da37740 1 -- start start 2024-09-10T07:53:45.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d96798bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2164200a50 con 0x563d96755140 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f21641ffc20 con 0x563d96766420 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2164200320 con 0x563d96766800 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f2163e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f2163e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36210/0 (socket says 172.21.15.3:36210) 2024-09-10T07:53:45.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.855+0000 7f2163e00640 1 -- 172.21.15.3:0/780958160 learned_addr learned my addr 172.21.15.3:0/780958160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 -- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 msgr2=0x563d96798bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d96798bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 -- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 msgr2=0x563d967986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 -- 172.21.15.3:0/780958160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d96798460 con 0x563d96755140 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2163e00640 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x563d96753060 tx=0x7f2154019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323f82ea865a8e20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2153400640 1 -- 172.21.15.3:0/780958160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154031070 con 0x563d96755140 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2153400640 1 -- 172.21.15.3:0/780958160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2154003810 con 0x563d96755140 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.856+0000 7f2153400640 1 -- 172.21.15.3:0/780958160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154003c00 con 0x563d96755140 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- 172.21.15.3:0/780958160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 msgr2=0x563d967654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x563d96753060 tx=0x7f2154019000 comp rx=0 tx=0).stop 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- 172.21.15.3:0/780958160 shutdown_connections 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d96798bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 --2- 172.21.15.3:0/780958160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d967654c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- 172.21.15.3:0/780958160 >> 172.21.15.3:0/780958160 conn(0x563d9674c480 msgr2=0x563d967a9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- 172.21.15.3:0/780958160 shutdown_connections 2024-09-10T07:53:45.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- 172.21.15.3:0/780958160 wait complete. 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.857+0000 7f216da37740 1 -- start start 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.858+0000 7f216da37740 1 Processor -- start 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.858+0000 7f216da37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.858+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f216da37740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f216da37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d96790450 con 0x563d96755140 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d96799cf0 con 0x563d96766420 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f216da37740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d9652a940 con 0x563d96766800 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f216da37740 10 client.0 _ll_register_callbacks cb 0x7f215c07a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36214/0 (socket says 172.21.15.3:36214) 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 -- 172.21.15.3:0/1184524265 learned_addr learned my addr 172.21.15.3:0/1184524265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2168c00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f215be00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 -- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 msgr2=0x563d967a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 -- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 msgr2=0x563d967a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 -- 172.21.15.3:0/1184524265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d96756730 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2163e00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f21540093d0 tx=0x7f2154004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a0687715d5dee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.859+0000 7f2150c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154031070 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2154005990 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2154003810 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f2154042050 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 10 client.8484 fetch_fsmap learned FSMap version 110 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x563d96750bb0 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2154003810 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 110) ==== 3554+0+0 (secure 0 0 0) 0x7f2154045020 con 0x563d96755140 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 10 client.8484 fetch_fsmap finished waiting for FSMap version 110 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 10 client.8484 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 20 client.8484 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967ade40 0x563d967ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 4 client.8484 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.860+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d966bc0e0 con 0x563d967ade40 2024-09-10T07:53:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f215be00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967ade40 0x563d967ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f215be00640 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967ade40 0x563d967ae220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=123aeff7edd59a0a server_cookie=365e81f213d3b19 in_seq=0 out_seq=0 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f2150c00640 10 client.8484 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f2150c00640 1 -- 172.21.15.3:0/1184524265 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563d966bc0e0 con 0x563d967ade40 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f2150c00640 10 client.8484 handle_command_reply: tid=0 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.861+0000 7f216da37740 1 client.8484 shutdown 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 msgr2=0x563d96813ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f21540093d0 tx=0x7f2154004730 comp rx=0 tx=0).stop 2024-09-10T07:53:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 shutdown_connections 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x563d967ade40 0x563d967ae220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d96766800 0x563d967a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d96766420 0x563d967a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 --2- 172.21.15.3:0/1184524265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d96755140 0x563d96813ae0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f2150c00640 0 client.8484 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 >> 172.21.15.3:0/1184524265 conn(0x563d9674c480 msgr2=0x563d96796ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 shutdown_connections 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 1 -- 172.21.15.3:0/1184524265 wait complete. 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.862+0000 7f216da37740 20 client.8484 trim_cache size 0 max 16384 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:45.865 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 msgr2=0x7f214006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 0x7f214006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2154009b40 tx=0x7f2154009190 comp rx=0 tx=0).stop 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 msgr2=0x7f21641e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f215c02ac70 tx=0x7f215c004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 shutdown_connections 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f214006be30 0x7f214006e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f216415e820 0x7f21641e7480 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21640cfac0 0x7f2164150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 --2- 172.21.15.3:0/1329752735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f216406bb20 0x7f21640cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 >> 172.21.15.3:0/1329752735 conn(0x7f2164072770 msgr2=0x7f2164160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 shutdown_connections 2024-09-10T07:53:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:45.863+0000 7f216ac00640 1 -- 172.21.15.3:0/1329752735 wait complete. 2024-09-10T07:53:45.878 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:45.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:46.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.034+0000 7fb99fe00640 1 Processor -- start 2024-09-10T07:53:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.034+0000 7fb99fe00640 1 -- start start 2024-09-10T07:53:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161de0 0x7fb9a00d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99fe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9a00d10f0 0x7fb9a00d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99fe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a00d3060 con 0x7fb9a0161a00 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99fe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a00d1ae0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00d3340 con 0x7fb9a0161de0 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99f400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9a00d10f0 0x7fb9a00d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99ea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb9a0161a00 0x7fb9a00cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49704/0 (socket says 172.21.15.3:49704) 2024-09-10T07:53:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.035+0000 7fb99ea00640 1 -- 172.21.15.3:0/124807973 learned_addr learned my addr 172.21.15.3:0/124807973 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.036+0000 7fb997e00640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161de0 0x7fb9a00d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.036+0000 7fb99f400640 1 -- 172.21.15.3:0/124807973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161de0 msgr2=0x7fb9a00d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.036+0000 7fb99f400640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161de0 0x7fb9a00d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.036+0000 7fb99f400640 1 -- 172.21.15.3:0/124807973 >> v1:172.21.15.3:6789/0 conn(0x7fb9a0161a00 legacy=0x7fb9a00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.036+0000 7fb99f400640 1 -- 172.21.15.3:0/124807973 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00d3620 con 0x7fb9a00d10f0 2024-09-10T07:53:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99f400640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a00d15a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb998009d90 tx=0x7fb99802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17ba4d61319eb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99e000640 1 -- 172.21.15.3:0/124807973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998007740 con 0x7fb9a00d10f0 2024-09-10T07:53:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99e000640 1 -- 172.21.15.3:0/124807973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9980078e0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99e000640 1 -- 172.21.15.3:0/124807973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998007cb0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99fe00640 1 -- 172.21.15.3:0/124807973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 msgr2=0x7fb9a00d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.037+0000 7fb99fe00640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a00d15a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb998009d90 tx=0x7fb99802a660 comp rx=0 tx=0).stop 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 -- 172.21.15.3:0/124807973 shutdown_connections 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a00d15a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 --2- 172.21.15.3:0/124807973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161de0 0x7fb9a00d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 -- 172.21.15.3:0/124807973 >> 172.21.15.3:0/124807973 conn(0x7fb9a0072770 msgr2=0x7fb9a006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 -- 172.21.15.3:0/124807973 shutdown_connections 2024-09-10T07:53:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.038+0000 7fb99fe00640 1 -- 172.21.15.3:0/124807973 wait complete. 2024-09-10T07:53:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.039+0000 7fb99fe00640 1 Processor -- start 2024-09-10T07:53:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.039+0000 7fb99fe00640 1 -- start start 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161a00 0x7fb9a0157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0161de0 0x7fb9a00cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00b79f0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00d3be0 con 0x7fb9a0161de0 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99fe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00b7b70 con 0x7fb9a0161a00 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36220/0 (socket says 172.21.15.3:36220) 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.040+0000 7fb99ea00640 1 -- 172.21.15.3:0/1968572786 learned_addr learned my addr 172.21.15.3:0/1968572786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb997e00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161a00 0x7fb9a0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99f400640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0161de0 0x7fb9a00cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 -- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161a00 msgr2=0x7fb9a0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161a00 0x7fb9a0157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 -- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0161de0 msgr2=0x7fb9a00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0161de0 0x7fb9a00cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 -- 172.21.15.3:0/1968572786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a0157850 con 0x7fb9a00d10f0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99ea00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c430 tx=0x7fb99000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb9a00d10f0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990005ce0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900072f0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.041+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a0157b30 con 0x7fb9a00d10f0 2024-09-10T07:53:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.042+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01580f0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.043+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a00b8f90 con 0x7fb9a00d10f0 2024-09-10T07:53:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.044+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb99001a070 con 0x7fb9a00d10f0 2024-09-10T07:53:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.045+0000 7fb997400640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 0x7fb97c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.045+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb99008a170 con 0x7fb9a00d10f0 2024-09-10T07:53:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.045+0000 7fb997e00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 0x7fb97c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.046+0000 7fb997e00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 0x7fb97c06e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b40f0 tx=0x7fb988008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.050+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb9a00b8f90 con 0x7fb9a00d10f0 2024-09-10T07:53:46.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.269+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a00d15a0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.269+0000 7fb997400640 1 -- 172.21.15.3:0/1968572786 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5266 (secure 0 0 0) 0x7fb9900584c0 con 0x7fb9a00d10f0 2024-09-10T07:53:46.269 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:46.269 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":110,"btime":"2024-09-10T07:53:41:661867+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:41.539501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:46.270 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 110 2024-09-10T07:53:46.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 msgr2=0x7fb97c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 0x7fb97c06e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b40f0 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 msgr2=0x7fb9a01f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c430 tx=0x7fb99000c8f0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 shutdown_connections 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb97c06be60 0x7fb97c06e320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9a0161de0 0x7fb9a00cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9a0161a00 0x7fb9a0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 --2- 172.21.15.3:0/1968572786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9a00d10f0 0x7fb9a01f7210 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 >> 172.21.15.3:0/1968572786 conn(0x7fb9a0072770 msgr2=0x7fb9a00b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 shutdown_connections 2024-09-10T07:53:46.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.272+0000 7fb99fe00640 1 -- 172.21.15.3:0/1968572786 wait complete. 2024-09-10T07:53:46.287 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:46.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:46.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.443+0000 7f5c8f200640 1 Processor -- start 2024-09-10T07:53:46.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.443+0000 7f5c8f200640 1 -- start start 2024-09-10T07:53:46.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.444+0000 7f5c8f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5c8815c720 0x7f5c88071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.444+0000 7f5c8f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c880cfdc0 0x7f5c8815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.444+0000 7f5c8f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c880cd7f0 con 0x7f5c88070f30 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.444+0000 7f5c8f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5c88072210 con 0x7f5c880cfdc0 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.444+0000 7f5c8f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f5c880cf880 con 0x7f5c8815c720 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c880cfdc0 0x7f5c8815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c7fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c88070f30 0x7f5c88073610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49708/0 (socket says 172.21.15.3:49708) 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c880cfdc0 0x7f5c8815cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36228/0 (socket says 172.21.15.3:36228) 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c7fe00640 1 -- 172.21.15.3:0/2900140075 learned_addr learned my addr 172.21.15.3:0/2900140075 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c87e00640 1 --2- 172.21.15.3:0/2900140075 >> v2:172.21.15.70:3300/0 conn(0x7f5c8815c720 0x7f5c88071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 -- 172.21.15.3:0/2900140075 >> v2:172.21.15.70:3300/0 conn(0x7f5c8815c720 msgr2=0x7f5c88071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 --2- 172.21.15.3:0/2900140075 >> v2:172.21.15.70:3300/0 conn(0x7f5c8815c720 0x7f5c88071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 -- 172.21.15.3:0/2900140075 >> v1:172.21.15.3:6789/0 conn(0x7f5c88070f30 legacy=0x7f5c88073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.445+0000 7f5c8d200640 1 -- 172.21.15.3:0/2900140075 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c880cda90 con 0x7f5c880cfdc0 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.446+0000 7f5c8d200640 1 --2- 172.21.15.3:0/2900140075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c880cfdc0 0x7f5c8815cb00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5c88075bd0 tx=0x7f5c8002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7837d256d0c715b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.446+0000 7f5c87400640 1 -- 172.21.15.3:0/2900140075 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80007740 con 0x7f5c880cfdc0 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c87400640 1 -- 172.21.15.3:0/2900140075 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c800078e0 con 0x7f5c880cfdc0 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c87400640 1 -- 172.21.15.3:0/2900140075 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80007cb0 con 0x7f5c880cfdc0 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 -- 172.21.15.3:0/2900140075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c880cfdc0 msgr2=0x7f5c8815cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 --2- 172.21.15.3:0/2900140075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c880cfdc0 0x7f5c8815cb00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5c88075bd0 tx=0x7f5c8002a680 comp rx=0 tx=0).stop 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 -- 172.21.15.3:0/2900140075 shutdown_connections 2024-09-10T07:53:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 --2- 172.21.15.3:0/2900140075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c880cfdc0 0x7f5c8815cb00 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 --2- 172.21.15.3:0/2900140075 >> v2:172.21.15.70:3300/0 conn(0x7f5c8815c720 0x7f5c88071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 -- 172.21.15.3:0/2900140075 >> 172.21.15.3:0/2900140075 conn(0x7f5c8806cfc0 msgr2=0x7f5c8806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 -- 172.21.15.3:0/2900140075 shutdown_connections 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.447+0000 7f5c8f200640 1 -- 172.21.15.3:0/2900140075 wait complete. 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.449+0000 7f5c8f200640 1 Processor -- start 2024-09-10T07:53:46.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.449+0000 7f5c8f200640 1 -- start start 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880cfdc0 0x7f5c8814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8815c720 0x7f5c8814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c88073d20 con 0x7f5c88070f30 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8806ed10 con 0x7f5c880cfdc0 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5c88073ea0 con 0x7f5c8815c720 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c87e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36236/0 (socket says 172.21.15.3:36236) 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c87e00640 1 -- 172.21.15.3:0/3680441159 learned_addr learned my addr 172.21.15.3:0/3680441159 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c8d200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8815c720 0x7f5c8814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.450+0000 7f5c7fe00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880cfdc0 0x7f5c8814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 -- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8815c720 msgr2=0x7f5c8814cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8815c720 0x7f5c8814cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 -- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880cfdc0 msgr2=0x7f5c8814bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880cfdc0 0x7f5c8814bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 -- 172.21.15.3:0/3680441159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8814c920 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c87e00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d830 tx=0x7f5c7800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800b6c0 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c78019070 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800ba80 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8814fd60 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.451+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c88150320 con 0x7f5c88070f30 2024-09-10T07:53:46.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.452+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c88071cd0 con 0x7f5c88070f30 2024-09-10T07:53:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.453+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c7801e460 con 0x7f5c88070f30 2024-09-10T07:53:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.454+0000 7f5c84c00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 0x7f5c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.454+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5c7808b3c0 con 0x7f5c88070f30 2024-09-10T07:53:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.454+0000 7f5c7fe00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 0x7f5c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.455+0000 7f5c7fe00640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 0x7f5c6406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5c88152b50 tx=0x7f5c70007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:46.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.459+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c78059720 con 0x7f5c88070f30 2024-09-10T07:53:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.714+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5c880cf2b0 con 0x7f5c88070f30 2024-09-10T07:53:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.715+0000 7f5c84c00640 1 -- 172.21.15.3:0/3680441159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1202 (secure 0 0 0) 0x7f5c7805da20 con 0x7f5c88070f30 2024-09-10T07:53:46.714 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:46.714 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":62,"data_bytes":662426,"bytes_used":271179776,"bytes_avail":772822933504,"bytes_total":773094113280,"write_bytes_sec":4094,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":111,"btime":"2024-09-10T07:53:46:679794+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-09-10T07:52:59.577824+0000","services":{}},"progress_events":{}} 2024-09-10T07:53:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 msgr2=0x7f5c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 0x7f5c6406e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5c88152b50 tx=0x7f5c70007e80 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 msgr2=0x7f5c8814a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d830 tx=0x7f5c7800dcf0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 shutdown_connections 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f5c6406be30 0x7f5c6406e2f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5c8815c720 0x7f5c8814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880cfdc0 0x7f5c8814bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 --2- 172.21.15.3:0/3680441159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88070f30 0x7f5c8814a8e0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 >> 172.21.15.3:0/3680441159 conn(0x7f5c8806cfc0 msgr2=0x7f5c8815eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 shutdown_connections 2024-09-10T07:53:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:46.717+0000 7f5c8f200640 1 -- 172.21.15.3:0/3680441159 wait complete. 2024-09-10T07:53:47.688 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:53:47.689+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:53:47.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.899+0000 7f0068200640 1 Processor -- start 2024-09-10T07:53:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.899+0000 7f0068200640 1 -- start start 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.900+0000 7f0068200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0060160710 0x7f00600cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.900+0000 7f0068200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 0x7f00600d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0068200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00600cfdd0 con 0x7f00600cf800 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0068200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f006015cc90 con 0x7f0060160710 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0068200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f006015cf70 con 0x7f0060160af0 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0060160710 0x7f00600cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0060160710 0x7f00600cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36238/0 (socket says 172.21.15.3:36238) 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 -- 172.21.15.3:0/2052504390 learned_addr learned my addr 172.21.15.3:0/2052504390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0064e00640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 0x7f00600d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 -- 172.21.15.3:0/2052504390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 msgr2=0x7f00600d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 0x7f00600d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 -- 172.21.15.3:0/2052504390 >> v1:172.21.15.70:6789/0 conn(0x7f00600cf800 legacy=0x7f006006bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0065800640 1 -- 172.21.15.3:0/2052504390 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006015d250 con 0x7f0060160710 2024-09-10T07:53:47.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.901+0000 7f0064e00640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 0x7f00600d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.902+0000 7f0065800640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0060160710 0x7f00600cf2c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0054009870 tx=0x7f005402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6c5e64dfa4bee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.902+0000 7f005fe00640 1 -- 172.21.15.3:0/2052504390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054037070 con 0x7f0060160710 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f005fe00640 1 -- 172.21.15.3:0/2052504390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054032440 con 0x7f0060160710 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f005fe00640 1 -- 172.21.15.3:0/2052504390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054032810 con 0x7f0060160710 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 -- 172.21.15.3:0/2052504390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0060160710 msgr2=0x7f00600cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0060160710 0x7f00600cf2c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0054009870 tx=0x7f005402a250 comp rx=0 tx=0).stop 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 -- 172.21.15.3:0/2052504390 shutdown_connections 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160af0 0x7f00600d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 --2- 172.21.15.3:0/2052504390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0060160710 0x7f00600cf2c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 -- 172.21.15.3:0/2052504390 >> 172.21.15.3:0/2052504390 conn(0x7f006006d000 msgr2=0x7f006006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 -- 172.21.15.3:0/2052504390 shutdown_connections 2024-09-10T07:53:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.903+0000 7f0068200640 1 -- 172.21.15.3:0/2052504390 wait complete. 2024-09-10T07:53:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.905+0000 7f0068200640 1 Processor -- start 2024-09-10T07:53:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.905+0000 7f0068200640 1 -- start start 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160710 0x7f00600b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060160af0 0x7f00600b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f006015d770 con 0x7f00600cf800 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0060073080 con 0x7f0060160af0 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0068200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00600b5450 con 0x7f0060160710 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0065800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0065800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36246/0 (socket says 172.21.15.3:36246) 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0065800640 1 -- 172.21.15.3:0/2529538696 learned_addr learned my addr 172.21.15.3:0/2529538696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0064e00640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160710 0x7f00600b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.906+0000 7f0066200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060160af0 0x7f00600b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 -- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160710 msgr2=0x7f00600b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160710 0x7f00600b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 -- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060160af0 msgr2=0x7f00600b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060160af0 0x7f00600b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 -- 172.21.15.3:0/2529538696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00600b35a0 con 0x7f00600cf800 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0065800640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f00540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054041020 con 0x7f00600cf800 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00540306c0 con 0x7f00600cf800 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054037040 con 0x7f00600cf800 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.907+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00600b3880 con 0x7f00600cf800 2024-09-10T07:53:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.908+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00600b4360 con 0x7f00600cf800 2024-09-10T07:53:47.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.908+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00601503d0 con 0x7f00600cf800 2024-09-10T07:53:47.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.909+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0054044020 con 0x7f00600cf800 2024-09-10T07:53:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.910+0000 7f005d600640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.910+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f00540ab620 con 0x7f00600cf800 2024-09-10T07:53:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.911+0000 7f0064e00640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:47.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.911+0000 7f0064e00640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 0x7f004006e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f006015fbf0 tx=0x7f004c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:47.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:47.915+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f005404b030 con 0x7f00600cf800 2024-09-10T07:53:48.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.132+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00600aed90 con 0x7f00600cf800 2024-09-10T07:53:48.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.133+0000 7f005d600640 1 -- 172.21.15.3:0/2529538696 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5269 (secure 0 0 0) 0x7f0054079fc0 con 0x7f00600cf800 2024-09-10T07:53:48.133 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:48.133 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":112,"btime":"2024-09-10T07:53:47:690474+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:47.280655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":6,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:48.133 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 112 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 msgr2=0x7f004006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 0x7f004006e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f006015fbf0 tx=0x7f004c008040 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 msgr2=0x7f006013edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f00540046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 shutdown_connections 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060160af0 0x7f00600b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0060160710 0x7f00600b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 --2- 172.21.15.3:0/2529538696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00600cf800 0x7f006013edc0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 >> 172.21.15.3:0/2529538696 conn(0x7f006006d000 msgr2=0x7f00600cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 shutdown_connections 2024-09-10T07:53:48.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.135+0000 7f0068200640 1 -- 172.21.15.3:0/2529538696 wait complete. 2024-09-10T07:53:48.150 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:48.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.305+0000 7f25ab200640 1 Processor -- start 2024-09-10T07:53:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.305+0000 7f25ab200640 1 -- start start 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25ab200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 0x7f25ac0d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25ab200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac0c9800 con 0x7f25ac06c230 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25ab200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac0d3300 con 0x7f25ac0d2cd0 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25ab200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25ac0d35e0 con 0x7f25ac15e420 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25aa800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25ac06c230 0x7f25ac15e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49722/0 (socket says 172.21.15.3:49722) 2024-09-10T07:53:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.306+0000 7f25aa800640 1 -- 172.21.15.3:0/507588496 learned_addr learned my addr 172.21.15.3:0/507588496 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9e00640 1 --2- 172.21.15.3:0/507588496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 0x7f25ac0d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324743137 0 0) 0x7f25ac0c9800 con 0x7f25ac06c230 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2590003520 con 0x7f25ac06c230 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359578763 0 0) 0x7f25ac0d3300 con 0x7f25ac0d2cd0 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ac0c9800 con 0x7f25ac0d2cd0 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 234184380 0 0) 0x7f2590003520 con 0x7f25ac06c230 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9e00640 1 -- 172.21.15.3:0/507588496 >> v1:172.21.15.70:6789/0 conn(0x7f25ac0d2cd0 legacy=0x7f25ac06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9e00640 1 -- 172.21.15.3:0/507588496 >> v1:172.21.15.3:6789/0 conn(0x7f25ac06c230 legacy=0x7f25ac15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.307+0000 7f25a9e00640 1 -- 172.21.15.3:0/507588496 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac0d38c0 con 0x7f25ac15e420 2024-09-10T07:53:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.308+0000 7f25a9e00640 1 --2- 172.21.15.3:0/507588496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 0x7f25ac0d2790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f259c0097d0 tx=0x7f259c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb49c7ace4a45c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c037070 con 0x7f25ac15e420 2024-09-10T07:53:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c032440 con 0x7f25ac15e420 2024-09-10T07:53:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25a9400640 1 -- 172.21.15.3:0/507588496 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c032810 con 0x7f25ac15e420 2024-09-10T07:53:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 -- 172.21.15.3:0/507588496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 msgr2=0x7f25ac0d2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 --2- 172.21.15.3:0/507588496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 0x7f25ac0d2790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f259c0097d0 tx=0x7f259c02a210 comp rx=0 tx=0).stop 2024-09-10T07:53:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 -- 172.21.15.3:0/507588496 shutdown_connections 2024-09-10T07:53:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 --2- 172.21.15.3:0/507588496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac15e420 0x7f25ac0d2790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 -- 172.21.15.3:0/507588496 >> 172.21.15.3:0/507588496 conn(0x7f25ac0732e0 msgr2=0x7f25ac062630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 -- 172.21.15.3:0/507588496 shutdown_connections 2024-09-10T07:53:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.309+0000 7f25ab200640 1 -- 172.21.15.3:0/507588496 wait complete. 2024-09-10T07:53:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.311+0000 7f25ab200640 1 Processor -- start 2024-09-10T07:53:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.311+0000 7f25ab200640 1 -- start start 2024-09-10T07:53:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac06c230 0x7f25ac153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25ac0d2cd0 0x7f25ac144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac169100 con 0x7f25ac15e420 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac0d3e80 con 0x7f25ac0d2cd0 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25ab200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f25ac169280 con 0x7f25ac06c230 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25aa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25aa800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36252/0 (socket says 172.21.15.3:36252) 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25aa800640 1 -- 172.21.15.3:0/514758600 learned_addr learned my addr 172.21.15.3:0/514758600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25a9e00640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac06c230 0x7f25ac153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.312+0000 7f25a3e00640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25ac0d2cd0 0x7f25ac144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 -- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac06c230 msgr2=0x7f25ac153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac06c230 0x7f25ac153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 -- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25ac0d2cd0 msgr2=0x7f25ac144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25ac0d2cd0 0x7f25ac144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 -- 172.21.15.3:0/514758600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac153760 con 0x7f25ac15e420 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25aa800640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c780 tx=0x7f25a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4018070 con 0x7f25ac15e420 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a400ce30 con 0x7f25ac15e420 2024-09-10T07:53:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a400a860 con 0x7f25ac15e420 2024-09-10T07:53:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.313+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1489f0 con 0x7f25ac15e420 2024-09-10T07:53:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.314+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac153a20 con 0x7f25ac15e420 2024-09-10T07:53:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.315+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac153c20 con 0x7f25ac15e420 2024-09-10T07:53:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.316+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25a401b070 con 0x7f25ac15e420 2024-09-10T07:53:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.316+0000 7f25a2000640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.316+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f25a408a340 con 0x7f25ac15e420 2024-09-10T07:53:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.317+0000 7f25a9e00640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.317+0000 7f25a9e00640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.321+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25ac153c20 con 0x7f25ac15e420 2024-09-10T07:53:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.574+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f25ac0d2790 con 0x7f25ac15e420 2024-09-10T07:53:48.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.575+0000 7f25a2000640 1 -- 172.21.15.3:0/514758600 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7f25a4058690 con 0x7f25ac15e420 2024-09-10T07:53:48.575 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:48.575 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":462,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":69,"data_bytes":700188,"bytes_used":271269888,"bytes_avail":772822843392,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9725,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":112,"btime":"2024-09-10T07:53:47:690474+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-09-10T07:53:47.589026+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 msgr2=0x7f258806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 msgr2=0x7f25ac144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c780 tx=0x7f25a400cc40 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 shutdown_connections 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25ac15e420 0x7f25ac144850 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25ac0d2cd0 0x7f25ac144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 --2- 172.21.15.3:0/514758600 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f25ac06c230 0x7f25ac153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 >> 172.21.15.3:0/514758600 conn(0x7f25ac0732e0 msgr2=0x7f25ac0b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 shutdown_connections 2024-09-10T07:53:48.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.577+0000 7f25ab200640 1 -- 172.21.15.3:0/514758600 wait complete. 2024-09-10T07:53:48.592 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:53:48.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:48.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub abort 2024-09-10T07:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.753+0000 7f5210a00640 1 Processor -- start 2024-09-10T07:53:48.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.753+0000 7f5210a00640 1 -- start start 2024-09-10T07:53:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.754+0000 7f5210a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.754+0000 7f5210a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 0x7f520c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.754+0000 7f5210a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c0b3b20 con 0x7f520c0b48a0 2024-09-10T07:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.754+0000 7f5210a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f520c0d0cb0 con 0x7f520c0d0840 2024-09-10T07:53:48.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.754+0000 7f5210a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f520c0b3dc0 con 0x7f520c1521d0 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 0x7f520c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 0x7f520c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46032/0 (socket says 172.21.15.3:46032) 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:33040/0 (socket says 172.21.15.3:33040) 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520be00640 1 -- 172.21.15.3:0/3012964360 learned_addr learned my addr 172.21.15.3:0/3012964360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 -- 172.21.15.3:0/3012964360 >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 msgr2=0x7f520c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 --2- 172.21.15.3:0/3012964360 >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 0x7f520c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 -- 172.21.15.3:0/3012964360 >> v1:172.21.15.70:6789/0 conn(0x7f520c0b48a0 legacy=0x7f520c0d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.755+0000 7f520b400640 1 -- 172.21.15.3:0/3012964360 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c0b40a0 con 0x7f520c1521d0 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.756+0000 7f520b400640 1 --2- 172.21.15.3:0/3012964360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0097a0 tx=0x7f51fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161841504c6f6b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f520aa00640 1 -- 172.21.15.3:0/3012964360 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc037070 con 0x7f520c1521d0 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f520aa00640 1 -- 172.21.15.3:0/3012964360 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc02ae50 con 0x7f520c1521d0 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f520aa00640 1 -- 172.21.15.3:0/3012964360 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc032660 con 0x7f520c1521d0 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 -- 172.21.15.3:0/3012964360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 msgr2=0x7f520c0cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 --2- 172.21.15.3:0/3012964360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0097a0 tx=0x7f51fc02a100 comp rx=0 tx=0).stop 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 -- 172.21.15.3:0/3012964360 shutdown_connections 2024-09-10T07:53:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 --2- 172.21.15.3:0/3012964360 >> v2:172.21.15.70:3300/0 conn(0x7f520c0d0840 0x7f520c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 --2- 172.21.15.3:0/3012964360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c1521d0 0x7f520c0cf150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 -- 172.21.15.3:0/3012964360 >> 172.21.15.3:0/3012964360 conn(0x7f520c0b1e90 msgr2=0x7f520c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 -- 172.21.15.3:0/3012964360 shutdown_connections 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.757+0000 7f5210a00640 1 -- 172.21.15.3:0/3012964360 wait complete. 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.759+0000 7f5210a00640 1 Processor -- start 2024-09-10T07:53:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.759+0000 7f5210a00640 1 -- start start 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c0d0840 0x7f520c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 0x7f520c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1525b0 con 0x7f520c0b48a0 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f520c0d0ed0 con 0x7f520c1521d0 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5210a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f520c152730 con 0x7f520c0d0840 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f520b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f520b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36258/0 (socket says 172.21.15.3:36258) 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f520b400640 1 -- 172.21.15.3:0/1526248475 learned_addr learned my addr 172.21.15.3:0/1526248475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.760+0000 7f5203e00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c0d0840 0x7f520c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520be00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 0x7f520c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 -- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c0d0840 msgr2=0x7f520c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c0d0840 0x7f520c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 -- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 msgr2=0x7f520c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 0x7f520c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 -- 172.21.15.3:0/1526248475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c153440 con 0x7f520c0b48a0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520be00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 0x7f520c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f520b400640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f51fc035040 tx=0x7f51fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f5203400640 1 -- 172.21.15.3:0/1526248475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc041070 con 0x7f520c0b48a0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f5203400640 1 -- 172.21.15.3:0/1526248475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc03c460 con 0x7f520c0b48a0 2024-09-10T07:53:48.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f5203400640 1 -- 172.21.15.3:0/1526248475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc037040 con 0x7f520c0b48a0 2024-09-10T07:53:48.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c153720 con 0x7f520c0b48a0 2024-09-10T07:53:48.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.761+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c153ce0 con 0x7f520c0b48a0 2024-09-10T07:53:48.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.763+0000 7f5203400640 1 -- 172.21.15.3:0/1526248475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51fc03a070 con 0x7f520c0b48a0 2024-09-10T07:53:48.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.764+0000 7f5203400640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 0x7f51e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.764+0000 7f5203400640 1 -- 172.21.15.3:0/1526248475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f51fc0ab240 con 0x7f520c0b48a0 2024-09-10T07:53:48.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.765+0000 7f5203e00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 0x7f51e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.765+0000 7f5215053740 1 Processor -- start 2024-09-10T07:53:48.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.765+0000 7f5215053740 1 -- start start 2024-09-10T07:53:48.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.765+0000 7f5203e00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 0x7f51e806e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f520c151310 tx=0x7f51f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b79a3e0 0x55b82b793240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b82b75fcf0 con 0x55b82b796410 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b82b6fb7f0 con 0x55b82b798c50 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b82b6a3800 con 0x55b82b79a3e0 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36260/0 (socket says 172.21.15.3:36260) 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 -- 172.21.15.3:0/752219666 learned_addr learned my addr 172.21.15.3:0/752219666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f520b400640 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f520be00640 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b79a3e0 0x55b82b793240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 -- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b79a3e0 msgr2=0x55b82b793240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b79a3e0 0x55b82b793240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 -- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 msgr2=0x55b82b7de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.766+0000 7f5203e00640 1 -- 172.21.15.3:0/752219666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82b4d87d0 con 0x55b82b796410 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.767+0000 7f5203e00640 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f51f40052c0 tx=0x7f51f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf7031285fb2346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.767+0000 7f51fb400640 1 -- 172.21.15.3:0/752219666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f400e500 con 0x55b82b796410 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.767+0000 7f51fb400640 1 -- 172.21.15.3:0/752219666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f400e6a0 con 0x55b82b796410 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.767+0000 7f51fb400640 1 -- 172.21.15.3:0/752219666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f400ea50 con 0x55b82b796410 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 -- 172.21.15.3:0/752219666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 msgr2=0x55b82b799030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f51f40052c0 tx=0x7f51f4011000 comp rx=0 tx=0).stop 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 -- 172.21.15.3:0/752219666 shutdown_connections 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b79a3e0 0x55b82b793240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b796410 0x55b82b799030 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 --2- 172.21.15.3:0/752219666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 -- 172.21.15.3:0/752219666 >> 172.21.15.3:0/752219666 conn(0x55b82b795410 msgr2=0x55b82b7dd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 -- 172.21.15.3:0/752219666 shutdown_connections 2024-09-10T07:53:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.768+0000 7f5215053740 1 -- 172.21.15.3:0/752219666 wait complete. 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.769+0000 7f5215053740 1 -- start start 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.769+0000 7f5215053740 1 Processor -- start 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b82b684750 con 0x55b82b79a3e0 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55b82b6f77b0 con 0x55b82b798c50 2024-09-10T07:53:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b82b7948b0 con 0x55b82b7c1160 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5215053740 10 client.0 _ll_register_callbacks cb 0x55b82b40dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36276/0 (socket says 172.21.15.3:36276) 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 -- 172.21.15.3:0/3389072026 learned_addr learned my addr 172.21.15.3:0/3389072026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f520b400640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f520be00640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 -- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 msgr2=0x55b82b7bc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7bc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 -- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 msgr2=0x55b82b7bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.770+0000 7f5203e00640 1 -- 172.21.15.3:0/3389072026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82b3ce0f0 con 0x55b82b79a3e0 2024-09-10T07:53:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f5203e00640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f51f4003e50 tx=0x7f51f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cf5d94f8d49ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4021920 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f51f4021ac0 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f400e6a0 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f51f4030050 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f5215053740 10 client.8523 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55b82b538f60 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f400e6a0 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.771+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7f51f4033020 con 0x55b82b79a3e0 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 10 client.8523 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:48.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 10 client.8523 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 20 client.8523 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7f58e0 0x55b82b7f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 4 client.8523 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82b708f20 con 0x55b82b7f58e0 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f520b400640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7f58e0 0x55b82b7f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f520b400640 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7f58e0 0x55b82b7f5cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d17c5d1f32891c server_cookie=1a6cbb12c7292940 in_seq=0 out_seq=0 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.772+0000 7f51efe00640 10 client.8523 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f51efe00640 1 -- 172.21.15.3:0/3389072026 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b82b708f20 con 0x55b82b7f58e0 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f51efe00640 10 client.8523 handle_command_reply: tid=0 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 client.8523 shutdown 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 msgr2=0x55b82b7bbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f51f4003e50 tx=0x7f51f4013000 comp rx=0 tx=0).stop 2024-09-10T07:53:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 shutdown_connections 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7f58e0 0x55b82b7f5cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7bc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7bbc90 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f51efe00640 0 client.8523 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 --2- 172.21.15.3:0/3389072026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 >> 172.21.15.3:0/3389072026 conn(0x55b82b796410 msgr2=0x55b82b7db940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 shutdown_connections 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.773+0000 7f5215053740 1 -- 172.21.15.3:0/3389072026 wait complete. 2024-09-10T07:53:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.774+0000 7f5215053740 20 client.8523 trim_cache size 0 max 16384 2024-09-10T07:53:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.791+0000 7f5215053740 1 Processor -- start 2024-09-10T07:53:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 -- start start 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7ba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520c167fc0 con 0x55b82b79a3e0 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f520c167160 con 0x55b82b798c50 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.792+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f520c167890 con 0x55b82b7c1160 2024-09-10T07:53:48.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36292/0 (socket says 172.21.15.3:36292) 2024-09-10T07:53:48.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 -- 172.21.15.3:0/1222840588 learned_addr learned my addr 172.21.15.3:0/1222840588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 -- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 msgr2=0x55b82b7c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 -- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 msgr2=0x55b82b7ba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 -- 172.21.15.3:0/1222840588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82b3a71b0 con 0x55b82b79a3e0 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f5203e00640 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f51f40079f0 tx=0x7f51f401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e1aeb1f9a4f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.793+0000 7f51faa00640 1 -- 172.21.15.3:0/1222840588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4021720 con 0x55b82b79a3e0 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f51faa00640 1 -- 172.21.15.3:0/1222840588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f40218c0 con 0x55b82b79a3e0 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f51faa00640 1 -- 172.21.15.3:0/1222840588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f400f040 con 0x55b82b79a3e0 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 -- 172.21.15.3:0/1222840588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 msgr2=0x55b82b7c0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f51f40079f0 tx=0x7f51f401d000 comp rx=0 tx=0).stop 2024-09-10T07:53:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 -- 172.21.15.3:0/1222840588 shutdown_connections 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b79a3e0 0x55b82b7c0630 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 --2- 172.21.15.3:0/1222840588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b798c50 0x55b82b7ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 -- 172.21.15.3:0/1222840588 >> 172.21.15.3:0/1222840588 conn(0x55b82b796410 msgr2=0x55b82b7ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 -- 172.21.15.3:0/1222840588 shutdown_connections 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.794+0000 7f5215053740 1 -- 172.21.15.3:0/1222840588 wait complete. 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.795+0000 7f5215053740 1 -- start start 2024-09-10T07:53:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.795+0000 7f5215053740 1 Processor -- start 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b79a3e0 0x55b82b7dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b82b7c36b0 con 0x55b82b798c50 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f520c15c460 con 0x55b82b79a3e0 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55b82b708f20 con 0x55b82b7c1160 2024-09-10T07:53:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5215053740 10 client.0 _ll_register_callbacks cb 0x7f51f4005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f520b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f520b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36300/0 (socket says 172.21.15.3:36300) 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f520b400640 1 -- 172.21.15.3:0/2479965771 learned_addr learned my addr 172.21.15.3:0/2479965771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f520be00640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.796+0000 7f5203e00640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b79a3e0 0x55b82b7dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 -- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 msgr2=0x55b82b7da0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7da0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 -- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b79a3e0 msgr2=0x55b82b7dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b79a3e0 0x55b82b7dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 -- 172.21.15.3:0/2479965771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82b7bd3e0 con 0x55b82b798c50 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f520b400640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f51fc079150 tx=0x7f51fc006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943980f3459e4479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f51efe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc04a040 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f51fc0a92d0 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f40218c0 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f51fc07d800 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f5215053740 10 client.8529 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.797+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55b82b792dd0 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f40218c0 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7f51fc04d060 con 0x55b82b798c50 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 10 client.8529 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 10 client.8529 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 20 client.8529 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7d7050 0x55b82b7da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 4 client.8529 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-10T07:53:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55b82b703270 con 0x55b82b7d7050 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5203e00640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7d7050 0x55b82b7da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f5203e00640 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7d7050 0x55b82b7da5f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c7b35ad484d0fa server_cookie=92c3a4f409f8ec9e in_seq=0 out_seq=0 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.798+0000 7f51efe00640 10 client.8529 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f51efe00640 1 -- 172.21.15.3:0/2479965771 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b82b703270 con 0x55b82b7d7050 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f51efe00640 10 client.8529 handle_command_reply: tid=0 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 client.8529 shutdown 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 msgr2=0x55b82b7db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f51fc079150 tx=0x7f51fc006d70 comp rx=0 tx=0).stop 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 shutdown_connections 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55b82b7d7050 0x55b82b7da5f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55b82b7c1160 0x55b82b7da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b82b79a3e0 0x55b82b7dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 --2- 172.21.15.3:0/2479965771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b82b798c50 0x55b82b7db940 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f51efe00640 0 client.8529 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 >> 172.21.15.3:0/2479965771 conn(0x55b82b796410 msgr2=0x55b82b7c0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 shutdown_connections 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 1 -- 172.21.15.3:0/2479965771 wait complete. 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.799+0000 7f5215053740 20 client.8529 trim_cache size 0 max 16384 2024-09-10T07:53:48.804 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stdout: "message": "", 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 msgr2=0x7f51e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 0x7f51e806e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f520c151310 tx=0x7f51f4009040 comp rx=0 tx=0).stop 2024-09-10T07:53:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 msgr2=0x7f520c149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f51fc035040 tx=0x7f51fc0046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 shutdown_connections 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f51e806be30 0x7f51e806e2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f520c1521d0 0x7f520c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f520c0d0840 0x7f520c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 --2- 172.21.15.3:0/1526248475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f520c0b48a0 0x7f520c149580 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 >> 172.21.15.3:0/1526248475 conn(0x7f520c0b1e90 msgr2=0x7f520c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 shutdown_connections 2024-09-10T07:53:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.800+0000 7f5210a00640 1 -- 172.21.15.3:0/1526248475 wait complete. 2024-09-10T07:53:48.814 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:48.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-10T07:53:48.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.972+0000 7fbe3e600640 1 Processor -- start 2024-09-10T07:53:48.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.973+0000 7fbe3e600640 1 -- start start 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe3e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe381451d0 0x7fbe3806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe3e600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe38145730 0x7fbe380b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe3e600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe380cd900 con 0x7fbe380cf270 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe3e600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3806d7c0 con 0x7fbe38145730 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe3e600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe380cdbe0 con 0x7fbe381451d0 2024-09-10T07:53:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe37e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe380cf270 0x7fbe380cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49738/0 (socket says 172.21.15.3:49738) 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe2ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe38145730 0x7fbe380b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.974+0000 7fbe37e00640 1 -- 172.21.15.3:0/2715867294 learned_addr learned my addr 172.21.15.3:0/2715867294 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe37400640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe381451d0 0x7fbe3806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2715867294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe381451d0 msgr2=0x7fbe3806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe381451d0 0x7fbe3806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2715867294 >> v1:172.21.15.3:6789/0 conn(0x7fbe380cf270 legacy=0x7fbe380cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2715867294 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3806da10 con 0x7fbe38145730 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.975+0000 7fbe36a00640 1 -- 172.21.15.3:0/2715867294 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856666337 0 0) 0x7fbe380cd900 con 0x7fbe380cf270 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38145730 0x7fbe380b3370 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbe200097a0 tx=0x7fbe2002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59e09fddecea4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe36a00640 1 -- 172.21.15.3:0/2715867294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20037070 con 0x7fbe38145730 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe36a00640 1 -- 172.21.15.3:0/2715867294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2002ae50 con 0x7fbe38145730 2024-09-10T07:53:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe36a00640 1 -- 172.21.15.3:0/2715867294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20032660 con 0x7fbe38145730 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe3e600640 1 -- 172.21.15.3:0/2715867294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38145730 msgr2=0x7fbe380b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.976+0000 7fbe3e600640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38145730 0x7fbe380b3370 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbe200097a0 tx=0x7fbe2002a100 comp rx=0 tx=0).stop 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 -- 172.21.15.3:0/2715867294 shutdown_connections 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38145730 0x7fbe380b3370 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 --2- 172.21.15.3:0/2715867294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe381451d0 0x7fbe3806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 -- 172.21.15.3:0/2715867294 >> 172.21.15.3:0/2715867294 conn(0x7fbe380b5cb0 msgr2=0x7fbe3806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 -- 172.21.15.3:0/2715867294 shutdown_connections 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.977+0000 7fbe3e600640 1 -- 172.21.15.3:0/2715867294 wait complete. 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.978+0000 7fbe3e600640 1 Processor -- start 2024-09-10T07:53:48.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.978+0000 7fbe3e600640 1 -- start start 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 0x7fbe3815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 0x7fbe380b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3806df30 con 0x7fbe381451d0 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbe380b38b0 con 0x7fbe380cf270 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.979+0000 7fbe3e600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3806e0b0 con 0x7fbe38145730 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36314/0 (socket says 172.21.15.3:36314) 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 -- 172.21.15.3:0/928970880 learned_addr learned my addr 172.21.15.3:0/928970880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe37e00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 0x7fbe380b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe37400640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 0x7fbe3815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 -- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 msgr2=0x7fbe380b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 0x7fbe380b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 -- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 msgr2=0x7fbe3815bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 0x7fbe3815bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 -- 172.21.15.3:0/928970880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3815e3a0 con 0x7fbe381451d0 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe37e00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 0x7fbe380b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe37400640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 0x7fbe3815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.980+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbe200097a0 tx=0x7fbe200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.981+0000 7fbe2fe00640 1 -- 172.21.15.3:0/928970880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20037040 con 0x7fbe381451d0 2024-09-10T07:53:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.981+0000 7fbe2fe00640 1 -- 172.21.15.3:0/928970880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20004030 con 0x7fbe381451d0 2024-09-10T07:53:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.981+0000 7fbe2fe00640 1 -- 172.21.15.3:0/928970880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20004420 con 0x7fbe381451d0 2024-09-10T07:53:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.981+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3815e680 con 0x7fbe381451d0 2024-09-10T07:53:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.981+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3815ec40 con 0x7fbe381451d0 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.983+0000 7fbe2fe00640 1 -- 172.21.15.3:0/928970880 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe20003610 con 0x7fbe381451d0 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.984+0000 7fbe2fe00640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 0x7fbe1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.984+0000 7fbe2fe00640 1 -- 172.21.15.3:0/928970880 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbe200abcb0 con 0x7fbe381451d0 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.984+0000 7fbe37400640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 0x7fbe1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.984+0000 7fbe41641740 1 Processor -- start 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.985+0000 7fbe41641740 1 -- start start 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.985+0000 7fbe37400640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 0x7fbe1406e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbe28006fd0 tx=0x7fbe28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.985+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b14b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe41641740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe41641740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56084b068660 con 0x56084b100480 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56084b010680 con 0x56084b11a420 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56084b064620 con 0x56084b11a800 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36316/0 (socket says 172.21.15.3:36316) 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 -- 172.21.15.3:0/1215774905 learned_addr learned my addr 172.21.15.3:0/1215774905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37400640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 -- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 msgr2=0x56084b14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 -- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 msgr2=0x56084b14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 -- 172.21.15.3:0/1215774905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084ae457d0 con 0x56084b100480 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37400640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.986+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbe30001e90 tx=0x7fbe3000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0da7758ca6ec50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe27e00640 1 -- 172.21.15.3:0/1215774905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300020a0 con 0x56084b100480 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe27e00640 1 -- 172.21.15.3:0/1215774905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe30004b60 con 0x56084b100480 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe27e00640 1 -- 172.21.15.3:0/1215774905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300070d0 con 0x56084b100480 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 -- 172.21.15.3:0/1215774905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 msgr2=0x56084b14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbe30001e90 tx=0x7fbe3000d520 comp rx=0 tx=0).stop 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 -- 172.21.15.3:0/1215774905 shutdown_connections 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b100480 0x56084b14b740 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 --2- 172.21.15.3:0/1215774905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.987+0000 7fbe41641740 1 -- 172.21.15.3:0/1215774905 >> 172.21.15.3:0/1215774905 conn(0x56084b101220 msgr2=0x56084b149840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.988+0000 7fbe41641740 1 -- 172.21.15.3:0/1215774905 shutdown_connections 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.988+0000 7fbe41641740 1 -- 172.21.15.3:0/1215774905 wait complete. 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.988+0000 7fbe41641740 1 -- start start 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.988+0000 7fbe41641740 1 Processor -- start 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b13fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56084b0ff3e0 con 0x56084b120710 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56084aff14c0 con 0x56084b11a420 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56084b0ff6c0 con 0x56084b11a800 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.989+0000 7fbe41641740 10 client.0 _ll_register_callbacks cb 0x56084b060e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36332/0 (socket says 172.21.15.3:36332) 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 -- 172.21.15.3:0/2642878931 learned_addr learned my addr 172.21.15.3:0/2642878931 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37400640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b13fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 -- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 msgr2=0x56084b13fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b13fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 -- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 msgr2=0x56084b146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 -- 172.21.15.3:0/2642878931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084ad7b3f0 con 0x56084b120710 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe37e00640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000da20 tx=0x7fbe30001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db462e254be9f8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.990+0000 7fbe24c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30023020 con 0x56084b120710 2024-09-10T07:53:48.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe30004b60 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbe300019a0 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fbe30007990 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 10 client.8544 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x56084ad3b0f0 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe30004b60 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fbe3001f470 con 0x56084b120710 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 10 client.8544 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 10 client.8544 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 20 client.8544 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b15d970 0x56084b15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 4 client.8544 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.991+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084b075d90 con 0x56084b15d970 2024-09-10T07:53:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.992+0000 7fbe37400640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b15d970 0x56084b15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:48.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.992+0000 7fbe37400640 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b15d970 0x56084b15dd80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97d898bfbacb1db server_cookie=1c89cfe2c539ee7 in_seq=0 out_seq=0 2024-09-10T07:53:48.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.992+0000 7fbe24c00640 10 client.8544 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe24c00640 1 -- 172.21.15.3:0/2642878931 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56084b075d90 con 0x56084b15d970 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe24c00640 10 client.8544 handle_command_reply: tid=0 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 client.8544 shutdown 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 msgr2=0x56084b120b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000da20 tx=0x7fbe30001630 comp rx=0 tx=0).stop 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 shutdown_connections 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b15d970 0x56084b15dd80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b120710 0x56084b120b90 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 --2- 172.21.15.3:0/2642878931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b11a420 0x56084b146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:48.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe24c00640 0 client.8544 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 >> 172.21.15.3:0/2642878931 conn(0x56084b100480 msgr2=0x56084b147ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 shutdown_connections 2024-09-10T07:53:49.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 1 -- 172.21.15.3:0/2642878931 wait complete. 2024-09-10T07:53:49.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:48.993+0000 7fbe41641740 20 client.8544 trim_cache size 0 max 16384 2024-09-10T07:53:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 Processor -- start 2024-09-10T07:53:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 -- start start 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a420 0x56084b0fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b11fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38200c60 con 0x56084b11a800 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ffe00 con 0x56084b120710 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.012+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbe38200530 con 0x56084b11a420 2024-09-10T07:53:49.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36338/0 (socket says 172.21.15.3:36338) 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2478896936 learned_addr learned my addr 172.21.15.3:0/2478896936 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe37e00640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b11fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe37400640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a420 0x56084b0fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a420 msgr2=0x56084b0fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a420 0x56084b0fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 msgr2=0x56084b11fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b11fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 -- 172.21.15.3:0/2478896936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084b0fa5e0 con 0x56084b11a800 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe37e00640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b11fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.013+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbe200aa140 tx=0x7fbe200761f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3783850e5947fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe27400640 1 -- 172.21.15.3:0/2478896936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2007e300 con 0x56084b11a800 2024-09-10T07:53:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe27400640 1 -- 172.21.15.3:0/2478896936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe200a8070 con 0x56084b11a800 2024-09-10T07:53:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe27400640 1 -- 172.21.15.3:0/2478896936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20082b30 con 0x56084b11a800 2024-09-10T07:53:49.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 -- 172.21.15.3:0/2478896936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 msgr2=0x56084b146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbe200aa140 tx=0x7fbe200761f0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 -- 172.21.15.3:0/2478896936 shutdown_connections 2024-09-10T07:53:49.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b11fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a800 0x56084b146ed0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 --2- 172.21.15.3:0/2478896936 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a420 0x56084b0fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 -- 172.21.15.3:0/2478896936 >> 172.21.15.3:0/2478896936 conn(0x56084b100480 msgr2=0x56084b147ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 -- 172.21.15.3:0/2478896936 shutdown_connections 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.014+0000 7fbe41641740 1 -- 172.21.15.3:0/2478896936 wait complete. 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.015+0000 7fbe41641740 1 -- start start 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.015+0000 7fbe41641740 1 Processor -- start 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b140750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38133160 con 0x56084b11a420 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56084b0fad50 con 0x56084b120710 2024-09-10T07:53:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56084b164c60 con 0x56084b11a800 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe41641740 10 client.0 _ll_register_callbacks cb 0x7fbe30001ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe37400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe37400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36340/0 (socket says 172.21.15.3:36340) 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe37400640 1 -- 172.21.15.3:0/3132120879 learned_addr learned my addr 172.21.15.3:0/3132120879 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe37e00640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.016+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 -- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 msgr2=0x56084b15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 -- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 msgr2=0x56084b140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 -- 172.21.15.3:0/3132120879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084b13fcb0 con 0x56084b11a420 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe37400640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ecd0 tx=0x7fbe28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266bd2f6c81fc976 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe24c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28032060 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbe28005990 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe200a8070 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fbe2800f040 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe41641740 10 client.8550 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.017+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x56084b14da90 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe200a8070 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fbe28012030 con 0x56084b11a420 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 10 client.8550 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 10 client.8550 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 20 client.8550 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b0fa810 0x56084b16d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 4 client.8550 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56084b0700e0 con 0x56084b0fa810 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.018+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b0fa810 0x56084b16d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe2ea00640 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b0fa810 0x56084b16d940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d31f8e81ad0edefb server_cookie=37ce96b0d8d4deb5 in_seq=0 out_seq=0 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe24c00640 10 client.8550 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe24c00640 1 -- 172.21.15.3:0/3132120879 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56084b0700e0 con 0x56084b0fa810 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe24c00640 10 client.8550 handle_command_reply: tid=0 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 client.8550 shutdown 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 msgr2=0x56084b1caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ecd0 tx=0x7fbe28004730 comp rx=0 tx=0).stop 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 shutdown_connections 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x56084b0fa810 0x56084b16d940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56084b120710 0x56084b140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56084b11a800 0x56084b15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 --2- 172.21.15.3:0/3132120879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56084b11a420 0x56084b1caed0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe24c00640 0 client.8550 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 >> 172.21.15.3:0/3132120879 conn(0x56084b100480 msgr2=0x56084b149a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 shutdown_connections 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 1 -- 172.21.15.3:0/3132120879 wait complete. 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.019+0000 7fbe41641740 20 client.8550 trim_cache size 0 max 16384 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:49.024 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.025 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 msgr2=0x7fbe1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 0x7fbe1406e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbe28006fd0 tx=0x7fbe28008040 comp rx=0 tx=0).stop 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 msgr2=0x7fbe381e66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbe200097a0 tx=0x7fbe200046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 shutdown_connections 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbe1406be30 0x7fbe1406e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe38145730 0x7fbe380b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe381451d0 0x7fbe381e66e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 --2- 172.21.15.3:0/928970880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe380cf270 0x7fbe3815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 >> 172.21.15.3:0/928970880 conn(0x7fbe380b5cb0 msgr2=0x7fbe381440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 shutdown_connections 2024-09-10T07:53:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.020+0000 7fbe3e600640 1 -- 172.21.15.3:0/928970880 wait complete. 2024-09-10T07:53:49.035 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:49.035 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:49.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-10T07:53:49.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.193+0000 7fa427800640 1 Processor -- start 2024-09-10T07:53:49.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.193+0000 7fa427800640 1 -- start start 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa427800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4200cf980 0x7fa42006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa427800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4200cd900 con 0x7fa420145730 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa427800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4200cdbe0 con 0x7fa4201451d0 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa427800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa42006d7c0 con 0x7fa4200cf980 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa41fe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa420145730 0x7fa4200cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49746/0 (socket says 172.21.15.3:49746) 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa41fe00640 1 -- 172.21.15.3:0/2902845522 learned_addr learned my addr 172.21.15.3:0/2902845522 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa425800640 1 --2- 172.21.15.3:0/2902845522 >> v2:172.21.15.70:3300/0 conn(0x7fa4200cf980 0x7fa42006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578710738 0 0) 0x7fa4200cd900 con 0x7fa420145730 2024-09-10T07:53:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.195+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa408003520 con 0x7fa420145730 2024-09-10T07:53:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1164464986 0 0) 0x7fa408003520 con 0x7fa420145730 2024-09-10T07:53:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 >> v2:172.21.15.70:3300/0 conn(0x7fa4200cf980 msgr2=0x7fa42006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 --2- 172.21.15.3:0/2902845522 >> v2:172.21.15.70:3300/0 conn(0x7fa4200cf980 0x7fa42006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 >> v1:172.21.15.70:6789/0 conn(0x7fa4201451d0 legacy=0x7fa4200b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42006da10 con 0x7fa420145730 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fa40c001670 con 0x7fa420145730 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 >> v1:172.21.15.3:6789/0 conn(0x7fa420145730 legacy=0x7fa4200cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4080039b0 0x7fa408003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa408004800 0x7fa408004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa40800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa42006d7c0 con 0x7fa4080051c0 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa40800c0f0 con 0x7fa408004800 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa40800c2f0 con 0x7fa4080039b0 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa425800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa40800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa41fe00640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa408004800 0x7fa408004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.196+0000 7fa424e00640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4080039b0 0x7fa408003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 -- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4080039b0 msgr2=0x7fa408003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4080039b0 0x7fa408003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 -- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa408004800 msgr2=0x7fa408004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa408004800 0x7fa408004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 -- 172.21.15.3:0/2902845522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa408003520 con 0x7fa4080051c0 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.197+0000 7fa425800640 1 -- 172.21.15.3:0/2902845522 reap_dead start 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa425800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa40800b9c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa418009fc0 tx=0x7fa408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa2ae07c415be99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418035020 con 0x7fa4080051c0 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa418003bd0 con 0x7fa4080051c0 2024-09-10T07:53:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa41f400640 1 -- 172.21.15.3:0/2902845522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418009c10 con 0x7fa4080051c0 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa427800640 1 -- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 msgr2=0x7fa40800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa427800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa40800b9c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa418009fc0 tx=0x7fa408005830 comp rx=0 tx=0).stop 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa427800640 1 -- 172.21.15.3:0/2902845522 shutdown_connections 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa427800640 1 --2- 172.21.15.3:0/2902845522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa40800b9c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.198+0000 7fa427800640 1 -- 172.21.15.3:0/2902845522 >> 172.21.15.3:0/2902845522 conn(0x7fa4200b5cb0 msgr2=0x7fa42006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.199+0000 7fa427800640 1 -- 172.21.15.3:0/2902845522 shutdown_connections 2024-09-10T07:53:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.199+0000 7fa427800640 1 -- 172.21.15.3:0/2902845522 wait complete. 2024-09-10T07:53:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.200+0000 7fa427800640 1 Processor -- start 2024-09-10T07:53:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.200+0000 7fa427800640 1 -- start start 2024-09-10T07:53:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 0x7fa4200ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 0x7fa42015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa42006d140 con 0x7fa4080051c0 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa42006cfc0 con 0x7fa42015eb60 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa427800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa42006d2c0 con 0x7fa4200ad3a0 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa424e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa424e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36356/0 (socket says 172.21.15.3:36356) 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.201+0000 7fa424e00640 1 -- 172.21.15.3:0/1334791815 learned_addr learned my addr 172.21.15.3:0/1334791815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa425800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 0x7fa42015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 0x7fa4200ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 -- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 msgr2=0x7fa4200ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 0x7fa4200ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 -- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 msgr2=0x7fa42015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 0x7fa42015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 -- 172.21.15.3:0/1334791815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42015fb60 con 0x7fa4080051c0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa425800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 0x7fa42015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 0x7fa4200ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa424e00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa41400f7f0 tx=0x7fa41400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa41cc00640 1 -- 172.21.15.3:0/1334791815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400fea0 con 0x7fa4080051c0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa41cc00640 1 -- 172.21.15.3:0/1334791815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa414018070 con 0x7fa4080051c0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa41cc00640 1 -- 172.21.15.3:0/1334791815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400d6d0 con 0x7fa4080051c0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.202+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42015f880 con 0x7fa4080051c0 2024-09-10T07:53:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.203+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4200aae00 con 0x7fa4080051c0 2024-09-10T07:53:49.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.204+0000 7fa41cc00640 1 -- 172.21.15.3:0/1334791815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa41400d870 con 0x7fa4080051c0 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.205+0000 7fa41cc00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 0x7fa40006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.205+0000 7fa41cc00640 1 -- 172.21.15.3:0/1334791815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa41408ad10 con 0x7fa4080051c0 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.206+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 0x7fa40006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.206+0000 7fa42a626740 1 Processor -- start 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.206+0000 7fa42a626740 1 -- start start 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.206+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 0x7fa40006e320 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa42015fff0 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44f490 0x55871e47df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e480740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55871e39d660 con 0x55871e44fbc0 2024-09-10T07:53:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55871e345680 con 0x55871e44f490 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55871e399620 con 0x55871e47fc20 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36370/0 (socket says 172.21.15.3:36370) 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.207+0000 7fa41fe00640 1 -- 172.21.15.3:0/1989092077 learned_addr learned my addr 172.21.15.3:0/1989092077 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 -- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 msgr2=0x55871e480740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e480740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 -- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44f490 msgr2=0x55871e47df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44f490 0x55871e47df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 -- 172.21.15.3:0/1989092077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871e17a7d0 con 0x55871e44fbc0 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa41fe00640 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0073b0 tx=0x7fa40c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afad6e5d079e3f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa411600640 1 -- 172.21.15.3:0/1989092077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c028070 con 0x55871e44fbc0 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa411600640 1 -- 172.21.15.3:0/1989092077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c015c40 con 0x55871e44fbc0 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.208+0000 7fa411600640 1 -- 172.21.15.3:0/1989092077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0143d0 con 0x55871e44fbc0 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 -- 172.21.15.3:0/1989092077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 msgr2=0x55871e480200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0073b0 tx=0x7fa40c014000 comp rx=0 tx=0).stop 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 -- 172.21.15.3:0/1989092077 shutdown_connections 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e480740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e480200 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 --2- 172.21.15.3:0/1989092077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44f490 0x55871e47df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 -- 172.21.15.3:0/1989092077 >> 172.21.15.3:0/1989092077 conn(0x55871e436220 msgr2=0x55871e47e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 -- 172.21.15.3:0/1989092077 shutdown_connections 2024-09-10T07:53:49.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.209+0000 7fa42a626740 1 -- 172.21.15.3:0/1989092077 wait complete. 2024-09-10T07:53:49.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.210+0000 7fa42a626740 1 -- start start 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.210+0000 7fa42a626740 1 Processor -- start 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e466360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e44e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55871e47cef0 con 0x55871e4675d0 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55871e3264c0 con 0x55871e44fbc0 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55871e47d1d0 con 0x55871e47fc20 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa42a626740 10 client.0 _ll_register_callbacks cb 0x55871e395e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa425800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa425800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36378/0 (socket says 172.21.15.3:36378) 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa425800640 1 -- 172.21.15.3:0/918984682 learned_addr learned my addr 172.21.15.3:0/918984682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa424e00640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e466360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.211+0000 7fa41fe00640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e44e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 -- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 msgr2=0x55871e44e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e44e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 -- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 msgr2=0x55871e466360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e466360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 -- 172.21.15.3:0/918984682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871e0b03f0 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa425800640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e230 tx=0x7fa418009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88d41a8418a725 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa406000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418035030 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa418032db0 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c015c40 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fa41804c050 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa42a626740 10 client.8562 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.212+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55871e0700f0 con 0x55871e4675d0 2024-09-10T07:53:49.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c015c40 con 0x55871e4675d0 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fa41804f020 con 0x55871e4675d0 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 10 client.8562 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 10 client.8562 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 20 client.8562 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e496a60 0x55871e496e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 4 client.8562 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.213+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871e3aad90 con 0x55871e496a60 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.214+0000 7fa424e00640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e496a60 0x55871e496e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.214+0000 7fa424e00640 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e496a60 0x55871e496e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dabae0ad5b012b81 server_cookie=f38930969988daf2 in_seq=0 out_seq=0 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.214+0000 7fa406000640 10 client.8562 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa406000640 1 -- 172.21.15.3:0/918984682 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa414077e80 con 0x55871e496a60 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa406000640 10 client.8562 handle_command_reply: tid=0 2024-09-10T07:53:49.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 client.8562 shutdown 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 msgr2=0x55871e467a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e230 tx=0x7fa418009750 comp rx=0 tx=0).stop 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 shutdown_connections 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e496a60 0x55871e496e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e467a50 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e44e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 --2- 172.21.15.3:0/918984682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e466360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa406000640 0 client.8562 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 >> 172.21.15.3:0/918984682 conn(0x55871e44f490 msgr2=0x55871e432f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 shutdown_connections 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.215+0000 7fa42a626740 1 -- 172.21.15.3:0/918984682 wait complete. 2024-09-10T07:53:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.216+0000 7fa42a626740 20 client.8562 trim_cache size 0 max 16384 2024-09-10T07:53:49.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 Processor -- start 2024-09-10T07:53:49.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 -- start start 2024-09-10T07:53:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e482840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e47bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4201fcd30 con 0x55871e4675d0 2024-09-10T07:53:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa420072e80 con 0x55871e44fbc0 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.234+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa4201fc600 con 0x55871e47fc20 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36392/0 (socket says 172.21.15.3:36392) 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 -- 172.21.15.3:0/4135126786 learned_addr learned my addr 172.21.15.3:0/4135126786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa424e00640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e482840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa425800640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e47bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 -- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 msgr2=0x55871e47bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e47bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 -- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 msgr2=0x55871e482840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e482840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 -- 172.21.15.3:0/4135126786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871e4356d0 con 0x55871e4675d0 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa424e00640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e482840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa425800640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e47bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.235+0000 7fa41fe00640 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009010 tx=0x7fa40c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83a2e4e2686aa6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa410c00640 1 -- 172.21.15.3:0/4135126786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c028040 con 0x55871e4675d0 2024-09-10T07:53:49.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa410c00640 1 -- 172.21.15.3:0/4135126786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c033070 con 0x55871e4675d0 2024-09-10T07:53:49.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa410c00640 1 -- 172.21.15.3:0/4135126786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02e8c0 con 0x55871e4675d0 2024-09-10T07:53:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 -- 172.21.15.3:0/4135126786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 msgr2=0x55871e4958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009010 tx=0x7fa40c015000 comp rx=0 tx=0).stop 2024-09-10T07:53:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 -- 172.21.15.3:0/4135126786 shutdown_connections 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e47fc20 0x55871e47bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e4675d0 0x55871e4958d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 --2- 172.21.15.3:0/4135126786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e44fbc0 0x55871e482840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 -- 172.21.15.3:0/4135126786 >> 172.21.15.3:0/4135126786 conn(0x55871e44f490 msgr2=0x55871e4801e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 -- 172.21.15.3:0/4135126786 shutdown_connections 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.236+0000 7fa42a626740 1 -- 172.21.15.3:0/4135126786 wait complete. 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.237+0000 7fa42a626740 1 -- start start 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.237+0000 7fa42a626740 1 Processor -- start 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e4675d0 0x55871e492ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e47fc20 0x55871e493430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55871e44bc80 con 0x55871e44fbc0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa4200ac880 con 0x55871e47fc20 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55871e433740 con 0x55871e4675d0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa42a626740 10 client.0 _ll_register_callbacks cb 0x7fa4080036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa424e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa424e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36394/0 (socket says 172.21.15.3:36394) 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.238+0000 7fa424e00640 1 -- 172.21.15.3:0/351338974 learned_addr learned my addr 172.21.15.3:0/351338974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 -- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e4675d0 msgr2=0x55871e492ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e4675d0 0x55871e492ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 -- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e47fc20 msgr2=0x55871e493430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e47fc20 0x55871e493430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 -- 172.21.15.3:0/351338974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871e493aa0 con 0x55871e44fbc0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa424e00640 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa414001eb0 tx=0x7fa4140055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6b24382882262f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa406000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414057220 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa414055590 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c033070 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fa414058370 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa42a626740 10 client.8568 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.239+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55871e467380 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c033070 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fa414058510 con 0x55871e44fbc0 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 10 client.8568 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 10 client.8568 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 20 client.8568 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e481ba0 0x55871e4397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 4 client.8568 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:49.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55871e3a50e0 con 0x55871e481ba0 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.240+0000 7fa41fe00640 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e481ba0 0x55871e4397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa41fe00640 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e481ba0 0x55871e4397d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e510fca368342d37 server_cookie=937b5fae43a256c4 in_seq=0 out_seq=0 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa406000640 10 client.8568 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa406000640 1 -- 172.21.15.3:0/351338974 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55871e3a50e0 con 0x55871e481ba0 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa406000640 10 client.8568 handle_command_reply: tid=0 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 client.8568 shutdown 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 msgr2=0x55871e481660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa414001eb0 tx=0x7fa4140055d0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 shutdown_connections 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55871e481ba0 0x55871e4397d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55871e47fc20 0x55871e493430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55871e4675d0 0x55871e492ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 --2- 172.21.15.3:0/351338974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55871e44fbc0 0x55871e481660 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa406000640 0 client.8568 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 >> 172.21.15.3:0/351338974 conn(0x55871e44f490 msgr2=0x55871e464fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 shutdown_connections 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.241+0000 7fa42a626740 1 -- 172.21.15.3:0/351338974 wait complete. 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa42a626740 20 client.8568 trim_cache size 0 max 16384 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:49.246 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:49.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 msgr2=0x7fa40006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 0x7fa40006e320 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa42015fff0 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 msgr2=0x7fa4201f1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa41400f7f0 tx=0x7fa41400fcb0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 shutdown_connections 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa40006be60 0x7fa40006e320 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa42015eb60 0x7fa42015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa4200ad3a0 0x7fa4200ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 --2- 172.21.15.3:0/1334791815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4080051c0 0x7fa4201f1280 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 >> 172.21.15.3:0/1334791815 conn(0x7fa4200b5cb0 msgr2=0x7fa4200b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 shutdown_connections 2024-09-10T07:53:49.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.242+0000 7fa427800640 1 -- 172.21.15.3:0/1334791815 wait complete. 2024-09-10T07:53:49.258 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:49.258 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:49.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T07:53:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.417+0000 7f76f7e00640 1 Processor -- start 2024-09-10T07:53:49.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.417+0000 7f76f7e00640 1 -- start start 2024-09-10T07:53:49.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76f7e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 0x7f76f80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76f7e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76f80d3860 0x7f76f80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76f7e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f80d3de0 con 0x7f76f80d4ae0 2024-09-10T07:53:49.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76f7e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80d4f50 con 0x7f76f80d3860 2024-09-10T07:53:49.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76f7e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f76f8161f50 con 0x7f76f8161b70 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76ee000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76f80d3860 0x7f76f80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76ee000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76f80d3860 0x7f76f80d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36398/0 (socket says 172.21.15.3:36398) 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.418+0000 7f76ee000640 1 -- 172.21.15.3:0/298090524 learned_addr learned my addr 172.21.15.3:0/298090524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76f6a00640 1 --2- 172.21.15.3:0/298090524 >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 0x7f76f80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76ee000640 1 -- 172.21.15.3:0/298090524 >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 msgr2=0x7f76f80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76ee000640 1 --2- 172.21.15.3:0/298090524 >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 0x7f76f80d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76ee000640 1 -- 172.21.15.3:0/298090524 >> v1:172.21.15.70:6789/0 conn(0x7f76f80d4ae0 legacy=0x7f76f80d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76ee000640 1 -- 172.21.15.3:0/298090524 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80d40c0 con 0x7f76f80d3860 2024-09-10T07:53:49.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.419+0000 7f76f6a00640 1 --2- 172.21.15.3:0/298090524 >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 0x7f76f80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76ee000640 1 --2- 172.21.15.3:0/298090524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f80d45a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f76f80d55c0 tx=0x7f76e002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda3f0543565a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76f6000640 1 -- 172.21.15.3:0/298090524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0037070 con 0x7f76f80d3860 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76f6000640 1 -- 172.21.15.3:0/298090524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e0032440 con 0x7f76f80d3860 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76f6000640 1 -- 172.21.15.3:0/298090524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0032810 con 0x7f76f80d3860 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76f7e00640 1 -- 172.21.15.3:0/298090524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 msgr2=0x7f76f80d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.420+0000 7f76f7e00640 1 --2- 172.21.15.3:0/298090524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f80d45a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f76f80d55c0 tx=0x7f76e002a250 comp rx=0 tx=0).stop 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 -- 172.21.15.3:0/298090524 shutdown_connections 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 --2- 172.21.15.3:0/298090524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f80d45a0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 --2- 172.21.15.3:0/298090524 >> v2:172.21.15.70:3300/0 conn(0x7f76f8161b70 0x7f76f80d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 -- 172.21.15.3:0/298090524 >> 172.21.15.3:0/298090524 conn(0x7f76f8062630 msgr2=0x7f76f8062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 -- 172.21.15.3:0/298090524 shutdown_connections 2024-09-10T07:53:49.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.421+0000 7f76f7e00640 1 -- 172.21.15.3:0/298090524 wait complete. 2024-09-10T07:53:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.422+0000 7f76f7e00640 1 Processor -- start 2024-09-10T07:53:49.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.422+0000 7f76f7e00640 1 -- start start 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76f80d4ae0 0x7f76f80cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 0x7f76f80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80c9fc0 con 0x7f76f80d3860 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80cf970 con 0x7f76f8161b70 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.423+0000 7f76f7e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80cf2d0 con 0x7f76f80d4ae0 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36406/0 (socket says 172.21.15.3:36406) 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 -- 172.21.15.3:0/4024927810 learned_addr learned my addr 172.21.15.3:0/4024927810 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f7400640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 0x7f76f80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76ee000640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76f80d4ae0 0x7f76f80cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 -- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76f80d4ae0 msgr2=0x7f76f80cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76f80d4ae0 0x7f76f80cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 -- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 msgr2=0x7f76f80b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 0x7f76f80b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 -- 172.21.15.3:0/4024927810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80cc630 con 0x7f76f80d3860 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f7400640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 0x7f76f80b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.424+0000 7f76f6a00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f76e800ce00 tx=0x7f76e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.425+0000 7f76ef400640 1 -- 172.21.15.3:0/4024927810 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e800f890 con 0x7f76f80d3860 2024-09-10T07:53:49.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.425+0000 7f76ef400640 1 -- 172.21.15.3:0/4024927810 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e800fa30 con 0x7f76f80d3860 2024-09-10T07:53:49.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.425+0000 7f76ef400640 1 -- 172.21.15.3:0/4024927810 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e800fe40 con 0x7f76f80d3860 2024-09-10T07:53:49.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.425+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f8156a40 con 0x7f76f80d3860 2024-09-10T07:53:49.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.425+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f8154d70 con 0x7f76f80d3860 2024-09-10T07:53:49.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.427+0000 7f76ef400640 1 -- 172.21.15.3:0/4024927810 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76e8018400 con 0x7f76f80d3860 2024-09-10T07:53:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.428+0000 7f76ef400640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 0x7f76d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.428+0000 7f76ef400640 1 -- 172.21.15.3:0/4024927810 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f76e808bc70 con 0x7f76f80d3860 2024-09-10T07:53:49.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.428+0000 7f76ee000640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 0x7f76d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.428+0000 7f7700a22740 1 Processor -- start 2024-09-10T07:53:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 -- start start 2024-09-10T07:53:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f76ee000640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 0x7f76d406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76e0034000 tx=0x7f76e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399a538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a399e0 0x560399a38240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5603999a1660 con 0x560399a83ea0 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x560399949680 con 0x560399a399e0 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.429+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56039999d620 con 0x560399a534f0 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36412/0 (socket says 172.21.15.3:36412) 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 -- 172.21.15.3:0/793714015 learned_addr learned my addr 172.21.15.3:0/793714015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76ee000640 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a399e0 0x560399a38240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f6a00640 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399a538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 -- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 msgr2=0x560399a538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399a538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 -- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a399e0 msgr2=0x560399a38240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a399e0 0x560399a38240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 -- 172.21.15.3:0/793714015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039977e7d0 con 0x560399a83ea0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.430+0000 7f76f7400640 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f76f0003e10 tx=0x7f76f00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24a1036efb798f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f76e7400640 1 -- 172.21.15.3:0/793714015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0005020 con 0x560399a83ea0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f76e7400640 1 -- 172.21.15.3:0/793714015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f00051c0 con 0x560399a83ea0 2024-09-10T07:53:49.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f76e7400640 1 -- 172.21.15.3:0/793714015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f000e950 con 0x560399a83ea0 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 -- 172.21.15.3:0/793714015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 msgr2=0x560399a843f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f76f0003e10 tx=0x7f76f00042d0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 -- 172.21.15.3:0/793714015 shutdown_connections 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a83ea0 0x560399a843f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a399e0 0x560399a38240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 --2- 172.21.15.3:0/793714015 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399a538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 -- 172.21.15.3:0/793714015 >> 172.21.15.3:0/793714015 conn(0x560399a3a410 msgr2=0x560399a82a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 -- 172.21.15.3:0/793714015 shutdown_connections 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.431+0000 7f7700a22740 1 -- 172.21.15.3:0/793714015 wait complete. 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.432+0000 7f7700a22740 1 -- start start 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.432+0000 7f7700a22740 1 Processor -- start 2024-09-10T07:53:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a534f0 0x560399a3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560399a38940 con 0x560399a42110 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56039992a4c0 con 0x560399a534f0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560399a38c20 con 0x560399a83ea0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f7700a22740 10 client.0 _ll_register_callbacks cb 0x560399999e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f76f7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f76f7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36422/0 (socket says 172.21.15.3:36422) 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f76f7400640 1 -- 172.21.15.3:0/1076225400 learned_addr learned my addr 172.21.15.3:0/1076225400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a534f0 0x560399a3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.433+0000 7f76ee000640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 -- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 msgr2=0x560399a68250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a68250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 -- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a534f0 msgr2=0x560399a3b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a534f0 0x560399a3b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 -- 172.21.15.3:0/1076225400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603996b43f0 con 0x560399a42110 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76f7400640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f76f0016040 tx=0x7f76f00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9473bd44242fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0023070 con 0x560399a42110 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76f00051c0 con 0x560399a42110 2024-09-10T07:53:49.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.434+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f76f000ed40 con 0x560399a42110 2024-09-10T07:53:49.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f76f002e050 con 0x560399a42110 2024-09-10T07:53:49.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 10 client.8580 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5603996740f0 con 0x560399a42110 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f00051c0 con 0x560399a42110 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7f76f0031020 con 0x560399a42110 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 10 client.8580 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 10 client.8580 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 20 client.8580 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a3ff00 0x560399a40310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 4 client.8580 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603999aed90 con 0x560399a3ff00 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.435+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a3ff00 0x560399a40310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.436+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a3ff00 0x560399a40310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0896689173324ff server_cookie=c67ef5655848fa4a in_seq=0 out_seq=0 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.436+0000 7f76dbe00640 10 client.8580 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f76dbe00640 1 -- 172.21.15.3:0/1076225400 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603999aed90 con 0x560399a3ff00 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f76dbe00640 10 client.8580 handle_command_reply: tid=0 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 client.8580 shutdown 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 msgr2=0x560399a3dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f76f0016040 tx=0x7f76f00036f0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 shutdown_connections 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a3dec0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a3ff00 0x560399a40310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 --2- 172.21.15.3:0/1076225400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a534f0 0x560399a3b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 >> 172.21.15.3:0/1076225400 conn(0x560399a399e0 msgr2=0x560399a82a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f76dbe00640 0 client.8580 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 shutdown_connections 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 1 -- 172.21.15.3:0/1076225400 wait complete. 2024-09-10T07:53:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.437+0000 7f7700a22740 20 client.8580 trim_cache size 0 max 16384 2024-09-10T07:53:49.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 Processor -- start 2024-09-10T07:53:49.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 -- start start 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 0x560399a67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a37330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8200b00 con 0x560399a534f0 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76f81ffca0 con 0x560399a42110 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.456+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76f82003d0 con 0x560399a83ea0 2024-09-10T07:53:49.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36434/0 (socket says 172.21.15.3:36434) 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 -- 172.21.15.3:0/3409874576 learned_addr learned my addr 172.21.15.3:0/3409874576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76f7400640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a37330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76f6a00640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 0x560399a67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 -- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 msgr2=0x560399a37330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a37330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 -- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 msgr2=0x560399a67d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 0x560399a67d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 -- 172.21.15.3:0/3409874576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560399a3bd20 con 0x560399a534f0 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76f7400640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a37330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76f6a00640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 0x560399a67d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76ee000640 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f76e0004630 tx=0x7f76e0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5927591567f722c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.457+0000 7f76e6a00640 1 -- 172.21.15.3:0/3409874576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0037040 con 0x560399a534f0 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f76e6a00640 1 -- 172.21.15.3:0/3409874576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e004f070 con 0x560399a534f0 2024-09-10T07:53:49.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f76e6a00640 1 -- 172.21.15.3:0/3409874576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e003c660 con 0x560399a534f0 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 -- 172.21.15.3:0/3409874576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 msgr2=0x560399a3ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f76e0004630 tx=0x7f76e0008030 comp rx=0 tx=0).stop 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 -- 172.21.15.3:0/3409874576 shutdown_connections 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a83ea0 0x560399a37330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a534f0 0x560399a3ddc0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 --2- 172.21.15.3:0/3409874576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a42110 0x560399a67d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 -- 172.21.15.3:0/3409874576 >> 172.21.15.3:0/3409874576 conn(0x560399a399e0 msgr2=0x560399a7fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 -- 172.21.15.3:0/3409874576 shutdown_connections 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.458+0000 7f7700a22740 1 -- 172.21.15.3:0/3409874576 wait complete. 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.459+0000 7f7700a22740 1 -- start start 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.459+0000 7f7700a22740 1 Processor -- start 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399b03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a83ea0 0x560399a51a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5603999aed90 con 0x560399a42110 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76f81f82e0 con 0x560399a83ea0 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x560399817940 con 0x560399a534f0 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f7700a22740 10 client.0 _ll_register_callbacks cb 0x7f76f0003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f76f6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f76f6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36446/0 (socket says 172.21.15.3:36446) 2024-09-10T07:53:49.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.460+0000 7f76f6a00640 1 -- 172.21.15.3:0/1236668085 learned_addr learned my addr 172.21.15.3:0/1236668085 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f7400640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a83ea0 0x560399a51a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76ee000640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399b03520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 -- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 msgr2=0x560399b03520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399b03520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 -- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a83ea0 msgr2=0x560399a51a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a83ea0 0x560399a51a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 -- 172.21.15.3:0/1236668085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560399a7f4f0 con 0x560399a42110 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76f6a00640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f76e8059ec0 tx=0x7f76e8059890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aa443b6c248474 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e801d040 con 0x560399a42110 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f76e80565a0 con 0x560399a42110 2024-09-10T07:53:49.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e004f070 con 0x560399a42110 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f76e8083050 con 0x560399a42110 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f7700a22740 10 client.8586 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.461+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x560399b03a60 con 0x560399a42110 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e004f070 con 0x560399a42110 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7f76e80625d0 con 0x560399a42110 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 10 client.8586 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 10 client.8586 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 20 client.8586 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a68200 0x560399a51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 4 client.8586 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560399a859c0 con 0x560399a68200 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.462+0000 7f76ee000640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a68200 0x560399a51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f76ee000640 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a68200 0x560399a51f40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45604060cec54b27 server_cookie=13434c780195d819 in_seq=0 out_seq=0 2024-09-10T07:53:49.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f76dbe00640 10 client.8586 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f76dbe00640 1 -- 172.21.15.3:0/1236668085 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560399a859c0 con 0x560399a68200 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f76dbe00640 10 client.8586 handle_command_reply: tid=0 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 client.8586 shutdown 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 msgr2=0x560399a86b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f76e8059ec0 tx=0x7f76e8059890 comp rx=0 tx=0).stop 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 shutdown_connections 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560399a83ea0 0x560399a51a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x560399a68200 0x560399a51f40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x560399a534f0 0x560399b03520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 --2- 172.21.15.3:0/1236668085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560399a42110 0x560399a86b20 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f76dbe00640 0 client.8586 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:49.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 >> 172.21.15.3:0/1236668085 conn(0x560399a399e0 msgr2=0x560399a37330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 shutdown_connections 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 1 -- 172.21.15.3:0/1236668085 wait complete. 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.463+0000 7f7700a22740 20 client.8586 trim_cache size 0 max 16384 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:49.468 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:49.469 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.469 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.469 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:49.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 msgr2=0x7f76d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 0x7f76d406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76e0034000 tx=0x7f76e002e000 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 msgr2=0x7f76f8152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f76e800ce00 tx=0x7f76e800a650 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 shutdown_connections 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f76d406be30 0x7f76d406e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76f8161b70 0x7f76f80b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76f80d4ae0 0x7f76f80cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 --2- 172.21.15.3:0/4024927810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76f80d3860 0x7f76f8152310 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 >> 172.21.15.3:0/4024927810 conn(0x7f76f8062630 msgr2=0x7f76f8062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 shutdown_connections 2024-09-10T07:53:49.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.464+0000 7f76f7e00640 1 -- 172.21.15.3:0/4024927810 wait complete. 2024-09-10T07:53:49.478 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:49.478 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:49.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-10T07:53:49.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.638+0000 7fad3be00640 1 Processor -- start 2024-09-10T07:53:49.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.638+0000 7fad3be00640 1 -- start start 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad3c15e840 0x7fad3c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3be00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c0ae3d0 con 0x7fad3c062d40 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3be00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c0ae690 con 0x7fad3c0adf90 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3be00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c0af730 con 0x7fad3c15e840 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fad3c062d40 0x7fad3c06bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49758/0 (socket says 172.21.15.3:49758) 2024-09-10T07:53:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.639+0000 7fad3b400640 1 -- 172.21.15.3:0/4195815054 learned_addr learned my addr 172.21.15.3:0/4195815054 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:49.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad3aa00640 1 --2- 172.21.15.3:0/4195815054 >> v2:172.21.15.70:3300/0 conn(0x7fad3c15e840 0x7fad3c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 814479071 0 0) 0x7fad3c0ae3d0 con 0x7fad3c062d40 2024-09-10T07:53:49.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad20003520 con 0x7fad3c062d40 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1928819409 0 0) 0x7fad20003520 con 0x7fad3c062d40 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 >> v2:172.21.15.70:3300/0 conn(0x7fad3c15e840 msgr2=0x7fad3c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 --2- 172.21.15.3:0/4195815054 >> v2:172.21.15.70:3300/0 conn(0x7fad3c15e840 0x7fad3c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 >> v1:172.21.15.70:6789/0 conn(0x7fad3c0adf90 legacy=0x7fad3c062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c0ae970 con 0x7fad3c062d40 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fad340019f0 con 0x7fad3c062d40 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 >> v1:172.21.15.3:6789/0 conn(0x7fad3c062d40 legacy=0x7fad3c06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 0x7fad20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200043b0 0x7fad20004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad20005080 0x7fad2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0af730 con 0x7fad200039b0 2024-09-10T07:53:49.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000bfe0 con 0x7fad20005080 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.640+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad2000c1e0 con 0x7fad200043b0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 0x7fad20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3b400640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad20005080 0x7fad2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3a000640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200043b0 0x7fad20004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 -- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200043b0 msgr2=0x7fad20004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200043b0 0x7fad20004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 -- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad20005080 msgr2=0x7fad2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad20005080 0x7fad2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3aa00640 1 -- 172.21.15.3:0/4195815054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad20003520 con 0x7fad200039b0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3b400640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad20005080 0x7fad2000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.641+0000 7fad3b400640 1 -- 172.21.15.3:0/4195815054 reap_dead start 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.642+0000 7fad3aa00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 0x7fad20003d90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009ac0 tx=0x7fad20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c426f9eda52c20d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.642+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c036070 con 0x7fad200039b0 2024-09-10T07:53:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.642+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c031400 con 0x7fad200039b0 2024-09-10T07:53:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.642+0000 7fad39600640 1 -- 172.21.15.3:0/4195815054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c009690 con 0x7fad200039b0 2024-09-10T07:53:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 -- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 msgr2=0x7fad20003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 0x7fad20003d90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009ac0 tx=0x7fad20005830 comp rx=0 tx=0).stop 2024-09-10T07:53:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 -- 172.21.15.3:0/4195815054 shutdown_connections 2024-09-10T07:53:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 --2- 172.21.15.3:0/4195815054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad200039b0 0x7fad20003d90 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 -- 172.21.15.3:0/4195815054 >> 172.21.15.3:0/4195815054 conn(0x7fad3c0d1240 msgr2=0x7fad3c0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 -- 172.21.15.3:0/4195815054 shutdown_connections 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.643+0000 7fad3be00640 1 -- 172.21.15.3:0/4195815054 wait complete. 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.644+0000 7fad3be00640 1 Processor -- start 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.644+0000 7fad3be00640 1 -- start start 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200039b0 0x7fad3c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c0c5c90 0x7fad3c160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0aef30 con 0x7fad3c0cc900 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c06f4c0 con 0x7fad3c0c5c90 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.645+0000 7fad3be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c0af210 con 0x7fad200039b0 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36474/0 (socket says 172.21.15.3:36474) 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 -- 172.21.15.3:0/986131974 learned_addr learned my addr 172.21.15.3:0/986131974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3aa00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200039b0 0x7fad3c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3a000640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c0c5c90 0x7fad3c160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 -- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200039b0 msgr2=0x7fad3c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200039b0 0x7fad3c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 -- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c0c5c90 msgr2=0x7fad3c160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c0c5c90 0x7fad3c160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 -- 172.21.15.3:0/986131974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c1608b0 con 0x7fad3c0cc900 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.646+0000 7fad3b400640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0cb820 tx=0x7fad3400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.647+0000 7fad32a00640 1 -- 172.21.15.3:0/986131974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3400eb30 con 0x7fad3c0cc900 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.647+0000 7fad32a00640 1 -- 172.21.15.3:0/986131974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3400ecd0 con 0x7fad3c0cc900 2024-09-10T07:53:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.647+0000 7fad32a00640 1 -- 172.21.15.3:0/986131974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34014680 con 0x7fad3c0cc900 2024-09-10T07:53:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.647+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c160b90 con 0x7fad3c0cc900 2024-09-10T07:53:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.647+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c161150 con 0x7fad3c0cc900 2024-09-10T07:53:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.649+0000 7fad32a00640 1 -- 172.21.15.3:0/986131974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad34014820 con 0x7fad3c0cc900 2024-09-10T07:53:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.650+0000 7fad32a00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 0x7fad1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.650+0000 7fad32a00640 1 -- 172.21.15.3:0/986131974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fad3408ab40 con 0x7fad3c0cc900 2024-09-10T07:53:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.650+0000 7fad44441740 1 Processor -- start 2024-09-10T07:53:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.650+0000 7fad3aa00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 0x7fad1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.650+0000 7fad44441740 1 -- start start 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad3aa00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 0x7fad1c06e320 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad2c033000 tx=0x7fad2c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 0x55924bafd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55924ba66660 con 0x55924bb184f0 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55924ba0e680 con 0x55924bafe9e0 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55924ba62620 con 0x55924bb48ea0 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36488/0 (socket says 172.21.15.3:36488) 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.651+0000 7fad3aa00640 1 -- 172.21.15.3:0/3640037097 learned_addr learned my addr 172.21.15.3:0/3640037097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3a000640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 0x55924bafd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3b400640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 -- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 msgr2=0x55924bb493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 -- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 msgr2=0x55924bafd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 0x55924bafd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 -- 172.21.15.3:0/3640037097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924b8437d0 con 0x55924bb184f0 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3a000640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 0x55924bafd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3b400640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad3aa00640 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009bb0 tx=0x7fad2c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b854a4f698ff06d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad27000640 1 -- 172.21.15.3:0/3640037097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c036040 con 0x55924bb184f0 2024-09-10T07:53:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad27000640 1 -- 172.21.15.3:0/3640037097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c037040 con 0x55924bb184f0 2024-09-10T07:53:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.652+0000 7fad27000640 1 -- 172.21.15.3:0/3640037097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c003b10 con 0x55924bb184f0 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 -- 172.21.15.3:0/3640037097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 msgr2=0x55924bb188d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009bb0 tx=0x7fad2c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 -- 172.21.15.3:0/3640037097 shutdown_connections 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bafe9e0 0x55924bafd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 --2- 172.21.15.3:0/3640037097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb184f0 0x55924bb188d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 -- 172.21.15.3:0/3640037097 >> 172.21.15.3:0/3640037097 conn(0x55924baff410 msgr2=0x55924bb47a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 -- 172.21.15.3:0/3640037097 shutdown_connections 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.653+0000 7fad44441740 1 -- 172.21.15.3:0/3640037097 wait complete. 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.654+0000 7fad44441740 1 -- start start 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.654+0000 7fad44441740 1 Processor -- start 2024-09-10T07:53:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb07e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55924bafd940 con 0x55924bb5b140 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55924b9ef4c0 con 0x55924bb184f0 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55924bafdc20 con 0x55924bb48ea0 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad44441740 10 client.0 _ll_register_callbacks cb 0x55924ba5ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36494/0 (socket says 172.21.15.3:36494) 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.655+0000 7fad3b400640 1 -- 172.21.15.3:0/314560364 learned_addr learned my addr 172.21.15.3:0/314560364 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3a000640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 -- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 msgr2=0x55924bb083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3aa00640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb07e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 -- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 msgr2=0x55924bb07e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb07e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 -- 172.21.15.3:0/314560364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924b7793f0 con 0x55924bb5b140 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3aa00640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb07e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3a000640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad3b400640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fad34089000 tx=0x7fad3405d190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22db7a8a6326b0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad1b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34014b10 con 0x55924bb5b140 2024-09-10T07:53:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fad340582e0 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad2c037040 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fad34014cb0 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad44441740 10 client.8598 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.656+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55924b7390f0 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c037040 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fad3406f070 con 0x55924bb5b140 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 10 client.8598 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 10 client.8598 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 20 client.8598 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bb43400 0x55924bb437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 4 client.8598 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924ba73d90 con 0x55924bb43400 2024-09-10T07:53:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.657+0000 7fad3aa00640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bb43400 0x55924bb437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.658+0000 7fad3aa00640 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bb43400 0x55924bb437e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fde4e9b387d1043 server_cookie=426f31881a7e340e in_seq=0 out_seq=0 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.658+0000 7fad1b400640 10 client.8598 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.658+0000 7fad1b400640 1 -- 172.21.15.3:0/314560364 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55924ba73d90 con 0x55924bb43400 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.658+0000 7fad1b400640 10 client.8598 handle_command_reply: tid=0 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.658+0000 7fad44441740 1 client.8598 shutdown 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 msgr2=0x55924bb5b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fad34089000 tx=0x7fad3405d190 comp rx=0 tx=0).stop 2024-09-10T07:53:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 shutdown_connections 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bb43400 0x55924bb437e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb5b140 0x55924bb5b5c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb48ea0 0x55924bb083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 --2- 172.21.15.3:0/314560364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb07e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad1b400640 0 client.8598 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 >> 172.21.15.3:0/314560364 conn(0x55924bafe9e0 msgr2=0x55924bb45410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 shutdown_connections 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 1 -- 172.21.15.3:0/314560364 wait complete. 2024-09-10T07:53:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.659+0000 7fad44441740 20 client.8598 trim_cache size 0 max 16384 2024-09-10T07:53:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.677+0000 7fad44441740 1 Processor -- start 2024-09-10T07:53:49.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.677+0000 7fad44441740 1 -- start start 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb184f0 0x55924bb176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 0x55924bafc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c201080 con 0x55924bb48ea0 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c200220 con 0x55924bb5b140 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c200950 con 0x55924bb184f0 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36502/0 (socket says 172.21.15.3:36502) 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.678+0000 7fad3a000640 1 -- 172.21.15.3:0/3605187580 learned_addr learned my addr 172.21.15.3:0/3605187580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3b400640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 0x55924bafc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3aa00640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb184f0 0x55924bb176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 -- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb184f0 msgr2=0x55924bb176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb184f0 0x55924bb176e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 -- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 msgr2=0x55924bafc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 0x55924bafc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 -- 172.21.15.3:0/3605187580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924bb05120 con 0x55924bb48ea0 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3b400640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 0x55924bafc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad3a000640 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fad28005470 tx=0x7fad280039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b58c6a9b565f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad26600640 1 -- 172.21.15.3:0/3605187580 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2801b020 con 0x55924bb48ea0 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad26600640 1 -- 172.21.15.3:0/3605187580 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28001660 con 0x55924bb48ea0 2024-09-10T07:53:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.679+0000 7fad26600640 1 -- 172.21.15.3:0/3605187580 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28001a50 con 0x55924bb48ea0 2024-09-10T07:53:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 -- 172.21.15.3:0/3605187580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 msgr2=0x55924bb5b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fad28005470 tx=0x7fad280039c0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 -- 172.21.15.3:0/3605187580 shutdown_connections 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb5b140 0x55924bafc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb5b560 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 --2- 172.21.15.3:0/3605187580 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb184f0 0x55924bb176e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 -- 172.21.15.3:0/3605187580 >> 172.21.15.3:0/3605187580 conn(0x55924bafe9e0 msgr2=0x55924baf88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 -- 172.21.15.3:0/3605187580 shutdown_connections 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.680+0000 7fad44441740 1 -- 172.21.15.3:0/3605187580 wait complete. 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.681+0000 7fad44441740 1 -- start start 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.681+0000 7fad44441740 1 Processor -- start 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb09550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 0x55924bbbfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55924bb171c0 con 0x55924bb48ea0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0c7260 con 0x55924bb184f0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55924bb58940 con 0x55924bb5b140 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad44441740 10 client.0 _ll_register_callbacks cb 0x7fad2c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36512/0 (socket says 172.21.15.3:36512) 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad3a000640 1 -- 172.21.15.3:0/1964066111 learned_addr learned my addr 172.21.15.3:0/1964066111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad3aa00640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb09550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.682+0000 7fad3b400640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 0x55924bbbfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 -- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 msgr2=0x55924bbbfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 0x55924bbbfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 -- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 msgr2=0x55924bb09550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb09550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 -- 172.21.15.3:0/1964066111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924bb43d20 con 0x55924bb48ea0 2024-09-10T07:53:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3b400640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 0x55924bbbfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad3a000640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fad28005440 tx=0x7fad28009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520d685814468c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad1b400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28029060 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fad28017870 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad28001660 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fad2802f050 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad44441740 10 client.8604 fetch_fsmap learned FSMap version 113 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.683+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55924bb43f30 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28001660 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 113) ==== 3558+0+0 (secure 0 0 0) 0x7fad28032020 con 0x55924bb48ea0 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 10 client.8604 fetch_fsmap finished waiting for FSMap version 113 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 10 client.8604 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 20 client.8604 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bbc6380 0x55924bbc6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 4 client.8604 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55924ba6e0e0 con 0x55924bbc6380 2024-09-10T07:53:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.684+0000 7fad3aa00640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bbc6380 0x55924bbc6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad3aa00640 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bbc6380 0x55924bbc6760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7845bc08f362b937 server_cookie=61626fcd85ac260a in_seq=0 out_seq=0 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad1b400640 10 client.8604 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad1b400640 1 -- 172.21.15.3:0/1964066111 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55924ba6e0e0 con 0x55924bbc6380 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad1b400640 10 client.8604 handle_command_reply: tid=0 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 client.8604 shutdown 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 msgr2=0x55924bb437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fad28005440 tx=0x7fad28009040 comp rx=0 tx=0).stop 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 shutdown_connections 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55924bbc6380 0x55924bbc6760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55924bb5b140 0x55924bbbfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55924bb48ea0 0x55924bb437e0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 --2- 172.21.15.3:0/1964066111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55924bb184f0 0x55924bb09550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 >> 172.21.15.3:0/1964066111 conn(0x55924bafe9e0 msgr2=0x55924bb45400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad1b400640 0 client.8604 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 shutdown_connections 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 1 -- 172.21.15.3:0/1964066111 wait complete. 2024-09-10T07:53:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.685+0000 7fad44441740 20 client.8604 trim_cache size 0 max 16384 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.690 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:49.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 msgr2=0x7fad1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 0x7fad1c06e320 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad2c033000 tx=0x7fad2c008030 comp rx=0 tx=0).stop 2024-09-10T07:53:49.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 msgr2=0x7fad3c0ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0cb820 tx=0x7fad3400e530 comp rx=0 tx=0).stop 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 shutdown_connections 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fad1c06be60 0x7fad1c06e320 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad3c0cc900 0x7fad3c0ccd90 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad3c0c5c90 0x7fad3c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 --2- 172.21.15.3:0/986131974 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad200039b0 0x7fad3c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 >> 172.21.15.3:0/986131974 conn(0x7fad3c0d1240 msgr2=0x7fad3c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 shutdown_connections 2024-09-10T07:53:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.686+0000 7fad3be00640 1 -- 172.21.15.3:0/986131974 wait complete. 2024-09-10T07:53:49.701 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:49.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:49.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-10T07:53:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.861+0000 7f349e200640 1 Processor -- start 2024-09-10T07:53:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.861+0000 7f349e200640 1 -- start start 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f349e200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f349815e900 0x7f34980b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f349e200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 0x7f34980b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f349e200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f349806d720 con 0x7f349815e520 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f349e200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34980b5b70 con 0x7f349815e900 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f349e200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f349806da00 con 0x7f349806bb20 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f3497400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f349815e520 0x7f34980b6fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49770/0 (socket says 172.21.15.3:49770) 2024-09-10T07:53:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f3497400640 1 -- 172.21.15.3:0/2428960112 learned_addr learned my addr 172.21.15.3:0/2428960112 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f348fe00640 1 --2- 172.21.15.3:0/2428960112 >> v2:172.21.15.3:3300/0 conn(0x7f349815e900 0x7f34980b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f3497e00640 1 --2- 172.21.15.3:0/2428960112 >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 0x7f34980b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f3496a00640 1 -- 172.21.15.3:0/2428960112 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057562338 0 0) 0x7f349806d720 con 0x7f349815e520 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f3496a00640 1 -- 172.21.15.3:0/2428960112 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f347c003520 con 0x7f349815e520 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f348fe00640 1 -- 172.21.15.3:0/2428960112 >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 msgr2=0x7f34980b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f348fe00640 1 --2- 172.21.15.3:0/2428960112 >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 0x7f34980b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f348fe00640 1 -- 172.21.15.3:0/2428960112 >> v1:172.21.15.3:6789/0 conn(0x7f349815e520 legacy=0x7f34980b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.863+0000 7f348fe00640 1 -- 172.21.15.3:0/2428960112 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498073b50 con 0x7f349815e900 2024-09-10T07:53:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.864+0000 7f3497e00640 1 --2- 172.21.15.3:0/2428960112 >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 0x7f34980b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.864+0000 7f348fe00640 1 --2- 172.21.15.3:0/2428960112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f34980b4e90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f34980d36f0 tx=0x7f348002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dee28a7e5b192b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f3496a00640 1 -- 172.21.15.3:0/2428960112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037070 con 0x7f349815e900 2024-09-10T07:53:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f3496a00640 1 -- 172.21.15.3:0/2428960112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480032440 con 0x7f349815e900 2024-09-10T07:53:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f3496a00640 1 -- 172.21.15.3:0/2428960112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348002ae60 con 0x7f349815e900 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 -- 172.21.15.3:0/2428960112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 msgr2=0x7f34980b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 --2- 172.21.15.3:0/2428960112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f34980b4e90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f34980d36f0 tx=0x7f348002a110 comp rx=0 tx=0).stop 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 -- 172.21.15.3:0/2428960112 shutdown_connections 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 --2- 172.21.15.3:0/2428960112 >> v2:172.21.15.70:3300/0 conn(0x7f349806bb20 0x7f34980b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 --2- 172.21.15.3:0/2428960112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f34980b4e90 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 -- 172.21.15.3:0/2428960112 >> 172.21.15.3:0/2428960112 conn(0x7f34980b4330 msgr2=0x7f34980b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.865+0000 7f349e200640 1 -- 172.21.15.3:0/2428960112 shutdown_connections 2024-09-10T07:53:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.866+0000 7f349e200640 1 -- 172.21.15.3:0/2428960112 wait complete. 2024-09-10T07:53:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.867+0000 7f349e200640 1 Processor -- start 2024-09-10T07:53:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.867+0000 7f349e200640 1 -- start start 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f349806bb20 0x7f3498144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f349815e520 0x7f34980ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34980b76c0 con 0x7f349815e900 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3498074110 con 0x7f349815e520 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f349e200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f34980b7840 con 0x7f349806bb20 2024-09-10T07:53:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f3497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f3497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36526/0 (socket says 172.21.15.3:36526) 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f3497e00640 1 -- 172.21.15.3:0/3731052562 learned_addr learned my addr 172.21.15.3:0/3731052562 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.868+0000 7f348fe00640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f349815e520 0x7f34980ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497400640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f349806bb20 0x7f3498144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 -- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f349806bb20 msgr2=0x7f3498144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f349806bb20 0x7f3498144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 -- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f349815e520 msgr2=0x7f34980ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f349815e520 0x7f34980ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 -- 172.21.15.3:0/3731052562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498144ed0 con 0x7f349815e900 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f3497e00640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3490007df0 tx=0x7f349000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f348f400640 1 -- 172.21.15.3:0/3731052562 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f349815e900 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f348f400640 1 -- 172.21.15.3:0/3731052562 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3490003990 con 0x7f349815e900 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f348f400640 1 -- 172.21.15.3:0/3731052562 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490003d80 con 0x7f349815e900 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34980ce7c0 con 0x7f349815e900 2024-09-10T07:53:49.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.869+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3498161020 con 0x7f349815e900 2024-09-10T07:53:49.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.873+0000 7f348f400640 1 -- 172.21.15.3:0/3731052562 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f349001d070 con 0x7f349815e900 2024-09-10T07:53:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.874+0000 7f348f400640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 0x7f347406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.874+0000 7f348f400640 1 -- 172.21.15.3:0/3731052562 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f349008aff0 con 0x7f349815e900 2024-09-10T07:53:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.874+0000 7f34a104b740 1 Processor -- start 2024-09-10T07:53:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.874+0000 7f3497400640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 0x7f347406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.874+0000 7f34a104b740 1 -- start start 2024-09-10T07:53:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f3497400640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 0x7f347406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3488005880 tx=0x7f3488009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebb7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 0x55c86ebb8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c86ead5660 con 0x55c86eb87800 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c86ea7d680 con 0x55c86eb87420 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c86ead1620 con 0x55c86eb6d480 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f348fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f348fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36538/0 (socket says 172.21.15.3:36538) 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f348fe00640 1 -- 172.21.15.3:0/2266021588 learned_addr learned my addr 172.21.15.3:0/2266021588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.875+0000 7f3497400640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebb7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f3497e00640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 0x55c86ebb8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 -- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 msgr2=0x55c86ebb8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 0x55c86ebb8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 -- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 msgr2=0x55c86ebb7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebb7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 -- 172.21.15.3:0/2266021588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86e8b27d0 con 0x55c86eb87800 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f3497e00640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 0x55c86ebb8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f348fe00640 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3480035040 tx=0x7f348002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c25b3a7edd54516 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f3487400640 1 -- 172.21.15.3:0/2266021588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037040 con 0x55c86eb87800 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f3487400640 1 -- 172.21.15.3:0/2266021588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348003c700 con 0x55c86eb87800 2024-09-10T07:53:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.876+0000 7f3487400640 1 -- 172.21.15.3:0/2266021588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348003caf0 con 0x55c86eb87800 2024-09-10T07:53:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 -- 172.21.15.3:0/2266021588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 msgr2=0x55c86ebb8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3480035040 tx=0x7f348002c000 comp rx=0 tx=0).stop 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 -- 172.21.15.3:0/2266021588 shutdown_connections 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb6d480 0x55c86ebb8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ebb8200 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 --2- 172.21.15.3:0/2266021588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebb7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 -- 172.21.15.3:0/2266021588 >> 172.21.15.3:0/2266021588 conn(0x55c86eb6e220 msgr2=0x55c86ebb6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 -- 172.21.15.3:0/2266021588 shutdown_connections 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.877+0000 7f34a104b740 1 -- 172.21.15.3:0/2266021588 wait complete. 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.878+0000 7f34a104b740 1 -- start start 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.878+0000 7f34a104b740 1 Processor -- start 2024-09-10T07:53:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87800 0x55c86ebc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86ebb13f0 0x55c86ebb39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c86eb6c3e0 con 0x55c86eb87420 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c86ea5e4c0 con 0x55c86eb87800 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c86eb6c6c0 con 0x55c86ebb13f0 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f34a104b740 10 client.0 _ll_register_callbacks cb 0x55c86eacde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36540/0 (socket says 172.21.15.3:36540) 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 -- 172.21.15.3:0/2578685397 learned_addr learned my addr 172.21.15.3:0/2578685397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497e00640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86ebb13f0 0x55c86ebb39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f348fe00640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87800 0x55c86ebc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 -- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86ebb13f0 msgr2=0x55c86ebb39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86ebb13f0 0x55c86ebb39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 -- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87800 msgr2=0x55c86ebc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87800 0x55c86ebc6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.879+0000 7f3497400640 1 -- 172.21.15.3:0/2578685397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86e7e83f0 con 0x55c86eb87420 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f3497400640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3488005ee0 tx=0x7f3488000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1ba300d0691b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488027060 con 0x55c86eb87420 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f3488010040 con 0x55c86eb87420 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348003c700 con 0x55c86eb87420 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f3488007560 con 0x55c86eb87420 2024-09-10T07:53:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f34a104b740 10 client.8619 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55c86e7a80f0 con 0x55c86eb87420 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348003c700 con 0x55c86eb87420 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.880+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f3488022430 con 0x55c86eb87420 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 10 client.8619 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 10 client.8619 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 20 client.8619 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86ebca4a0 0x55c86ebca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 4 client.8619 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86eae2d90 con 0x55c86ebca4a0 2024-09-10T07:53:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.881+0000 7f348fe00640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86ebca4a0 0x55c86ebca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.882+0000 7f348fe00640 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86ebca4a0 0x55c86ebca8e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2362aa7b37c2e026 server_cookie=b097aa0ea7df5605 in_seq=0 out_seq=0 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.882+0000 7f347be00640 10 client.8619 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.882+0000 7f347be00640 1 -- 172.21.15.3:0/2578685397 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c86eae2d90 con 0x55c86ebca4a0 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.882+0000 7f347be00640 10 client.8619 handle_command_reply: tid=0 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.882+0000 7f34a104b740 1 client.8619 shutdown 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 msgr2=0x55c86ebb2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3488005ee0 tx=0x7f3488000ed0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 shutdown_connections 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86ebca4a0 0x55c86ebca8e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86ebb13f0 0x55c86ebb39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87800 0x55c86ebc6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 --2- 172.21.15.3:0/2578685397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87420 0x55c86ebb2200 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f347be00640 0 client.8619 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 >> 172.21.15.3:0/2578685397 conn(0x55c86eb6d480 msgr2=0x55c86ebb4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 shutdown_connections 2024-09-10T07:53:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 1 -- 172.21.15.3:0/2578685397 wait complete. 2024-09-10T07:53:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.883+0000 7f34a104b740 20 client.8619 trim_cache size 0 max 16384 2024-09-10T07:53:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.901+0000 7f34a104b740 1 Processor -- start 2024-09-10T07:53:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 -- start start 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 0x55c86ebc66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3498200a90 con 0x55c86ebb13f0 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f34981ffc30 con 0x55c86eb87420 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.902+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3498200360 con 0x55c86eb87800 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36548/0 (socket says 172.21.15.3:36548) 2024-09-10T07:53:49.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 -- 172.21.15.3:0/322395309 learned_addr learned my addr 172.21.15.3:0/322395309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497400640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f348fe00640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 0x55c86ebc66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 -- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 msgr2=0x55c86ebc66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 0x55c86ebc66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 -- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 msgr2=0x55c86ebafbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebafbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 -- 172.21.15.3:0/322395309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86eb6d990 con 0x55c86ebb13f0 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f348fe00640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 0x55c86ebc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3497e00640 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3490007dc0 tx=0x7f349005d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486c31f2ccd6d87b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3486a00640 1 -- 172.21.15.3:0/322395309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490055a10 con 0x55c86ebb13f0 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3486a00640 1 -- 172.21.15.3:0/322395309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3490071170 con 0x55c86ebb13f0 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.903+0000 7f3486a00640 1 -- 172.21.15.3:0/322395309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490055bb0 con 0x55c86ebb13f0 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 -- 172.21.15.3:0/322395309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 msgr2=0x55c86ebb2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3490007dc0 tx=0x7f349005d820 comp rx=0 tx=0).stop 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 -- 172.21.15.3:0/322395309 shutdown_connections 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86ebb13f0 0x55c86ebb2f10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87800 0x55c86ebc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 --2- 172.21.15.3:0/322395309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86eb87420 0x55c86ebafbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 -- 172.21.15.3:0/322395309 >> 172.21.15.3:0/322395309 conn(0x55c86eb6d480 msgr2=0x55c86eb6cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 -- 172.21.15.3:0/322395309 shutdown_connections 2024-09-10T07:53:49.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.904+0000 7f34a104b740 1 -- 172.21.15.3:0/322395309 wait complete. 2024-09-10T07:53:49.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.905+0000 7f34a104b740 1 -- start start 2024-09-10T07:53:49.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.905+0000 7f34a104b740 1 Processor -- start 2024-09-10T07:53:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 0x55c86ebade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 0x55c86eb858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34981fecf0 con 0x55c86eb87800 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55c86eb86b10 con 0x55c86ebb13f0 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55c86ec35970 con 0x55c86eb87420 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f34a104b740 10 client.0 _ll_register_callbacks cb 0x7f3488001c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36560/0 (socket says 172.21.15.3:36560) 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 -- 172.21.15.3:0/2287152093 learned_addr learned my addr 172.21.15.3:0/2287152093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f3497400640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 0x55c86ebade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f3497e00640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 0x55c86eb858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 -- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 msgr2=0x55c86ebade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 0x55c86ebade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 -- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 msgr2=0x55c86eb858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 0x55c86eb858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f348fe00640 1 -- 172.21.15.3:0/2287152093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86ebaf600 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f3497400640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 0x55c86ebade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.906+0000 7f3497e00640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 0x55c86eb858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f348fe00640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3480009860 tx=0x7f3480008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ea4342242486db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f347be00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037040 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f3480004340 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3490071170 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f3480059050 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f34a104b740 10 client.8625 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55c86ec32ea0 con 0x55c86eb87800 2024-09-10T07:53:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.907+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3490071170 con 0x55c86eb87800 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f348005c020 con 0x55c86eb87800 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 10 client.8625 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 10 client.8625 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 20 client.8625 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86eb85e10 0x55c86eb717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 4 client.8625 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c86eadd0e0 con 0x55c86eb85e10 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.908+0000 7f3497400640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86eb85e10 0x55c86eb717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f3497400640 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86eb85e10 0x55c86eb717d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bb4884ba3ef0a83 server_cookie=1f7c3e88adea908d in_seq=0 out_seq=0 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f347be00640 10 client.8625 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f347be00640 1 -- 172.21.15.3:0/2287152093 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c86eadd0e0 con 0x55c86eb85e10 2024-09-10T07:53:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f347be00640 10 client.8625 handle_command_reply: tid=0 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 client.8625 shutdown 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 msgr2=0x55c86ec32960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3480009860 tx=0x7f3480008030 comp rx=0 tx=0).stop 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 shutdown_connections 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55c86eb85e10 0x55c86eb717d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c86ebb13f0 0x55c86eb858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c86eb87800 0x55c86ec32960 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f347be00640 0 client.8625 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 --2- 172.21.15.3:0/2287152093 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55c86eb87420 0x55c86ebade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 >> 172.21.15.3:0/2287152093 conn(0x55c86eb6d480 msgr2=0x55c86ebb4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 shutdown_connections 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.909+0000 7f34a104b740 1 -- 172.21.15.3:0/2287152093 wait complete. 2024-09-10T07:53:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f34a104b740 20 client.8625 trim_cache size 0 max 16384 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:49.914 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 msgr2=0x7f347406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 0x7f347406e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3488005880 tx=0x7f3488009040 comp rx=0 tx=0).stop 2024-09-10T07:53:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 msgr2=0x7f349815ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3490007df0 tx=0x7f349000d4a0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 shutdown_connections 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f347406be30 0x7f347406e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f349815e900 0x7f349815ca70 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f349815e520 0x7f34980ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 --2- 172.21.15.3:0/3731052562 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f349806bb20 0x7f3498144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 >> 172.21.15.3:0/3731052562 conn(0x7f34980b4330 msgr2=0x7f34980b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 shutdown_connections 2024-09-10T07:53:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:49.910+0000 7f349e200640 1 -- 172.21.15.3:0/3731052562 wait complete. 2024-09-10T07:53:49.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:49.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:49.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T07:53:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.084+0000 7fbee7000640 1 Processor -- start 2024-09-10T07:53:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.084+0000 7fbee7000640 1 -- start start 2024-09-10T07:53:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbee7000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbee015c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbee7000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee015eff0 con 0x7fbee00b4c90 2024-09-10T07:53:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbee7000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee015f2b0 con 0x7fbee015b230 2024-09-10T07:53:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbee7000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbee015fbe0 con 0x7fbee0072c50 2024-09-10T07:53:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbedf400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbee00b4c90 0x7fbee0071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49786/0 (socket says 172.21.15.3:49786) 2024-09-10T07:53:50.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbedf400640 1 -- 172.21.15.3:0/2611046967 learned_addr learned my addr 172.21.15.3:0/2611046967 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbee5000640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbee015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560859559 0 0) 0x7fbee015eff0 con 0x7fbee00b4c90 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.086+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbec4005730 con 0x7fbee00b4c90 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936037485 0 0) 0x7fbee015f2b0 con 0x7fbee015b230 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee015eff0 con 0x7fbee015b230 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2857412930 0 0) 0x7fbec4005730 con 0x7fbee00b4c90 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 msgr2=0x7fbee015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbee015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 >> v1:172.21.15.70:6789/0 conn(0x7fbee015b230 legacy=0x7fbee00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee015f590 con 0x7fbee00b4c90 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7fbecc001670 con 0x7fbee00b4c90 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 >> v1:172.21.15.3:6789/0 conn(0x7fbee00b4c90 legacy=0x7fbee0071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbec4005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbec40062e0 0x7fbec4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec4006f50 0x7fbec40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbee015fbe0 con 0x7fbec40062e0 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbec400dcb0 con 0x7fbec4006f50 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbec400deb0 con 0x7fbee0072c50 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedf400640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbec40062e0 0x7fbec4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbedfe00640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbec4005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.087+0000 7fbee5000640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec4006f50 0x7fbec40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.088+0000 7fbedf400640 1 -- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 msgr2=0x7fbec4005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.088+0000 7fbedf400640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee0072c50 0x7fbec4005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.088+0000 7fbedf400640 1 -- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec4006f50 msgr2=0x7fbec40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.088+0000 7fbedf400640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec4006f50 0x7fbec40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.088+0000 7fbedf400640 1 -- 172.21.15.3:0/2611046967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec4005730 con 0x7fbec40062e0 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbedf400640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbec40062e0 0x7fbec4006a10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c1c0 tx=0x7fbec4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5813dce49fc86a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc007cc0 con 0x7fbec40062e0 2024-09-10T07:53:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbecc005af0 con 0x7fbec40062e0 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbedea00640 1 -- 172.21.15.3:0/2611046967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc007070 con 0x7fbec40062e0 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 -- 172.21.15.3:0/2611046967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbec40062e0 msgr2=0x7fbec4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 --2- 172.21.15.3:0/2611046967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbec40062e0 0x7fbec4006a10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c1c0 tx=0x7fbec4007a40 comp rx=0 tx=0).stop 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee5000640 1 -- 172.21.15.3:0/2611046967 reap_dead start 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 -- 172.21.15.3:0/2611046967 shutdown_connections 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 -- 172.21.15.3:0/2611046967 >> 172.21.15.3:0/2611046967 conn(0x7fbee00b5cb0 msgr2=0x7fbee006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 -- 172.21.15.3:0/2611046967 shutdown_connections 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.089+0000 7fbee7000640 1 -- 172.21.15.3:0/2611046967 wait complete. 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.091+0000 7fbee7000640 1 Processor -- start 2024-09-10T07:53:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.091+0000 7fbee7000640 1 -- start start 2024-09-10T07:53:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40062e0 0x7fbee00b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee014b5a0 0x7fbee00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbee015b790 con 0x7fbee014ba20 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbee015b610 con 0x7fbec40062e0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee7000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbee006e1e0 con 0x7fbee014b5a0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36584/0 (socket says 172.21.15.3:36584) 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.092+0000 7fbee5000640 1 -- 172.21.15.3:0/451714757 learned_addr learned my addr 172.21.15.3:0/451714757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbedfe00640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40062e0 0x7fbee00b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbedf400640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee014b5a0 0x7fbee00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 -- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee014b5a0 msgr2=0x7fbee00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee014b5a0 0x7fbee00ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 -- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40062e0 msgr2=0x7fbee00b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40062e0 0x7fbee00b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 -- 172.21.15.3:0/451714757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee00b1530 con 0x7fbee014ba20 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee5000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbee0073100 tx=0x7fbed800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbed7e00640 1 -- 172.21.15.3:0/451714757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8019070 con 0x7fbee014ba20 2024-09-10T07:53:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbed7e00640 1 -- 172.21.15.3:0/451714757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8011e90 con 0x7fbee014ba20 2024-09-10T07:53:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee00b1810 con 0x7fbee014ba20 2024-09-10T07:53:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbed7e00640 1 -- 172.21.15.3:0/451714757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003bd0 con 0x7fbee014ba20 2024-09-10T07:53:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.093+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee00b1dd0 con 0x7fbee014ba20 2024-09-10T07:53:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.097+0000 7fbed7e00640 1 -- 172.21.15.3:0/451714757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbed801c070 con 0x7fbee014ba20 2024-09-10T07:53:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.097+0000 7fbed7e00640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 0x7fbec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbed7e00640 1 -- 172.21.15.3:0/451714757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbed808b700 con 0x7fbee014ba20 2024-09-10T07:53:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbee9ece740 1 Processor -- start 2024-09-10T07:53:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbedfe00640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 0x7fbec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbee9ece740 1 -- start start 2024-09-10T07:53:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbedfe00640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 0x7fbec006e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbed00055b0 tx=0x7fbed0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.098+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e731221480 0x55e73126c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee9ece740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e731189660 con 0x55e73123b800 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e731131680 con 0x55e73123b420 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e731185620 con 0x55e731221480 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46396/0 (socket says 172.21.15.3:46396) 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 -- 172.21.15.3:0/277059555 learned_addr learned my addr 172.21.15.3:0/277059555 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbee5000640 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e731221480 0x55e73126c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 -- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e731221480 msgr2=0x55e73126c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e731221480 0x55e73126c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 -- 172.21.15.3:0/277059555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 msgr2=0x55e73126c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedfe00640 1 -- 172.21.15.3:0/277059555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e730f667d0 con 0x55e73123b420 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.099+0000 7fbedf400640 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbedfe00640 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001940 tx=0x7fbed0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a41efad54b56d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbed4c00640 1 -- 172.21.15.3:0/277059555 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0028070 con 0x55e73123b420 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbed4c00640 1 -- 172.21.15.3:0/277059555 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed0015c00 con 0x55e73123b420 2024-09-10T07:53:50.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbed4c00640 1 -- 172.21.15.3:0/277059555 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed00143d0 con 0x55e73123b420 2024-09-10T07:53:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbee9ece740 1 -- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 msgr2=0x55e73126bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.100+0000 7fbee9ece740 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001940 tx=0x7fbed0014000 comp rx=0 tx=0).stop 2024-09-10T07:53:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 -- 172.21.15.3:0/277059555 shutdown_connections 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e731221480 0x55e73126c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 --2- 172.21.15.3:0/277059555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e73126bc40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 -- 172.21.15.3:0/277059555 >> 172.21.15.3:0/277059555 conn(0x55e731222220 msgr2=0x55e73126a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 -- 172.21.15.3:0/277059555 shutdown_connections 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 -- 172.21.15.3:0/277059555 wait complete. 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.101+0000 7fbee9ece740 1 -- start start 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 Processor -- start 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e731266c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73127e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e7312203e0 con 0x55e73123b800 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e7311124c0 con 0x55e73123b420 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.102+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e7312206c0 con 0x55e73122acd0 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbee9ece740 10 client.0 _ll_register_callbacks cb 0x55e731181e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36592/0 (socket says 172.21.15.3:36592) 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e731266c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 -- 172.21.15.3:0/3890761247 learned_addr learned my addr 172.21.15.3:0/3890761247 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbee5000640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73127e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 -- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 msgr2=0x55e73127e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73127e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 -- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 msgr2=0x55e731266c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e731266c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.103+0000 7fbedf400640 1 -- 172.21.15.3:0/3890761247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e730e9c3f0 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbedf400640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02ada0 tx=0x7fbecc001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dae819aa0d221c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0047e0 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbecc004980 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed0015c00 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7fbecc04c050 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbee9ece740 10 client.8637 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55e730e5c0f0 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed0015c00 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.104+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7fbecc04f020 con 0x55e73123b800 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 10 client.8637 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 10 client.8637 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 20 client.8637 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e731265870 0x55e731265ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 4 client.8637 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e731196d90 con 0x55e731265870 2024-09-10T07:53:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbedfe00640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e731265870 0x55e731265ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbedfe00640 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e731265870 0x55e731265ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9571405226a8f289 server_cookie=741e65b61ed747f6 in_seq=0 out_seq=0 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.105+0000 7fbec9600640 10 client.8637 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbec9600640 1 -- 172.21.15.3:0/3890761247 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e731196d90 con 0x55e731265870 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbec9600640 10 client.8637 handle_command_reply: tid=0 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 client.8637 shutdown 2024-09-10T07:53:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 msgr2=0x55e73122a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02ada0 tx=0x7fbecc001a30 comp rx=0 tx=0).stop 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 shutdown_connections 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e731265870 0x55e731265ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73127e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73122a790 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3890761247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b420 0x55e731266c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbec9600640 0 client.8637 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.106+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 >> 172.21.15.3:0/3890761247 conn(0x55e731221480 msgr2=0x55e73126bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.107+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 shutdown_connections 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.107+0000 7fbee9ece740 1 -- 172.21.15.3:0/3890761247 wait complete. 2024-09-10T07:53:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.107+0000 7fbee9ece740 20 client.8637 trim_cache size 0 max 16384 2024-09-10T07:53:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 Processor -- start 2024-09-10T07:53:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 -- start start 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 0x55e73128fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73123b420 0x55e73126bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0201570 con 0x55e73123b800 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0200710 con 0x55e73122acd0 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.125+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbee0200e40 con 0x55e73123b420 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36598/0 (socket says 172.21.15.3:36598) 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 -- 172.21.15.3:0/621276533 learned_addr learned my addr 172.21.15.3:0/621276533 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbedfe00640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 0x55e73128fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbedf400640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73123b420 0x55e73126bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 -- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73123b420 msgr2=0x55e73126bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73123b420 0x55e73126bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 -- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 msgr2=0x55e73128fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 0x55e73128fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 -- 172.21.15.3:0/621276533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7312668f0 con 0x55e73123b800 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbedfe00640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 0x55e73128fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbee5000640 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbed8055ea0 tx=0x7fbed805df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbddffb16807924c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.126+0000 7fbecbe00640 1 -- 172.21.15.3:0/621276533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8023040 con 0x55e73123b800 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbecbe00640 1 -- 172.21.15.3:0/621276533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8003d70 con 0x55e73123b800 2024-09-10T07:53:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbecbe00640 1 -- 172.21.15.3:0/621276533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8060070 con 0x55e73123b800 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 -- 172.21.15.3:0/621276533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 msgr2=0x55e73126c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbed8055ea0 tx=0x7fbed805df30 comp rx=0 tx=0).stop 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 -- 172.21.15.3:0/621276533 shutdown_connections 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b800 0x55e73126c0e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73123b420 0x55e73126bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 --2- 172.21.15.3:0/621276533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73122acd0 0x55e73128fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 -- 172.21.15.3:0/621276533 >> 172.21.15.3:0/621276533 conn(0x55e731221480 msgr2=0x55e731265c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 -- 172.21.15.3:0/621276533 shutdown_connections 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.127+0000 7fbee9ece740 1 -- 172.21.15.3:0/621276533 wait complete. 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.128+0000 7fbee9ece740 1 -- start start 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.128+0000 7fbee9ece740 1 Processor -- start 2024-09-10T07:53:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73121f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b800 0x55e7312e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e7311bfc10 con 0x55e73123b420 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01545f0 con 0x55e73123b800 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e730fff940 con 0x55e73122acd0 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee9ece740 10 client.0 _ll_register_callbacks cb 0x7fbec400dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbedf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbedf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36600/0 (socket says 172.21.15.3:36600) 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbedf400640 1 -- 172.21.15.3:0/3340622050 learned_addr learned my addr 172.21.15.3:0/3340622050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbee5000640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b800 0x55e7312e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.129+0000 7fbedfe00640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73121f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 -- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 msgr2=0x55e73121f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73121f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 -- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b800 msgr2=0x55e7312e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b800 0x55e7312e98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 -- 172.21.15.3:0/3340622050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7312902f0 con 0x55e73123b420 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbedf400640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02ada0 tx=0x7fbecc007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8933957f1032ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbec9600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc007cc0 con 0x55e73123b420 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fbecc0072d0 con 0x55e73123b420 2024-09-10T07:53:50.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.130+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed8003d70 con 0x55e73123b420 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7fbecc0074d0 con 0x55e73123b420 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 10 client.8643 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55e73126e240 con 0x55e73123b420 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8003d70 con 0x55e73123b420 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7fbecc062080 con 0x55e73123b420 2024-09-10T07:53:50.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 10 client.8643 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 10 client.8643 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 20 client.8643 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e73126e8a0 0x55e7312e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 4 client.8643 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7311910e0 con 0x55e73126e8a0 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.131+0000 7fbedfe00640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e73126e8a0 0x55e7312e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbedfe00640 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e73126e8a0 0x55e7312e9e10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfcd8952deea052 server_cookie=58bd1befebf744ff in_seq=0 out_seq=0 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbec9600640 10 client.8643 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbec9600640 1 -- 172.21.15.3:0/3340622050 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e7311910e0 con 0x55e73126e8a0 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbec9600640 10 client.8643 handle_command_reply: tid=0 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 client.8643 shutdown 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 msgr2=0x55e7312e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02ada0 tx=0x7fbecc007e60 comp rx=0 tx=0).stop 2024-09-10T07:53:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 shutdown_connections 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e73123b800 0x55e7312e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e73123b420 0x55e7312e87b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x55e73126e8a0 0x55e7312e9e10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 --2- 172.21.15.3:0/3340622050 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e73122acd0 0x55e73121f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 >> 172.21.15.3:0/3340622050 conn(0x55e731221480 msgr2=0x55e73123a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.132+0000 7fbec9600640 0 client.8643 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 shutdown_connections 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee9ece740 1 -- 172.21.15.3:0/3340622050 wait complete. 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee9ece740 20 client.8643 trim_cache size 0 max 16384 2024-09-10T07:53:50.136 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:50.137 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 msgr2=0x7fbec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 0x7fbec006e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbed00055b0 tx=0x7fbed0009040 comp rx=0 tx=0).stop 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 msgr2=0x7fbee00aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fbee0073100 tx=0x7fbed800f580 comp rx=0 tx=0).stop 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 shutdown_connections 2024-09-10T07:53:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbec006be30 0x7fbec006e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee014ba20 0x7fbee00aede0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbee014b5a0 0x7fbee00ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 --2- 172.21.15.3:0/451714757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40062e0 0x7fbee00b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.133+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 >> 172.21.15.3:0/451714757 conn(0x7fbee00b5cb0 msgr2=0x7fbee0155460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.134+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 shutdown_connections 2024-09-10T07:53:50.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.134+0000 7fbee7000640 1 -- 172.21.15.3:0/451714757 wait complete. 2024-09-10T07:53:50.149 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:50.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.307+0000 7ffaa6600640 1 Processor -- start 2024-09-10T07:53:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.307+0000 7ffaa6600640 1 -- start start 2024-09-10T07:53:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.308+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 0x7ffaa00ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.308+0000 7ffaa6600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa00ccd80 con 0x7ffaa015b110 2024-09-10T07:53:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffaa6600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa00cd060 con 0x7ffaa015b4f0 2024-09-10T07:53:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffaa6600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa00b44f0 con 0x7ffaa00d08f0 2024-09-10T07:53:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffaa015b110 0x7ffaa00b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49792/0 (socket says 172.21.15.3:49792) 2024-09-10T07:53:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9f400640 1 -- 172.21.15.3:0/2710267117 learned_addr learned my addr 172.21.15.3:0/2710267117 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9fe00640 1 --2- 172.21.15.3:0/2710267117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 0x7ffaa00ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888566065 0 0) 0x7ffaa00ccd80 con 0x7ffaa015b110 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa84003520 con 0x7ffaa015b110 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636888174 0 0) 0x7ffaa00cd060 con 0x7ffaa015b4f0 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa00ccd80 con 0x7ffaa015b4f0 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9fe00640 1 -- 172.21.15.3:0/2710267117 >> v1:172.21.15.70:6789/0 conn(0x7ffaa015b4f0 legacy=0x7ffaa00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9fe00640 1 -- 172.21.15.3:0/2710267117 >> v1:172.21.15.3:6789/0 conn(0x7ffaa015b110 legacy=0x7ffaa00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.309+0000 7ffa9fe00640 1 -- 172.21.15.3:0/2710267117 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa00cd340 con 0x7ffaa00d08f0 2024-09-10T07:53:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.310+0000 7ffa9fe00640 1 --2- 172.21.15.3:0/2710267117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 0x7ffaa00ceb50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009dc0 tx=0x7ffa9802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf37da557b05e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98007740 con 0x7ffaa00d08f0 2024-09-10T07:53:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa980078e0 con 0x7ffaa00d08f0 2024-09-10T07:53:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffa9ea00640 1 -- 172.21.15.3:0/2710267117 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98007cb0 con 0x7ffaa00d08f0 2024-09-10T07:53:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 -- 172.21.15.3:0/2710267117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 msgr2=0x7ffaa00ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 --2- 172.21.15.3:0/2710267117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 0x7ffaa00ceb50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009dc0 tx=0x7ffa9802a6d0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 -- 172.21.15.3:0/2710267117 shutdown_connections 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 --2- 172.21.15.3:0/2710267117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa00d08f0 0x7ffaa00ceb50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 -- 172.21.15.3:0/2710267117 >> 172.21.15.3:0/2710267117 conn(0x7ffaa006d000 msgr2=0x7ffaa006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.311+0000 7ffaa6600640 1 -- 172.21.15.3:0/2710267117 shutdown_connections 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.312+0000 7ffaa6600640 1 -- 172.21.15.3:0/2710267117 wait complete. 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.313+0000 7ffaa6600640 1 Processor -- start 2024-09-10T07:53:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.313+0000 7ffaa6600640 1 -- start start 2024-09-10T07:53:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa00d08f0 0x7ffaa014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa015b110 0x7ffaa0153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0073e90 con 0x7ffaa015b4f0 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0073d10 con 0x7ffaa00d08f0 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffaa6600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa0074010 con 0x7ffaa015b110 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffa9fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffa9fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36612/0 (socket says 172.21.15.3:36612) 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffa9fe00640 1 -- 172.21.15.3:0/4053505541 learned_addr learned my addr 172.21.15.3:0/4053505541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffa9f400640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa00d08f0 0x7ffaa014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.314+0000 7ffa97e00640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa015b110 0x7ffaa0153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 -- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa015b110 msgr2=0x7ffaa0153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa015b110 0x7ffaa0153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 -- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa00d08f0 msgr2=0x7ffaa014b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa00d08f0 0x7ffaa014b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 -- 172.21.15.3:0/4053505541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa01539c0 con 0x7ffaa015b4f0 2024-09-10T07:53:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa9fe00640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009ee0 tx=0x7ffa98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98040070 con 0x7ffaa015b4f0 2024-09-10T07:53:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa98031ac0 con 0x7ffaa015b4f0 2024-09-10T07:53:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa0156a00 con 0x7ffaa015b4f0 2024-09-10T07:53:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.315+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98045460 con 0x7ffaa015b4f0 2024-09-10T07:53:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.316+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa015b970 con 0x7ffaa015b4f0 2024-09-10T07:53:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.317+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa00b3d10 con 0x7ffaa015b4f0 2024-09-10T07:53:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.317+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa98039040 con 0x7ffaa015b4f0 2024-09-10T07:53:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.318+0000 7ffa97400640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 0x7ffa7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.318+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7ffa980ac440 con 0x7ffaa015b4f0 2024-09-10T07:53:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.319+0000 7ffa9f400640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 0x7ffa7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.319+0000 7ffa9f400640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 0x7ffa7c06e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffa90000f60 tx=0x7ffa90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.322+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa9807a850 con 0x7ffaa015b4f0 2024-09-10T07:53:50.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.544+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaa0158c40 con 0x7ffaa015b4f0 2024-09-10T07:53:50.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.545+0000 7ffa97400640 1 -- 172.21.15.3:0/4053505541 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7ffa9807eb50 con 0x7ffaa015b4f0 2024-09-10T07:53:50.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:50.545 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:50.545 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 msgr2=0x7ffa7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 0x7ffa7c06e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffa90000f60 tx=0x7ffa90009040 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 msgr2=0x7ffaa00ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009ee0 tx=0x7ffa98004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 shutdown_connections 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa7c06be30 0x7ffa7c06e2f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa015b4f0 0x7ffaa00ae8a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa015b110 0x7ffaa0153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 --2- 172.21.15.3:0/4053505541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa00d08f0 0x7ffaa014b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 >> 172.21.15.3:0/4053505541 conn(0x7ffaa006d000 msgr2=0x7ffaa00b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 shutdown_connections 2024-09-10T07:53:50.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.547+0000 7ffaa6600640 1 -- 172.21.15.3:0/4053505541 wait complete. 2024-09-10T07:53:50.562 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:50.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.718+0000 7f36dcc00640 1 Processor -- start 2024-09-10T07:53:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.718+0000 7f36dcc00640 1 -- start start 2024-09-10T07:53:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.719+0000 7f36dcc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f36d8070f30 0x7f36d8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.719+0000 7f36dcc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 0x7f36d815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.719+0000 7f36dcc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d80cd7f0 con 0x7f36d815c720 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.719+0000 7f36dcc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f36d8072210 con 0x7f36d8070f30 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.719+0000 7f36dcc00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f36d80cf880 con 0x7f36d80cfdc0 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36d5e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f36d815c720 0x7f36d8073610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49802/0 (socket says 172.21.15.3:49802) 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36d5e00640 1 -- 172.21.15.3:0/2046292030 learned_addr learned my addr 172.21.15.3:0/2046292030 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36cfe00640 1 --2- 172.21.15.3:0/2046292030 >> v2:172.21.15.3:3300/0 conn(0x7f36d8070f30 0x7f36d8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36d6800640 1 --2- 172.21.15.3:0/2046292030 >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 0x7f36d815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36d5400640 1 -- 172.21.15.3:0/2046292030 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 180857757 0 0) 0x7f36d80cd7f0 con 0x7f36d815c720 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36cfe00640 1 -- 172.21.15.3:0/2046292030 >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 msgr2=0x7f36d815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36cfe00640 1 --2- 172.21.15.3:0/2046292030 >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 0x7f36d815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36cfe00640 1 -- 172.21.15.3:0/2046292030 >> v1:172.21.15.3:6789/0 conn(0x7f36d815c720 legacy=0x7f36d8073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36cfe00640 1 -- 172.21.15.3:0/2046292030 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d80cda90 con 0x7f36d8070f30 2024-09-10T07:53:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.720+0000 7f36d6800640 1 --2- 172.21.15.3:0/2046292030 >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 0x7f36d815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.721+0000 7f36cfe00640 1 --2- 172.21.15.3:0/2046292030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8070f30 0x7f36d8071cd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f36d806dd40 tx=0x7f36c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4004c3a7f1808d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.721+0000 7f36d5400640 1 -- 172.21.15.3:0/2046292030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0037070 con 0x7f36d8070f30 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.721+0000 7f36d5400640 1 -- 172.21.15.3:0/2046292030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0032440 con 0x7f36d8070f30 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.721+0000 7f36d5400640 1 -- 172.21.15.3:0/2046292030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0032810 con 0x7f36d8070f30 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 -- 172.21.15.3:0/2046292030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8070f30 msgr2=0x7f36d8071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2046292030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8070f30 0x7f36d8071cd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f36d806dd40 tx=0x7f36c002a1a0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 -- 172.21.15.3:0/2046292030 shutdown_connections 2024-09-10T07:53:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2046292030 >> v2:172.21.15.70:3300/0 conn(0x7f36d80cfdc0 0x7f36d815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2046292030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8070f30 0x7f36d8071cd0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 -- 172.21.15.3:0/2046292030 >> 172.21.15.3:0/2046292030 conn(0x7f36d806cfc0 msgr2=0x7f36d806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 -- 172.21.15.3:0/2046292030 shutdown_connections 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.722+0000 7f36dcc00640 1 -- 172.21.15.3:0/2046292030 wait complete. 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.724+0000 7f36dcc00640 1 Processor -- start 2024-09-10T07:53:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.724+0000 7f36dcc00640 1 -- start start 2024-09-10T07:53:50.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d8070f30 0x7f36d8157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 0x7f36d81e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8073d20 con 0x7f36d815c720 2024-09-10T07:53:50.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f36d806ed10 con 0x7f36d8070f30 2024-09-10T07:53:50.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36dcc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f36d8073ea0 con 0x7f36d80cfdc0 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36d6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36d6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36628/0 (socket says 172.21.15.3:36628) 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36d6800640 1 -- 172.21.15.3:0/2265174532 learned_addr learned my addr 172.21.15.3:0/2265174532 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36cfe00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 0x7f36d81e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.725+0000 7f36d5e00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d8070f30 0x7f36d8157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 -- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 msgr2=0x7f36d81e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 0x7f36d81e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 -- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d8070f30 msgr2=0x7f36d8157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d8070f30 0x7f36d8157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 -- 172.21.15.3:0/2265174532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d8154520 con 0x7f36d815c720 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36cfe00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 0x7f36d81e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36d6800640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f36d0007df0 tx=0x7f36d000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d001a070 con 0x7f36d815c720 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d0003990 con 0x7f36d815c720 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d81e55a0 con 0x7f36d815c720 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0003d80 con 0x7f36d815c720 2024-09-10T07:53:50.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.726+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d81e5b60 con 0x7f36d815c720 2024-09-10T07:53:50.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.727+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d80cf2b0 con 0x7f36d815c720 2024-09-10T07:53:50.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.728+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36d001d070 con 0x7f36d815c720 2024-09-10T07:53:50.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.729+0000 7f36ce000640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 0x7f36b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:50.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.729+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f36d008b170 con 0x7f36d815c720 2024-09-10T07:53:50.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.729+0000 7f36d5e00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 0x7f36b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:50.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.730+0000 7f36d5e00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 0x7f36b406e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36c8000f60 tx=0x7f36c8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.733+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f36d0010070 con 0x7f36d815c720 2024-09-10T07:53:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.982+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f36d8071cb0 con 0x7f36d815c720 2024-09-10T07:53:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.983+0000 7f36ce000640 1 -- 172.21.15.3:0/2265174532 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7f36d00593a0 con 0x7f36d815c720 2024-09-10T07:53:50.983 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:50.983 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":464,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700188,"bytes_used":271269888,"bytes_avail":772822843392,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7507,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-09-10T07:53:47.589026+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 msgr2=0x7f36b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 0x7f36b406e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36c8000f60 tx=0x7f36c8007d50 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 msgr2=0x7f36d8153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f36d0007df0 tx=0x7f36d000d4a0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 shutdown_connections 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f36b406be30 0x7f36b406e2f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d815c720 0x7f36d8153fe0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36d80cfdc0 0x7f36d81e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 --2- 172.21.15.3:0/2265174532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d8070f30 0x7f36d8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:50.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 >> 172.21.15.3:0/2265174532 conn(0x7f36d806cfc0 msgr2=0x7f36d80b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 shutdown_connections 2024-09-10T07:53:50.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:50.985+0000 7f36dcc00640 1 -- 172.21.15.3:0/2265174532 wait complete. 2024-09-10T07:53:51.000 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:53:51.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:53:51.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-09-10T07:53:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.161+0000 7f6669a00640 1 Processor -- start 2024-09-10T07:53:51.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.161+0000 7f6669a00640 1 -- start start 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.162+0000 7f6669a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 0x7f66640d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.162+0000 7f6669a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66640d4f00 con 0x7f66640cfe90 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6669a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66640d3de0 con 0x7f6664161b70 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6669a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6664161f50 con 0x7f66640d45a0 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6669000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66640cfe90 0x7f66640d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49812/0 (socket says 172.21.15.3:49812) 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6669000640 1 -- 172.21.15.3:0/1653091046 learned_addr learned my addr 172.21.15.3:0/1653091046 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f665b400640 1 --2- 172.21.15.3:0/1653091046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 0x7f66640d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952715716 0 0) 0x7f66640d4f00 con 0x7f66640cfe90 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6648003520 con 0x7f66640cfe90 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796652085 0 0) 0x7f66640d3de0 con 0x7f6664161b70 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66640d4f00 con 0x7f6664161b70 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f665b400640 1 -- 172.21.15.3:0/1653091046 >> v1:172.21.15.70:6789/0 conn(0x7f6664161b70 legacy=0x7f66640d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.163+0000 7f665b400640 1 -- 172.21.15.3:0/1653091046 >> v1:172.21.15.3:6789/0 conn(0x7f66640cfe90 legacy=0x7f66640d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.164+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3931189928 0 0) 0x7f6648003520 con 0x7f66640cfe90 2024-09-10T07:53:51.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.164+0000 7f665b400640 1 -- 172.21.15.3:0/1653091046 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66640d40c0 con 0x7f66640d45a0 2024-09-10T07:53:51.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.165+0000 7f665b400640 1 --2- 172.21.15.3:0/1653091046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 0x7f66640d49c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097d0 tx=0x7f664c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f6cb72ef757e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.165+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c037070 con 0x7f66640d45a0 2024-09-10T07:53:51.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.165+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c02ac30 con 0x7f66640d45a0 2024-09-10T07:53:51.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.165+0000 7f6663400640 1 -- 172.21.15.3:0/1653091046 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0019f0 con 0x7f66640d45a0 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 -- 172.21.15.3:0/1653091046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 msgr2=0x7f66640d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 --2- 172.21.15.3:0/1653091046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 0x7f66640d49c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097d0 tx=0x7f664c02a1a0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 -- 172.21.15.3:0/1653091046 shutdown_connections 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 --2- 172.21.15.3:0/1653091046 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66640d45a0 0x7f66640d49c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 -- 172.21.15.3:0/1653091046 >> 172.21.15.3:0/1653091046 conn(0x7f6664062630 msgr2=0x7f6664062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 -- 172.21.15.3:0/1653091046 shutdown_connections 2024-09-10T07:53:51.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.166+0000 7f6669a00640 1 -- 172.21.15.3:0/1653091046 wait complete. 2024-09-10T07:53:51.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.167+0000 7f6669a00640 1 Processor -- start 2024-09-10T07:53:51.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.167+0000 7f6669a00640 1 -- start start 2024-09-10T07:53:51.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66640cfe90 0x7f6664151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6664161b70 0x7f66640ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66640c9fc0 con 0x7f66640d45a0 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66640cf970 con 0x7f66640cfe90 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.168+0000 7f6669a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66640d3a30 con 0x7f6664161b70 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36630/0 (socket says 172.21.15.3:36630) 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 -- 172.21.15.3:0/2499109471 learned_addr learned my addr 172.21.15.3:0/2499109471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f6663e00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66640cfe90 0x7f6664151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f6669000640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6664161b70 0x7f66640ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 -- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6664161b70 msgr2=0x7f66640ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6664161b70 0x7f66640ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 -- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66640cfe90 msgr2=0x7f6664151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66640cfe90 0x7f6664151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 -- 172.21.15.3:0/2499109471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66640b41b0 con 0x7f66640d45a0 2024-09-10T07:53:51.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.169+0000 7f665b400640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f664c0098f0 tx=0x7f664c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.170+0000 7f6660c00640 1 -- 172.21.15.3:0/2499109471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c037050 con 0x7f66640d45a0 2024-09-10T07:53:51.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.170+0000 7f6660c00640 1 -- 172.21.15.3:0/2499109471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c043070 con 0x7f66640d45a0 2024-09-10T07:53:51.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.170+0000 7f6660c00640 1 -- 172.21.15.3:0/2499109471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c03e690 con 0x7f66640d45a0 2024-09-10T07:53:51.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.170+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66640b4490 con 0x7f66640d45a0 2024-09-10T07:53:51.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.170+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66640b4a50 con 0x7f66640d45a0 2024-09-10T07:53:51.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.172+0000 7f6660c00640 1 -- 172.21.15.3:0/2499109471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f664c03e830 con 0x7f66640d45a0 2024-09-10T07:53:51.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.172+0000 7f6660c00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 0x7f664006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.173+0000 7f6660c00640 1 -- 172.21.15.3:0/2499109471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f664c0ab750 con 0x7f66640d45a0 2024-09-10T07:53:51.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.173+0000 7f6663e00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 0x7f664006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.173+0000 7f666de83740 1 Processor -- start 2024-09-10T07:53:51.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.173+0000 7f666de83740 1 -- start start 2024-09-10T07:53:51.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f6663e00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 0x7f664006e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f66540011e0 tx=0x7f6654009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadad9e0 0x5652dadac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652dad15660 con 0x5652dadc74f0 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652dacbd680 con 0x5652dadad9e0 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.174+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652dad11620 con 0x5652dadf7ea0 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36642/0 (socket says 172.21.15.3:36642) 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 -- 172.21.15.3:0/1164507193 learned_addr learned my addr 172.21.15.3:0/1164507193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6669000640 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f665b400640 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadad9e0 0x5652dadac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 -- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 msgr2=0x5652dadf83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 -- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadad9e0 msgr2=0x5652dadac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadad9e0 0x5652dadac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 -- 172.21.15.3:0/1164507193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652daaf27d0 con 0x5652dadc74f0 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.175+0000 7f6663e00640 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f665400fc00 tx=0x7f6654012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251469c22a058169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f6658c00640 1 -- 172.21.15.3:0/1164507193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654025070 con 0x5652dadc74f0 2024-09-10T07:53:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f6658c00640 1 -- 172.21.15.3:0/1164507193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665401b050 con 0x5652dadc74f0 2024-09-10T07:53:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f6658c00640 1 -- 172.21.15.3:0/1164507193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654020410 con 0x5652dadc74f0 2024-09-10T07:53:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 -- 172.21.15.3:0/1164507193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 msgr2=0x5652dadc78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f665400fc00 tx=0x7f6654012000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 -- 172.21.15.3:0/1164507193 shutdown_connections 2024-09-10T07:53:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadad9e0 0x5652dadac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 --2- 172.21.15.3:0/1164507193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652dadc78d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 -- 172.21.15.3:0/1164507193 >> 172.21.15.3:0/1164507193 conn(0x5652dadae410 msgr2=0x5652dadf6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 -- 172.21.15.3:0/1164507193 shutdown_connections 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.176+0000 7f666de83740 1 -- 172.21.15.3:0/1164507193 wait complete. 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.177+0000 7f666de83740 1 -- start start 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.177+0000 7f666de83740 1 Processor -- start 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadc74f0 0x5652dae078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 0x5652daded2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5652dadac940 con 0x5652dadf1870 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652dac9e4c0 con 0x5652dadf7ea0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652dadacc20 con 0x5652dadc74f0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f666de83740 10 client.0 _ll_register_callbacks cb 0x5652dad0de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f6669000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f6669000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36656/0 (socket says 172.21.15.3:36656) 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.178+0000 7f6669000640 1 -- 172.21.15.3:0/3217752522 learned_addr learned my addr 172.21.15.3:0/3217752522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6663e00640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadc74f0 0x5652dae078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f665b400640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 0x5652daded2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 -- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadc74f0 msgr2=0x5652dae078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadc74f0 0x5652dae078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 -- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 msgr2=0x5652daded2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 0x5652daded2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 -- 172.21.15.3:0/3217752522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652daa283f0 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f665b400640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 0x5652daded2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6669000640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f665c0076a0 tx=0x7f665c007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b285b82e07e98b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6651600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c018030 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f665c010b50 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.179+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665401b050 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f665c021050 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 10 client.8670 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5652da9e80f0 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665401b050 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f665c024020 con 0x5652dadf1870 2024-09-10T07:53:51.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 10 client.8670 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 10 client.8670 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 20 client.8670 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652daddb9f0 0x5652daddbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 4 client.8670 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652dad22d90 con 0x5652daddb9f0 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.180+0000 7f6663e00640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652daddb9f0 0x5652daddbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.181+0000 7f6663e00640 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652daddb9f0 0x5652daddbe00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b321cebe156817 server_cookie=c41e734445cc7750 in_seq=0 out_seq=0 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.181+0000 7f6651600640 10 client.8670 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.181+0000 7f6651600640 1 -- 172.21.15.3:0/3217752522 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652dad22d90 con 0x5652daddb9f0 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.181+0000 7f6651600640 10 client.8670 handle_command_reply: tid=0 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 client.8670 shutdown 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 msgr2=0x5652dadf3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f665c0076a0 tx=0x7f665c007b60 comp rx=0 tx=0).stop 2024-09-10T07:53:51.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 shutdown_connections 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652dadf3b40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf7ea0 0x5652daded2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652daddb9f0 0x5652daddbe00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 --2- 172.21.15.3:0/3217752522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadc74f0 0x5652dae078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 >> 172.21.15.3:0/3217752522 conn(0x5652dadad9e0 msgr2=0x5652dadab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f6651600640 0 client.8670 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 shutdown_connections 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 1 -- 172.21.15.3:0/3217752522 wait complete. 2024-09-10T07:53:51.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.182+0000 7f666de83740 20 client.8670 trim_cache size 0 max 16384 2024-09-10T07:53:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.200+0000 7f666de83740 1 Processor -- start 2024-09-10T07:53:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.200+0000 7f666de83740 1 -- start start 2024-09-10T07:53:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 0x5652dada7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dae1ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6664200940 con 0x5652dadc74f0 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66641ffcf0 con 0x5652dadf1870 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6664200210 con 0x5652dadf7ea0 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f6663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f6663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36664/0 (socket says 172.21.15.3:36664) 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.201+0000 7f6663e00640 1 -- 172.21.15.3:0/3856045072 learned_addr learned my addr 172.21.15.3:0/3856045072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 -- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 msgr2=0x5652dae1ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dae1ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f665b400640 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 0x5652dada7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 -- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 msgr2=0x5652dada7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 0x5652dada7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 -- 172.21.15.3:0/3856045072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652dadf4670 con 0x5652dadc74f0 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f665b400640 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 0x5652dada7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6663e00640 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6654014820 tx=0x7f6654015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41729bfa10f34dff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6653e00640 1 -- 172.21.15.3:0/3856045072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654025070 con 0x5652dadc74f0 2024-09-10T07:53:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6653e00640 1 -- 172.21.15.3:0/3856045072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665401b040 con 0x5652dadc74f0 2024-09-10T07:53:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.202+0000 7f6653e00640 1 -- 172.21.15.3:0/3856045072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665402e870 con 0x5652dadc74f0 2024-09-10T07:53:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 -- 172.21.15.3:0/3856045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 msgr2=0x5652daddbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6654014820 tx=0x7f6654015000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 -- 172.21.15.3:0/3856045072 shutdown_connections 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dae1ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadf1870 0x5652dada7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 --2- 172.21.15.3:0/3856045072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadc74f0 0x5652daddbdd0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 -- 172.21.15.3:0/3856045072 >> 172.21.15.3:0/3856045072 conn(0x5652dadad9e0 msgr2=0x5652dadaf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 -- 172.21.15.3:0/3856045072 shutdown_connections 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.203+0000 7f666de83740 1 -- 172.21.15.3:0/3856045072 wait complete. 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.204+0000 7f666de83740 1 -- start start 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.204+0000 7f666de83740 1 Processor -- start 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadc74f0 0x5652dae74c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66640cb6c0 con 0x5652dadf1870 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5652dada8050 con 0x5652dadc74f0 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5652dae1b3c0 con 0x5652dadf7ea0 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f666de83740 10 client.0 _ll_register_callbacks cb 0x7f665c001c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36670/0 (socket says 172.21.15.3:36670) 2024-09-10T07:53:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 -- 172.21.15.3:0/3732096473 learned_addr learned my addr 172.21.15.3:0/3732096473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f6663e00640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadc74f0 0x5652dae74c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f6669000640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 -- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 msgr2=0x5652dadf8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 -- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadc74f0 msgr2=0x5652dae74c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadc74f0 0x5652dae74c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f665b400640 1 -- 172.21.15.3:0/3732096473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652dadf9400 con 0x5652dadf1870 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.205+0000 7f6669000640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f665b400640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f664c035040 tx=0x7f664c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585781465073918 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c07dda0 con 0x5652dadf1870 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f664c079c50 con 0x5652dadf1870 2024-09-10T07:53:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665401b040 con 0x5652dadf1870 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f664c088050 con 0x5652dadf1870 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 10 client.8676 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5652dadf6cd0 con 0x5652dadf1870 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665401b040 con 0x5652dadf1870 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f664c08b020 con 0x5652dadf1870 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 10 client.8676 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 10 client.8676 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 20 client.8676 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652dadf36a0 0x5652dadf88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 4 client.8676 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.206+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x5652dad1d0e0 con 0x5652dadf36a0 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f6663e00640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652dadf36a0 0x5652dadf88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f6663e00640 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652dadf36a0 0x5652dadf88c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8b67d4006e314c server_cookie=50a407b43f5cf0fe in_seq=0 out_seq=0 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f6651600640 10 client.8676 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f6651600640 1 -- 172.21.15.3:0/3732096473 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5652dad1d0e0 con 0x5652dadf36a0 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f6651600640 10 client.8676 handle_command_reply: tid=0 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.207+0000 7f666de83740 1 client.8676 shutdown 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 msgr2=0x5652daddd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f664c035040 tx=0x7f664c009520 comp rx=0 tx=0).stop 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 shutdown_connections 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x5652dadf36a0 0x5652dadf88c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5652dadf7ea0 0x5652dadf8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5652dadf1870 0x5652daddd440 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 --2- 172.21.15.3:0/3732096473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5652dadc74f0 0x5652dae74c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 >> 172.21.15.3:0/3732096473 conn(0x5652dadad9e0 msgr2=0x5652daddbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f6651600640 0 client.8676 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 shutdown_connections 2024-09-10T07:53:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 1 -- 172.21.15.3:0/3732096473 wait complete. 2024-09-10T07:53:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.208+0000 7f666de83740 20 client.8676 trim_cache size 0 max 16384 2024-09-10T07:53:51.212 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0 2024-09-10T07:53:51.212 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 msgr2=0x7f664006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 0x7f664006e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f66540011e0 tx=0x7f6654009040 comp rx=0 tx=0).stop 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 msgr2=0x7f6664153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f664c0098f0 tx=0x7f664c0046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 shutdown_connections 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f664006be60 0x7f664006e320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6664161b70 0x7f66640ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66640d45a0 0x7f6664153690 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 --2- 172.21.15.3:0/2499109471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66640cfe90 0x7f6664151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 >> 172.21.15.3:0/2499109471 conn(0x7f6664062630 msgr2=0x7f66640d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 shutdown_connections 2024-09-10T07:53:51.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.209+0000 7f6669a00640 1 -- 172.21.15.3:0/2499109471 wait complete. 2024-09-10T07:53:51.224 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:51.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-10T07:53:51.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.385+0000 7f8a36600640 1 Processor -- start 2024-09-10T07:53:51.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.385+0000 7f8a36600640 1 -- start start 2024-09-10T07:53:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.386+0000 7f8a36600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8a3006bb20 0x7f8a3006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.386+0000 7f8a36600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a300d2ed0 con 0x7f8a3015e820 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.386+0000 7f8a36600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a300d31b0 con 0x7f8a300cf3b0 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.386+0000 7f8a36600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3006c4b0 con 0x7f8a3006bb20 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2f400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a3015e820 0x7f8a300d3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49828/0 (socket says 172.21.15.3:49828) 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2f400640 1 -- 172.21.15.3:0/3465353483 learned_addr learned my addr 172.21.15.3:0/3465353483 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3465353483 >> v2:172.21.15.3:3300/0 conn(0x7f8a3006bb20 0x7f8a3006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2ea00640 1 -- 172.21.15.3:0/3465353483 >> v1:172.21.15.70:6789/0 conn(0x7f8a300cf3b0 legacy=0x7f8a300cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2ea00640 1 -- 172.21.15.3:0/3465353483 >> v1:172.21.15.3:6789/0 conn(0x7f8a3015e820 legacy=0x7f8a300d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2ea00640 1 -- 172.21.15.3:0/3465353483 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a300d3490 con 0x7f8a3006bb20 2024-09-10T07:53:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.387+0000 7f8a2e000640 1 -- 172.21.15.3:0/3465353483 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839708081 0 0) 0x7f8a300d2ed0 con 0x7f8a3015e820 2024-09-10T07:53:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.388+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3465353483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a3006bf70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097a0 tx=0x7f8a1c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507c6644ec686d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.388+0000 7f8a2e000640 1 -- 172.21.15.3:0/3465353483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c037070 con 0x7f8a3006bb20 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.388+0000 7f8a2e000640 1 -- 172.21.15.3:0/3465353483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c032440 con 0x7f8a3006bb20 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.388+0000 7f8a2e000640 1 -- 172.21.15.3:0/3465353483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c032810 con 0x7f8a3006bb20 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 -- 172.21.15.3:0/3465353483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 msgr2=0x7f8a3006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 --2- 172.21.15.3:0/3465353483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a3006bf70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097a0 tx=0x7f8a1c02a230 comp rx=0 tx=0).stop 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 -- 172.21.15.3:0/3465353483 shutdown_connections 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 --2- 172.21.15.3:0/3465353483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a3006bf70 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 -- 172.21.15.3:0/3465353483 >> 172.21.15.3:0/3465353483 conn(0x7f8a30072770 msgr2=0x7f8a300d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 -- 172.21.15.3:0/3465353483 shutdown_connections 2024-09-10T07:53:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.389+0000 7f8a36600640 1 -- 172.21.15.3:0/3465353483 wait complete. 2024-09-10T07:53:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.391+0000 7f8a36600640 1 Processor -- start 2024-09-10T07:53:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.391+0000 7f8a36600640 1 -- start start 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a300cf3b0 0x7f8a300abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a3015e820 0x7f8a300b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300d3a50 con 0x7f8a3006bb20 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300d2c80 con 0x7f8a3015e820 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a36600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8a300b7730 con 0x7f8a300cf3b0 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36692/0 (socket says 172.21.15.3:36692) 2024-09-10T07:53:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 -- 172.21.15.3:0/3811954485 learned_addr learned my addr 172.21.15.3:0/3811954485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a300cf3b0 0x7f8a300abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a3015e820 0x7f8a300b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 -- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a300cf3b0 msgr2=0x7f8a300abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a300cf3b0 0x7f8a300abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 -- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a3015e820 msgr2=0x7f8a300b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a3015e820 0x7f8a300b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.392+0000 7f8a2f400640 1 -- 172.21.15.3:0/3811954485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a30160b20 con 0x7f8a3006bb20 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c3f0 tx=0x7f8a2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a27400640 1 -- 172.21.15.3:0/3811954485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200074e0 con 0x7f8a3006bb20 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a27400640 1 -- 172.21.15.3:0/3811954485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a200047e0 con 0x7f8a3006bb20 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a27400640 1 -- 172.21.15.3:0/3811954485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20004c10 con 0x7f8a3006bb20 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a30160e00 con 0x7f8a3006bb20 2024-09-10T07:53:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.393+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301616a0 con 0x7f8a3006bb20 2024-09-10T07:53:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.395+0000 7f8a27400640 1 -- 172.21.15.3:0/3811954485 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a20011070 con 0x7f8a3006bb20 2024-09-10T07:53:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.396+0000 7f8a27400640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 0x7f8a1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.396+0000 7f8a27400640 1 -- 172.21.15.3:0/3811954485 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8a2008aa90 con 0x7f8a3006bb20 2024-09-10T07:53:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.396+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 0x7f8a1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.397+0000 7f8a3948c740 1 Processor -- start 2024-09-10T07:53:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.397+0000 7f8a3948c740 1 -- start start 2024-09-10T07:53:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.397+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 0x7f8a1006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f978480 0x55d93f9c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d93f8e0660 con 0x55d93f992420 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d93f888680 con 0x55d93f978480 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d93f8dc620 con 0x55d93f992800 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36698/0 (socket says 172.21.15.3:36698) 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 -- 172.21.15.3:0/3860169629 learned_addr learned my addr 172.21.15.3:0/3860169629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f978480 0x55d93f9c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 -- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 msgr2=0x55d93f9c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 -- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f978480 msgr2=0x55d93f9c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f978480 0x55d93f9c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 -- 172.21.15.3:0/3860169629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93f6bd7d0 con 0x55d93f992420 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.398+0000 7f8a2f400640 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a20059fd0 tx=0x7f8a200593a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef653c9e814c6903 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a1be00640 1 -- 172.21.15.3:0/3860169629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2005d050 con 0x55d93f992420 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a1be00640 1 -- 172.21.15.3:0/3860169629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20061880 con 0x55d93f992420 2024-09-10T07:53:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a1be00640 1 -- 172.21.15.3:0/3860169629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20056d40 con 0x55d93f992420 2024-09-10T07:53:51.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 -- 172.21.15.3:0/3860169629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 msgr2=0x55d93f9c2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a20059fd0 tx=0x7f8a200593a0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 -- 172.21.15.3:0/3860169629 shutdown_connections 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f978480 0x55d93f9c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 --2- 172.21.15.3:0/3860169629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992420 0x55d93f9c2c40 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 -- 172.21.15.3:0/3860169629 >> 172.21.15.3:0/3860169629 conn(0x55d93f979220 msgr2=0x55d93f9c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 -- 172.21.15.3:0/3860169629 shutdown_connections 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.399+0000 7f8a3948c740 1 -- 172.21.15.3:0/3860169629 wait complete. 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.400+0000 7f8a3948c740 1 -- start start 2024-09-10T07:53:51.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.400+0000 7f8a3948c740 1 Processor -- start 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 0x55d93f9a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d93f9773e0 con 0x55d93f9be690 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d93f8694c0 con 0x55d93f992420 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d93f9776c0 con 0x55d93f992800 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a3948c740 10 client.0 _ll_register_callbacks cb 0x55d93f8d8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36706/0 (socket says 172.21.15.3:36706) 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.401+0000 7f8a2fe00640 1 -- 172.21.15.3:0/2403364292 learned_addr learned my addr 172.21.15.3:0/2403364292 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2f400640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 0x55d93f9a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 -- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 msgr2=0x55d93f9be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 -- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 msgr2=0x55d93f9a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 0x55d93f9a8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 -- 172.21.15.3:0/2403364292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93f5f33f0 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2f400640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 0x55d93f9a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005980 tx=0x7f8a28001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d393d099cc9dda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2801b060 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8a28016b10 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a20061880 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f8a28016d10 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a3948c740 10 client.8691 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.402+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55d93f5b30f0 con 0x55d93f9be690 2024-09-10T07:53:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20061880 con 0x55d93f9be690 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f8a280124d0 con 0x55d93f9be690 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 10 client.8691 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 10 client.8691 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 20 client.8691 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9aadc0 0x55d93f9d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 4 client.8691 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93f8edd90 con 0x55d93f9aadc0 2024-09-10T07:53:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.403+0000 7f8a2f400640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9aadc0 0x55d93f9d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.404+0000 7f8a2f400640 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9aadc0 0x55d93f9d5080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfae0a7d77b01295 server_cookie=ec9c964c06301acf in_seq=0 out_seq=0 2024-09-10T07:53:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.404+0000 7f8a18c00640 10 client.8691 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.404+0000 7f8a18c00640 1 -- 172.21.15.3:0/2403364292 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d93f8edd90 con 0x55d93f9aadc0 2024-09-10T07:53:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.404+0000 7f8a18c00640 10 client.8691 handle_command_reply: tid=0 2024-09-10T07:53:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.404+0000 7f8a3948c740 1 client.8691 shutdown 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 msgr2=0x55d93f9bcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005980 tx=0x7f8a28001930 comp rx=0 tx=0).stop 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 shutdown_connections 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9aadc0 0x55d93f9d5080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9bcec0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992800 0x55d93f9be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 --2- 172.21.15.3:0/2403364292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992420 0x55d93f9a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a18c00640 0 client.8691 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 >> 172.21.15.3:0/2403364292 conn(0x55d93f978480 msgr2=0x55d93f9c24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 shutdown_connections 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 1 -- 172.21.15.3:0/2403364292 wait complete. 2024-09-10T07:53:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.405+0000 7f8a3948c740 20 client.8691 trim_cache size 0 max 16384 2024-09-10T07:53:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.423+0000 7f8a3948c740 1 Processor -- start 2024-09-10T07:53:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.423+0000 7f8a3948c740 1 -- start start 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93f9e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 0x55d93f9a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30200b00 con 0x55d93f9be690 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a301ffcc0 con 0x55d93f992800 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8a302003d0 con 0x55d93f992420 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2024-09-10T07:53:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.424+0000 7f8a2fe00640 1 -- 172.21.15.3:0/1926377397 learned_addr learned my addr 172.21.15.3:0/1926377397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2f400640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93f9e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 -- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 msgr2=0x55d93f9e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 0x55d93f9a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93f9e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 -- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 msgr2=0x55d93f9a6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 0x55d93f9a6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 -- 172.21.15.3:0/1926377397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93f9a71c0 con 0x55d93f9be690 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2f400640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93f9e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 0x55d93f9a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a28012a80 tx=0x7f8a28010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beed991ac61ad0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a1b400640 1 -- 172.21.15.3:0/1926377397 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28003d50 con 0x55d93f9be690 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a1b400640 1 -- 172.21.15.3:0/1926377397 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a28001e30 con 0x55d93f9be690 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.425+0000 7f8a1b400640 1 -- 172.21.15.3:0/1926377397 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a280053f0 con 0x55d93f9be690 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- 172.21.15.3:0/1926377397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 msgr2=0x55d93f9d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a28012a80 tx=0x7f8a28010000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- 172.21.15.3:0/1926377397 shutdown_connections 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f9be690 0x55d93f9d1f70 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f992800 0x55d93f9a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 --2- 172.21.15.3:0/1926377397 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93f9e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- 172.21.15.3:0/1926377397 >> 172.21.15.3:0/1926377397 conn(0x55d93f978480 msgr2=0x55d93f9c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- 172.21.15.3:0/1926377397 shutdown_connections 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- 172.21.15.3:0/1926377397 wait complete. 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.426+0000 7f8a3948c740 1 -- start start 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.427+0000 7f8a3948c740 1 Processor -- start 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93fa40b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f9be690 0x55d93f9ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3015fa50 con 0x55d93f992800 2024-09-10T07:53:51.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55d93f9d4e10 con 0x55d93f9be690 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55d93fa41450 con 0x55d93f992420 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a3948c740 10 client.0 _ll_register_callbacks cb 0x7f8a28003b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36738/0 (socket says 172.21.15.3:36738) 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2ea00640 1 -- 172.21.15.3:0/897123945 learned_addr learned my addr 172.21.15.3:0/897123945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93fa40b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2fe00640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f9be690 0x55d93f9ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 -- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f9be690 msgr2=0x55d93f9ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f9be690 0x55d93f9ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 -- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 msgr2=0x55d93fa3fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2f400640 1 -- 172.21.15.3:0/897123945 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55d93f976100 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.428+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a2f400640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93fa40b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a20061350 tx=0x7f8a2005b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ebd23e76c211f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.70:3301/0 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20061880 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mon.2 v2:172.21.15.70:3301/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8a20089030 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a28001e30 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f8a20082030 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a3948c740 10 client.7304 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.429+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55d93fa41250 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mon.2 v2:172.21.15.70:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a28001e30 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mon.2 v2:172.21.15.70:3301/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f8a20055310 con 0x55d93f992420 2024-09-10T07:53:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 10 client.7304 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 10 client.7304 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.a 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 20 client.7304 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9bae00 0x55d93f9d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 4 client.7304 mds_command: new command op to 6875 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 --> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d93f8e80e0 con 0x55d93f9bae00 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.430+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9bae00 0x55d93f9d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a2ea00640 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9bae00 0x55d93f9d1f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bba673b46b555bc server_cookie=7cc5a532b4aca1f9 in_seq=0 out_seq=0 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a18c00640 10 client.7304 ms_handle_connect on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a18c00640 1 -- 172.21.15.3:0/897123945 <== mds.1 v2:172.21.15.3:6834/3004970849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d93f8e80e0 con 0x55d93f9bae00 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a18c00640 10 client.7304 handle_command_reply: tid=0 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 client.7304 shutdown 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 msgr2=0x55d93fa40b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93fa40b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a20061350 tx=0x7f8a2005b040 comp rx=0 tx=0).stop 2024-09-10T07:53:51.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 shutdown_connections 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:6834/3004970849,v1:172.21.15.3:6836/3004970849] conn(0x55d93f9bae00 0x55d93f9d1f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d93f9be690 0x55d93f9ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d93f992800 0x55d93fa3fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 --2- 172.21.15.3:0/897123945 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55d93f992420 0x55d93fa40b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a18c00640 0 client.7304 ms_handle_reset on v2:172.21.15.3:6834/3004970849 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 >> 172.21.15.3:0/897123945 conn(0x55d93f978480 msgr2=0x55d93f9a8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 shutdown_connections 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.431+0000 7f8a3948c740 1 -- 172.21.15.3:0/897123945 wait complete. 2024-09-10T07:53:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a3948c740 20 client.7304 trim_cache size 0 max 16384 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.440 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 msgr2=0x7f8a1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 0x7f8a1006e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c02e000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 msgr2=0x7f8a301549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c3f0 tx=0x7f8a2000c8b0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 shutdown_connections 2024-09-10T07:53:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8a1006be30 0x7f8a1006e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a3015e820 0x7f8a300b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a300cf3b0 0x7f8a300abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 --2- 172.21.15.3:0/3811954485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3006bb20 0x7f8a301549a0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 >> 172.21.15.3:0/3811954485 conn(0x7f8a30072770 msgr2=0x7f8a300cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 shutdown_connections 2024-09-10T07:53:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.432+0000 7f8a36600640 1 -- 172.21.15.3:0/3811954485 wait complete. 2024-09-10T07:53:51.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:51.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:51.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-10T07:53:51.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.609+0000 7f52c6400640 1 Processor -- start 2024-09-10T07:53:51.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.609+0000 7f52c6400640 1 -- start start 2024-09-10T07:53:51.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c6400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 0x7f52c00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c6400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c0071760 con 0x7f52c015e530 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c6400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c006c660 con 0x7f52c006d700 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c6400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52c006db70 con 0x7f52c00b71d0 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c5000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52c015e530 0x7f52c0071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49836/0 (socket says 172.21.15.3:49836) 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c5000640 1 -- 172.21.15.3:0/816619468 learned_addr learned my addr 172.21.15.3:0/816619468 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.610+0000 7f52c5a00640 1 --2- 172.21.15.3:0/816619468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 0x7f52c00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472906145 0 0) 0x7f52c0071760 con 0x7f52c015e530 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a8003520 con 0x7f52c015e530 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4194733751 0 0) 0x7f52c006c660 con 0x7f52c006d700 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52c0071760 con 0x7f52c006d700 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52c5a00640 1 -- 172.21.15.3:0/816619468 >> v1:172.21.15.70:6789/0 conn(0x7f52c006d700 legacy=0x7f52c00b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52c5a00640 1 -- 172.21.15.3:0/816619468 >> v1:172.21.15.3:6789/0 conn(0x7f52c015e530 legacy=0x7f52c0071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1853466653 0 0) 0x7f52a8003520 con 0x7f52c015e530 2024-09-10T07:53:51.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.611+0000 7f52c5a00640 1 -- 172.21.15.3:0/816619468 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c006c940 con 0x7f52c00b71d0 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.612+0000 7f52c5a00640 1 --2- 172.21.15.3:0/816619468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 0x7f52c00ae8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009dc0 tx=0x7f52b802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37f1ffdd020acac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4001690 con 0x7f52c00b71d0 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b8007540 con 0x7f52c00b71d0 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52bf400640 1 -- 172.21.15.3:0/816619468 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8007910 con 0x7f52c00b71d0 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 -- 172.21.15.3:0/816619468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 msgr2=0x7f52c00ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 --2- 172.21.15.3:0/816619468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 0x7f52c00ae8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009dc0 tx=0x7f52b802a660 comp rx=0 tx=0).stop 2024-09-10T07:53:51.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 -- 172.21.15.3:0/816619468 shutdown_connections 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 --2- 172.21.15.3:0/816619468 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c00b71d0 0x7f52c00ae8a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 -- 172.21.15.3:0/816619468 >> 172.21.15.3:0/816619468 conn(0x7f52c006f210 msgr2=0x7f52c006f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 -- 172.21.15.3:0/816619468 shutdown_connections 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.613+0000 7f52c6400640 1 -- 172.21.15.3:0/816619468 wait complete. 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.615+0000 7f52c6400640 1 Processor -- start 2024-09-10T07:53:51.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.615+0000 7f52c6400640 1 -- start start 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 0x7f52c00b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 0x7f52c015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52c006d1e0 con 0x7f52c015e530 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52c006cf00 con 0x7f52c00b71d0 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c6400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52c015e910 con 0x7f52c006d700 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36750/0 (socket says 172.21.15.3:36750) 2024-09-10T07:53:51.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.616+0000 7f52c5a00640 1 -- 172.21.15.3:0/1396727922 learned_addr learned my addr 172.21.15.3:0/1396727922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5000640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 0x7f52c00b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 -- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 msgr2=0x7f52c00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52bfe00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 0x7f52c015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 0x7f52c00b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 -- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 msgr2=0x7f52c015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 0x7f52c015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 -- 172.21.15.3:0/1396727922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c015fa70 con 0x7f52c015e530 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5000640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 0x7f52c00b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52bfe00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 0x7f52c015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c5a00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009ee0 tx=0x7f52b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52bcc00640 1 -- 172.21.15.3:0/1396727922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8040070 con 0x7f52c015e530 2024-09-10T07:53:51.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52bcc00640 1 -- 172.21.15.3:0/1396727922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b8003b90 con 0x7f52c015e530 2024-09-10T07:53:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52bcc00640 1 -- 172.21.15.3:0/1396727922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8007e90 con 0x7f52c015e530 2024-09-10T07:53:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c00b4930 con 0x7f52c015e530 2024-09-10T07:53:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.617+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c0157020 con 0x7f52c015e530 2024-09-10T07:53:51.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.619+0000 7f52bcc00640 1 -- 172.21.15.3:0/1396727922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52b8039040 con 0x7f52c015e530 2024-09-10T07:53:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.620+0000 7f52bcc00640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 0x7f52a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.620+0000 7f52bcc00640 1 -- 172.21.15.3:0/1396727922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f52b80ac1a0 con 0x7f52c015e530 2024-09-10T07:53:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.621+0000 7f52c5000640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 0x7f52a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.621+0000 7f52ca9d8740 1 Processor -- start 2024-09-10T07:53:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.621+0000 7f52ca9d8740 1 -- start start 2024-09-10T07:53:51.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.621+0000 7f52c5000640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 0x7f52a006e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000f60 tx=0x7f52b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1b874f0 0x55e3b1b878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b6d9e0 0x55e3b1b6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e3b1ad5660 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e3b1a7d680 con 0x55e3b1b6d9e0 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e3b1ad1620 con 0x55e3b1b874f0 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36756/0 (socket says 172.21.15.3:36756) 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.622+0000 7f52c5a00640 1 -- 172.21.15.3:0/2504968128 learned_addr learned my addr 172.21.15.3:0/2504968128 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 -- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1b874f0 msgr2=0x55e3b1b878d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1b874f0 0x55e3b1b878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 -- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b6d9e0 msgr2=0x55e3b1b6c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b6d9e0 0x55e3b1b6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 -- 172.21.15.3:0/2504968128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b18b27d0 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52c5a00640 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f52b807a000 tx=0x7f52b8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a2301a217fecc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52b1600640 1 -- 172.21.15.3:0/2504968128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8049070 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52b1600640 1 -- 172.21.15.3:0/2504968128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b807a440 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.623+0000 7f52b1600640 1 -- 172.21.15.3:0/2504968128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b807e7f0 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 -- 172.21.15.3:0/2504968128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 msgr2=0x55e3b1bb83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f52b807a000 tx=0x7f52b8009b90 comp rx=0 tx=0).stop 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 -- 172.21.15.3:0/2504968128 shutdown_connections 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1bb83f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b6d9e0 0x55e3b1b6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/2504968128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1b874f0 0x55e3b1b878d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 -- 172.21.15.3:0/2504968128 >> 172.21.15.3:0/2504968128 conn(0x55e3b1b6e410 msgr2=0x55e3b1bb6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 -- 172.21.15.3:0/2504968128 shutdown_connections 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.624+0000 7f52ca9d8740 1 -- 172.21.15.3:0/2504968128 wait complete. 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.625+0000 7f52ca9d8740 1 -- start start 2024-09-10T07:53:51.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.625+0000 7f52ca9d8740 1 Processor -- start 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1bb3540 0x55e3b1bb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e3b1b6c940 con 0x55e3b1b874f0 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55e3b1a5e4c0 con 0x55e3b1bb3540 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e3b1b6cc20 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52ca9d8740 10 client.0 _ll_register_callbacks cb 0x55e3b1acde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36768/0 (socket says 172.21.15.3:36768) 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 -- 172.21.15.3:0/4058048346 learned_addr learned my addr 172.21.15.3:0/4058048346 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5a00640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1bb3540 0x55e3b1bb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52bfe00640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 -- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 msgr2=0x55e3b1bada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 -- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1bb3540 msgr2=0x55e3b1bb1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1bb3540 0x55e3b1bb1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.626+0000 7f52c5000640 1 -- 172.21.15.3:0/4058048346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b17e83f0 con 0x55e3b1b874f0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52c5000640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005b00 tx=0x7f52b4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52f063d2fccb171 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52a6000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4021620 con 0x55e3b1b874f0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f52b4012410 con 0x55e3b1b874f0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b807a440 con 0x55e3b1b874f0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f52b4012650 con 0x55e3b1b874f0 2024-09-10T07:53:51.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52ca9d8740 10 client.8709 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.627+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55e3b17a80f0 con 0x55e3b1b874f0 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b807a440 con 0x55e3b1b874f0 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f52b4029050 con 0x55e3b1b874f0 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 10 client.8709 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 10 client.8709 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 20 client.8709 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1bc9f80 0x55e3b1bca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 4 client.8709 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b1ae2d90 con 0x55e3b1bc9f80 2024-09-10T07:53:51.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.628+0000 7f52bfe00640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1bc9f80 0x55e3b1bca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.629+0000 7f52bfe00640 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1bc9f80 0x55e3b1bca360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e936acbbe435e3b3 server_cookie=c66a41943a490ccb in_seq=0 out_seq=0 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.629+0000 7f52a6000640 10 client.8709 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52a6000640 1 -- 172.21.15.3:0/4058048346 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3b1ae2d90 con 0x55e3b1bc9f80 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52a6000640 10 client.8709 handle_command_reply: tid=0 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 client.8709 shutdown 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 msgr2=0x55e3b1b9bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005b00 tx=0x7f52b4005f00 comp rx=0 tx=0).stop 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 shutdown_connections 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1bc9f80 0x55e3b1bca360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1bb3540 0x55e3b1bb1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52a6000640 0 client.8709 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/4058048346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1b874f0 0x55e3b1b9bad0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.630+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 >> 172.21.15.3:0/4058048346 conn(0x55e3b1b6d9e0 msgr2=0x55e3b1bb6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.631+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 shutdown_connections 2024-09-10T07:53:51.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.631+0000 7f52ca9d8740 1 -- 172.21.15.3:0/4058048346 wait complete. 2024-09-10T07:53:51.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.631+0000 7f52ca9d8740 20 client.8709 trim_cache size 0 max 16384 2024-09-10T07:53:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.649+0000 7f52ca9d8740 1 Processor -- start 2024-09-10T07:53:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.649+0000 7f52ca9d8740 1 -- start start 2024-09-10T07:53:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1b87970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0200aa0 con 0x55e3b1bb3540 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ffc40 con 0x55e3b1b874f0 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0200370 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36784/0 (socket says 172.21.15.3:36784) 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.650+0000 7f52bfe00640 1 -- 172.21.15.3:0/1922186451 learned_addr learned my addr 172.21.15.3:0/1922186451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 -- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 msgr2=0x55e3b1bafbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 -- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 msgr2=0x55e3b1b87970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52c5000640 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1b87970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1b87970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 -- 172.21.15.3:0/1922186451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b1badda0 con 0x55e3b1bb3540 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52c5000640 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1b87970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52bfe00640 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f52c006bbd0 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bbbf92794f1bef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52b0c00640 1 -- 172.21.15.3:0/1922186451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0036b0 con 0x55e3b1bb3540 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52b0c00640 1 -- 172.21.15.3:0/1922186451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac003850 con 0x55e3b1bb3540 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.651+0000 7f52b0c00640 1 -- 172.21.15.3:0/1922186451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac003c40 con 0x55e3b1bb3540 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 -- 172.21.15.3:0/1922186451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 msgr2=0x55e3b1bacb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f52c006bbd0 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 -- 172.21.15.3:0/1922186451 shutdown_connections 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb7ea0 0x55e3b1bafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb3540 0x55e3b1bacb50 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/1922186451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1b87970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 -- 172.21.15.3:0/1922186451 >> 172.21.15.3:0/1922186451 conn(0x55e3b1b6d9e0 msgr2=0x55e3b1b67ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 -- 172.21.15.3:0/1922186451 shutdown_connections 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.652+0000 7f52ca9d8740 1 -- 172.21.15.3:0/1922186451 wait complete. 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.653+0000 7f52ca9d8740 1 -- start start 2024-09-10T07:53:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.653+0000 7f52ca9d8740 1 Processor -- start 2024-09-10T07:53:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1bda650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 0x55e3b1bb8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e3b1bad4c0 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0157b90 con 0x55e3b1b874f0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55e3b1bb8f50 con 0x55e3b1bb3540 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52ca9d8740 10 client.0 _ll_register_callbacks cb 0x7f52b4001ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36788/0 (socket says 172.21.15.3:36788) 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 -- 172.21.15.3:0/641804306 learned_addr learned my addr 172.21.15.3:0/641804306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5000640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1bda650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52bfe00640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 0x55e3b1bb8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 -- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 msgr2=0x55e3b1bb8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 0x55e3b1bb8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 -- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 msgr2=0x55e3b1bda650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1bda650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52c5a00640 1 -- 172.21.15.3:0/641804306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b1bd9ec0 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.654+0000 7f52bfe00640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 0x55e3b1bb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52c5a00640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f52b8082af0 tx=0x7f52b80aa590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b64c3bedb4dacbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8049030 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f52b807a6a0 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac003850 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f52b8083020 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52ca9d8740 10 client.8715 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55e3b1bdab90 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac003850 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f52b8082b90 con 0x55e3b1bb7ea0 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52ca9d8740 10 client.8715 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.655+0000 7f52ca9d8740 10 client.8715 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.656+0000 7f52ca9d8740 20 client.8715 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.656+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1b6b870 0x55e3b1c2a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.656+0000 7f52ca9d8740 4 client.8715 mds_command: new command op to 6872 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.656+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 --> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3b1b9d920 con 0x55e3b1b6b870 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.656+0000 7f52c5000640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1b6b870 0x55e3b1c2a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52c5000640 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1b6b870 0x55e3b1c2a970 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f24e38e111bb5eb8 server_cookie=ae6fcfb8a3a034f3 in_seq=0 out_seq=0 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52a6000640 10 client.8715 ms_handle_connect on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52a6000640 1 -- 172.21.15.3:0/641804306 <== mds.2 v2:172.21.15.70:6833/3268462742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e3b1b9d920 con 0x55e3b1b6b870 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52a6000640 10 client.8715 handle_command_reply: tid=0 2024-09-10T07:53:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 client.8715 shutdown 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 msgr2=0x55e3b1b6b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f52b8082af0 tx=0x7f52b80aa590 comp rx=0 tx=0).stop 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 shutdown_connections 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:6833/3268462742,v1:172.21.15.70:6835/3268462742] conn(0x55e3b1b6b870 0x55e3b1c2a970 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e3b1bb7ea0 0x55e3b1b6b330 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55e3b1bb3540 0x55e3b1bb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 --2- 172.21.15.3:0/641804306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e3b1b874f0 0x55e3b1bda650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52a6000640 0 client.8715 ms_handle_reset on v2:172.21.15.70:6833/3268462742 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 >> 172.21.15.3:0/641804306 conn(0x55e3b1b6d9e0 msgr2=0x55e3b1bacb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 shutdown_connections 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 1 -- 172.21.15.3:0/641804306 wait complete. 2024-09-10T07:53:51.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.657+0000 7f52ca9d8740 20 client.8715 trim_cache size 0 max 16384 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 msgr2=0x7f52a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 0x7f52a006e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000f60 tx=0x7f52b4009040 comp rx=0 tx=0).stop 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 msgr2=0x7f52c01438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009ee0 tx=0x7f52b8004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 shutdown_connections 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f52a006be30 0x7f52a006e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52c015e530 0x7f52c01438e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c00b71d0 0x7f52c015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 --2- 172.21.15.3:0/1396727922 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52c006d700 0x7f52c00b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 >> 172.21.15.3:0/1396727922 conn(0x7f52c006f210 msgr2=0x7f52c015a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 shutdown_connections 2024-09-10T07:53:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.658+0000 7f52c6400640 1 -- 172.21.15.3:0/1396727922 wait complete. 2024-09-10T07:53:51.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:51.673 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-10T07:53:51.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T07:53:51.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.835+0000 7f93a6c00640 1 Processor -- start 2024-09-10T07:53:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.835+0000 7f93a6c00640 1 -- start start 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93a00d45a0 0x7f93a00d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a00d4f00 con 0x7f93a00cfe90 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a00d3de0 con 0x7f93a0161b70 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f93a0161f50 con 0x7f93a00d45a0 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f93a00cfe90 0x7f93a00d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49844/0 (socket says 172.21.15.3:49844) 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a6200640 1 -- 172.21.15.3:0/1696036975 learned_addr learned my addr 172.21.15.3:0/1696036975 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.836+0000 7f93a4e00640 1 --2- 172.21.15.3:0/1696036975 >> v2:172.21.15.70:3300/0 conn(0x7f93a00d45a0 0x7f93a00d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.837+0000 7f939fe00640 1 -- 172.21.15.3:0/1696036975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515212325 0 0) 0x7f93a00d4f00 con 0x7f93a00cfe90 2024-09-10T07:53:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.837+0000 7f939fe00640 1 -- 172.21.15.3:0/1696036975 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9388003520 con 0x7f93a00cfe90 2024-09-10T07:53:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.837+0000 7f93a4e00640 1 -- 172.21.15.3:0/1696036975 >> v1:172.21.15.70:6789/0 conn(0x7f93a0161b70 legacy=0x7f93a00d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.837+0000 7f93a4e00640 1 -- 172.21.15.3:0/1696036975 >> v1:172.21.15.3:6789/0 conn(0x7f93a00cfe90 legacy=0x7f93a00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.837+0000 7f93a4e00640 1 -- 172.21.15.3:0/1696036975 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a00d40c0 con 0x7f93a00d45a0 2024-09-10T07:53:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.838+0000 7f93a4e00640 1 --2- 172.21.15.3:0/1696036975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00d45a0 0x7f93a00d49c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f938c0097a0 tx=0x7f938c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8ae1c7a7b010b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.838+0000 7f939fe00640 1 -- 172.21.15.3:0/1696036975 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c037070 con 0x7f93a00d45a0 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.838+0000 7f939fe00640 1 -- 172.21.15.3:0/1696036975 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c032440 con 0x7f93a00d45a0 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.838+0000 7f939fe00640 1 -- 172.21.15.3:0/1696036975 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c032810 con 0x7f93a00d45a0 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 -- 172.21.15.3:0/1696036975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00d45a0 msgr2=0x7f93a00d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 --2- 172.21.15.3:0/1696036975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00d45a0 0x7f93a00d49c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f938c0097a0 tx=0x7f938c02a230 comp rx=0 tx=0).stop 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 -- 172.21.15.3:0/1696036975 shutdown_connections 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 --2- 172.21.15.3:0/1696036975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00d45a0 0x7f93a00d49c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 -- 172.21.15.3:0/1696036975 >> 172.21.15.3:0/1696036975 conn(0x7f93a0062630 msgr2=0x7f93a0062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 -- 172.21.15.3:0/1696036975 shutdown_connections 2024-09-10T07:53:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.839+0000 7f93a6c00640 1 -- 172.21.15.3:0/1696036975 wait complete. 2024-09-10T07:53:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.841+0000 7f93a6c00640 1 Processor -- start 2024-09-10T07:53:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.841+0000 7f93a6c00640 1 -- start start 2024-09-10T07:53:51.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00cfe90 0x7f93a01f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a00d45a0 0x7f93a00b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f93a00c9fc0 con 0x7f93a0161b70 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f93a00cf970 con 0x7f93a00cfe90 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f93a00d3a30 con 0x7f93a00d45a0 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36804/0 (socket says 172.21.15.3:36804) 2024-09-10T07:53:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 -- 172.21.15.3:0/941357779 learned_addr learned my addr 172.21.15.3:0/941357779 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a5800640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00cfe90 0x7f93a01f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a4e00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a00d45a0 0x7f93a00b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 -- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a00d45a0 msgr2=0x7f93a00b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a00d45a0 0x7f93a00b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 -- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00cfe90 msgr2=0x7f93a01f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00cfe90 0x7f93a01f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.842+0000 7f93a6200640 1 -- 172.21.15.3:0/941357779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a00b65b0 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f93a6200640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f939800cb70 tx=0x7f939800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f939d600640 1 -- 172.21.15.3:0/941357779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398017070 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f939d600640 1 -- 172.21.15.3:0/941357779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939800ad40 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f939d600640 1 -- 172.21.15.3:0/941357779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398012620 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a00b6890 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.843+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a00b6e50 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.845+0000 7f939d600640 1 -- 172.21.15.3:0/941357779 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f939801a070 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.846+0000 7f939d600640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 0x7f938006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.846+0000 7f939d600640 1 -- 172.21.15.3:0/941357779 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f939808ad20 con 0x7f93a0161b70 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.846+0000 7f93a5800640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 0x7f938006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.847+0000 7f93ab242740 1 Processor -- start 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.847+0000 7f93ab242740 1 -- start start 2024-09-10T07:53:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.847+0000 7f93a5800640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 0x7f938006e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9394000f60 tx=0x7f9394009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4869e0 0x556e5a485240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556e5a3ee660 con 0x556e5a4a04f0 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556e5a396680 con 0x556e5a4d0ea0 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556e5a3ea620 con 0x556e5a4869e0 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36810/0 (socket says 172.21.15.3:36810) 2024-09-10T07:53:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 -- 172.21.15.3:0/4259167627 learned_addr learned my addr 172.21.15.3:0/4259167627 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a4e00640 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4869e0 0x556e5a485240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a6200640 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 -- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4869e0 msgr2=0x556e5a485240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4869e0 0x556e5a485240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 -- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 msgr2=0x556e5a4d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.848+0000 7f93a5800640 1 -- 172.21.15.3:0/4259167627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5a1cb7d0 con 0x556e5a4a04f0 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f93a5800640 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f939400fa40 tx=0x7f9394012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f66060348bb747 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f9392000640 1 -- 172.21.15.3:0/4259167627 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394025070 con 0x556e5a4a04f0 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f9392000640 1 -- 172.21.15.3:0/4259167627 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939401b050 con 0x556e5a4a04f0 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f9392000640 1 -- 172.21.15.3:0/4259167627 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394014de0 con 0x556e5a4a04f0 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f93ab242740 1 -- 172.21.15.3:0/4259167627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 msgr2=0x556e5a4a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.849+0000 7f93ab242740 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f939400fa40 tx=0x7f9394012000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 -- 172.21.15.3:0/4259167627 shutdown_connections 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4869e0 0x556e5a485240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 --2- 172.21.15.3:0/4259167627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a4a08d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 -- 172.21.15.3:0/4259167627 >> 172.21.15.3:0/4259167627 conn(0x556e5a487410 msgr2=0x556e5a4cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 -- 172.21.15.3:0/4259167627 shutdown_connections 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 -- 172.21.15.3:0/4259167627 wait complete. 2024-09-10T07:53:51.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.850+0000 7f93ab242740 1 -- start start 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 Processor -- start 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4a04f0 0x556e5a4ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556e5a485940 con 0x556e5a4e3b00 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556e5a3774c0 con 0x556e5a4a04f0 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.851+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556e5a485c20 con 0x556e5a4d0ea0 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93ab242740 10 client.0 _ll_register_callbacks cb 0x556e5a3e6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36820/0 (socket says 172.21.15.3:36820) 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 -- 172.21.15.3:0/1606125085 learned_addr learned my addr 172.21.15.3:0/1606125085 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a5800640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4a04f0 0x556e5a4ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a4e00640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 -- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 msgr2=0x556e5a4e35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4e35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 -- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4a04f0 msgr2=0x556e5a4ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4a04f0 0x556e5a4ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 -- 172.21.15.3:0/1606125085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5a1013f0 con 0x556e5a4e3b00 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a4e00640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4e35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f93a6200640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f93980891b0 tx=0x7f9398059120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5687ba9877bc2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.852+0000 7f9386a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398061880 con 0x556e5a4e3b00 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9398004b60 con 0x556e5a4e3b00 2024-09-10T07:53:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939401b050 con 0x556e5a4e3b00 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f9398004d60 con 0x556e5a4e3b00 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 10 client.8727 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x556e5a0c10f0 con 0x556e5a4e3b00 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939401b050 con 0x556e5a4e3b00 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f9398058430 con 0x556e5a4e3b00 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 10 client.8727 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 10 client.8727 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 20 client.8727 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a4c8a00 0x556e5a4c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 4 client.8727 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.853+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e5a3fbd90 con 0x556e5a4c8a00 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.854+0000 7f93a5800640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a4c8a00 0x556e5a4c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.854+0000 7f93a5800640 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a4c8a00 0x556e5a4c8e10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50acd0064068cebb server_cookie=93b456a8fa7beae7 in_seq=0 out_seq=0 2024-09-10T07:53:51.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.854+0000 7f9386a00640 10 client.8727 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f9386a00640 1 -- 172.21.15.3:0/1606125085 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e5a3fbd90 con 0x556e5a4c8a00 2024-09-10T07:53:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f9386a00640 10 client.8727 handle_command_reply: tid=0 2024-09-10T07:53:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 client.8727 shutdown 2024-09-10T07:53:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 msgr2=0x556e5a4b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f93980891b0 tx=0x7f9398059120 comp rx=0 tx=0).stop 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 shutdown_connections 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4e3b00 0x556e5a4b4c10 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a4c8a00 0x556e5a4c8e10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4e35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 --2- 172.21.15.3:0/1606125085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4a04f0 0x556e5a4ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f9386a00640 0 client.8727 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 >> 172.21.15.3:0/1606125085 conn(0x556e5a4869e0 msgr2=0x556e5a484140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.855+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 shutdown_connections 2024-09-10T07:53:51.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.856+0000 7f93ab242740 1 -- 172.21.15.3:0/1606125085 wait complete. 2024-09-10T07:53:51.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.856+0000 7f93ab242740 20 client.8727 trim_cache size 0 max 16384 2024-09-10T07:53:51.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.874+0000 7f93ab242740 1 Processor -- start 2024-09-10T07:53:51.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.874+0000 7f93ab242740 1 -- start start 2024-09-10T07:53:51.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4e3b00 0x556e5a4d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0200950 con 0x556e5a4a04f0 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01ffaf0 con 0x556e5a4e3b00 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f93a0200220 con 0x556e5a4d0ea0 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36836/0 (socket says 172.21.15.3:36836) 2024-09-10T07:53:51.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 -- 172.21.15.3:0/3776297641 learned_addr learned my addr 172.21.15.3:0/3776297641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 -- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 msgr2=0x556e5a4f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 -- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4e3b00 msgr2=0x556e5a4d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4e3b00 0x556e5a4d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 -- 172.21.15.3:0/3776297641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5a4c6320 con 0x556e5a4a04f0 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.875+0000 7f93a5800640 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9394008fd0 tx=0x7f9394015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb94be11d644f4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f9391600640 1 -- 172.21.15.3:0/3776297641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939403c070 con 0x556e5a4a04f0 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f9391600640 1 -- 172.21.15.3:0/3776297641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939401b040 con 0x556e5a4a04f0 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f9391600640 1 -- 172.21.15.3:0/3776297641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394025040 con 0x556e5a4a04f0 2024-09-10T07:53:51.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 -- 172.21.15.3:0/3776297641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 msgr2=0x556e5a49fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9394008fd0 tx=0x7f9394015000 comp rx=0 tx=0).stop 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 -- 172.21.15.3:0/3776297641 shutdown_connections 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4e3b00 0x556e5a4d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4d0ea0 0x556e5a4f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 --2- 172.21.15.3:0/3776297641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a49fa40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 -- 172.21.15.3:0/3776297641 >> 172.21.15.3:0/3776297641 conn(0x556e5a4869e0 msgr2=0x556e5a4c6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 -- 172.21.15.3:0/3776297641 shutdown_connections 2024-09-10T07:53:51.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.876+0000 7f93ab242740 1 -- 172.21.15.3:0/3776297641 wait complete. 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.877+0000 7f93ab242740 1 -- start start 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.877+0000 7f93ab242740 1 Processor -- start 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4e3b00 0x556e5a54eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556e5a4cd1e0 con 0x556e5a4a04f0 2024-09-10T07:53:51.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01448a0 con 0x556e5a4d0ea0 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556e5a4b4560 con 0x556e5a4e3b00 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93ab242740 10 client.0 _ll_register_callbacks cb 0x7f938c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36850/0 (socket says 172.21.15.3:36850) 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.878+0000 7f93a5800640 1 -- 172.21.15.3:0/3392001261 learned_addr learned my addr 172.21.15.3:0/3392001261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a4e00640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a6200640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4e3b00 0x556e5a54eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 -- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4e3b00 msgr2=0x556e5a54eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4e3b00 0x556e5a54eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 -- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 msgr2=0x556e5a4b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 -- 172.21.15.3:0/3392001261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5a4f2880 con 0x556e5a4a04f0 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93a5800640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f939400fa40 tx=0x7f9394001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548f64c5d205d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f9386a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-10T07:53:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939403c070 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f939402e850 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.879+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939401b040 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f9394049050 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 10 client.8733 fetch_fsmap learned FSMap version 114 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x556e5a54f050 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939401b040 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 114) ==== 3558+0+0 (secure 0 0 0) 0x7f939404c020 con 0x556e5a4a04f0 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 10 client.8733 fetch_fsmap finished waiting for FSMap version 114 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 10 client.8733 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 20 client.8733 populate_metadata read hostname 'smithi003' 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a534370 0x556e5a534780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:51.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 4 client.8733 mds_command: new command op to 7543 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 --> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e5a4d29c0 con 0x556e5a534370 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.880+0000 7f93a4e00640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a534370 0x556e5a534780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93a4e00640 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a534370 0x556e5a534780 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46fdef5e77221fa server_cookie=633effed6d2def8d in_seq=0 out_seq=0 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f9386a00640 10 client.8733 ms_handle_connect on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f9386a00640 1 -- 172.21.15.3:0/3392001261 <== mds.0 v2:172.21.15.3:6835/3773991953 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556e5a4d29c0 con 0x556e5a534370 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f9386a00640 10 client.8733 handle_command_reply: tid=0 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 client.8733 shutdown 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 msgr2=0x556e5a548c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f939400fa40 tx=0x7f9394001080 comp rx=0 tx=0).stop 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 shutdown_connections 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:6835/3773991953,v1:172.21.15.3:6837/3773991953] conn(0x556e5a534370 0x556e5a534780 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556e5a4e3b00 0x556e5a54eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556e5a4d0ea0 0x556e5a4b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 --2- 172.21.15.3:0/3392001261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e5a4a04f0 0x556e5a548c20 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f9386a00640 0 client.8733 ms_handle_reset on v2:172.21.15.3:6835/3773991953 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 >> 172.21.15.3:0/3392001261 conn(0x556e5a4869e0 msgr2=0x556e5a4c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 shutdown_connections 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.881+0000 7f93ab242740 1 -- 172.21.15.3:0/3392001261 wait complete. 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93ab242740 20 client.8733 trim_cache size 0 max 16384 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-10T07:53:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6": { 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/d1/d2/d3", 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "7a4de7f6-6cf6-4308-9c71-dc74cd48b4b6", 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-10T07:53:51.891 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 msgr2=0x7f938006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 0x7f938006e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9394000f60 tx=0x7f9394009040 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 msgr2=0x7f93a01f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f939800cb70 tx=0x7f939800a430 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 shutdown_connections 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f938006be30 0x7f938006e2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a0161b70 0x7f93a01f75d0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f93a00d45a0 0x7f93a00b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 --2- 172.21.15.3:0/941357779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93a00cfe90 0x7f93a01f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:51.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 >> 172.21.15.3:0/941357779 conn(0x7f93a0062630 msgr2=0x7f93a015cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 shutdown_connections 2024-09-10T07:53:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:51.882+0000 7f93a6c00640 1 -- 172.21.15.3:0/941357779 wait complete. 2024-09-10T07:53:51.898 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T07:53:51.898 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:52.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.054+0000 7f79e7600640 1 Processor -- start 2024-09-10T07:53:52.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.054+0000 7f79e7600640 1 -- start start 2024-09-10T07:53:52.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e7600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79e00d08f0 0x7f79e00ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e7600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e00ccd80 con 0x7f79e015b110 2024-09-10T07:53:52.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e7600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e00cd060 con 0x7f79e015b4f0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e7600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f79e00b44f0 con 0x7f79e00d08f0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e4c00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79e015b110 0x7f79e00b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49858/0 (socket says 172.21.15.3:49858) 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e4c00640 1 -- 172.21.15.3:0/2477512523 learned_addr learned my addr 172.21.15.3:0/2477512523 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e5600640 1 --2- 172.21.15.3:0/2477512523 >> v2:172.21.15.3:3300/0 conn(0x7f79e00d08f0 0x7f79e00ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79df400640 1 -- 172.21.15.3:0/2477512523 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243553824 0 0) 0x7f79e00ccd80 con 0x7f79e015b110 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79df400640 1 -- 172.21.15.3:0/2477512523 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79c8005730 con 0x7f79e015b110 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e5600640 1 -- 172.21.15.3:0/2477512523 >> v1:172.21.15.70:6789/0 conn(0x7f79e015b4f0 legacy=0x7f79e00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e5600640 1 -- 172.21.15.3:0/2477512523 >> v1:172.21.15.3:6789/0 conn(0x7f79e015b110 legacy=0x7f79e00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.056+0000 7f79e5600640 1 -- 172.21.15.3:0/2477512523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e00cd340 con 0x7f79e00d08f0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.057+0000 7f79e5600640 1 --2- 172.21.15.3:0/2477512523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e00d08f0 0x7f79e00ceb50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009d90 tx=0x7f79d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19b0400a9df339b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79df400640 1 -- 172.21.15.3:0/2477512523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8007740 con 0x7f79e00d08f0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79df400640 1 -- 172.21.15.3:0/2477512523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d80078e0 con 0x7f79e00d08f0 2024-09-10T07:53:52.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79df400640 1 -- 172.21.15.3:0/2477512523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8007cb0 con 0x7f79e00d08f0 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 -- 172.21.15.3:0/2477512523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e00d08f0 msgr2=0x7f79e00ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 --2- 172.21.15.3:0/2477512523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e00d08f0 0x7f79e00ceb50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009d90 tx=0x7f79d802a6f0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 -- 172.21.15.3:0/2477512523 shutdown_connections 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 --2- 172.21.15.3:0/2477512523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e00d08f0 0x7f79e00ceb50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 -- 172.21.15.3:0/2477512523 >> 172.21.15.3:0/2477512523 conn(0x7f79e006d000 msgr2=0x7f79e006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 -- 172.21.15.3:0/2477512523 shutdown_connections 2024-09-10T07:53:52.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.058+0000 7f79e7600640 1 -- 172.21.15.3:0/2477512523 wait complete. 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.060+0000 7f79e7600640 1 Processor -- start 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.060+0000 7f79e7600640 1 -- start start 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e00d08f0 0x7f79e015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e015b110 0x7f79e00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0073e90 con 0x7f79e015b4f0 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0073d10 con 0x7f79e00d08f0 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e7600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f79e0074010 con 0x7f79e015b110 2024-09-10T07:53:52.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e4c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e00d08f0 0x7f79e015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36864/0 (socket says 172.21.15.3:36864) 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.061+0000 7f79e5600640 1 -- 172.21.15.3:0/731587700 learned_addr learned my addr 172.21.15.3:0/731587700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79dfe00640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e015b110 0x7f79e00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 -- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e015b110 msgr2=0x7f79e00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e015b110 0x7f79e00ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 -- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e00d08f0 msgr2=0x7f79e015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e00d08f0 0x7f79e015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 -- 172.21.15.3:0/731587700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e015be10 con 0x7f79e015b4f0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79e5600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009d90 tx=0x7f79d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d803f070 con 0x7f79e015b4f0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.062+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d8031c40 con 0x7f79e015b4f0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.063+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8009450 con 0x7f79e015b4f0 2024-09-10T07:53:52.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.063+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e015c0b0 con 0x7f79e015b4f0 2024-09-10T07:53:52.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.063+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e015c670 con 0x7f79e015b4f0 2024-09-10T07:53:52.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.064+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e00b3d10 con 0x7f79e015b4f0 2024-09-10T07:53:52.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.065+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79d8003990 con 0x7f79e015b4f0 2024-09-10T07:53:52.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.066+0000 7f79dcc00640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 0x7f79c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.066+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f79d80abb60 con 0x7f79e015b4f0 2024-09-10T07:53:52.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.066+0000 7f79e4c00640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 0x7f79c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.067+0000 7f79e4c00640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 0x7f79c006e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f79d40011e0 tx=0x7f79d4007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.071+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f79d8079eb0 con 0x7f79e015b4f0 2024-09-10T07:53:52.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.292+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e006c400 con 0x7f79e015b4f0 2024-09-10T07:53:52.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.292+0000 7f79dcc00640 1 -- 172.21.15.3:0/731587700 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f79d807e1b0 con 0x7f79e015b4f0 2024-09-10T07:53:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:52.292 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:52.292 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 msgr2=0x7f79c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 0x7f79c006e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f79d40011e0 tx=0x7f79d4007d30 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 msgr2=0x7f79e00aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009d90 tx=0x7f79d8004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 shutdown_connections 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f79c006be30 0x7f79c006e2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e015b4f0 0x7f79e00aede0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f79e015b110 0x7f79e00ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 --2- 172.21.15.3:0/731587700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79e00d08f0 0x7f79e015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 >> 172.21.15.3:0/731587700 conn(0x7f79e006d000 msgr2=0x7f79e00d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 shutdown_connections 2024-09-10T07:53:52.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.295+0000 7f79e7600640 1 -- 172.21.15.3:0/731587700 wait complete. 2024-09-10T07:53:52.311 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:52.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.467+0000 7fb199e00640 1 Processor -- start 2024-09-10T07:53:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.467+0000 7fb199e00640 1 -- start start 2024-09-10T07:53:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.468+0000 7fb199e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb19415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.468+0000 7fb199e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.468+0000 7fb199e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1940cf4e0 con 0x7fb19415e7c0 2024-09-10T07:53:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.468+0000 7fb199e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb1940cfd80 con 0x7fb1940d4a40 2024-09-10T07:53:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.468+0000 7fb199e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb1940cf7c0 con 0x7fb1940d4660 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb193400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb19415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb193400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:46684/0 (socket says 172.21.15.3:46684) 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb19415e7c0 0x7fb1940ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:53730/0 (socket says 172.21.15.3:53730) 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb193400640 1 -- 172.21.15.3:0/1995514979 learned_addr learned my addr 172.21.15.3:0/1995514979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192a00640 1 -- 172.21.15.3:0/1995514979 >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 msgr2=0x7fb1940d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192a00640 1 --2- 172.21.15.3:0/1995514979 >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192a00640 1 -- 172.21.15.3:0/1995514979 >> v1:172.21.15.70:6789/0 conn(0x7fb19415e7c0 legacy=0x7fb1940ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb192a00640 1 -- 172.21.15.3:0/1995514979 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940cfaa0 con 0x7fb1940d4660 2024-09-10T07:53:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.469+0000 7fb193400640 1 --2- 172.21.15.3:0/1995514979 >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.470+0000 7fb192a00640 1 --2- 172.21.15.3:0/1995514979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb19415f6a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1840097a0 tx=0x7fb18402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0192b29a887ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb191600640 1 -- 172.21.15.3:0/1995514979 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184037070 con 0x7fb1940d4660 2024-09-10T07:53:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb191600640 1 -- 172.21.15.3:0/1995514979 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184032440 con 0x7fb1940d4660 2024-09-10T07:53:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb191600640 1 -- 172.21.15.3:0/1995514979 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184032810 con 0x7fb1940d4660 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 -- 172.21.15.3:0/1995514979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 msgr2=0x7fb19415f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 --2- 172.21.15.3:0/1995514979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb19415f6a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1840097a0 tx=0x7fb18402a180 comp rx=0 tx=0).stop 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 -- 172.21.15.3:0/1995514979 shutdown_connections 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 --2- 172.21.15.3:0/1995514979 >> v2:172.21.15.70:3300/0 conn(0x7fb1940d4a40 0x7fb1940d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 --2- 172.21.15.3:0/1995514979 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb19415f6a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 -- 172.21.15.3:0/1995514979 >> 172.21.15.3:0/1995514979 conn(0x7fb194072560 msgr2=0x7fb194072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 -- 172.21.15.3:0/1995514979 shutdown_connections 2024-09-10T07:53:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.471+0000 7fb199e00640 1 -- 172.21.15.3:0/1995514979 wait complete. 2024-09-10T07:53:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.473+0000 7fb199e00640 1 Processor -- start 2024-09-10T07:53:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.473+0000 7fb199e00640 1 -- start start 2024-09-10T07:53:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb194158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1940d4a40 0x7fb1940b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1940d3020 con 0x7fb19415e7c0 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb1940d2ea0 con 0x7fb1940d4a40 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb199e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb1940d31a0 con 0x7fb1940d4660 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb193400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36880/0 (socket says 172.21.15.3:36880) 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.474+0000 7fb193400640 1 -- 172.21.15.3:0/2306044082 learned_addr learned my addr 172.21.15.3:0/2306044082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb192000640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1940d4a40 0x7fb1940b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb192a00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb194158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 -- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 msgr2=0x7fb194158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb194158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 -- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1940d4a40 msgr2=0x7fb1940b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1940d4a40 0x7fb1940b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 -- 172.21.15.3:0/2306044082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940b6fc0 con 0x7fb19415e7c0 2024-09-10T07:53:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb192a00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb194158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb193400640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00efc0 tx=0x7fb18c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00cbe0 con 0x7fb19415e7c0 2024-09-10T07:53:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18c00cd80 con 0x7fb19415e7c0 2024-09-10T07:53:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c003bb0 con 0x7fb19415e7c0 2024-09-10T07:53:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.475+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb194158df0 con 0x7fb19415e7c0 2024-09-10T07:53:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.476+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1941593b0 con 0x7fb19415e7c0 2024-09-10T07:53:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.477+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb194161b30 con 0x7fb19415e7c0 2024-09-10T07:53:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.477+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb18c01b030 con 0x7fb19415e7c0 2024-09-10T07:53:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.478+0000 7fb18b400640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 0x7fb17406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.478+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb18c08b210 con 0x7fb19415e7c0 2024-09-10T07:53:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.479+0000 7fb192a00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 0x7fb17406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:52.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.479+0000 7fb192a00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 0x7fb17406e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb184034000 tx=0x7fb18402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:52.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.483+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb18c022020 con 0x7fb19415e7c0 2024-09-10T07:53:52.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.734+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb1940d2940 con 0x7fb19415e7c0 2024-09-10T07:53:52.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.734+0000 7fb18b400640 1 -- 172.21.15.3:0/2306044082 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7fb18c010040 con 0x7fb19415e7c0 2024-09-10T07:53:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:52.734 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":466,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700762,"bytes_used":271269888,"bytes_avail":772822843392,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7592,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-09-10T07:53:47.589026+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:52.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.736+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 msgr2=0x7fb17406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.736+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 0x7fb17406e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb184034000 tx=0x7fb18402e000 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.736+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 msgr2=0x7fb194160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.736+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00efc0 tx=0x7fb18c00c490 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 shutdown_connections 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb17406be30 0x7fb17406e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb19415e7c0 0x7fb194160820 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1940d4a40 0x7fb1940b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 --2- 172.21.15.3:0/2306044082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb1940d4660 0x7fb194158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 >> 172.21.15.3:0/2306044082 conn(0x7fb194072560 msgr2=0x7fb1940cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 shutdown_connections 2024-09-10T07:53:52.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:52.737+0000 7fb199e00640 1 -- 172.21.15.3:0/2306044082 wait complete. 2024-09-10T07:53:52.752 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-10T07:53:52.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:53:52.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:53:53.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.917+0000 7fe432600640 1 Processor -- start 2024-09-10T07:53:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.917+0000 7fe432600640 1 -- start start 2024-09-10T07:53:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe432600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe42c15e520 0x7fe42c073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe432600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0b7110 0x7fe42c06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe432600640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42c071780 con 0x7fe42c0d1b70 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe432600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe42c0b7860 con 0x7fe42c15e520 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe432600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe42c073e90 con 0x7fe42c0b7110 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe42be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe42c0d1b70 0x7fe42c0b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45316/0 (socket says 172.21.15.3:45316) 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe42be00640 1 -- 172.21.15.3:0/3083294688 learned_addr learned my addr 172.21.15.3:0/3083294688 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe431200640 1 --2- 172.21.15.3:0/3083294688 >> v2:172.21.15.70:3300/0 conn(0x7fe42c15e520 0x7fe42c073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.918+0000 7fe431c00640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0b7110 0x7fe42c06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924464784 0 0) 0x7fe42c071780 con 0x7fe42c0d1b70 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe414003520 con 0x7fe42c0d1b70 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe431200640 1 -- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0b7110 msgr2=0x7fe42c06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe431200640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0b7110 0x7fe42c06bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:53.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe431200640 1 -- 172.21.15.3:0/3083294688 >> v1:172.21.15.3:6789/0 conn(0x7fe42c0d1b70 legacy=0x7fe42c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2294244653 0 0) 0x7fe414003520 con 0x7fe42c0d1b70 2024-09-10T07:53:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.919+0000 7fe431200640 1 -- 172.21.15.3:0/3083294688 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c0709c0 con 0x7fe42c15e520 2024-09-10T07:53:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.920+0000 7fe431200640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c15e520 0x7fe42c073950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe4200097a0 tx=0x7fe42002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f313af1d56f525 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420037070 con 0x7fe42c15e520 2024-09-10T07:53:53.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42002aba0 con 0x7fe42c15e520 2024-09-10T07:53:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe42b400640 1 -- 172.21.15.3:0/3083294688 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418001670 con 0x7fe42c15e520 2024-09-10T07:53:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 -- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c15e520 msgr2=0x7fe42c073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c15e520 0x7fe42c073950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe4200097a0 tx=0x7fe42002a110 comp rx=0 tx=0).stop 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 -- 172.21.15.3:0/3083294688 shutdown_connections 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0b7110 0x7fe42c06bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 --2- 172.21.15.3:0/3083294688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c15e520 0x7fe42c073950 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.921+0000 7fe432600640 1 -- 172.21.15.3:0/3083294688 >> 172.21.15.3:0/3083294688 conn(0x7fe42c0b41e0 msgr2=0x7fe42c0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.922+0000 7fe432600640 1 -- 172.21.15.3:0/3083294688 shutdown_connections 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.922+0000 7fe432600640 1 -- 172.21.15.3:0/3083294688 wait complete. 2024-09-10T07:53:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.923+0000 7fe432600640 1 Processor -- start 2024-09-10T07:53:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.923+0000 7fe432600640 1 -- start start 2024-09-10T07:53:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 0x7fe42c162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0d1b70 0x7fe42c148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c071260 con 0x7fe42c15e520 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c070f80 con 0x7fe42c0b7110 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.924+0000 7fe432600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe42c15e900 con 0x7fe42c0d1b70 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52642/0 (socket says 172.21.15.3:52642) 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 -- 172.21.15.3:0/2843280309 learned_addr learned my addr 172.21.15.3:0/2843280309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe42be00640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0d1b70 0x7fe42c148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431200640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 0x7fe42c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 -- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0d1b70 msgr2=0x7fe42c148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0d1b70 0x7fe42c148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 -- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 msgr2=0x7fe42c162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 0x7fe42c162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 -- 172.21.15.3:0/2843280309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c15ee10 con 0x7fe42c15e520 2024-09-10T07:53:53.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431200640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 0x7fe42c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe431c00640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe42400efc0 tx=0x7fe42400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424018020 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe424014480 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424014870 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.925+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c15f0f0 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.926+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c15f6b0 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.927+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c158e30 con 0x7fe42c15e520 2024-09-10T07:53:53.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.928+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe424005d10 con 0x7fe42c15e520 2024-09-10T07:53:53.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.929+0000 7fe428c00640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:53.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.929+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe42408b920 con 0x7fe42c15e520 2024-09-10T07:53:53.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.929+0000 7fe431200640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:53.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.930+0000 7fe431200640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 0x7fe40c06e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe420034000 tx=0x7fe42002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:53.933+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe42c158e30 con 0x7fe42c15e520 2024-09-10T07:53:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.153+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe42c073950 con 0x7fe42c15e520 2024-09-10T07:53:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.154+0000 7fe428c00640 1 -- 172.21.15.3:0/2843280309 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7fe424022070 con 0x7fe42c15e520 2024-09-10T07:53:54.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:54.154 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 msgr2=0x7fe40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 0x7fe40c06e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe420034000 tx=0x7fe42002e000 comp rx=0 tx=0).stop 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 msgr2=0x7fe42c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe42400efc0 tx=0x7fe42400c490 comp rx=0 tx=0).stop 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 shutdown_connections 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42c15e520 0x7fe42c0b2a80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe42c0d1b70 0x7fe42c148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 --2- 172.21.15.3:0/2843280309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42c0b7110 0x7fe42c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.156+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 >> 172.21.15.3:0/2843280309 conn(0x7fe42c0b41e0 msgr2=0x7fe42c15ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.157+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 shutdown_connections 2024-09-10T07:53:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.157+0000 7fe432600640 1 -- 172.21.15.3:0/2843280309 wait complete. 2024-09-10T07:53:54.171 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:54.172 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.327+0000 7f87ed400640 1 Processor -- start 2024-09-10T07:53:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.327+0000 7f87ed400640 1 -- start start 2024-09-10T07:53:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87ed400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e806ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87ed400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87e80b4aa0 0x7f87e80cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87ed400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e80ccd40 con 0x7f87e80cf8f0 2024-09-10T07:53:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87ed400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f87e806f250 con 0x7f87e80b4aa0 2024-09-10T07:53:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87ed400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80cd020 con 0x7f87e815b410 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87e5e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87e80b4aa0 0x7f87e80cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87e5e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87e80b4aa0 0x7f87e80cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52652/0 (socket says 172.21.15.3:52652) 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87e5e00640 1 -- 172.21.15.3:0/651485861 learned_addr learned my addr 172.21.15.3:0/651485861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.328+0000 7f87e6800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e806ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.329+0000 7f87e5e00640 1 -- 172.21.15.3:0/651485861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 msgr2=0x7f87e806ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.329+0000 7f87e5e00640 1 --2- 172.21.15.3:0/651485861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e806ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.329+0000 7f87e5e00640 1 -- 172.21.15.3:0/651485861 >> v1:172.21.15.70:6789/0 conn(0x7f87e80cf8f0 legacy=0x7f87e815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.329+0000 7f87e5e00640 1 -- 172.21.15.3:0/651485861 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e80cd300 con 0x7f87e80b4aa0 2024-09-10T07:53:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.330+0000 7f87e5e00640 1 --2- 172.21.15.3:0/651485861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e80cf3b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f87d400bb10 tx=0x7f87d402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a890f730ea0b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.330+0000 7f87e5400640 1 -- 172.21.15.3:0/651485861 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4036020 con 0x7f87e80b4aa0 2024-09-10T07:53:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.330+0000 7f87e5400640 1 -- 172.21.15.3:0/651485861 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d4032820 con 0x7f87e80b4aa0 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.330+0000 7f87e5400640 1 -- 172.21.15.3:0/651485861 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4032bf0 con 0x7f87e80b4aa0 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 -- 172.21.15.3:0/651485861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 msgr2=0x7f87e80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 --2- 172.21.15.3:0/651485861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e80cf3b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f87d400bb10 tx=0x7f87d402c920 comp rx=0 tx=0).stop 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 -- 172.21.15.3:0/651485861 shutdown_connections 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 --2- 172.21.15.3:0/651485861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e80cf3b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 --2- 172.21.15.3:0/651485861 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e806ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 -- 172.21.15.3:0/651485861 >> 172.21.15.3:0/651485861 conn(0x7f87e806cfc0 msgr2=0x7f87e806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 -- 172.21.15.3:0/651485861 shutdown_connections 2024-09-10T07:53:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.331+0000 7f87ed400640 1 -- 172.21.15.3:0/651485861 wait complete. 2024-09-10T07:53:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.332+0000 7f87ed400640 1 Processor -- start 2024-09-10T07:53:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.333+0000 7f87ed400640 1 -- start start 2024-09-10T07:53:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.333+0000 7f87ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.333+0000 7f87ed400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e80cf8f0 0x7f87e813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.333+0000 7f87ed400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e8153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87ed400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80cdba0 con 0x7f87e80b4aa0 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87ed400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80cd8c0 con 0x7f87e80cf8f0 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87ed400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80b46a0 con 0x7f87e815b410 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52668/0 (socket says 172.21.15.3:52668) 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 -- 172.21.15.3:0/3161263213 learned_addr learned my addr 172.21.15.3:0/3161263213 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e5e00640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e80cf8f0 0x7f87e813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e7200640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e8153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 -- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 msgr2=0x7f87e8153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e8153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 -- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e80cf8f0 msgr2=0x7f87e813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e80cf8f0 0x7f87e813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 -- 172.21.15.3:0/3161263213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8154aa0 con 0x7f87e80b4aa0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.334+0000 7f87e6800640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e870 tx=0x7f87d800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.335+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8017070 con 0x7f87e80b4aa0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.335+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d800c940 con 0x7f87e80b4aa0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.335+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800cd30 con 0x7f87e80b4aa0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.335+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e8154d80 con 0x7f87e80b4aa0 2024-09-10T07:53:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.335+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e8152a40 con 0x7f87e80b4aa0 2024-09-10T07:53:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.336+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8072540 con 0x7f87e80b4aa0 2024-09-10T07:53:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.337+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87d801a070 con 0x7f87e80b4aa0 2024-09-10T07:53:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.338+0000 7f87dea00640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.338+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f87d808b170 con 0x7f87e80b4aa0 2024-09-10T07:53:54.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.338+0000 7f87e5e00640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:54.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.339+0000 7f87e5e00640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 0x7f87c806e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f87d4037000 tx=0x7f87d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.343+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87d8059a80 con 0x7f87e80b4aa0 2024-09-10T07:53:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.596+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f87e80b5550 con 0x7f87e80b4aa0 2024-09-10T07:53:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.596+0000 7f87dea00640 1 -- 172.21.15.3:0/3161263213 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2022 (secure 0 0 0) 0x7f87d8059380 con 0x7f87e80b4aa0 2024-09-10T07:53:54.596 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:54.596 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":468,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":271269888,"bytes_avail":772822843392,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7507,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-09-10T07:53:53.590371+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 msgr2=0x7f87c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 0x7f87c806e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f87d4037000 tx=0x7f87d4008040 comp rx=0 tx=0).stop 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 msgr2=0x7f87e8072c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e870 tx=0x7f87d800ed30 comp rx=0 tx=0).stop 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 shutdown_connections 2024-09-10T07:53:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f87c806be30 0x7f87c806e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87e815b410 0x7f87e8153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e80cf8f0 0x7f87e813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 --2- 172.21.15.3:0/3161263213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87e80b4aa0 0x7f87e8072c90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.598+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 >> 172.21.15.3:0/3161263213 conn(0x7f87e806cfc0 msgr2=0x7f87e815a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.599+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 shutdown_connections 2024-09-10T07:53:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:54.599+0000 7f87ed400640 1 -- 172.21.15.3:0/3161263213 wait complete. 2024-09-10T07:53:54.614 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:53:55.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:55.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.784+0000 7ff138400640 1 Processor -- start 2024-09-10T07:53:55.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.784+0000 7ff138400640 1 -- start start 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.785+0000 7ff138400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff13015b110 0x7ff1300b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.785+0000 7ff138400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300d08f0 0x7ff1300cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.785+0000 7ff138400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300b4250 con 0x7ff13015b4f0 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.785+0000 7ff138400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff1300b44f0 con 0x7ff13015b110 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.785+0000 7ff138400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300ccd80 con 0x7ff1300d08f0 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff13015b110 0x7ff1300b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff136400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300d08f0 0x7ff1300cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135000640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff13015b4f0 0x7ff1300d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:57134/0 (socket says 172.21.15.3:57134) 2024-09-10T07:53:55.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135000640 1 -- 172.21.15.3:0/3542752344 learned_addr learned my addr 172.21.15.3:0/3542752344 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135a00640 1 -- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300d08f0 msgr2=0x7ff1300cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135a00640 1 --2- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300d08f0 0x7ff1300cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135a00640 1 -- 172.21.15.3:0/3542752344 >> v1:172.21.15.70:6789/0 conn(0x7ff13015b4f0 legacy=0x7ff1300d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff135a00640 1 -- 172.21.15.3:0/3542752344 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300cd060 con 0x7ff13015b110 2024-09-10T07:53:55.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.786+0000 7ff12fe00640 1 -- 172.21.15.3:0/3542752344 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217282277 0 0) 0x7ff1300b4250 con 0x7ff13015b4f0 2024-09-10T07:53:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.787+0000 7ff135a00640 1 --2- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300b3d10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1300b4670 tx=0x7ff12402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7df4806488283a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff12fe00640 1 -- 172.21.15.3:0/3542752344 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124037070 con 0x7ff13015b110 2024-09-10T07:53:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff12fe00640 1 -- 172.21.15.3:0/3542752344 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124032440 con 0x7ff13015b110 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff12fe00640 1 -- 172.21.15.3:0/3542752344 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124032810 con 0x7ff13015b110 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 -- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 msgr2=0x7ff1300b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 --2- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300b3d10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1300b4670 tx=0x7ff12402a180 comp rx=0 tx=0).stop 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 -- 172.21.15.3:0/3542752344 shutdown_connections 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 --2- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300d08f0 0x7ff1300cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 --2- 172.21.15.3:0/3542752344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300b3d10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 -- 172.21.15.3:0/3542752344 >> 172.21.15.3:0/3542752344 conn(0x7ff13006d000 msgr2=0x7ff13006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 -- 172.21.15.3:0/3542752344 shutdown_connections 2024-09-10T07:53:55.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.788+0000 7ff138400640 1 -- 172.21.15.3:0/3542752344 wait complete. 2024-09-10T07:53:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.790+0000 7ff138400640 1 Processor -- start 2024-09-10T07:53:55.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.790+0000 7ff138400640 1 -- start start 2024-09-10T07:53:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff13015b4f0 0x7ff1300aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff130073d10 con 0x7ff1300d08f0 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300cd620 con 0x7ff13015b110 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff138400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff130073e90 con 0x7ff13015b4f0 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff135a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff135a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52684/0 (socket says 172.21.15.3:52684) 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.791+0000 7ff135a00640 1 -- 172.21.15.3:0/3842440895 learned_addr learned my addr 172.21.15.3:0/3842440895 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff136400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff13015b4f0 0x7ff1300aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135000640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 -- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff13015b4f0 msgr2=0x7ff1300aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff13015b4f0 0x7ff1300aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 -- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 msgr2=0x7ff1300ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 -- 172.21.15.3:0/3842440895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13006c6c0 con 0x7ff1300d08f0 2024-09-10T07:53:55.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135000640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.792+0000 7ff135a00640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.793+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124037040 con 0x7ff1300d08f0 2024-09-10T07:53:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.793+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124042070 con 0x7ff1300d08f0 2024-09-10T07:53:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.793+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124030920 con 0x7ff1300d08f0 2024-09-10T07:53:55.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.793+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1301f40b0 con 0x7ff1300d08f0 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.793+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13006c3e0 con 0x7ff1300d08f0 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.794+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1300b3d10 con 0x7ff1300d08f0 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.795+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff124030ac0 con 0x7ff1300d08f0 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.796+0000 7ff12d600640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.796+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7ff1240ab7d0 con 0x7ff1300d08f0 2024-09-10T07:53:55.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.796+0000 7ff135000640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:55.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.796+0000 7ff135000640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 0x7ff11006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff130159260 tx=0x7ff11c007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:55.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:55.800+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff12407a0e0 con 0x7ff1300d08f0 2024-09-10T07:53:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.022+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1300ce5f0 con 0x7ff1300d08f0 2024-09-10T07:53:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.023+0000 7ff12d600640 1 -- 172.21.15.3:0/3842440895 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7ff1240799e0 con 0x7ff1300d08f0 2024-09-10T07:53:56.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:56.022 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 msgr2=0x7ff11006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 0x7ff11006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff130159260 tx=0x7ff11c007cc0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 msgr2=0x7ff1301542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff1240046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 shutdown_connections 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ff11006be30 0x7ff11006e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff13015b4f0 0x7ff1300aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff13015b110 0x7ff1300ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 --2- 172.21.15.3:0/3842440895 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1300d08f0 0x7ff1301542a0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 >> 172.21.15.3:0/3842440895 conn(0x7ff13006d000 msgr2=0x7ff1300ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 shutdown_connections 2024-09-10T07:53:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.025+0000 7ff138400640 1 -- 172.21.15.3:0/3842440895 wait complete. 2024-09-10T07:53:56.040 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:56.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:56.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.197+0000 7f9969000640 1 Processor -- start 2024-09-10T07:53:56.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.197+0000 7f9969000640 1 -- start start 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f9969000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 0x7f99640d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f9969000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996406c9c0 con 0x7f996406c530 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f9969000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99640d2770 con 0x7f996415e560 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f9969000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f99640d2a50 con 0x7f99640ce720 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f995be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f996406c530 0x7f99640d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45318/0 (socket says 172.21.15.3:45318) 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f995be00640 1 -- 172.21.15.3:0/3291626809 learned_addr learned my addr 172.21.15.3:0/3291626809 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:56.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.198+0000 7f9963e00640 1 --2- 172.21.15.3:0/3291626809 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 0x7f99640d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:56.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621689714 0 0) 0x7f996406c9c0 con 0x7f996406c530 2024-09-10T07:53:56.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9948003520 con 0x7f996406c530 2024-09-10T07:53:56.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3505425586 0 0) 0x7f99640d2770 con 0x7f996415e560 2024-09-10T07:53:56.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f996406c9c0 con 0x7f996415e560 2024-09-10T07:53:56.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9963e00640 1 -- 172.21.15.3:0/3291626809 >> v1:172.21.15.70:6789/0 conn(0x7f996415e560 legacy=0x7f99640b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9963e00640 1 -- 172.21.15.3:0/3291626809 >> v1:172.21.15.3:6789/0 conn(0x7f996406c530 legacy=0x7f99640d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.199+0000 7f9963e00640 1 -- 172.21.15.3:0/3291626809 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99640d2d30 con 0x7f99640ce720 2024-09-10T07:53:56.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.200+0000 7f9963e00640 1 --2- 172.21.15.3:0/3291626809 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 0x7f99640d3530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f995c009dc0 tx=0x7f995c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d928e374b23a0e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.200+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007740 con 0x7f99640ce720 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.200+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c0078e0 con 0x7f99640ce720 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.200+0000 7f9962a00640 1 -- 172.21.15.3:0/3291626809 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007cb0 con 0x7f99640ce720 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 -- 172.21.15.3:0/3291626809 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 msgr2=0x7f99640d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 --2- 172.21.15.3:0/3291626809 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 0x7f99640d3530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f995c009dc0 tx=0x7f995c02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 -- 172.21.15.3:0/3291626809 shutdown_connections 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 --2- 172.21.15.3:0/3291626809 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f99640ce720 0x7f99640d3530 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 -- 172.21.15.3:0/3291626809 >> 172.21.15.3:0/3291626809 conn(0x7f99640b6050 msgr2=0x7f99640b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:56.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 -- 172.21.15.3:0/3291626809 shutdown_connections 2024-09-10T07:53:56.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.201+0000 7f9969000640 1 -- 172.21.15.3:0/3291626809 wait complete. 2024-09-10T07:53:56.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.203+0000 7f9969000640 1 Processor -- start 2024-09-10T07:53:56.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.203+0000 7f9969000640 1 -- start start 2024-09-10T07:53:56.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 0x7f9964161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f996415e560 0x7f99640d4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99640d1760 con 0x7f99640d4a40 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f99640d15e0 con 0x7f99640ce720 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9969000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f99640cf3d0 con 0x7f996415e560 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52690/0 (socket says 172.21.15.3:52690) 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 -- 172.21.15.3:0/2729487890 learned_addr learned my addr 172.21.15.3:0/2729487890 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963400640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 0x7f9964161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f995be00640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f996415e560 0x7f99640d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:56.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 -- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f996415e560 msgr2=0x7f99640d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f996415e560 0x7f99640d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.204+0000 7f9963e00640 1 -- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 msgr2=0x7f9964161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9963e00640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 0x7f9964161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9963e00640 1 -- 172.21.15.3:0/2729487890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99640ce4f0 con 0x7f99640d4a40 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9963400640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 0x7f9964161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9963e00640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f995c009ee0 tx=0x7f995c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c040070 con 0x7f99640d4a40 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c031bc0 con 0x7f99640d4a40 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c009450 con 0x7f99640d4a40 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99640c86a0 con 0x7f99640d4a40 2024-09-10T07:53:56.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.205+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99640c8c60 con 0x7f99640d4a40 2024-09-10T07:53:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.206+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99640d0ed0 con 0x7f99640d4a40 2024-09-10T07:53:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.207+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f995c039040 con 0x7f99640d4a40 2024-09-10T07:53:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.208+0000 7f995b400640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 0x7f994006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:56.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.208+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f995c0ac670 con 0x7f99640d4a40 2024-09-10T07:53:56.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.208+0000 7f9963400640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 0x7f994006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:56.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.209+0000 7f9963400640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 0x7f994006e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f99540011e0 tx=0x7f9954009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:56.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.212+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f995c07b010 con 0x7f99640d4a40 2024-09-10T07:53:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.464+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f996415a410 con 0x7f99640d4a40 2024-09-10T07:53:56.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.465+0000 7f995b400640 1 -- 172.21.15.3:0/2729487890 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2022 (secure 0 0 0) 0x7f995c07a910 con 0x7f99640d4a40 2024-09-10T07:53:56.464 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:56.465 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":271269888,"bytes_avail":772822843392,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-09-10T07:53:53.590371+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 msgr2=0x7f994006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 0x7f994006e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f99540011e0 tx=0x7f9954009040 comp rx=0 tx=0).stop 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 msgr2=0x7f99640cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f995c009ee0 tx=0x7f995c004a40 comp rx=0 tx=0).stop 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 shutdown_connections 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f994006be30 0x7f994006e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99640d4a40 0x7f99640cd7a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f996415e560 0x7f99640d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 --2- 172.21.15.3:0/2729487890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99640ce720 0x7f9964161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 >> 172.21.15.3:0/2729487890 conn(0x7f99640b6050 msgr2=0x7f996415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 shutdown_connections 2024-09-10T07:53:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:56.467+0000 7f9969000640 1 -- 172.21.15.3:0/2729487890 wait complete. 2024-09-10T07:53:56.476 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:53:56.477+0000 7fabc0200640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T07:53:56.482 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:53:57.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:57.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.653+0000 7f98cfc00640 1 Processor -- start 2024-09-10T07:53:57.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.653+0000 7f98cfc00640 1 -- start start 2024-09-10T07:53:57.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cfc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98c80d07c0 0x7f98c8167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cfc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c80cfdd0 con 0x7f98c8160710 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cfc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c815cc90 con 0x7f98c8160af0 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cfc00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f98c815cf70 con 0x7f98c80d07c0 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cdc00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98c80d07c0 0x7f98c8167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cd200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98c8160710 0x7f98c80cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45320/0 (socket says 172.21.15.3:45320) 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.655+0000 7f98cd200640 1 -- 172.21.15.3:0/2508821387 learned_addr learned my addr 172.21.15.3:0/2508821387 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:57.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.656+0000 7f98cdc00640 1 -- 172.21.15.3:0/2508821387 >> v1:172.21.15.70:6789/0 conn(0x7f98c8160af0 legacy=0x7f98c80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.656+0000 7f98cdc00640 1 -- 172.21.15.3:0/2508821387 >> v1:172.21.15.3:6789/0 conn(0x7f98c8160710 legacy=0x7f98c80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.656+0000 7f98cdc00640 1 -- 172.21.15.3:0/2508821387 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c815d250 con 0x7f98c80d07c0 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.656+0000 7f98c7e00640 1 -- 172.21.15.3:0/2508821387 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251070627 0 0) 0x7f98c80cfdd0 con 0x7f98c8160710 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.657+0000 7f98cdc00640 1 --2- 172.21.15.3:0/2508821387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c80d07c0 0x7f98c8167090 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc88e0f10aed77d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.657+0000 7f98c7e00640 1 -- 172.21.15.3:0/2508821387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007740 con 0x7f98c80d07c0 2024-09-10T07:53:57.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.657+0000 7f98c7e00640 1 -- 172.21.15.3:0/2508821387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c00078e0 con 0x7f98c80d07c0 2024-09-10T07:53:57.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.657+0000 7f98c7e00640 1 -- 172.21.15.3:0/2508821387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007cb0 con 0x7f98c80d07c0 2024-09-10T07:53:57.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 -- 172.21.15.3:0/2508821387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c80d07c0 msgr2=0x7f98c8167090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 --2- 172.21.15.3:0/2508821387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c80d07c0 0x7f98c8167090 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c002a6f0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 -- 172.21.15.3:0/2508821387 shutdown_connections 2024-09-10T07:53:57.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 --2- 172.21.15.3:0/2508821387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c80d07c0 0x7f98c8167090 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 -- 172.21.15.3:0/2508821387 >> 172.21.15.3:0/2508821387 conn(0x7f98c806d000 msgr2=0x7f98c806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:57.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 -- 172.21.15.3:0/2508821387 shutdown_connections 2024-09-10T07:53:57.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.658+0000 7f98cfc00640 1 -- 172.21.15.3:0/2508821387 wait complete. 2024-09-10T07:53:57.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.659+0000 7f98cfc00640 1 Processor -- start 2024-09-10T07:53:57.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 -- start start 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98c80d07c0 0x7f98c81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98c8160af0 0x7f98c8152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f98c815d770 con 0x7f98c8160710 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.660+0000 7f98cfc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8073080 con 0x7f98c80d07c0 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98cfc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f98c80cd710 con 0x7f98c8160af0 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52710/0 (socket says 172.21.15.3:52710) 2024-09-10T07:53:57.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 -- 172.21.15.3:0/401813915 learned_addr learned my addr 172.21.15.3:0/401813915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98cd200640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98c80d07c0 0x7f98c81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98cdc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98c8160af0 0x7f98c8152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 -- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98c8160af0 msgr2=0x7f98c8152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98c8160af0 0x7f98c8152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 -- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98c80d07c0 msgr2=0x7f98c81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98c80d07c0 0x7f98c81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.661+0000 7f98bfe00640 1 -- 172.21.15.3:0/401813915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c81548f0 con 0x7f98c8160710 2024-09-10T07:53:57.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98bfe00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c430 tx=0x7f98b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0017070 con 0x7f98c8160710 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b00052d0 con 0x7f98c8160710 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0003830 con 0x7f98c8160710 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c8154bd0 con 0x7f98c8160710 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.662+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c80cd9f0 con 0x7f98c8160710 2024-09-10T07:53:57.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.663+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c80cfa60 con 0x7f98c8160710 2024-09-10T07:53:57.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.664+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98b001a070 con 0x7f98c8160710 2024-09-10T07:53:57.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.665+0000 7f98c5600640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:57.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.665+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f98b008a240 con 0x7f98c8160710 2024-09-10T07:53:57.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.665+0000 7f98cd200640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:57.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.666+0000 7f98cd200640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 0x7f98a406e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007f70 tx=0x7f98b8007f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:57.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.670+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98b0058470 con 0x7f98c8160710 2024-09-10T07:53:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.894+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98c815d8f0 con 0x7f98c8160710 2024-09-10T07:53:57.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.895+0000 7f98c5600640 1 -- 172.21.15.3:0/401813915 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f98b005c770 con 0x7f98c8160710 2024-09-10T07:53:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 msgr2=0x7f98a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 0x7f98a406e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007f70 tx=0x7f98b8007f00 comp rx=0 tx=0).stop 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 msgr2=0x7f98c81529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c430 tx=0x7f98b000c8f0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 shutdown_connections 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f98c8160af0 0x7f98c8152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98c8160710 0x7f98c81529c0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 --2- 172.21.15.3:0/401813915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98c80d07c0 0x7f98c81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.897+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 >> 172.21.15.3:0/401813915 conn(0x7f98c806d000 msgr2=0x7f98c80b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.898+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 shutdown_connections 2024-09-10T07:53:57.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:57.898+0000 7f98cfc00640 1 -- 172.21.15.3:0/401813915 wait complete. 2024-09-10T07:53:57.913 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:57.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.069+0000 7fdb83200640 1 Processor -- start 2024-09-10T07:53:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.069+0000 7fdb83200640 1 -- start start 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb83200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb8406bb20 0x7fdb8406bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb83200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb840c9800 con 0x7fdb8415e800 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb83200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb840d3300 con 0x7fdb8415e420 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb83200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8406c4a0 con 0x7fdb8406bb20 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb81400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb8406bb20 0x7fdb8406bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb82800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdb8415e800 0x7fdb8415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45328/0 (socket says 172.21.15.3:45328) 2024-09-10T07:53:58.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.070+0000 7fdb82800640 1 -- 172.21.15.3:0/1210124382 learned_addr learned my addr 172.21.15.3:0/1210124382 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.071+0000 7fdb81400640 1 -- 172.21.15.3:0/1210124382 >> v1:172.21.15.70:6789/0 conn(0x7fdb8415e420 legacy=0x7fdb840d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.071+0000 7fdb81400640 1 -- 172.21.15.3:0/1210124382 >> v1:172.21.15.3:6789/0 conn(0x7fdb8415e800 legacy=0x7fdb8415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.071+0000 7fdb81400640 1 -- 172.21.15.3:0/1210124382 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840d35e0 con 0x7fdb8406bb20 2024-09-10T07:53:58.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.071+0000 7fdb80a00640 1 -- 172.21.15.3:0/1210124382 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806087389 0 0) 0x7fdb840c9800 con 0x7fdb8415e800 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.072+0000 7fdb81400640 1 --2- 172.21.15.3:0/1210124382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8406bb20 0x7fdb8406bf60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fdb700097a0 tx=0x7fdb7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02d1777e00ea720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.072+0000 7fdb80a00640 1 -- 172.21.15.3:0/1210124382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70037070 con 0x7fdb8406bb20 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.072+0000 7fdb80a00640 1 -- 172.21.15.3:0/1210124382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb70032440 con 0x7fdb8406bb20 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.072+0000 7fdb80a00640 1 -- 172.21.15.3:0/1210124382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70032810 con 0x7fdb8406bb20 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 -- 172.21.15.3:0/1210124382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8406bb20 msgr2=0x7fdb8406bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 --2- 172.21.15.3:0/1210124382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8406bb20 0x7fdb8406bf60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fdb700097a0 tx=0x7fdb7002a230 comp rx=0 tx=0).stop 2024-09-10T07:53:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 -- 172.21.15.3:0/1210124382 shutdown_connections 2024-09-10T07:53:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 --2- 172.21.15.3:0/1210124382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8406bb20 0x7fdb8406bf60 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 -- 172.21.15.3:0/1210124382 >> 172.21.15.3:0/1210124382 conn(0x7fdb840732e0 msgr2=0x7fdb84062630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 -- 172.21.15.3:0/1210124382 shutdown_connections 2024-09-10T07:53:58.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.073+0000 7fdb83200640 1 -- 172.21.15.3:0/1210124382 wait complete. 2024-09-10T07:53:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.074+0000 7fdb83200640 1 Processor -- start 2024-09-10T07:53:58.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 -- start start 2024-09-10T07:53:58.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 0x7fdb8414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb8415e800 0x7fdb841520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840d3e80 con 0x7fdb8415e420 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840d3ba0 con 0x7fdb8406bb20 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.075+0000 7fdb83200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdb840d2ea0 con 0x7fdb8415e800 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52728/0 (socket says 172.21.15.3:52728) 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 -- 172.21.15.3:0/2101744301 learned_addr learned my addr 172.21.15.3:0/2101744301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81e00640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 0x7fdb8414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb82800640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb8415e800 0x7fdb841520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:58.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 -- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb8415e800 msgr2=0x7fdb841520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb8415e800 0x7fdb841520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 -- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 msgr2=0x7fdb8414b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 0x7fdb8414b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 -- 172.21.15.3:0/2101744301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8414b5c0 con 0x7fdb8415e420 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81e00640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 0x7fdb8414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:58.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.076+0000 7fdb81400640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098c0 tx=0x7fdb700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.077+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70037040 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.077+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb70042070 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.077+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb840cf450 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.077+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70030a70 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.077+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb840cfa10 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.078+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8406bf60 con 0x7fdb8415e420 2024-09-10T07:53:58.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.079+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb70030c10 con 0x7fdb8415e420 2024-09-10T07:53:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.080+0000 7fdb7a000640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 0x7fdb6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.080+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fdb700aba00 con 0x7fdb8415e420 2024-09-10T07:53:58.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.080+0000 7fdb81e00640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 0x7fdb6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.081+0000 7fdb81e00640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 0x7fdb6406e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdb84152780 tx=0x7fdb74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.084+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb7007a310 con 0x7fdb8415e420 2024-09-10T07:53:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.337+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdb8415d640 con 0x7fdb8415e420 2024-09-10T07:53:58.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.338+0000 7fdb7a000640 1 -- 172.21.15.3:0/2101744301 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1981 (secure 0 0 0) 0x7fdb70079c10 con 0x7fdb8415e420 2024-09-10T07:53:58.337 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:58.337 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":271278080,"bytes_avail":772822835200,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-09-10T07:53:57.591421+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 msgr2=0x7fdb6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 0x7fdb6406e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdb84152780 tx=0x7fdb74009040 comp rx=0 tx=0).stop 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 msgr2=0x7fdb84151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098c0 tx=0x7fdb700046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 shutdown_connections 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fdb6406be30 0x7fdb6406e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdb8415e800 0x7fdb841520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb8415e420 0x7fdb84151b70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 --2- 172.21.15.3:0/2101744301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb8406bb20 0x7fdb8414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:58.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 >> 172.21.15.3:0/2101744301 conn(0x7fdb840732e0 msgr2=0x7fdb840d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:58.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 shutdown_connections 2024-09-10T07:53:58.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:58.340+0000 7fdb83200640 1 -- 172.21.15.3:0/2101744301 wait complete. 2024-09-10T07:53:58.355 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:53:59.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.523+0000 7f7684200640 1 Processor -- start 2024-09-10T07:53:59.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.523+0000 7f7684200640 1 -- start start 2024-09-10T07:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.524+0000 7f7684200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f767c161b30 0x7f767c15f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7684200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7684200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767c15fbb0 con 0x7f767c15fdb0 2024-09-10T07:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7684200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f767c0b36a0 con 0x7f767c161b30 2024-09-10T07:53:59.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7684200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f767c0b3980 con 0x7f767c06c230 2024-09-10T07:53:59.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7681800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f767c161b30 0x7f767c15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7682200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7682200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:55462/0 (socket says 172.21.15.3:55462) 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7682200640 1 -- 172.21.15.3:0/905538835 learned_addr learned my addr 172.21.15.3:0/905538835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.525+0000 7f7680e00640 1 --1- 172.21.15.3:0/905538835 >> v1:172.21.15.70:6789/0 conn(0x7f767c15fdb0 0x7f767c06bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:57156/0 (socket says 172.21.15.3:57156) 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f7681800640 1 -- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 msgr2=0x7f767c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f7681800640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f7681800640 1 -- 172.21.15.3:0/905538835 >> v1:172.21.15.70:6789/0 conn(0x7f767c15fdb0 legacy=0x7f767c06bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f7681800640 1 -- 172.21.15.3:0/905538835 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c0b3c60 con 0x7f767c161b30 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f767be00640 1 -- 172.21.15.3:0/905538835 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212826344 0 0) 0x7f7668001110 con 0x7f767c15fdb0 2024-09-10T07:53:59.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.526+0000 7f7682200640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:53:59.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.527+0000 7f7681800640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c15f670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f76700097a0 tx=0x7f767002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=268d0a76512012f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.527+0000 7f767be00640 1 -- 172.21.15.3:0/905538835 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670037070 con 0x7f767c161b30 2024-09-10T07:53:59.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.527+0000 7f767be00640 1 -- 172.21.15.3:0/905538835 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670032440 con 0x7f767c161b30 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.527+0000 7f767be00640 1 -- 172.21.15.3:0/905538835 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670032810 con 0x7f767c161b30 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 -- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 msgr2=0x7f767c15f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c15f670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f76700097a0 tx=0x7f767002a180 comp rx=0 tx=0).stop 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 -- 172.21.15.3:0/905538835 shutdown_connections 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c06c230 0x7f767c0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 --2- 172.21.15.3:0/905538835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c15f670 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 -- 172.21.15.3:0/905538835 >> 172.21.15.3:0/905538835 conn(0x7f767c06d000 msgr2=0x7f767c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 -- 172.21.15.3:0/905538835 shutdown_connections 2024-09-10T07:53:59.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.528+0000 7f7684200640 1 -- 172.21.15.3:0/905538835 wait complete. 2024-09-10T07:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.529+0000 7f7684200640 1 Processor -- start 2024-09-10T07:53:59.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.530+0000 7f7684200640 1 -- start start 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 0x7f767c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f767c161f10 con 0x7f767c06c230 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f767c0b4220 con 0x7f767c161b30 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7684200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f767c0aefa0 con 0x7f767c15fdb0 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52738/0 (socket says 172.21.15.3:52738) 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 -- 172.21.15.3:0/1340538347 learned_addr learned my addr 172.21.15.3:0/1340538347 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:59.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7680e00640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 0x7f767c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7682200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 -- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 msgr2=0x7f767c150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 0x7f767c150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 -- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 msgr2=0x7f767c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7681800640 1 -- 172.21.15.3:0/1340538347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c1423c0 con 0x7f767c06c230 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7680e00640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 0x7f767c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.531+0000 7f7682200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:53:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7681800640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f76700098c0 tx=0x7f76700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670037040 con 0x7f767c06c230 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670030500 con 0x7f767c06c230 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76700308d0 con 0x7f767c06c230 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c151950 con 0x7f767c06c230 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.532+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c151f10 con 0x7f767c06c230 2024-09-10T07:53:59.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.533+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767c152c80 con 0x7f767c06c230 2024-09-10T07:53:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.534+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7670044020 con 0x7f767c06c230 2024-09-10T07:53:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.535+0000 7f7679600640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 0x7f765c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.535+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f76700ab150 con 0x7f767c06c230 2024-09-10T07:53:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.535+0000 7f7680e00640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 0x7f765c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.536+0000 7f7680e00640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 0x7f765c06e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f767c14ea80 tx=0x7f7668007a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.539+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f767c152c80 con 0x7f767c06c230 2024-09-10T07:53:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.760+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f767c0b3300 con 0x7f767c06c230 2024-09-10T07:53:59.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.760+0000 7f7679600640 1 -- 172.21.15.3:0/1340538347 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f767c1423c0 con 0x7f767c06c230 2024-09-10T07:53:59.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:53:59.761 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:53:59.761 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:53:59.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 msgr2=0x7f765c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 0x7f765c06e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f767c14ea80 tx=0x7f7668007a30 comp rx=0 tx=0).stop 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 msgr2=0x7f767c141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f76700098c0 tx=0x7f76700046c0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 shutdown_connections 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f765c06be30 0x7f765c06e2f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c161b30 0x7f767c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f767c15fdb0 0x7f767c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 --2- 172.21.15.3:0/1340538347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06c230 0x7f767c141e80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.762+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 >> 172.21.15.3:0/1340538347 conn(0x7f767c06d000 msgr2=0x7f767c15ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:59.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.763+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 shutdown_connections 2024-09-10T07:53:59.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.763+0000 7f7684200640 1 -- 172.21.15.3:0/1340538347 wait complete. 2024-09-10T07:53:59.778 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:53:59.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:53:59.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.934+0000 7fc029e00640 1 Processor -- start 2024-09-10T07:53:59.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.934+0000 7fc029e00640 1 -- start start 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc024145d00 0x7fc0240cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc024073330 0x7fc024160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc024074200 con 0x7fc024073750 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc024073c40 con 0x7fc024145d00 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc024073f20 con 0x7fc024073330 2024-09-10T07:53:59.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc028a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc024145d00 0x7fc0240cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc024073750 0x7fc0240714b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45338/0 (socket says 172.21.15.3:45338) 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc029400640 1 -- 172.21.15.3:0/482412406 learned_addr learned my addr 172.21.15.3:0/482412406 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.935+0000 7fc023e00640 1 --2- 172.21.15.3:0/482412406 >> v2:172.21.15.70:3300/0 conn(0x7fc024073330 0x7fc024160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.936+0000 7fc028a00640 1 -- 172.21.15.3:0/482412406 >> v2:172.21.15.70:3300/0 conn(0x7fc024073330 msgr2=0x7fc024160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.936+0000 7fc028a00640 1 --2- 172.21.15.3:0/482412406 >> v2:172.21.15.70:3300/0 conn(0x7fc024073330 0x7fc024160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.936+0000 7fc028a00640 1 -- 172.21.15.3:0/482412406 >> v1:172.21.15.3:6789/0 conn(0x7fc024073750 legacy=0x7fc0240714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.936+0000 7fc028a00640 1 -- 172.21.15.3:0/482412406 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02415ff30 con 0x7fc024145d00 2024-09-10T07:53:59.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.936+0000 7fc023400640 1 -- 172.21.15.3:0/482412406 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193916001 0 0) 0x7fc024074200 con 0x7fc024073750 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.937+0000 7fc028a00640 1 --2- 172.21.15.3:0/482412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024145d00 0x7fc0240cfbe0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ce190 tx=0x7fc01802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b58dc26a0585873 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.937+0000 7fc023400640 1 -- 172.21.15.3:0/482412406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018037070 con 0x7fc024145d00 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.937+0000 7fc023400640 1 -- 172.21.15.3:0/482412406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc018032440 con 0x7fc024145d00 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.937+0000 7fc023400640 1 -- 172.21.15.3:0/482412406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018032810 con 0x7fc024145d00 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 -- 172.21.15.3:0/482412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024145d00 msgr2=0x7fc0240cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 --2- 172.21.15.3:0/482412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024145d00 0x7fc0240cfbe0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ce190 tx=0x7fc01802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 -- 172.21.15.3:0/482412406 shutdown_connections 2024-09-10T07:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 --2- 172.21.15.3:0/482412406 >> v2:172.21.15.70:3300/0 conn(0x7fc024073330 0x7fc024160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 --2- 172.21.15.3:0/482412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024145d00 0x7fc0240cfbe0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 -- 172.21.15.3:0/482412406 >> 172.21.15.3:0/482412406 conn(0x7fc02406bb20 msgr2=0x7fc02406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 -- 172.21.15.3:0/482412406 shutdown_connections 2024-09-10T07:53:59.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.938+0000 7fc029e00640 1 -- 172.21.15.3:0/482412406 wait complete. 2024-09-10T07:53:59.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.939+0000 7fc029e00640 1 Processor -- start 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 -- start start 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 0x7fc024157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc024145d00 0x7fc02413a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc024160df0 con 0x7fc024073750 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc024160c70 con 0x7fc024073330 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.940+0000 7fc029e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc024071bc0 con 0x7fc024145d00 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52754/0 (socket says 172.21.15.3:52754) 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 -- 172.21.15.3:0/1382210654 learned_addr learned my addr 172.21.15.3:0/1382210654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 -- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc024145d00 msgr2=0x7fc02413a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc024145d00 0x7fc02413a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 -- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 msgr2=0x7fc024157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc028a00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 0x7fc024157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 0x7fc024157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 -- 172.21.15.3:0/1382210654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241588e0 con 0x7fc024073750 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc028a00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 0x7fc024157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.941+0000 7fc023e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d5f0 tx=0x7fc01000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.942+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100125f0 con 0x7fc024073750 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.942+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc010019070 con 0x7fc024073750 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.942+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100129d0 con 0x7fc024073750 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.942+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc024158bc0 con 0x7fc024073750 2024-09-10T07:53:59.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.942+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc024159180 con 0x7fc024073750 2024-09-10T07:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.943+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024160730 con 0x7fc024073750 2024-09-10T07:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.944+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc010012b70 con 0x7fc024073750 2024-09-10T07:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.945+0000 7fc020c00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 0x7fc00406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.945+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fc01008b310 con 0x7fc024073750 2024-09-10T07:53:59.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.945+0000 7fc028a00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 0x7fc00406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:53:59.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.945+0000 7fc028a00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 0x7fc00406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc018034000 tx=0x7fc01802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:53:59.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:53:59.949+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc010059c20 con 0x7fc024073750 2024-09-10T07:54:00.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.204+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc0240cdd10 con 0x7fc024073750 2024-09-10T07:54:00.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.204+0000 7fc020c00640 1 -- 172.21.15.3:0/1382210654 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1962 (secure 0 0 0) 0x7fc010059520 con 0x7fc024073750 2024-09-10T07:54:00.204 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:00.205 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":473,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":271278080,"bytes_avail":772822835200,"bytes_total":773094113280,"write_bytes_sec":1109,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-09-10T07:53:57.591421+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 msgr2=0x7fc00406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 0x7fc00406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc018034000 tx=0x7fc01802e000 comp rx=0 tx=0).stop 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 msgr2=0x7fc02413b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d5f0 tx=0x7fc01000dab0 comp rx=0 tx=0).stop 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 shutdown_connections 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc00406be30 0x7fc00406e2f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc024145d00 0x7fc02413a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:00.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc024073750 0x7fc02413b360 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 --2- 172.21.15.3:0/1382210654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc024073330 0x7fc024157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 >> 172.21.15.3:0/1382210654 conn(0x7fc02406bb20 msgr2=0x7fc0240cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 shutdown_connections 2024-09-10T07:54:00.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:00.207+0000 7fc029e00640 1 -- 172.21.15.3:0/1382210654 wait complete. 2024-09-10T07:54:00.223 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:54:01.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:01.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.392+0000 7f3a73800640 1 Processor -- start 2024-09-10T07:54:01.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.392+0000 7f3a73800640 1 -- start start 2024-09-10T07:54:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.393+0000 7f3a73800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.393+0000 7f3a73800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 0x7f3a6c0d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.393+0000 7f3a73800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c0b70b0 con 0x7f3a6c0d1a40 2024-09-10T07:54:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.393+0000 7f3a73800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c0b78b0 con 0x7f3a6c162cd0 2024-09-10T07:54:01.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.393+0000 7f3a73800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c0b62f0 con 0x7f3a6c15e970 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a6be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 0x7f3a6c0d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a6be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 0x7f3a6c0d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:47878/0 (socket says 172.21.15.3:47878) 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a71800640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3a6c0d1a40 0x7f3a6c0d2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.3:57172/0 (socket says 172.21.15.3:57172) 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a6be00640 1 -- 172.21.15.3:0/82524028 learned_addr learned my addr 172.21.15.3:0/82524028 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:55492/0 (socket says 172.21.15.3:55492) 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 -- 172.21.15.3:0/82524028 >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 msgr2=0x7f3a6c0d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 --2- 172.21.15.3:0/82524028 >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 0x7f3a6c0d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 -- 172.21.15.3:0/82524028 >> v1:172.21.15.70:6789/0 conn(0x7f3a6c0d1a40 legacy=0x7f3a6c0d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a70e00640 1 -- 172.21.15.3:0/82524028 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c0b65d0 con 0x7f3a6c15e970 2024-09-10T07:54:01.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.394+0000 7f3a6b400640 1 -- 172.21.15.3:0/82524028 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22837169 0 0) 0x7f3a64001490 con 0x7f3a6c0d1a40 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.395+0000 7f3a70e00640 1 --2- 172.21.15.3:0/82524028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a600097a0 tx=0x7f3a6002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ed5a54d9d9025e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a6b400640 1 -- 172.21.15.3:0/82524028 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60037070 con 0x7f3a6c15e970 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a6b400640 1 -- 172.21.15.3:0/82524028 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6002ae50 con 0x7f3a6c15e970 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a6b400640 1 -- 172.21.15.3:0/82524028 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60032660 con 0x7f3a6c15e970 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 -- 172.21.15.3:0/82524028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 msgr2=0x7f3a6c0b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 --2- 172.21.15.3:0/82524028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3a600097a0 tx=0x7f3a6002a100 comp rx=0 tx=0).stop 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 -- 172.21.15.3:0/82524028 shutdown_connections 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 --2- 172.21.15.3:0/82524028 >> v2:172.21.15.70:3300/0 conn(0x7f3a6c162cd0 0x7f3a6c0d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 --2- 172.21.15.3:0/82524028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c15e970 0x7f3a6c0b37a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 -- 172.21.15.3:0/82524028 >> 172.21.15.3:0/82524028 conn(0x7f3a6c06bd00 msgr2=0x7f3a6c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 -- 172.21.15.3:0/82524028 shutdown_connections 2024-09-10T07:54:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.396+0000 7f3a73800640 1 -- 172.21.15.3:0/82524028 wait complete. 2024-09-10T07:54:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.398+0000 7f3a73800640 1 Processor -- start 2024-09-10T07:54:01.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.398+0000 7f3a73800640 1 -- start start 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c0d1a40 0x7f3a6c1f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a6c162cd0 0x7f3a6c1f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0cfd50 con 0x7f3a6c15e970 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0b6b90 con 0x7f3a6c162cd0 2024-09-10T07:54:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a73800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c163130 con 0x7f3a6c0d1a40 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a6be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a6be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52768/0 (socket says 172.21.15.3:52768) 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a6be00640 1 -- 172.21.15.3:0/402218183 learned_addr learned my addr 172.21.15.3:0/402218183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a70e00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c0d1a40 0x7f3a6c1f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.399+0000 7f3a71800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a6c162cd0 0x7f3a6c1f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 -- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c0d1a40 msgr2=0x7f3a6c1f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c0d1a40 0x7f3a6c1f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 -- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a6c162cd0 msgr2=0x7f3a6c1f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a6c162cd0 0x7f3a6c1f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 -- 172.21.15.3:0/402218183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c15ff50 con 0x7f3a6c15e970 2024-09-10T07:54:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a6be00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e8a0 tx=0x7f3a5800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a58017070 con 0x7f3a6c15e970 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5800c940 con 0x7f3a6c15e970 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c0ce5a0 con 0x7f3a6c15e970 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800cd10 con 0x7f3a6c15e970 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.400+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c15f6b0 con 0x7f3a6c15e970 2024-09-10T07:54:01.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.401+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c1f50d0 con 0x7f3a6c15e970 2024-09-10T07:54:01.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.403+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a5801a070 con 0x7f3a6c15e970 2024-09-10T07:54:01.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.403+0000 7f3a68c00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 0x7f3a4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.403+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f3a5808ac50 con 0x7f3a6c15e970 2024-09-10T07:54:01.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.404+0000 7f3a70e00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 0x7f3a4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.404+0000 7f3a70e00640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 0x7f3a4c06e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3a60034000 tx=0x7f3a6002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.408+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a58090020 con 0x7f3a6c15e970 2024-09-10T07:54:01.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.632+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a6c15cff0 con 0x7f3a6c15e970 2024-09-10T07:54:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.632+0000 7f3a68c00640 1 -- 172.21.15.3:0/402218183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f3a58058fb0 con 0x7f3a6c15e970 2024-09-10T07:54:01.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:01.632 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:01.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.634+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 msgr2=0x7f3a4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.634+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 0x7f3a4c06e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3a60034000 tx=0x7f3a6002e000 comp rx=0 tx=0).stop 2024-09-10T07:54:01.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.634+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 msgr2=0x7f3a6c1f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.634+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e8a0 tx=0x7f3a5800ed60 comp rx=0 tx=0).stop 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 shutdown_connections 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3a4c06be30 0x7f3a4c06e2f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a6c162cd0 0x7f3a6c1f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a6c15e970 0x7f3a6c1f5e70 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 --2- 172.21.15.3:0/402218183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3a6c0d1a40 0x7f3a6c1f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 >> 172.21.15.3:0/402218183 conn(0x7f3a6c06bd00 msgr2=0x7f3a6c0d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 shutdown_connections 2024-09-10T07:54:01.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.635+0000 7f3a73800640 1 -- 172.21.15.3:0/402218183 wait complete. 2024-09-10T07:54:01.651 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:54:01.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:54:01.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.808+0000 7f48b4e00640 1 Processor -- start 2024-09-10T07:54:01.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.808+0000 7f48b4e00640 1 -- start start 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48b4e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48b4e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48b00b76a0 0x7f48b00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48b4e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b01620d0 con 0x7f48b0161cf0 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48b4e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f48b0072140 con 0x7f48b00b76a0 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48b4e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f48b00b7b30 con 0x7f48b00cf680 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48ae800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48b00b76a0 0x7f48b00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48ae800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48b00b76a0 0x7f48b00d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52782/0 (socket says 172.21.15.3:52782) 2024-09-10T07:54:01.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.809+0000 7f48ae800640 1 -- 172.21.15.3:0/851225848 learned_addr learned my addr 172.21.15.3:0/851225848 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.810+0000 7f48ad400640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.810+0000 7f48ae800640 1 -- 172.21.15.3:0/851225848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 msgr2=0x7f48b00d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.810+0000 7f48ae800640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.810+0000 7f48ae800640 1 -- 172.21.15.3:0/851225848 >> v1:172.21.15.70:6789/0 conn(0x7f48b0161cf0 legacy=0x7f48b00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.810+0000 7f48ae800640 1 -- 172.21.15.3:0/851225848 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b00d41e0 con 0x7f48b00b76a0 2024-09-10T07:54:01.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.811+0000 7f48ae800640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b00b76a0 0x7f48b00d3260 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009e50 tx=0x7f48a802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907106fe17425f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.811+0000 7f48aca00640 1 -- 172.21.15.3:0/851225848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8036070 con 0x7f48b00b76a0 2024-09-10T07:54:01.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.811+0000 7f48aca00640 1 -- 172.21.15.3:0/851225848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8031600 con 0x7f48b00b76a0 2024-09-10T07:54:01.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.811+0000 7f48aca00640 1 -- 172.21.15.3:0/851225848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a80319d0 con 0x7f48b00b76a0 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 -- 172.21.15.3:0/851225848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b00b76a0 msgr2=0x7f48b00d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b00b76a0 0x7f48b00d3260 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009e50 tx=0x7f48a802a680 comp rx=0 tx=0).stop 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 -- 172.21.15.3:0/851225848 shutdown_connections 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b00b76a0 0x7f48b00d3260 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 --2- 172.21.15.3:0/851225848 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 -- 172.21.15.3:0/851225848 >> 172.21.15.3:0/851225848 conn(0x7f48b00d4710 msgr2=0x7f48b00d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 -- 172.21.15.3:0/851225848 shutdown_connections 2024-09-10T07:54:01.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.812+0000 7f48b4e00640 1 -- 172.21.15.3:0/851225848 wait complete. 2024-09-10T07:54:01.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.813+0000 7f48b4e00640 1 Processor -- start 2024-09-10T07:54:01.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.813+0000 7f48b4e00640 1 -- start start 2024-09-10T07:54:01.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b00b76a0 0x7f48b014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0072950 con 0x7f48b0161cf0 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0072670 con 0x7f48b00b76a0 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.814+0000 7f48b4e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f48b0072e70 con 0x7f48b00cf680 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52790/0 (socket says 172.21.15.3:52790) 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 -- 172.21.15.3:0/4020012978 learned_addr learned my addr 172.21.15.3:0/4020012978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ad400640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ade00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b00b76a0 0x7f48b014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 -- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 msgr2=0x7f48b00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 -- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b00b76a0 msgr2=0x7f48b014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:01.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b00b76a0 0x7f48b014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:01.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.815+0000 7f48ae800640 1 -- 172.21.15.3:0/4020012978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b00ca1d0 con 0x7f48b0161cf0 2024-09-10T07:54:01.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48ae800640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009f70 tx=0x7f48a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a803f020 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8037040 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8036070 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b00ca4b0 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.816+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b00caa70 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.817+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b00d3b00 con 0x7f48b0161cf0 2024-09-10T07:54:01.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.818+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48a8042020 con 0x7f48b0161cf0 2024-09-10T07:54:01.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.819+0000 7f48a6000640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 0x7f489006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:01.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.819+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f48a80ab220 con 0x7f48b0161cf0 2024-09-10T07:54:01.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.819+0000 7f48ade00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 0x7f489006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:01.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.819+0000 7f48ade00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 0x7f489006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f48a00016e0 tx=0x7f48a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:01.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:01.823+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f48a8049030 con 0x7f48b0161cf0 2024-09-10T07:54:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.076+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f48b00b48d0 con 0x7f48b0161cf0 2024-09-10T07:54:02.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.076+0000 7f48a6000640 1 -- 172.21.15.3:0/4020012978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1961 (secure 0 0 0) 0x7f48a80794e0 con 0x7f48b0161cf0 2024-09-10T07:54:02.076 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":475,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":706277,"bytes_used":271278080,"bytes_avail":772822835200,"bytes_total":773094113280,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-09-10T07:53:57.591421+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-10T07:54:02.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 msgr2=0x7f489006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 0x7f489006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f48a00016e0 tx=0x7f48a0009040 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 msgr2=0x7f48b015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009f70 tx=0x7f48a8004a40 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 shutdown_connections 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f489006be60 0x7f489006e320 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b0161cf0 0x7f48b015ba20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f48b00cf680 0x7f48b00c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 --2- 172.21.15.3:0/4020012978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b00b76a0 0x7f48b014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.078+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 >> 172.21.15.3:0/4020012978 conn(0x7f48b00d4710 msgr2=0x7f48b00b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.079+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 shutdown_connections 2024-09-10T07:54:02.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:02.079+0000 7f48b4e00640 1 -- 172.21.15.3:0/4020012978 wait complete. 2024-09-10T07:54:02.094 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-10T07:54:03.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:03.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.261+0000 7fd2a9000640 1 Processor -- start 2024-09-10T07:54:03.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.261+0000 7fd2a9000640 1 -- start start 2024-09-10T07:54:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.262+0000 7fd2a9000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a9000640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a40b4250 con 0x7fd2a40d08f0 2024-09-10T07:54:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a9000640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a40ccd80 con 0x7fd2a415b4f0 2024-09-10T07:54:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a9000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a40b44f0 con 0x7fd2a415b110 2024-09-10T07:54:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a2800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd2a40d08f0 0x7fd2a40cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45350/0 (socket says 172.21.15.3:45350) 2024-09-10T07:54:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a2800640 1 -- 172.21.15.3:0/723017140 learned_addr learned my addr 172.21.15.3:0/723017140 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a1e00640 1 --2- 172.21.15.3:0/723017140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200055336 0 0) 0x7fd2a40b4250 con 0x7fd2a40d08f0 2024-09-10T07:54:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd288003520 con 0x7fd2a40d08f0 2024-09-10T07:54:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140216847 0 0) 0x7fd2a40ccd80 con 0x7fd2a415b4f0 2024-09-10T07:54:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.263+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a40b4250 con 0x7fd2a415b4f0 2024-09-10T07:54:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.264+0000 7fd2a1e00640 1 -- 172.21.15.3:0/723017140 >> v1:172.21.15.70:6789/0 conn(0x7fd2a415b4f0 legacy=0x7fd2a40d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.264+0000 7fd2a1e00640 1 -- 172.21.15.3:0/723017140 >> v1:172.21.15.3:6789/0 conn(0x7fd2a40d08f0 legacy=0x7fd2a40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.264+0000 7fd2a1e00640 1 -- 172.21.15.3:0/723017140 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a40cd060 con 0x7fd2a415b110 2024-09-10T07:54:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a1e00640 1 --2- 172.21.15.3:0/723017140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a40b3d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2940097d0 tx=0x7fd29402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ed95d5125bb801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294037070 con 0x7fd2a415b110 2024-09-10T07:54:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd294032440 con 0x7fd2a415b110 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a1400640 1 -- 172.21.15.3:0/723017140 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940327f0 con 0x7fd2a415b110 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a9000640 1 -- 172.21.15.3:0/723017140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 msgr2=0x7fd2a40b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.265+0000 7fd2a9000640 1 --2- 172.21.15.3:0/723017140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a40b3d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2940097d0 tx=0x7fd29402a210 comp rx=0 tx=0).stop 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.266+0000 7fd2a9000640 1 -- 172.21.15.3:0/723017140 shutdown_connections 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.266+0000 7fd2a9000640 1 --2- 172.21.15.3:0/723017140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a40b3d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.266+0000 7fd2a9000640 1 -- 172.21.15.3:0/723017140 >> 172.21.15.3:0/723017140 conn(0x7fd2a406d000 msgr2=0x7fd2a406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.266+0000 7fd2a9000640 1 -- 172.21.15.3:0/723017140 shutdown_connections 2024-09-10T07:54:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.266+0000 7fd2a9000640 1 -- 172.21.15.3:0/723017140 wait complete. 2024-09-10T07:54:03.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.267+0000 7fd2a9000640 1 Processor -- start 2024-09-10T07:54:03.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.267+0000 7fd2a9000640 1 -- start start 2024-09-10T07:54:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a4151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2a415b4f0 0x7fd2a4151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4073d10 con 0x7fd2a40d08f0 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a40cd620 con 0x7fd2a415b4f0 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a9000640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a4073e90 con 0x7fd2a415b110 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.268+0000 7fd2a1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52800/0 (socket says 172.21.15.3:52800) 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 -- 172.21.15.3:0/3148593762 learned_addr learned my addr 172.21.15.3:0/3148593762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a2800640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2a415b4f0 0x7fd2a4151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd29be00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a4151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 -- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 msgr2=0x7fd2a4151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a4151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 -- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2a415b4f0 msgr2=0x7fd2a4151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2a415b4f0 0x7fd2a4151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 -- 172.21.15.3:0/3148593762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a415a940 con 0x7fd2a40d08f0 2024-09-10T07:54:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd29be00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a4151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd2a1e00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd29402a250 tx=0x7fd2940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.269+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294041020 con 0x7fd2a40d08f0 2024-09-10T07:54:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.270+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29403d6e0 con 0x7fd2a40d08f0 2024-09-10T07:54:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.270+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294037040 con 0x7fd2a40d08f0 2024-09-10T07:54:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.270+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a4154700 con 0x7fd2a40d08f0 2024-09-10T07:54:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.270+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a4153e60 con 0x7fd2a40d08f0 2024-09-10T07:54:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.271+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a415c3d0 con 0x7fd2a40d08f0 2024-09-10T07:54:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.272+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd294044020 con 0x7fd2a40d08f0 2024-09-10T07:54:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.273+0000 7fd29a000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 0x7fd28006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.273+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd2940ab270 con 0x7fd2a40d08f0 2024-09-10T07:54:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.273+0000 7fd29be00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 0x7fd28006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.274+0000 7fd29be00640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 0x7fd28006e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd2a416a090 tx=0x7fd28c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.277+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd29404b070 con 0x7fd2a40d08f0 2024-09-10T07:54:03.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.498+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a40b3d10 con 0x7fd2a40d08f0 2024-09-10T07:54:03.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.498+0000 7fd29a000640 1 -- 172.21.15.3:0/3148593762 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7fd294079b90 con 0x7fd2a40d08f0 2024-09-10T07:54:03.498 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:03.498 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:03.499 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 msgr2=0x7fd28006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 0x7fd28006e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd2a416a090 tx=0x7fd28c009040 comp rx=0 tx=0).stop 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 msgr2=0x7fd2a415a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd29402a250 tx=0x7fd2940046c0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 shutdown_connections 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fd28006be30 0x7fd28006e2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2a415b4f0 0x7fd2a4151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd2a415b110 0x7fd2a4151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 --2- 172.21.15.3:0/3148593762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a40d08f0 0x7fd2a415a400 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 >> 172.21.15.3:0/3148593762 conn(0x7fd2a406d000 msgr2=0x7fd2a416a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:03.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 shutdown_connections 2024-09-10T07:54:03.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.501+0000 7fd2a9000640 1 -- 172.21.15.3:0/3148593762 wait complete. 2024-09-10T07:54:03.519 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'a', 'd'] 2024-09-10T07:54:03.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T07:54:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.674+0000 7f4501400640 1 Processor -- start 2024-09-10T07:54:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.674+0000 7f4501400640 1 -- start start 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.675+0000 7f4501400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 0x7f44fc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.675+0000 7f4501400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0cd7f0 con 0x7f44fc0cfdc0 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.675+0000 7f4501400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0cf880 con 0x7f44fc15c720 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.675+0000 7f4501400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc072210 con 0x7f44fc070f30 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.675+0000 7f44fb200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44fc0cfdc0 0x7f44fc15cb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45360/0 (socket says 172.21.15.3:45360) 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44fb200640 1 -- 172.21.15.3:0/738522045 learned_addr learned my addr 172.21.15.3:0/738522045 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:03.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9e00640 1 --2- 172.21.15.3:0/738522045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 0x7f44fc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 381603634 0 0) 0x7f44fc0cd7f0 con 0x7f44fc0cfdc0 2024-09-10T07:54:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e0003520 con 0x7f44fc0cfdc0 2024-09-10T07:54:03.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251737419 0 0) 0x7f44fc0cf880 con 0x7f44fc15c720 2024-09-10T07:54:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44fc0cd7f0 con 0x7f44fc15c720 2024-09-10T07:54:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9e00640 1 -- 172.21.15.3:0/738522045 >> v1:172.21.15.70:6789/0 conn(0x7f44fc15c720 legacy=0x7f44fc073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1103430877 0 0) 0x7f44e0003520 con 0x7f44fc0cfdc0 2024-09-10T07:54:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9e00640 1 -- 172.21.15.3:0/738522045 >> v1:172.21.15.3:6789/0 conn(0x7f44fc0cfdc0 legacy=0x7f44fc15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.676+0000 7f44f9e00640 1 -- 172.21.15.3:0/738522045 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc0cda90 con 0x7f44fc070f30 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f44f9e00640 1 --2- 172.21.15.3:0/738522045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 0x7f44fc071cd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097d0 tx=0x7f44e802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f0425dcb6a52c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8037070 con 0x7f44fc070f30 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e802ac30 con 0x7f44fc070f30 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f44f9400640 1 -- 172.21.15.3:0/738522045 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40019f0 con 0x7f44fc070f30 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f4501400640 1 -- 172.21.15.3:0/738522045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 msgr2=0x7f44fc071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f4501400640 1 --2- 172.21.15.3:0/738522045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 0x7f44fc071cd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097d0 tx=0x7f44e802a1a0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f4501400640 1 -- 172.21.15.3:0/738522045 shutdown_connections 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f4501400640 1 --2- 172.21.15.3:0/738522045 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc070f30 0x7f44fc071cd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.678+0000 7f4501400640 1 -- 172.21.15.3:0/738522045 >> 172.21.15.3:0/738522045 conn(0x7f44fc06cfc0 msgr2=0x7f44fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.679+0000 7f4501400640 1 -- 172.21.15.3:0/738522045 shutdown_connections 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.679+0000 7f4501400640 1 -- 172.21.15.3:0/738522045 wait complete. 2024-09-10T07:54:03.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.680+0000 7f4501400640 1 Processor -- start 2024-09-10T07:54:03.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.680+0000 7f4501400640 1 -- start start 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44fc070f30 0x7f44fc148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc0cfdc0 0x7f44fc1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc073d20 con 0x7f44fc15c720 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06ed10 con 0x7f44fc070f30 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f4501400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc073ea0 con 0x7f44fc0cfdc0 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f44fb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f44fb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52806/0 (socket says 172.21.15.3:52806) 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.681+0000 7f44fb200640 1 -- 172.21.15.3:0/1059877373 learned_addr learned my addr 172.21.15.3:0/1059877373 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44f9e00640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc0cfdc0 0x7f44fc1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fa800640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44fc070f30 0x7f44fc148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 -- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc0cfdc0 msgr2=0x7f44fc1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc0cfdc0 0x7f44fc1521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 -- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44fc070f30 msgr2=0x7f44fc148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44fc070f30 0x7f44fc148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 -- 172.21.15.3:0/1059877373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc152f90 con 0x7f44fc15c720 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44fb200640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001350 tx=0x7f44f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4017070 con 0x7f44fc15c720 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f400ad40 con 0x7f44fc15c720 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4012620 con 0x7f44fc15c720 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc153270 con 0x7f44fc15c720 2024-09-10T07:54:03.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.682+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc153830 con 0x7f44fc15c720 2024-09-10T07:54:03.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.683+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc06e630 con 0x7f44fc15c720 2024-09-10T07:54:03.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.684+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44f401a070 con 0x7f44fc15c720 2024-09-10T07:54:03.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.685+0000 7f44f2a00640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 0x7f44dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:03.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.685+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f44f408ad80 con 0x7f44fc15c720 2024-09-10T07:54:03.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.686+0000 7f44fa800640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 0x7f44dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:03.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.686+0000 7f44fa800640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 0x7f44dc06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0011e0 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:03.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.690+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44fc06e630 con 0x7f44fc15c720 2024-09-10T07:54:03.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.940+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f44fc0cf880 con 0x7f44fc15c720 2024-09-10T07:54:03.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.940+0000 7f44f2a00640 1 -- 172.21.15.3:0/1059877373 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1868 (secure 0 0 0) 0x7f44f4059720 con 0x7f44fc15c720 2024-09-10T07:54:03.940 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:03.940 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"46339922-75e0-4945-8bb6-1acaef196937","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":477,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1725954381,"num_in_osds":8,"osd_in_since":1725954369,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":712043,"bytes_used":271278080,"bytes_avail":772822835200,"bytes_total":773094113280,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6872},{"filesystem_id":7,"rank":1,"name":"a","status":"up:active","gid":6875},{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7543}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":33,"modified":"2024-09-10T07:54:03.592809+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T07:54:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 msgr2=0x7f44dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 0x7f44dc06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0011e0 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2024-09-10T07:54:03.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 msgr2=0x7f44fc152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f44f4001350 tx=0x7f44f400a430 comp rx=0 tx=0).stop 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 shutdown_connections 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f44dc06be30 0x7f44dc06e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc15c720 0x7f44fc152730 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f44fc0cfdc0 0x7f44fc1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 --2- 172.21.15.3:0/1059877373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44fc070f30 0x7f44fc148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.942+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 >> 172.21.15.3:0/1059877373 conn(0x7f44fc06cfc0 msgr2=0x7f44fc15b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.943+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 shutdown_connections 2024-09-10T07:54:03.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:03.943+0000 7f4501400640 1 -- 172.21.15.3:0/1059877373 wait complete. 2024-09-10T07:54:03.958 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-10T07:54:03.958 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T07:54:03.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:54:04.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T07:54:04.147 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T07:54:04.148 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:54:04.148 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:54:04.148 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:04.183 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:54:04.184 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:04.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T07:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.455+0000 7f4b4de00640 1 Processor -- start 2024-09-10T07:54:04.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.456+0000 7f4b4de00640 1 -- start start 2024-09-10T07:54:04.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b4de00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4b4815e970 0x7f4b480b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b4de00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4b48162cd0 0x7f4b480d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b4de00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b480b70b0 con 0x7f4b480d1a40 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b4de00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4b480d1e90 con 0x7f4b48162cd0 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b4de00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4b480b78b0 con 0x7f4b4815e970 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b3e000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4b48162cd0 0x7f4b480d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b47400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4b480d1a40 0x7f4b480d2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42948/0 (socket says 172.21.15.3:42948) 2024-09-10T07:54:04.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b47400640 1 -- 172.21.15.3:0/3225227674 learned_addr learned my addr 172.21.15.3:0/3225227674 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b46a00640 1 --2- 172.21.15.3:0/3225227674 >> v2:172.21.15.70:3300/0 conn(0x7f4b4815e970 0x7f4b480b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b3e000640 1 -- 172.21.15.3:0/3225227674 >> v2:172.21.15.70:3300/0 conn(0x7f4b4815e970 msgr2=0x7f4b480b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b3e000640 1 --2- 172.21.15.3:0/3225227674 >> v2:172.21.15.70:3300/0 conn(0x7f4b4815e970 0x7f4b480b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b3e000640 1 -- 172.21.15.3:0/3225227674 >> v1:172.21.15.3:6789/0 conn(0x7f4b480d1a40 legacy=0x7f4b480d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.458+0000 7f4b3e000640 1 -- 172.21.15.3:0/3225227674 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b480b62f0 con 0x7f4b48162cd0 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.457+0000 7f4b46000640 1 -- 172.21.15.3:0/3225227674 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771313327 0 0) 0x7f4b40001490 con 0x7f4b480d1a40 2024-09-10T07:54:04.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.458+0000 7f4b3e000640 1 --2- 172.21.15.3:0/3225227674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48162cd0 0x7f4b480d1500 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f073ea9ad3cb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b46000640 1 -- 172.21.15.3:0/3225227674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30037070 con 0x7f4b48162cd0 2024-09-10T07:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b46000640 1 -- 172.21.15.3:0/3225227674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3002ae50 con 0x7f4b48162cd0 2024-09-10T07:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b46000640 1 -- 172.21.15.3:0/3225227674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30032660 con 0x7f4b48162cd0 2024-09-10T07:54:04.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 -- 172.21.15.3:0/3225227674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48162cd0 msgr2=0x7f4b480d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 --2- 172.21.15.3:0/3225227674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48162cd0 0x7f4b480d1500 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002a140 comp rx=0 tx=0).stop 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 -- 172.21.15.3:0/3225227674 shutdown_connections 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 --2- 172.21.15.3:0/3225227674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48162cd0 0x7f4b480d1500 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 --2- 172.21.15.3:0/3225227674 >> v2:172.21.15.70:3300/0 conn(0x7f4b4815e970 0x7f4b480b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 -- 172.21.15.3:0/3225227674 >> 172.21.15.3:0/3225227674 conn(0x7f4b4806bd00 msgr2=0x7f4b4806c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 -- 172.21.15.3:0/3225227674 shutdown_connections 2024-09-10T07:54:04.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.459+0000 7f4b4de00640 1 -- 172.21.15.3:0/3225227674 wait complete. 2024-09-10T07:54:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.461+0000 7f4b4de00640 1 Processor -- start 2024-09-10T07:54:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.461+0000 7f4b4de00640 1 -- start start 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b480d1a40 0x7f4b4815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b48162cd0 0x7f4b480ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4b480b6b90 con 0x7f4b4815e970 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4b480b68b0 con 0x7f4b480d1a40 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b4de00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4b48163160 con 0x7f4b48162cd0 2024-09-10T07:54:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b3e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b3e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46498/0 (socket says 172.21.15.3:46498) 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b3e000640 1 -- 172.21.15.3:0/1594906688 learned_addr learned my addr 172.21.15.3:0/1594906688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.462+0000 7f4b46a00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b480d1a40 0x7f4b4815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b47400640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b48162cd0 0x7f4b480ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 -- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b48162cd0 msgr2=0x7f4b480ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b48162cd0 0x7f4b480ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 -- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b480d1a40 msgr2=0x7f4b4815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b480d1a40 0x7f4b4815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 -- 172.21.15.3:0/1594906688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4815cc50 con 0x7f4b4815e970 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3e000640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4b300098c0 tx=0x7f4b300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30041020 con 0x7f4b4815e970 2024-09-10T07:54:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3003c670 con 0x7f4b4815e970 2024-09-10T07:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30037040 con 0x7f4b4815e970 2024-09-10T07:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.463+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b480cb8d0 con 0x7f4b4815e970 2024-09-10T07:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.464+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b480cbe90 con 0x7f4b4815e970 2024-09-10T07:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.465+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b480b37a0 con 0x7f4b4815e970 2024-09-10T07:54:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.466+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b30044020 con 0x7f4b4815e970 2024-09-10T07:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.466+0000 7f4b3fe00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 0x7f4b2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.467+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4b300ab570 con 0x7f4b4815e970 2024-09-10T07:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.467+0000 7f4b46a00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 0x7f4b2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.467+0000 7f4b46a00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 0x7f4b2406e320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:04.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.471+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b3004b030 con 0x7f4b4815e970 2024-09-10T07:54:04.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.667+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b48069f70 con 0x7f4b4815e970 2024-09-10T07:54:04.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.667+0000 7f4b3fe00640 1 -- 172.21.15.3:0/1594906688 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f4b300a9e20 con 0x7f4b4815e970 2024-09-10T07:54:04.668 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-10T07:54:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.669+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 msgr2=0x7f4b2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.669+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 0x7f4b2406e320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38008040 comp rx=0 tx=0).stop 2024-09-10T07:54:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.669+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 msgr2=0x7f4b481f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:04.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.669+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4b300098c0 tx=0x7f4b300046c0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 shutdown_connections 2024-09-10T07:54:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4b2406be60 0x7f4b2406e320 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4b48162cd0 0x7f4b480ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b4815e970 0x7f4b481f4370 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 --2- 172.21.15.3:0/1594906688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b480d1a40 0x7f4b4815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 >> 172.21.15.3:0/1594906688 conn(0x7f4b4806bd00 msgr2=0x7f4b480b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 shutdown_connections 2024-09-10T07:54:04.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:04.670+0000 7f4b4de00640 1 -- 172.21.15.3:0/1594906688 wait complete. 2024-09-10T07:54:04.686 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T07:54:04.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:04.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T07:54:04.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:04.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:04.752 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T07:54:04.790 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:54:04.790 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:54:04.790 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:04.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:54:04.830 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:54:04.830 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:54:04.830 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:04.865 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:54:04.865 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T07:54:04.866 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:54:04.866 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:04.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:54:04.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:05.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.065+0000 7f829f200640 1 Processor -- start 2024-09-10T07:54:05.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.065+0000 7f829f200640 1 -- start start 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 0x7f82980d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82980d05f0 0x7f8298072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82980cfdd0 con 0x7f8298160710 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f829815cc90 con 0x7f82980d05f0 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f829815cf70 con 0x7f8298160af0 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f829d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82980d05f0 0x7f8298072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f8297e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8298160710 0x7f82980cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42952/0 (socket says 172.21.15.3:42952) 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f8297e00640 1 -- 172.21.15.3:0/2784679945 learned_addr learned my addr 172.21.15.3:0/2784679945 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.066+0000 7f828f400640 1 --2- 172.21.15.3:0/2784679945 >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 0x7f82980d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f829d200640 1 -- 172.21.15.3:0/2784679945 >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 msgr2=0x7f82980d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f829d200640 1 --2- 172.21.15.3:0/2784679945 >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 0x7f82980d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f829d200640 1 -- 172.21.15.3:0/2784679945 >> v1:172.21.15.3:6789/0 conn(0x7f8298160710 legacy=0x7f82980cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f829d200640 1 -- 172.21.15.3:0/2784679945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829815d250 con 0x7f82980d05f0 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f828f400640 1 --2- 172.21.15.3:0/2784679945 >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 0x7f82980d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:05.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.067+0000 7f8297400640 1 -- 172.21.15.3:0/2784679945 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533452860 0 0) 0x7f82980cfdd0 con 0x7f8298160710 2024-09-10T07:54:05.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.068+0000 7f829d200640 1 --2- 172.21.15.3:0/2784679945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82980d05f0 0x7f8298072070 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f829806df70 tx=0x7f829002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832d77b92f6a9893 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.068+0000 7f8297400640 1 -- 172.21.15.3:0/2784679945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290007740 con 0x7f82980d05f0 2024-09-10T07:54:05.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.068+0000 7f8297400640 1 -- 172.21.15.3:0/2784679945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82900078e0 con 0x7f82980d05f0 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.068+0000 7f8297400640 1 -- 172.21.15.3:0/2784679945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290007cb0 con 0x7f82980d05f0 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 -- 172.21.15.3:0/2784679945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82980d05f0 msgr2=0x7f8298072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 --2- 172.21.15.3:0/2784679945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82980d05f0 0x7f8298072070 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f829806df70 tx=0x7f829002a680 comp rx=0 tx=0).stop 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 -- 172.21.15.3:0/2784679945 shutdown_connections 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 --2- 172.21.15.3:0/2784679945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82980d05f0 0x7f8298072070 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 --2- 172.21.15.3:0/2784679945 >> v2:172.21.15.70:3300/0 conn(0x7f8298160af0 0x7f82980d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 -- 172.21.15.3:0/2784679945 >> 172.21.15.3:0/2784679945 conn(0x7f829806d000 msgr2=0x7f829806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 -- 172.21.15.3:0/2784679945 shutdown_connections 2024-09-10T07:54:05.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.069+0000 7f829f200640 1 -- 172.21.15.3:0/2784679945 wait complete. 2024-09-10T07:54:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.070+0000 7f829f200640 1 Processor -- start 2024-09-10T07:54:05.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 -- start start 2024-09-10T07:54:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82980d05f0 0x7f8298152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8298160710 0x7f8298155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f829815d770 con 0x7f8298160af0 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f829815c960 con 0x7f82980d05f0 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.071+0000 7f829f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8298155e50 con 0x7f8298160710 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46508/0 (socket says 172.21.15.3:46508) 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 -- 172.21.15.3:0/3833164773 learned_addr learned my addr 172.21.15.3:0/3833164773 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f8297e00640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82980d05f0 0x7f8298152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f828f400640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8298160710 0x7f8298155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 -- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8298160710 msgr2=0x7f8298155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8298160710 0x7f8298155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 -- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82980d05f0 msgr2=0x7f8298152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82980d05f0 0x7f8298152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 -- 172.21.15.3:0/3833164773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8298156130 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.072+0000 7f829d200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e80 tx=0x7f8290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.073+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290040070 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.073+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8290031c40 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.073+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290009450 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.073+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8298156410 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.073+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981569d0 con 0x7f8298160af0 2024-09-10T07:54:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.074+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8298152e60 con 0x7f8298160af0 2024-09-10T07:54:05.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.075+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8290039040 con 0x7f8298160af0 2024-09-10T07:54:05.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.076+0000 7f8294c00640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 0x7f827406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.076+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f82900ac100 con 0x7f8298160af0 2024-09-10T07:54:05.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.076+0000 7f8297e00640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 0x7f827406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.077+0000 7f8297e00640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 0x7f827406e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8288000f60 tx=0x7f8288007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.080+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f829007a3c0 con 0x7f8298160af0 2024-09-10T07:54:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.303+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82980cfa60 con 0x7f8298160af0 2024-09-10T07:54:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.303+0000 7f8294c00640 1 -- 172.21.15.3:0/3833164773 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f829007e6c0 con 0x7f8298160af0 2024-09-10T07:54:05.303 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:05.303 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:05.303 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 msgr2=0x7f827406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 0x7f827406e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8288000f60 tx=0x7f8288007d50 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 msgr2=0x7f82981e7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e80 tx=0x7f8290004a40 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 shutdown_connections 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f827406be30 0x7f827406e2f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8298160af0 0x7f82981e7f40 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8298160710 0x7f8298155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 --2- 172.21.15.3:0/3833164773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82980d05f0 0x7f8298152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 >> 172.21.15.3:0/3833164773 conn(0x7f829806d000 msgr2=0x7f82980b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:05.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 shutdown_connections 2024-09-10T07:54:05.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.306+0000 7f829f200640 1 -- 172.21.15.3:0/3833164773 wait complete. 2024-09-10T07:54:05.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:05.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.478+0000 7f4f8d600640 1 Processor -- start 2024-09-10T07:54:05.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.478+0000 7f4f8d600640 1 -- start start 2024-09-10T07:54:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f8d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f8d600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f8815fdb0 0x7f4f8806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f8d600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f8815fbb0 con 0x7f4f881601f0 2024-09-10T07:54:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f8d600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f4f880b36a0 con 0x7f4f8815fdb0 2024-09-10T07:54:05.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f8d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f880b3980 con 0x7f4f88161b30 2024-09-10T07:54:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f86a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f86a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:58080/0 (socket says 172.21.15.3:58080) 2024-09-10T07:54:05.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f86a00640 1 -- 172.21.15.3:0/3383234503 learned_addr learned my addr 172.21.15.3:0/3383234503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.480+0000 7f4f7fe00640 1 --2- 172.21.15.3:0/3383234503 >> v2:172.21.15.70:3300/0 conn(0x7f4f8815fdb0 0x7f4f8806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.481+0000 7f4f86a00640 1 -- 172.21.15.3:0/3383234503 >> v2:172.21.15.70:3300/0 conn(0x7f4f8815fdb0 msgr2=0x7f4f8806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.481+0000 7f4f86a00640 1 --2- 172.21.15.3:0/3383234503 >> v2:172.21.15.70:3300/0 conn(0x7f4f8815fdb0 0x7f4f8806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.481+0000 7f4f86a00640 1 -- 172.21.15.3:0/3383234503 >> v1:172.21.15.70:6789/0 conn(0x7f4f881601f0 legacy=0x7f4f880b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.481+0000 7f4f86a00640 1 -- 172.21.15.3:0/3383234503 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f880b3c60 con 0x7f4f88161b30 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f86a00640 1 --2- 172.21.15.3:0/3383234503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009870 tx=0x7f4f7802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cef0ab54f94aa5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f86000640 1 -- 172.21.15.3:0/3383234503 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78037070 con 0x7f4f88161b30 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f86000640 1 -- 172.21.15.3:0/3383234503 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78032440 con 0x7f4f88161b30 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f86000640 1 -- 172.21.15.3:0/3383234503 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78032810 con 0x7f4f88161b30 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f8d600640 1 -- 172.21.15.3:0/3383234503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 msgr2=0x7f4f8815f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.482+0000 7f4f8d600640 1 --2- 172.21.15.3:0/3383234503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009870 tx=0x7f4f7802a250 comp rx=0 tx=0).stop 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 -- 172.21.15.3:0/3383234503 shutdown_connections 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 --2- 172.21.15.3:0/3383234503 >> v2:172.21.15.70:3300/0 conn(0x7f4f8815fdb0 0x7f4f8806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 --2- 172.21.15.3:0/3383234503 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f88161b30 0x7f4f8815f670 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 -- 172.21.15.3:0/3383234503 >> 172.21.15.3:0/3383234503 conn(0x7f4f8806d000 msgr2=0x7f4f8806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 -- 172.21.15.3:0/3383234503 shutdown_connections 2024-09-10T07:54:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.483+0000 7f4f8d600640 1 -- 172.21.15.3:0/3383234503 wait complete. 2024-09-10T07:54:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.484+0000 7f4f8d600640 1 Processor -- start 2024-09-10T07:54:05.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.484+0000 7f4f8d600640 1 -- start start 2024-09-10T07:54:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f8815fdb0 0x7f4f88072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f881601f0 0x7f4f88073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88161f10 con 0x7f4f88161b30 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f880b4220 con 0x7f4f8815fdb0 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.485+0000 7f4f8d600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f4f880aefa0 con 0x7f4f881601f0 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46510/0 (socket says 172.21.15.3:46510) 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 -- 172.21.15.3:0/2982875342 learned_addr learned my addr 172.21.15.3:0/2982875342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f7fe00640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f881601f0 0x7f4f88073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f86a00640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f8815fdb0 0x7f4f88072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 -- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f881601f0 msgr2=0x7f4f88073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f881601f0 0x7f4f88073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 -- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f8815fdb0 msgr2=0x7f4f88072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f8815fdb0 0x7f4f88072ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.486+0000 7f4f87400640 1 -- 172.21.15.3:0/2982875342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8815c340 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f87400640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c7a0 tx=0x7f4f8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80003b70 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f80003d30 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000aa90 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f88073410 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.487+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8815bd80 con 0x7f4f88161b30 2024-09-10T07:54:05.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.488+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f880b3300 con 0x7f4f88161b30 2024-09-10T07:54:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.489+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f8000ac30 con 0x7f4f88161b30 2024-09-10T07:54:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.490+0000 7f4f7ea00640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 0x7f4f6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.490+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4f8008abe0 con 0x7f4f88161b30 2024-09-10T07:54:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.490+0000 7f4f86a00640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 0x7f4f6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.491+0000 7f4f86a00640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 0x7f4f6406e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4f78034000 tx=0x7f4f7802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.494+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f8815fbb0 con 0x7f4f88161b30 2024-09-10T07:54:05.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.717+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f88161730 con 0x7f4f88161b30 2024-09-10T07:54:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.717+0000 7f4f7ea00640 1 -- 172.21.15.3:0/2982875342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f4f80058fd0 con 0x7f4f88161b30 2024-09-10T07:54:05.717 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:05.717 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:05.717 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.719+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 msgr2=0x7f4f6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.719+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 0x7f4f6406e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4f78034000 tx=0x7f4f7802e000 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.719+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 msgr2=0x7f4f880ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.719+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c7a0 tx=0x7f4f8000cc60 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 shutdown_connections 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f4f6406be30 0x7f4f6406e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f88161b30 0x7f4f880ce080 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4f881601f0 0x7f4f88073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 --2- 172.21.15.3:0/2982875342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f8815fdb0 0x7f4f88072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 >> 172.21.15.3:0/2982875342 conn(0x7f4f8806d000 msgr2=0x7f4f88160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 shutdown_connections 2024-09-10T07:54:05.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.720+0000 7f4f8d600640 1 -- 172.21.15.3:0/2982875342 wait complete. 2024-09-10T07:54:05.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:54:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.893+0000 7f47d2200640 1 Processor -- start 2024-09-10T07:54:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.893+0000 7f47d2200640 1 -- start start 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47cc15e860 0x7f47cc0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 0x7f47cc0d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0d3780 con 0x7f47cc15e860 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0cee40 con 0x7f47cc0cfbf0 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d2200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc0cf120 con 0x7f47cc0d4150 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d0e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47cc15e860 0x7f47cc0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d0e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47cc15e860 0x7f47cc0cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46520/0 (socket says 172.21.15.3:46520) 2024-09-10T07:54:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.894+0000 7f47d0e00640 1 -- 172.21.15.3:0/662561795 learned_addr learned my addr 172.21.15.3:0/662561795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:05.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d1800640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47c3e00640 1 --2- 172.21.15.3:0/662561795 >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 0x7f47cc0d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d0e00640 1 -- 172.21.15.3:0/662561795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 msgr2=0x7f47cc0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d0e00640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc0b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d0e00640 1 -- 172.21.15.3:0/662561795 >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 msgr2=0x7f47cc0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d0e00640 1 --2- 172.21.15.3:0/662561795 >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 0x7f47cc0d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47d0e00640 1 -- 172.21.15.3:0/662561795 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0d3410 con 0x7f47cc15e860 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.895+0000 7f47c3e00640 1 --2- 172.21.15.3:0/662561795 >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 0x7f47cc0d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47d0e00640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cf6b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0d0ad0 tx=0x7f47bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8fb288d5e76c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47cbe00640 1 -- 172.21.15.3:0/662561795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc037070 con 0x7f47cc15e860 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47cbe00640 1 -- 172.21.15.3:0/662561795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc032440 con 0x7f47cc15e860 2024-09-10T07:54:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47cbe00640 1 -- 172.21.15.3:0/662561795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc032810 con 0x7f47cc15e860 2024-09-10T07:54:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47d2200640 1 -- 172.21.15.3:0/662561795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 msgr2=0x7f47cc0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.896+0000 7f47d2200640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cf6b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0d0ad0 tx=0x7f47bc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 -- 172.21.15.3:0/662561795 shutdown_connections 2024-09-10T07:54:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc0b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 --2- 172.21.15.3:0/662561795 >> v2:172.21.15.70:3300/0 conn(0x7f47cc0cfbf0 0x7f47cc0d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 --2- 172.21.15.3:0/662561795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cf6b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 -- 172.21.15.3:0/662561795 >> 172.21.15.3:0/662561795 conn(0x7f47cc0d4650 msgr2=0x7f47cc0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 -- 172.21.15.3:0/662561795 shutdown_connections 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.897+0000 7f47d2200640 1 -- 172.21.15.3:0/662561795 wait complete. 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.898+0000 7f47d2200640 1 Processor -- start 2024-09-10T07:54:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.898+0000 7f47d2200640 1 -- start start 2024-09-10T07:54:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47cc0cfbf0 0x7f47cc1fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc1fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0b7dc0 con 0x7f47cc15e860 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0d2460 con 0x7f47cc0cfbf0 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.899+0000 7f47d2200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc0b7f40 con 0x7f47cc0d4150 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46534/0 (socket says 172.21.15.3:46534) 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 -- 172.21.15.3:0/1733165227 learned_addr learned my addr 172.21.15.3:0/1733165227 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d0e00640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47cc0cfbf0 0x7f47cc1fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47c3e00640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc1fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 -- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 msgr2=0x7f47cc1fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc1fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 -- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47cc0cfbf0 msgr2=0x7f47cc1fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47cc0cfbf0 0x7f47cc1fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 -- 172.21.15.3:0/1733165227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0ccdd0 con 0x7f47cc15e860 2024-09-10T07:54:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47c3e00640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc1fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47d1800640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007e30 tx=0x7f47c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.900+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4014620 con 0x7f47cc15e860 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.901+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c4019070 con 0x7f47cc15e860 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.901+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc1f9b40 con 0x7f47cc15e860 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.901+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4014a00 con 0x7f47cc15e860 2024-09-10T07:54:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.901+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc070860 con 0x7f47cc15e860 2024-09-10T07:54:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.902+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc0b7880 con 0x7f47cc15e860 2024-09-10T07:54:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.903+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47c4014ba0 con 0x7f47cc15e860 2024-09-10T07:54:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.903+0000 7f47c9600640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 0x7f47a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.903+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f47c408b6c0 con 0x7f47cc15e860 2024-09-10T07:54:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.904+0000 7f47d0e00640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 0x7f47a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.904+0000 7f47d0e00640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 0x7f47a806e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f47bc034000 tx=0x7f47bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:05.908+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f47c405a060 con 0x7f47cc15e860 2024-09-10T07:54:06.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.091+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47cc0b84a0 con 0x7f47cc15e860 2024-09-10T07:54:06.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.093+0000 7f47c9600640 1 -- 172.21.15.3:0/1733165227 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16607 (secure 0 0 0) 0x7f47c4059960 con 0x7f47cc15e860 2024-09-10T07:54:06.092 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:06.093 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":102,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:12.227124+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:53:08.843582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T07:53:09.632431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:54:06.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 msgr2=0x7f47a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 0x7f47a806e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f47bc034000 tx=0x7f47bc02e000 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 msgr2=0x7f47cc0cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007e30 tx=0x7f47c400e4b0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 shutdown_connections 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f47a806be30 0x7f47a806e2f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47cc15e860 0x7f47cc0cc0e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47cc0d4150 0x7f47cc1fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 --2- 172.21.15.3:0/1733165227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47cc0cfbf0 0x7f47cc1fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 >> 172.21.15.3:0/1733165227 conn(0x7f47cc0d4650 msgr2=0x7f47cc0b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 shutdown_connections 2024-09-10T07:54:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.095+0000 7f47d2200640 1 -- 172.21.15.3:0/1733165227 wait complete. 2024-09-10T07:54:06.111 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:54:06.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.268+0000 7f82b8c00640 1 Processor -- start 2024-09-10T07:54:06.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.268+0000 7f82b8c00640 1 -- start start 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.269+0000 7f82b8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.269+0000 7f82b8c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82b40d07c0 0x7f82b4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.269+0000 7f82b8c00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b40cfdd0 con 0x7f82b4160af0 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b8c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f82b415cc90 con 0x7f82b40d07c0 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f82b415cf70 con 0x7f82b4160710 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82abe00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f82b4160af0 0x7f82b40d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42962/0 (socket says 172.21.15.3:42962) 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82abe00640 1 -- 172.21.15.3:0/1520705845 learned_addr learned my addr 172.21.15.3:0/1520705845 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1e00640 1 --2- 172.21.15.3:0/1520705845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b2800640 1 --2- 172.21.15.3:0/1520705845 >> v2:172.21.15.70:3300/0 conn(0x7f82b40d07c0 0x7f82b4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1400640 1 -- 172.21.15.3:0/1520705845 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022440251 0 0) 0x7f82b40cfdd0 con 0x7f82b4160af0 2024-09-10T07:54:06.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1400640 1 -- 172.21.15.3:0/1520705845 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8298003520 con 0x7f82b4160af0 2024-09-10T07:54:06.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1e00640 1 -- 172.21.15.3:0/1520705845 >> v2:172.21.15.70:3300/0 conn(0x7f82b40d07c0 msgr2=0x7f82b4072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:06.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1e00640 1 --2- 172.21.15.3:0/1520705845 >> v2:172.21.15.70:3300/0 conn(0x7f82b40d07c0 0x7f82b4072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1e00640 1 -- 172.21.15.3:0/1520705845 >> v1:172.21.15.3:6789/0 conn(0x7f82b4160af0 legacy=0x7f82b40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.270+0000 7f82b1e00640 1 -- 172.21.15.3:0/1520705845 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b415d250 con 0x7f82b4160710 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.271+0000 7f82b1e00640 1 --2- 172.21.15.3:0/1520705845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b40cf2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097a0 tx=0x7f82a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ec7c4b07fa0788 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b1400640 1 -- 172.21.15.3:0/1520705845 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4037070 con 0x7f82b4160710 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b1400640 1 -- 172.21.15.3:0/1520705845 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a4032440 con 0x7f82b4160710 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b1400640 1 -- 172.21.15.3:0/1520705845 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4032810 con 0x7f82b4160710 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 -- 172.21.15.3:0/1520705845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 msgr2=0x7f82b40cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 --2- 172.21.15.3:0/1520705845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b40cf2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097a0 tx=0x7f82a402a180 comp rx=0 tx=0).stop 2024-09-10T07:54:06.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 -- 172.21.15.3:0/1520705845 shutdown_connections 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 --2- 172.21.15.3:0/1520705845 >> v2:172.21.15.70:3300/0 conn(0x7f82b40d07c0 0x7f82b4072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 --2- 172.21.15.3:0/1520705845 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b40cf2c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 -- 172.21.15.3:0/1520705845 >> 172.21.15.3:0/1520705845 conn(0x7f82b406d000 msgr2=0x7f82b406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 -- 172.21.15.3:0/1520705845 shutdown_connections 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.272+0000 7f82b8c00640 1 -- 172.21.15.3:0/1520705845 wait complete. 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.274+0000 7f82b8c00640 1 Processor -- start 2024-09-10T07:54:06.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.274+0000 7f82b8c00640 1 -- start start 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82b40d07c0 0x7f82b4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b41e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82b415d770 con 0x7f82b4160af0 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f82b40cf800 con 0x7f82b40d07c0 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f82b40b3b10 con 0x7f82b4160710 2024-09-10T07:54:06.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46536/0 (socket says 172.21.15.3:46536) 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b2800640 1 -- 172.21.15.3:0/3620352508 learned_addr learned my addr 172.21.15.3:0/3620352508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82abe00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b41e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.275+0000 7f82b1e00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82b40d07c0 0x7f82b4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 -- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 msgr2=0x7f82b41e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b41e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 -- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82b40d07c0 msgr2=0x7f82b4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82b40d07c0 0x7f82b4152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 -- 172.21.15.3:0/3620352508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b40b3df0 con 0x7f82b4160af0 2024-09-10T07:54:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b2800640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00efc0 tx=0x7f82ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac018020 con 0x7f82b4160af0 2024-09-10T07:54:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ac014480 con 0x7f82b4160af0 2024-09-10T07:54:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac014870 con 0x7f82b4160af0 2024-09-10T07:54:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.276+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b40b40d0 con 0x7f82b4160af0 2024-09-10T07:54:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.277+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41e62b0 con 0x7f82b4160af0 2024-09-10T07:54:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.278+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b40b0ed0 con 0x7f82b4160af0 2024-09-10T07:54:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.278+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ac005d10 con 0x7f82b4160af0 2024-09-10T07:54:06.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.279+0000 7f82aa000640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.279+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f82ac08b8c0 con 0x7f82b4160af0 2024-09-10T07:54:06.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.280+0000 7f82b1e00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.280+0000 7f82b1e00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 0x7f829006e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82a4034000 tx=0x7f82a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.284+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82ac022070 con 0x7f82b4160af0 2024-09-10T07:54:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.503+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82b40cd2a0 con 0x7f82b4160af0 2024-09-10T07:54:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.504+0000 7f82aa000640 1 -- 172.21.15.3:0/3620352508 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f82ac010040 con 0x7f82b4160af0 2024-09-10T07:54:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:06.504 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 msgr2=0x7f829006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 0x7f829006e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82a4034000 tx=0x7f82a402e000 comp rx=0 tx=0).stop 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 msgr2=0x7f82b40ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00efc0 tx=0x7f82ac00c490 comp rx=0 tx=0).stop 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 shutdown_connections 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f829006be60 0x7f829006e320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4160af0 0x7f82b40ae8d0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f82b4160710 0x7f82b41e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 --2- 172.21.15.3:0/3620352508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82b40d07c0 0x7f82b4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 >> 172.21.15.3:0/3620352508 conn(0x7f82b406d000 msgr2=0x7f82b40b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:06.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 shutdown_connections 2024-09-10T07:54:06.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.506+0000 7f82b8c00640 1 -- 172.21.15.3:0/3620352508 wait complete. 2024-09-10T07:54:06.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.680+0000 7f54bb200640 1 Processor -- start 2024-09-10T07:54:06.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.680+0000 7f54bb200640 1 -- start start 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54bb200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54bc15ca80 0x7f54bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54bb200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54bc0ce730 con 0x7f54bc071cd0 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54bb200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54bc073e60 con 0x7f54bc145010 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54bb200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f54bc0ced10 con 0x7f54bc15ca80 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54ba800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f54bc071cd0 0x7f54bc0cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42972/0 (socket says 172.21.15.3:42972) 2024-09-10T07:54:06.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.681+0000 7f54ba800640 1 -- 172.21.15.3:0/4141503779 learned_addr learned my addr 172.21.15.3:0/4141503779 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b3e00640 1 --2- 172.21.15.3:0/4141503779 >> v2:172.21.15.70:3300/0 conn(0x7f54bc15ca80 0x7f54bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407451181 0 0) 0x7f54bc0ce730 con 0x7f54bc071cd0 2024-09-10T07:54:06.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a0003520 con 0x7f54bc071cd0 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3497679534 0 0) 0x7f54a0003520 con 0x7f54bc071cd0 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 >> v2:172.21.15.70:3300/0 conn(0x7f54bc15ca80 msgr2=0x7f54bc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 --2- 172.21.15.3:0/4141503779 >> v2:172.21.15.70:3300/0 conn(0x7f54bc15ca80 0x7f54bc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 >> v1:172.21.15.70:6789/0 conn(0x7f54bc145010 legacy=0x7f54bc073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc0ce9a0 con 0x7f54bc071cd0 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f54b40019f0 con 0x7f54bc071cd0 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 >> v1:172.21.15.3:6789/0 conn(0x7f54bc071cd0 legacy=0x7f54bc0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a0003a20 0x7f54a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54a00051c0 0x7f54a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc0ced10 con 0x7f54a0004800 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f54a000c0f0 con 0x7f54a0003a20 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.682+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54a000c2f0 con 0x7f54a00051c0 2024-09-10T07:54:06.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b9e00640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a0003a20 0x7f54a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54ba800640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54a00051c0 0x7f54a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 -- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54a00051c0 msgr2=0x7f54a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54a00051c0 0x7f54a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 -- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a0003a20 msgr2=0x7f54a0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a0003a20 0x7f54a0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54b3e00640 1 -- 172.21.15.3:0/4141503779 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a0003520 con 0x7f54a0004800 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.683+0000 7f54ba800640 1 -- 172.21.15.3:0/4141503779 reap_dead start 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.684+0000 7f54b3e00640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54a0004c80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f54bc06fa90 tx=0x7f54a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3dfe664f1a59b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.684+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4036070 con 0x7f54a0004800 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.684+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a4031400 con 0x7f54a0004800 2024-09-10T07:54:06.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.684+0000 7f54b9400640 1 -- 172.21.15.3:0/4141503779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4009690 con 0x7f54a0004800 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 -- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 msgr2=0x7f54a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54a0004c80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f54bc06fa90 tx=0x7f54a0005830 comp rx=0 tx=0).stop 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 -- 172.21.15.3:0/4141503779 shutdown_connections 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 --2- 172.21.15.3:0/4141503779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54a0004c80 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 -- 172.21.15.3:0/4141503779 >> 172.21.15.3:0/4141503779 conn(0x7f54bc06bb20 msgr2=0x7f54bc06bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 -- 172.21.15.3:0/4141503779 shutdown_connections 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.685+0000 7f54bb200640 1 -- 172.21.15.3:0/4141503779 wait complete. 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.686+0000 7f54bb200640 1 Processor -- start 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.687+0000 7f54bb200640 1 -- start start 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.687+0000 7f54bb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.687+0000 7f54bb200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 0x7f54bc143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.687+0000 7f54bb200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 0x7f54bc0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.687+0000 7f54bb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc145570 con 0x7f54a0004800 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54bb200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1453f0 con 0x7f54bc160b50 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54bb200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f54bc073410 con 0x7f54bc1606d0 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46550/0 (socket says 172.21.15.3:46550) 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 -- 172.21.15.3:0/3160209655 learned_addr learned my addr 172.21.15.3:0/3160209655 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b3e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 0x7f54bc143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54ba800640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 0x7f54bc0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 -- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 msgr2=0x7f54bc143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 0x7f54bc143800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 -- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 msgr2=0x7f54bc0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 0x7f54bc0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 -- 172.21.15.3:0/3160209655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc141520 con 0x7f54a0004800 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b3e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 0x7f54bc143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54ba800640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 0x7f54bc0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.688+0000 7f54b9e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00f820 tx=0x7f54ac00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.689+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0050d0 con 0x7f54a0004800 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.689+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ac018070 con 0x7f54a0004800 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.689+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005490 con 0x7f54a0004800 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.689+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc141860 con 0x7f54a0004800 2024-09-10T07:54:06.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.689+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc1420a0 con 0x7f54a0004800 2024-09-10T07:54:06.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.690+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54bc0737a0 con 0x7f54a0004800 2024-09-10T07:54:06.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.691+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54ac00d730 con 0x7f54a0004800 2024-09-10T07:54:06.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.692+0000 7f54b2000640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 0x7f549806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:06.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.692+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f54ac08acf0 con 0x7f54a0004800 2024-09-10T07:54:06.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.692+0000 7f54b3e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 0x7f549806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:06.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.693+0000 7f54b3e00640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 0x7f549806e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f54a4033000 tx=0x7f54a4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:06.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.697+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54bc141860 con 0x7f54a0004800 2024-09-10T07:54:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.919+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54bc0b1690 con 0x7f54a0004800 2024-09-10T07:54:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.920+0000 7f54b2000640 1 -- 172.21.15.3:0/3160209655 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5271 (secure 0 0 0) 0x7f54ac059690 con 0x7f54a0004800 2024-09-10T07:54:06.920 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:06.920 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":114,"btime":"2024-09-10T07:53:49:701494+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T07:53:10.636980+0000","modified":"2024-09-10T07:53:49.685486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7543,"mds_1":6875,"mds_2":6872},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6872":{"gid":6872,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":12,"addr":"172.21.15.70:6835/3268462742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3268462742},{"type":"v1","addr":"172.21.15.70:6835","nonce":3268462742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6875":{"gid":6875,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6836/3004970849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3004970849},{"type":"v1","addr":"172.21.15.3:6836","nonce":3004970849}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7543":{"gid":7543,"name":"c","rank":0,"incarnation":100,"state":"up:active","state_seq":12,"addr":"172.21.15.3:6837/3773991953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3773991953},{"type":"v1","addr":"172.21.15.3:6837","nonce":3773991953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7543,"qdb_cluster":[6872,6875,7543]},"id":7}]} 2024-09-10T07:54:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 114 2024-09-10T07:54:06.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 msgr2=0x7f549806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 0x7f549806e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f54a4033000 tx=0x7f54a4008030 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 msgr2=0x7f54bc1425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00f820 tx=0x7f54ac00fce0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 shutdown_connections 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f549806be30 0x7f549806e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc160b50 0x7f54bc0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.922+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54bc1606d0 0x7f54bc143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.923+0000 7f54bb200640 1 --2- 172.21.15.3:0/3160209655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54a0004800 0x7f54bc1425a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.923+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 >> 172.21.15.3:0/3160209655 conn(0x7f54bc06bb20 msgr2=0x7f54bc0b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.923+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 shutdown_connections 2024-09-10T07:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:06.923+0000 7f54bb200640 1 -- 172.21.15.3:0/3160209655 wait complete. 2024-09-10T07:54:06.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:54:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.094+0000 7f8b4cc00640 1 Processor -- start 2024-09-10T07:54:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.094+0000 7f8b4cc00640 1 -- start start 2024-09-10T07:54:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.095+0000 7f8b4cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.095+0000 7f8b4cc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b4815b4f0 0x7f8b480d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.095+0000 7f8b4cc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b480b4250 con 0x7f8b48073d10 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.095+0000 7f8b4cc00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8b480b44f0 con 0x7f8b4815b4f0 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.095+0000 7f8b4cc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b480ccd80 con 0x7f8b4815b110 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b46800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8b48073d10 0x7f8b4815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42988/0 (socket says 172.21.15.3:42988) 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b46800640 1 -- 172.21.15.3:0/324455653 learned_addr learned my addr 172.21.15.3:0/324455653 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45e00640 1 --2- 172.21.15.3:0/324455653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b3fe00640 1 --2- 172.21.15.3:0/324455653 >> v2:172.21.15.70:3300/0 conn(0x7f8b4815b4f0 0x7f8b480d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45400640 1 -- 172.21.15.3:0/324455653 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576627330 0 0) 0x7f8b480b4250 con 0x7f8b48073d10 2024-09-10T07:54:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45400640 1 -- 172.21.15.3:0/324455653 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b2c005730 con 0x7f8b48073d10 2024-09-10T07:54:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45e00640 1 -- 172.21.15.3:0/324455653 >> v2:172.21.15.70:3300/0 conn(0x7f8b4815b4f0 msgr2=0x7f8b480d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45e00640 1 --2- 172.21.15.3:0/324455653 >> v2:172.21.15.70:3300/0 conn(0x7f8b4815b4f0 0x7f8b480d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45e00640 1 -- 172.21.15.3:0/324455653 >> v1:172.21.15.3:6789/0 conn(0x7f8b48073d10 legacy=0x7f8b4815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.096+0000 7f8b45e00640 1 -- 172.21.15.3:0/324455653 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b480cd060 con 0x7f8b4815b110 2024-09-10T07:54:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.097+0000 7f8b45e00640 1 --2- 172.21.15.3:0/324455653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480b3d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097a0 tx=0x7f8b3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25f5f755d6d7c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b45400640 1 -- 172.21.15.3:0/324455653 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38037070 con 0x7f8b4815b110 2024-09-10T07:54:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b45400640 1 -- 172.21.15.3:0/324455653 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38032440 con 0x7f8b4815b110 2024-09-10T07:54:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b45400640 1 -- 172.21.15.3:0/324455653 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38032810 con 0x7f8b4815b110 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 -- 172.21.15.3:0/324455653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 msgr2=0x7f8b480b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/324455653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480b3d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097a0 tx=0x7f8b3802a180 comp rx=0 tx=0).stop 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 -- 172.21.15.3:0/324455653 shutdown_connections 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/324455653 >> v2:172.21.15.70:3300/0 conn(0x7f8b4815b4f0 0x7f8b480d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/324455653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480b3d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 -- 172.21.15.3:0/324455653 >> 172.21.15.3:0/324455653 conn(0x7f8b4806d000 msgr2=0x7f8b4806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 -- 172.21.15.3:0/324455653 shutdown_connections 2024-09-10T07:54:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.098+0000 7f8b4cc00640 1 -- 172.21.15.3:0/324455653 wait complete. 2024-09-10T07:54:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.100+0000 7f8b4cc00640 1 Processor -- start 2024-09-10T07:54:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.100+0000 7f8b4cc00640 1 -- start start 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4815b4f0 0x7f8b4815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8b480ceb50 con 0x7f8b48073d10 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8b480cd620 con 0x7f8b4815b4f0 2024-09-10T07:54:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b4cc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b480cecd0 con 0x7f8b4815b110 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b45e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b45e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46558/0 (socket says 172.21.15.3:46558) 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.101+0000 7f8b45e00640 1 -- 172.21.15.3:0/859413758 learned_addr learned my addr 172.21.15.3:0/859413758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 -- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 msgr2=0x7f8b480ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 -- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4815b4f0 msgr2=0x7f8b4815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4815b4f0 0x7f8b4815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 -- 172.21.15.3:0/859413758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b480af210 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b45e00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8b38035040 tx=0x7f8b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38041030 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3803d4e0 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b480af4f0 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38037040 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.102+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4815bab0 con 0x7f8b48073d10 2024-09-10T07:54:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.104+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b480b3d10 con 0x7f8b48073d10 2024-09-10T07:54:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.104+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b38044070 con 0x7f8b48073d10 2024-09-10T07:54:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.105+0000 7f8b3e000640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 0x7f8b2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.105+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8b380ab500 con 0x7f8b48073d10 2024-09-10T07:54:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.106+0000 7f8b3fe00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 0x7f8b2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.106+0000 7f8b3fe00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 0x7f8b2406e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006a90 tx=0x7f8b30006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.110+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b38079ec0 con 0x7f8b48073d10 2024-09-10T07:54:07.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.298+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b480afee0 con 0x7f8b48073d10 2024-09-10T07:54:07.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.300+0000 7f8b3e000640 1 -- 172.21.15.3:0/859413758 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16607 (secure 0 0 0) 0x7f8b380797c0 con 0x7f8b48073d10 2024-09-10T07:54:07.300 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:07.300 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":102,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:53:12.227124+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T07:53:08.843582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T07:53:09.632431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 msgr2=0x7f8b2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 0x7f8b2406e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006a90 tx=0x7f8b30006a20 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 msgr2=0x7f8b4814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8b38035040 tx=0x7f8b380046c0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 shutdown_connections 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8b2406be30 0x7f8b2406e2f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b4815b4f0 0x7f8b4815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b4815b110 0x7f8b480ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 --2- 172.21.15.3:0/859413758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b48073d10 0x7f8b4814bcd0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 >> 172.21.15.3:0/859413758 conn(0x7f8b4806d000 msgr2=0x7f8b4806d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 shutdown_connections 2024-09-10T07:54:07.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.302+0000 7f8b4cc00640 1 -- 172.21.15.3:0/859413758 wait complete. 2024-09-10T07:54:07.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T07:54:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.476+0000 7f6ec3200640 1 Processor -- start 2024-09-10T07:54:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.476+0000 7f6ec3200640 1 -- start start 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec415e860 0x7f6ec40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ec40cfbf0 0x7f6ec40d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ec40d4150 0x7f6ec40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec40d3780 con 0x7f6ec40d4150 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec40cee40 con 0x7f6ec40cfbf0 2024-09-10T07:54:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.477+0000 7f6ec3200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec40cf120 con 0x7f6ec415e860 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ec40d4150 0x7f6ec40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ec40d4150 0x7f6ec40b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46560/0 (socket says 172.21.15.3:46560) 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 -- 172.21.15.3:0/706216041 learned_addr learned my addr 172.21.15.3:0/706216041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec1e00640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec415e860 0x7f6ec40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ebbe00640 1 --2- 172.21.15.3:0/706216041 >> v2:172.21.15.70:3300/0 conn(0x7f6ec40cfbf0 0x7f6ec40d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 -- 172.21.15.3:0/706216041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec415e860 msgr2=0x7f6ec40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec415e860 0x7f6ec40cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 -- 172.21.15.3:0/706216041 >> v2:172.21.15.70:3300/0 conn(0x7f6ec40cfbf0 msgr2=0x7f6ec40d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 --2- 172.21.15.3:0/706216041 >> v2:172.21.15.70:3300/0 conn(0x7f6ec40cfbf0 0x7f6ec40d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.478+0000 7f6ec2800640 1 -- 172.21.15.3:0/706216041 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec40d3410 con 0x7f6ec40d4150 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.479+0000 7f6ec2800640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec40d4150 0x7f6ec40b7880 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4075b70 tx=0x7f6ebc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611eeab0e53ea80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.479+0000 7f6ec1400640 1 -- 172.21.15.3:0/706216041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc036070 con 0x7f6ec40d4150 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec1400640 1 -- 172.21.15.3:0/706216041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc031600 con 0x7f6ec40d4150 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec1400640 1 -- 172.21.15.3:0/706216041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0319d0 con 0x7f6ec40d4150 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 -- 172.21.15.3:0/706216041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec40d4150 msgr2=0x7f6ec40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec40d4150 0x7f6ec40b7880 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4075b70 tx=0x7f6ebc02a6a0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 -- 172.21.15.3:0/706216041 shutdown_connections 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec40d4150 0x7f6ec40b7880 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 --2- 172.21.15.3:0/706216041 >> v2:172.21.15.70:3300/0 conn(0x7f6ec40cfbf0 0x7f6ec40d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 --2- 172.21.15.3:0/706216041 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec415e860 0x7f6ec40cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 -- 172.21.15.3:0/706216041 >> 172.21.15.3:0/706216041 conn(0x7f6ec40d4650 msgr2=0x7f6ec40d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:07.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 -- 172.21.15.3:0/706216041 shutdown_connections 2024-09-10T07:54:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.480+0000 7f6ec3200640 1 -- 172.21.15.3:0/706216041 wait complete. 2024-09-10T07:54:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.482+0000 7f6ec3200640 1 Processor -- start 2024-09-10T07:54:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.482+0000 7f6ec3200640 1 -- start start 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec40cfbf0 0x7f6ec4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 0x7f6ec415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec40b7dc0 con 0x7f6ec415e860 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec40d2460 con 0x7f6ec40cfbf0 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec3200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec40b7f40 con 0x7f6ec40d4150 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46572/0 (socket says 172.21.15.3:46572) 2024-09-10T07:54:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ec2800640 1 -- 172.21.15.3:0/807961963 learned_addr learned my addr 172.21.15.3:0/807961963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.483+0000 7f6ebbe00640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 0x7f6ec415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec1e00640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec40cfbf0 0x7f6ec4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 -- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 msgr2=0x7f6ec415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 0x7f6ec415c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 -- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec40cfbf0 msgr2=0x7f6ec4157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec40cfbf0 0x7f6ec4157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 -- 172.21.15.3:0/807961963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec415b5f0 con 0x7f6ec415e860 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ebbe00640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 0x7f6ec415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec2800640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc034040 tx=0x7f6ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc041020 con 0x7f6ec415e860 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0077a0 con 0x7f6ec415e860 2024-09-10T07:54:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc036040 con 0x7f6ec415e860 2024-09-10T07:54:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec41579a0 con 0x7f6ec415e860 2024-09-10T07:54:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.484+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec4158520 con 0x7f6ec415e860 2024-09-10T07:54:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.486+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec4158ec0 con 0x7f6ec415e860 2024-09-10T07:54:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.487+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ebc044020 con 0x7f6ec415e860 2024-09-10T07:54:07.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.487+0000 7f6eba000640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 0x7f6ea006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.487+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f6ebc0aba20 con 0x7f6ec415e860 2024-09-10T07:54:07.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.488+0000 7f6ec1e00640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 0x7f6ea006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.488+0000 7f6ec1e00640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 0x7f6ea006e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009a30 tx=0x7f6eb4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.492+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ec4158ec0 con 0x7f6ec415e860 2024-09-10T07:54:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.711+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6ec4072c80 con 0x7f6ec415e860 2024-09-10T07:54:07.710 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-10T07:54:07.711+0000 7fabbd000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T07:54:07.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.713+0000 7f6eba000640 1 -- 172.21.15.3:0/807961963 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7f6ebc04b030 con 0x7f6ec415e860 2024-09-10T07:54:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T07:54:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.716+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 msgr2=0x7f6ea006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.716+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 0x7f6ea006e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009a30 tx=0x7f6eb4008040 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.716+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 msgr2=0x7f6ec4157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.716+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc034040 tx=0x7f6ebc004a40 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 shutdown_connections 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f6ea006be30 0x7f6ea006e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ec415e860 0x7f6ec4157460 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6ec40d4150 0x7f6ec415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 --2- 172.21.15.3:0/807961963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec40cfbf0 0x7f6ec4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 >> 172.21.15.3:0/807961963 conn(0x7f6ec40d4650 msgr2=0x7f6ec4073480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 shutdown_connections 2024-09-10T07:54:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.717+0000 7f6ec3200640 1 -- 172.21.15.3:0/807961963 wait complete. 2024-09-10T07:54:07.727 INFO:tasks.ceph.mds.d.smithi070.stderr: -1289> 2024-09-10T07:53:44.973+0000 7f458f200640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x555e8ab87600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.727 INFO:tasks.ceph.mds.d.smithi070.stderr: -1275> 2024-09-10T07:53:44.973+0000 7f458f200640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/AsLsXsFsx@2},l=7669 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x555e8ab87b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.735 INFO:tasks.ceph.mds.c.smithi003.stderr: -2838> 2024-09-10T07:53:44.973+0000 7f2ccb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1) n(v3 rc2024-09-10T07:53:28.810097+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55c7157c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-10T07:53:28.810097+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.735 INFO:tasks.ceph.mds.c.smithi003.stderr: -2736> 2024-09-10T07:53:44.974+0000 7f2cd3400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1) n(v4 rc2024-09-10T07:53:28.810097+0000 5=0+5) (inest sync->mix g=1,2) caps={7669=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x55c7157c7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.536101+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-10T07:53:28.810097+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.738 INFO:tasks.ceph.mds.a.smithi003.stderr: -1392> 2024-09-10T07:53:44.973+0000 7fea99c00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v2 rc2024-09-10T07:53:28.810097+0000 4=0+4) (inest lock) (ifile excl) caps={7669=pAsLsXsFsx/pAsLsXsFsxcral@4},l=7669 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x562063eb1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.810097+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-10T07:53:28.810097+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.738 INFO:tasks.ceph.mds.a.smithi003.stderr: -1254> 2024-09-10T07:53:44.975+0000 7feaa1e00640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1) n(v1 rc2024-09-10T07:53:28.810097+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7669=pAsLsXsFs/pAsLsXsFsxcral@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x562063eb1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-10T07:53:28.537101+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-10T07:53:28.537101+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-10T07:53:28.810097+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-10T07:54:07.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T07:54:07.774 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-09-10T07:54:07.804 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-10T07:54:07.806 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-10T07:54:07.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.910+0000 7f8e14200640 1 Processor -- start 2024-09-10T07:54:07.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.910+0000 7f8e14200640 1 -- start start 2024-09-10T07:54:07.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e14200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e14200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0c145730 0x7f8e0c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e14200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0c0cd900 con 0x7f8e0c0cf270 2024-09-10T07:54:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e14200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0c06d7c0 con 0x7f8e0c145730 2024-09-10T07:54:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e14200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0cdbe0 con 0x7f8e0c1451d0 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.911+0000 7f8e10e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0c145730 0x7f8e0c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e12200640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8e0c0cf270 0x7f8e0c0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43026/0 (socket says 172.21.15.3:43026) 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e10e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8e0c145730 0x7f8e0c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46612/0 (socket says 172.21.15.3:46612) 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e12200640 1 -- 172.21.15.3:0/3965876278 learned_addr learned my addr 172.21.15.3:0/3965876278 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e11800640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e10e00640 1 -- 172.21.15.3:0/3965876278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 msgr2=0x7f8e0c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e10e00640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e10e00640 1 -- 172.21.15.3:0/3965876278 >> v1:172.21.15.3:6789/0 conn(0x7f8e0c0cf270 legacy=0x7f8e0c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e10e00640 1 -- 172.21.15.3:0/3965876278 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c06da10 con 0x7f8e0c145730 2024-09-10T07:54:07.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.912+0000 7f8e0be00640 1 -- 172.21.15.3:0/3965876278 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013831703 0 0) 0x7f8e0c0cd900 con 0x7f8e0c0cf270 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.913+0000 7f8e10e00640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0b3370 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e059a06aba76cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.913+0000 7f8e0be00640 1 -- 172.21.15.3:0/3965876278 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037070 con 0x7f8e0c145730 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.913+0000 7f8e0be00640 1 -- 172.21.15.3:0/3965876278 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df802ae50 con 0x7f8e0c145730 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.913+0000 7f8e0be00640 1 -- 172.21.15.3:0/3965876278 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8032660 con 0x7f8e0c145730 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 -- 172.21.15.3:0/3965876278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 msgr2=0x7f8e0c0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0b3370 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a100 comp rx=0 tx=0).stop 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 -- 172.21.15.3:0/3965876278 shutdown_connections 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0b3370 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 --2- 172.21.15.3:0/3965876278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 -- 172.21.15.3:0/3965876278 >> 172.21.15.3:0/3965876278 conn(0x7f8e0c0b5cb0 msgr2=0x7f8e0c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 -- 172.21.15.3:0/3965876278 shutdown_connections 2024-09-10T07:54:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.914+0000 7f8e14200640 1 -- 172.21.15.3:0/3965876278 wait complete. 2024-09-10T07:54:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.916+0000 7f8e14200640 1 Processor -- start 2024-09-10T07:54:07.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.916+0000 7f8e14200640 1 -- start start 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.916+0000 7f8e14200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0c0cf270 0x7f8e0c1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e14200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e14200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e14200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c06df30 con 0x7f8e0c145730 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e14200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0b38b0 con 0x7f8e0c0cf270 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e14200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c06e0b0 con 0x7f8e0c1451d0 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46626/0 (socket says 172.21.15.3:46626) 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 -- 172.21.15.3:0/3247361683 learned_addr learned my addr 172.21.15.3:0/3247361683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e10e00640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e11800640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0c0cf270 0x7f8e0c1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 -- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 msgr2=0x7f8e0c14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 -- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0c0cf270 msgr2=0x7f8e0c1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0c0cf270 0x7f8e0c1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.917+0000 7f8e12200640 1 -- 172.21.15.3:0/3247361683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1f4d50 con 0x7f8e0c145730 2024-09-10T07:54:07.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e12200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c7a0 tx=0x7f8e0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04018070 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0400a640 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400aab0 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c150030 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.918+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c13b6a0 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.919+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c06cfc0 con 0x7f8e0c145730 2024-09-10T07:54:07.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.920+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e0400ac50 con 0x7f8e0c145730 2024-09-10T07:54:07.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.921+0000 7f8e09600640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:07.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.921+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5967+0+0 (secure 0 0 0) 0x7f8e0408a5e0 con 0x7f8e0c145730 2024-09-10T07:54:07.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.921+0000 7f8e11800640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:07.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.922+0000 7f8e11800640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 0x7f8dec06e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c15c3f0 tx=0x7f8e00009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:07.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:07.925+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e040587a0 con 0x7f8e0c145730 2024-09-10T07:54:08.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.143+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8e0c1504f0 con 0x7f8e0c145730 2024-09-10T07:54:08.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.778+0000 7f8e09600640 1 -- 172.21.15.3:0/3247361683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) ==== 104+0+0 (secure 0 0 0) 0x7f8e0405caa0 con 0x7f8e0c145730 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 msgr2=0x7f8dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 0x7f8dec06e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c15c3f0 tx=0x7f8e00009190 comp rx=0 tx=0).stop 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 msgr2=0x7f8e0c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c7a0 tx=0x7f8e0400cc60 comp rx=0 tx=0).stop 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 shutdown_connections 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e0c145730 0x7f8e0c0ae8a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8e0c1451d0 0x7f8e0c14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 --2- 172.21.15.3:0/3247361683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0c0cf270 0x7f8e0c1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 >> 172.21.15.3:0/3247361683 conn(0x7f8e0c0b5cb0 msgr2=0x7f8e0c06e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:08.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 shutdown_connections 2024-09-10T07:54:08.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.782+0000 7f8e14200640 1 -- 172.21.15.3:0/3247361683 wait complete. 2024-09-10T07:54:08.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T07:54:08.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.968+0000 7f8718e00640 1 Processor -- start 2024-09-10T07:54:08.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.968+0000 7f8718e00640 1 -- start start 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8718e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f871415e460 0x7f87140d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8718e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87140cf4d0 0x7f87140d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8718e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140b7730 con 0x7f87140d1450 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8718e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f87140d3760 con 0x7f87140cf4d0 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8718e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f87140d3a00 con 0x7f871415e460 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8712a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87140cf4d0 0x7f87140d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8713e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f87140d1450 0x7f871406bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43032/0 (socket says 172.21.15.3:43032) 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.969+0000 7f8713e00640 1 -- 172.21.15.3:0/2110197931 learned_addr learned my addr 172.21.15.3:0/2110197931 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8713400640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f871415e460 0x7f87140d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8712a00640 1 -- 172.21.15.3:0/2110197931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f871415e460 msgr2=0x7f87140d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8712a00640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f871415e460 0x7f87140d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8712a00640 1 -- 172.21.15.3:0/2110197931 >> v1:172.21.15.3:6789/0 conn(0x7f87140d1450 legacy=0x7f871406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8712000640 1 -- 172.21.15.3:0/2110197931 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782935583 0 0) 0x7f87140b7730 con 0x7f87140d1450 2024-09-10T07:54:08.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.970+0000 7f8712a00640 1 -- 172.21.15.3:0/2110197931 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87140d3ce0 con 0x7f87140cf4d0 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.971+0000 7f8712a00640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87140cf4d0 0x7f87140d0f10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f87000097a0 tx=0x7f870002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdd35a480677cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.971+0000 7f8712000640 1 -- 172.21.15.3:0/2110197931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700037070 con 0x7f87140cf4d0 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.971+0000 7f8712000640 1 -- 172.21.15.3:0/2110197931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8700032440 con 0x7f87140cf4d0 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.971+0000 7f8712000640 1 -- 172.21.15.3:0/2110197931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700032810 con 0x7f87140cf4d0 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 -- 172.21.15.3:0/2110197931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87140cf4d0 msgr2=0x7f87140d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87140cf4d0 0x7f87140d0f10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f87000097a0 tx=0x7f870002a210 comp rx=0 tx=0).stop 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 -- 172.21.15.3:0/2110197931 shutdown_connections 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87140cf4d0 0x7f87140d0f10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 --2- 172.21.15.3:0/2110197931 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f871415e460 0x7f87140d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 -- 172.21.15.3:0/2110197931 >> 172.21.15.3:0/2110197931 conn(0x7f87140b6050 msgr2=0x7f87140b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:08.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 -- 172.21.15.3:0/2110197931 shutdown_connections 2024-09-10T07:54:08.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.972+0000 7f8718e00640 1 -- 172.21.15.3:0/2110197931 wait complete. 2024-09-10T07:54:08.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.974+0000 7f8718e00640 1 Processor -- start 2024-09-10T07:54:08.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.974+0000 7f8718e00640 1 -- start start 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87140cf4d0 0x7f871415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87140d1450 0x7f8714156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f871415e840 con 0x7f871415e460 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f87140d42a0 con 0x7f87140d1450 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8718e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f871415e9c0 con 0x7f87140cf4d0 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46642/0 (socket says 172.21.15.3:46642) 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 -- 172.21.15.3:0/4278848875 learned_addr learned my addr 172.21.15.3:0/4278848875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8712a00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87140d1450 0x7f8714156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713400640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87140cf4d0 0x7f871415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 -- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87140cf4d0 msgr2=0x7f871415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87140cf4d0 0x7f871415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 -- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87140d1450 msgr2=0x7f8714156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:08.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87140d1450 0x7f8714156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:08.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.975+0000 7f8713e00640 1 -- 172.21.15.3:0/4278848875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871415ece0 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f8713e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c800 tx=0x7f870c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c018070 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c004d80 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c00a880 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871415efc0 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.976+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871415f580 con 0x7f871415e460 2024-09-10T07:54:08.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.977+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87140d0f10 con 0x7f871415e460 2024-09-10T07:54:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.978+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f870c00aa20 con 0x7f871415e460 2024-09-10T07:54:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.979+0000 7f870b400640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 0x7f86f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.979+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5967+0+0 (secure 0 0 0) 0x7f870c08a410 con 0x7f871415e460 2024-09-10T07:54:08.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.979+0000 7f8713400640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 0x7f86f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:08.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.980+0000 7f8713400640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 0x7f86f406e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8704009aa0 tx=0x7f8704009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:08.983+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f870c058690 con 0x7f871415e460 2024-09-10T07:54:09.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.165+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8714159880 con 0x7f871415e460 2024-09-10T07:54:09.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.795+0000 7f870b400640 1 -- 172.21.15.3:0/4278848875 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f870c05c990 con 0x7f871415e460 2024-09-10T07:54:09.795 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 msgr2=0x7f86f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 0x7f86f406e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8704009aa0 tx=0x7f8704009190 comp rx=0 tx=0).stop 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 msgr2=0x7f87141571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c800 tx=0x7f870c00ccc0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 shutdown_connections 2024-09-10T07:54:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f86f406be60 0x7f86f406e320 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871415e460 0x7f87141571e0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87140d1450 0x7f8714156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 --2- 172.21.15.3:0/4278848875 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f87140cf4d0 0x7f871415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.798+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 >> 172.21.15.3:0/4278848875 conn(0x7f87140b6050 msgr2=0x7f87140b9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.799+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 shutdown_connections 2024-09-10T07:54:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.799+0000 7f8718e00640 1 -- 172.21.15.3:0/4278848875 wait complete. 2024-09-10T07:54:09.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T07:54:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.983+0000 7f7fef800640 1 Processor -- start 2024-09-10T07:54:09.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.983+0000 7f7fef800640 1 -- start start 2024-09-10T07:54:09.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.984+0000 7f7fef800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fef800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 0x7f7fe80722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fef800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe80cf1f0 con 0x7f7fe815c530 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fef800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe80d00c0 con 0x7f7fe80b24d0 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fef800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe80b2930 con 0x7f7fe80cd910 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fece00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7fe815c530 0x7f7fe80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43042/0 (socket says 172.21.15.3:43042) 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fece00640 1 -- 172.21.15.3:0/2286056179 learned_addr learned my addr 172.21.15.3:0/2286056179 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fe7e00640 1 --2- 172.21.15.3:0/2286056179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fed800640 1 --2- 172.21.15.3:0/2286056179 >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 0x7f7fe80722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fe7400640 1 -- 172.21.15.3:0/2286056179 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018745273 0 0) 0x7f7fe80cf1f0 con 0x7f7fe815c530 2024-09-10T07:54:09.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.985+0000 7f7fe7400640 1 -- 172.21.15.3:0/2286056179 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd0003520 con 0x7f7fe815c530 2024-09-10T07:54:09.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.986+0000 7f7fe7e00640 1 -- 172.21.15.3:0/2286056179 >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 msgr2=0x7f7fe80722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:09.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.986+0000 7f7fe7e00640 1 --2- 172.21.15.3:0/2286056179 >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 0x7f7fe80722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.986+0000 7f7fe7e00640 1 -- 172.21.15.3:0/2286056179 >> v1:172.21.15.3:6789/0 conn(0x7f7fe815c530 legacy=0x7f7fe80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.986+0000 7f7fe7e00640 1 -- 172.21.15.3:0/2286056179 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe80ce150 con 0x7f7fe80cd910 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.986+0000 7f7fed800640 1 --2- 172.21.15.3:0/2286056179 >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 0x7f7fe80722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fe7e00640 1 --2- 172.21.15.3:0/2286056179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80b1f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40097a0 tx=0x7f7fd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897c61b7e3f7bfc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fe7400640 1 -- 172.21.15.3:0/2286056179 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4037070 con 0x7f7fe80cd910 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fe7400640 1 -- 172.21.15.3:0/2286056179 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4032440 con 0x7f7fe80cd910 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fe7400640 1 -- 172.21.15.3:0/2286056179 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40327f0 con 0x7f7fe80cd910 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fef800640 1 -- 172.21.15.3:0/2286056179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 msgr2=0x7f7fe80b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.987+0000 7f7fef800640 1 --2- 172.21.15.3:0/2286056179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80b1f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40097a0 tx=0x7f7fd402a180 comp rx=0 tx=0).stop 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 -- 172.21.15.3:0/2286056179 shutdown_connections 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 --2- 172.21.15.3:0/2286056179 >> v2:172.21.15.70:3300/0 conn(0x7f7fe80b24d0 0x7f7fe80722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 --2- 172.21.15.3:0/2286056179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80b1f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 -- 172.21.15.3:0/2286056179 >> 172.21.15.3:0/2286056179 conn(0x7f7fe8062cc0 msgr2=0x7f7fe806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:09.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 -- 172.21.15.3:0/2286056179 shutdown_connections 2024-09-10T07:54:09.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.988+0000 7f7fef800640 1 -- 172.21.15.3:0/2286056179 wait complete. 2024-09-10T07:54:09.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.989+0000 7f7fef800640 1 Processor -- start 2024-09-10T07:54:09.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.989+0000 7f7fef800640 1 -- start start 2024-09-10T07:54:09.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fe80b24d0 0x7f7fe814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80ce9f0 con 0x7f7fe815c530 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80ce710 con 0x7f7fe80b24d0 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fef800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe80cecd0 con 0x7f7fe80cd910 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.990+0000 7f7fed800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46644/0 (socket says 172.21.15.3:46644) 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 -- 172.21.15.3:0/810853348 learned_addr learned my addr 172.21.15.3:0/810853348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fe7e00640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fece00640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fe80b24d0 0x7f7fe814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 -- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 msgr2=0x7f7fe80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 -- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fe80b24d0 msgr2=0x7f7fe814bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:09.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fe80b24d0 0x7f7fe814bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:09.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 -- 172.21.15.3:0/810853348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe80b4b20 con 0x7f7fe815c530 2024-09-10T07:54:09.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fed800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009f00 tx=0x7f7fe000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0018030 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe000ce50 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.991+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0003bb0 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.992+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe814c680 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.992+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe814cc40 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.993+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe80b1f90 con 0x7f7fe815c530 2024-09-10T07:54:09.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.994+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fe001b070 con 0x7f7fe815c530 2024-09-10T07:54:09.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.994+0000 7f7fe4c00640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:09.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.994+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5529+0+0 (secure 0 0 0) 0x7f7fe008b930 con 0x7f7fe815c530 2024-09-10T07:54:09.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.995+0000 7f7fece00640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:09.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.995+0000 7f7fece00640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 0x7f7fc806e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005880 tx=0x7f7fdc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:09.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:09.999+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7fe0010040 con 0x7f7fe815c530 2024-09-10T07:54:10.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.188+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7fe80cf720 con 0x7f7fe815c530 2024-09-10T07:54:10.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.797+0000 7f7fe4c00640 1 -- 172.21.15.3:0/810853348 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f7fe0012340 con 0x7f7fe815c530 2024-09-10T07:54:10.797 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 msgr2=0x7f7fc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 0x7f7fc806e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005880 tx=0x7f7fdc009040 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 msgr2=0x7f7fe80c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009f00 tx=0x7f7fe000c490 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 shutdown_connections 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7fc806be60 0x7f7fc806e320 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fe815c530 0x7f7fe80c89d0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fe80cd910 0x7f7fe80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 --2- 172.21.15.3:0/810853348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fe80b24d0 0x7f7fe814bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 >> 172.21.15.3:0/810853348 conn(0x7f7fe8062cc0 msgr2=0x7f7fe806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:10.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 shutdown_connections 2024-09-10T07:54:10.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.800+0000 7f7fef800640 1 -- 172.21.15.3:0/810853348 wait complete. 2024-09-10T07:54:10.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T07:54:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.985+0000 7fa3f4a00640 1 Processor -- start 2024-09-10T07:54:10.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.985+0000 7fa3f4a00640 1 -- start start 2024-09-10T07:54:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.986+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f00b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.986+0000 7fa3f4a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f006bb20 0x7fa3f006bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.986+0000 7fa3f4a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f00b8680 con 0x7fa3f0148040 2024-09-10T07:54:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.986+0000 7fa3f4a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f00b88f0 con 0x7fa3f006bb20 2024-09-10T07:54:10.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.986+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f00b8bd0 con 0x7fa3f00b96b0 2024-09-10T07:54:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f006bb20 0x7fa3f006bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f006bb20 0x7fa3f006bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46648/0 (socket says 172.21.15.3:46648) 2024-09-10T07:54:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 -- 172.21.15.3:0/3237518543 learned_addr learned my addr 172.21.15.3:0/3237518543 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3e7e00640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f00b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 -- 172.21.15.3:0/3237518543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 msgr2=0x7fa3f00b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f00b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 -- 172.21.15.3:0/3237518543 >> v1:172.21.15.70:6789/0 conn(0x7fa3f0148040 legacy=0x7fa3f0162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3ef200640 1 -- 172.21.15.3:0/3237518543 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f00b8eb0 con 0x7fa3f006bb20 2024-09-10T07:54:10.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.987+0000 7fa3e7e00640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f00b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.988+0000 7fa3ef200640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f006bfb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009e50 tx=0x7fa3e802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1666ded11dfc6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3ede00640 1 -- 172.21.15.3:0/3237518543 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8036070 con 0x7fa3f006bb20 2024-09-10T07:54:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3ede00640 1 -- 172.21.15.3:0/3237518543 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e8031600 con 0x7fa3f006bb20 2024-09-10T07:54:10.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3ede00640 1 -- 172.21.15.3:0/3237518543 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80319d0 con 0x7fa3f006bb20 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 -- 172.21.15.3:0/3237518543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 msgr2=0x7fa3f006bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f006bfb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009e50 tx=0x7fa3e802a630 comp rx=0 tx=0).stop 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 -- 172.21.15.3:0/3237518543 shutdown_connections 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f006bfb0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/3237518543 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f00b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 -- 172.21.15.3:0/3237518543 >> 172.21.15.3:0/3237518543 conn(0x7fa3f0062a50 msgr2=0x7fa3f0072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 -- 172.21.15.3:0/3237518543 shutdown_connections 2024-09-10T07:54:10.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.989+0000 7fa3f4a00640 1 -- 172.21.15.3:0/3237518543 wait complete. 2024-09-10T07:54:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.991+0000 7fa3f4a00640 1 Processor -- start 2024-09-10T07:54:10.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.991+0000 7fa3f4a00640 1 -- start start 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f013e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0148040 0x7fa3f00d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f015ec10 con 0x7fa3f006bb20 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0148420 con 0x7fa3f0148040 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f015ed90 con 0x7fa3f00b96b0 2024-09-10T07:54:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46658/0 (socket says 172.21.15.3:46658) 2024-09-10T07:54:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3ee800640 1 -- 172.21.15.3:0/991771749 learned_addr learned my addr 172.21.15.3:0/991771749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3ef200640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0148040 0x7fa3f00d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.993+0000 7fa3e7e00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f013e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 -- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 msgr2=0x7fa3f013e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f013e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 -- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0148040 msgr2=0x7fa3f00d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0148040 0x7fa3f00d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 -- 172.21.15.3:0/991771749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f0140410 con 0x7fa3f006bb20 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.994+0000 7fa3ee800640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b420 tx=0x7fa3e000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.995+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011830 con 0x7fa3f006bb20 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.995+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e0016070 con 0x7fa3f006bb20 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.995+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01406f0 con 0x7fa3f006bb20 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.995+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0011c10 con 0x7fa3f006bb20 2024-09-10T07:54:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.995+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f0140cb0 con 0x7fa3f006bb20 2024-09-10T07:54:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.996+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f00b68a0 con 0x7fa3f006bb20 2024-09-10T07:54:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.998+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3e001b460 con 0x7fa3f006bb20 2024-09-10T07:54:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.999+0000 7fa3e6a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 0x7fa3cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.999+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fa3e008a630 con 0x7fa3f006bb20 2024-09-10T07:54:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.999+0000 7fa3e7e00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 0x7fa3cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:10.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:10.999+0000 7fa3e7e00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 0x7fa3cc06e320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00d5470 tx=0x7fa3d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:11.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.003+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3e0020020 con 0x7fa3f006bb20 2024-09-10T07:54:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.185+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa3f00b70a0 con 0x7fa3f006bb20 2024-09-10T07:54:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.189+0000 7fa3e6a00640 1 -- 172.21.15.3:0/991771749 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fa3e0058b40 con 0x7fa3f006bb20 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 msgr2=0x7fa3cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 0x7fa3cc06e320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00d5470 tx=0x7fa3d8008040 comp rx=0 tx=0).stop 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 msgr2=0x7fa3f013fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b420 tx=0x7fa3e000b8e0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 shutdown_connections 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa3cc06be60 0x7fa3cc06e320 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0148040 0x7fa3f00d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa3f00b96b0 0x7fa3f013e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 --2- 172.21.15.3:0/991771749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f006bb20 0x7fa3f013fdc0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 >> 172.21.15.3:0/991771749 conn(0x7fa3f0062a50 msgr2=0x7fa3f0072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 shutdown_connections 2024-09-10T07:54:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.191+0000 7fa3f4a00640 1 -- 172.21.15.3:0/991771749 wait complete. 2024-09-10T07:54:11.207 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-10T07:54:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.364+0000 7ffb7ae00640 1 Processor -- start 2024-09-10T07:54:11.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.364+0000 7ffb7ae00640 1 -- start start 2024-09-10T07:54:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb7ae00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb740d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb7ae00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb740ccd80 con 0x7ffb7415b110 2024-09-10T07:54:11.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb7ae00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb740cd060 con 0x7ffb74073d10 2024-09-10T07:54:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb7ae00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb740b44f0 con 0x7ffb7415b4f0 2024-09-10T07:54:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb73e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb7415b110 0x7ffb740b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43052/0 (socket says 172.21.15.3:43052) 2024-09-10T07:54:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb73e00640 1 -- 172.21.15.3:0/272166300 learned_addr learned my addr 172.21.15.3:0/272166300 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:11.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb6be00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb740d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259842825 0 0) 0x7ffb740ccd80 con 0x7ffb7415b110 2024-09-10T07:54:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.366+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb58003520 con 0x7ffb7415b110 2024-09-10T07:54:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418103849 0 0) 0x7ffb740cd060 con 0x7ffb74073d10 2024-09-10T07:54:11.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb740ccd80 con 0x7ffb74073d10 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1266010813 0 0) 0x7ffb58003520 con 0x7ffb7415b110 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 msgr2=0x7ffb740d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb740d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 >> v1:172.21.15.70:6789/0 conn(0x7ffb74073d10 legacy=0x7ffb740cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb740cd340 con 0x7ffb7415b110 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7ffb64001670 con 0x7ffb7415b110 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 >> v1:172.21.15.3:6789/0 conn(0x7ffb7415b110 legacy=0x7ffb740b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb58003b70 0x7ffb58003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740b44f0 con 0x7ffb58004d40 2024-09-10T07:54:11.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5800bad0 con 0x7ffb58003b70 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5800bcd0 con 0x7ffb7415b4f0 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb78e00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb73e00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb58003b70 0x7ffb58003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.367+0000 7ffb6be00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.368+0000 7ffb78e00640 1 -- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 msgr2=0x7ffb58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.368+0000 7ffb78e00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb7415b4f0 0x7ffb58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.368+0000 7ffb78e00640 1 -- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb58003b70 msgr2=0x7ffb58003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.368+0000 7ffb78e00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb58003b70 0x7ffb58003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.368+0000 7ffb78e00640 1 -- 172.21.15.3:0/272166300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb58003520 con 0x7ffb58004d40 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb78e00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001fa0 tx=0x7ffb58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82b92bc88b2b0af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c001bf0 con 0x7ffb58004d40 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6c003b70 con 0x7ffb58004d40 2024-09-10T07:54:11.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb73400640 1 -- 172.21.15.3:0/272166300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0050d0 con 0x7ffb58004d40 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb7ae00640 1 -- 172.21.15.3:0/272166300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb58004d40 msgr2=0x7ffb580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/272166300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb58004d40 0x7ffb580051c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001fa0 tx=0x7ffb58005830 comp rx=0 tx=0).stop 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.369+0000 7ffb78e00640 1 -- 172.21.15.3:0/272166300 reap_dead start 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.370+0000 7ffb7ae00640 1 -- 172.21.15.3:0/272166300 shutdown_connections 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.370+0000 7ffb7ae00640 1 -- 172.21.15.3:0/272166300 >> 172.21.15.3:0/272166300 conn(0x7ffb7406d000 msgr2=0x7ffb7406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.370+0000 7ffb7ae00640 1 -- 172.21.15.3:0/272166300 shutdown_connections 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.370+0000 7ffb7ae00640 1 -- 172.21.15.3:0/272166300 wait complete. 2024-09-10T07:54:11.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.371+0000 7ffb7ae00640 1 Processor -- start 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.371+0000 7ffb7ae00640 1 -- start start 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 0x7ffb741f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb741f4c70 0x7ffb74153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740d0360 con 0x7ffb741f4350 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740d01e0 con 0x7ffb740adfc0 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.372+0000 7ffb7ae00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffb740d04e0 con 0x7ffb741f4c70 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46670/0 (socket says 172.21.15.3:46670) 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 -- 172.21.15.3:0/2247394788 learned_addr learned my addr 172.21.15.3:0/2247394788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb78e00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb741f4c70 0x7ffb74153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb6be00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 0x7ffb741f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 -- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb741f4c70 msgr2=0x7ffb74153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb741f4c70 0x7ffb74153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 -- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 msgr2=0x7ffb741f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 0x7ffb741f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 -- 172.21.15.3:0/2247394788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb740ae600 con 0x7ffb741f4350 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb6be00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 0x7ffb741f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb73e00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ffb64004f50 tx=0x7ffb640074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.373+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64007740 con 0x7ffb741f4350 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.374+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb64018070 con 0x7ffb741f4350 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.374+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400fb60 con 0x7ffb741f4350 2024-09-10T07:54:11.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.374+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb740ae8e0 con 0x7ffb741f4350 2024-09-10T07:54:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.374+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb740aeea0 con 0x7ffb741f4350 2024-09-10T07:54:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.375+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb740d0810 con 0x7ffb741f4350 2024-09-10T07:54:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.376+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb6400fd00 con 0x7ffb741f4350 2024-09-10T07:54:11.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.377+0000 7ffb70c00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 0x7ffb5006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.377+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7ffb6408ac10 con 0x7ffb741f4350 2024-09-10T07:54:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.377+0000 7ffb6be00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 0x7ffb5006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:11.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.377+0000 7ffb6be00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 0x7ffb5006e490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffb74071a20 tx=0x7ffb5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:11.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.381+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb64059800 con 0x7ffb741f4350 2024-09-10T07:54:11.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.636+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7ffb740b2100 con 0x7ffb741f4350 2024-09-10T07:54:11.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.808+0000 7ffb70c00640 1 -- 172.21.15.3:0/2247394788 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v398) ==== 165+0+0 (secure 0 0 0) 0x7ffb64059100 con 0x7ffb741f4350 2024-09-10T07:54:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 msgr2=0x7ffb5006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 0x7ffb5006e490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffb74071a20 tx=0x7ffb5c008040 comp rx=0 tx=0).stop 2024-09-10T07:54:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 msgr2=0x7ffb740ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ffb64004f50 tx=0x7ffb640074f0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 shutdown_connections 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffb5006bfd0 0x7ffb5006e490 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffb741f4c70 0x7ffb74153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb740adfc0 0x7ffb741f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 --2- 172.21.15.3:0/2247394788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb741f4350 0x7ffb740ada80 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.810+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 >> 172.21.15.3:0/2247394788 conn(0x7ffb7406d000 msgr2=0x7ffb740b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.811+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 shutdown_connections 2024-09-10T07:54:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:11.811+0000 7ffb7ae00640 1 -- 172.21.15.3:0/2247394788 wait complete. 2024-09-10T07:54:11.826 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-10T07:54:11.827 INFO:tasks.cephfs_test_runner: 2024-09-10T07:54:11.827 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T07:54:11.827 INFO:tasks.cephfs_test_runner:Ran 6 tests in 429.338s 2024-09-10T07:54:11.827 INFO:tasks.cephfs_test_runner: 2024-09-10T07:54:11.827 INFO:tasks.cephfs_test_runner:OK 2024-09-10T07:54:11.827 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T07:54:11.838 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T07:54:11.876 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T07:54:11.876 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:54:11.876 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:11.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:54:11.913 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-10T07:54:11.913 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:11.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T07:54:11.988 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:54:11.989 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:12.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:54:12.028 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T07:54:12.028 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T07:54:12.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T07:54:12.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.107 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:54:12.141 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T07:54:12.142 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T07:54:12.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T07:54:12.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T07:54:12.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-10T07:54:12.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.815 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-10T07:54:12.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-10T07:54:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-10T07:54:12.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:12.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.955 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:54:12.995 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T07:54:12.995 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T07:54:12.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:12.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T07:54:13.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:13.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T07:54:13.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:13.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-09-10T07:54:13.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:13.646 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T07:54:13.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T07:54:13.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-09-10T07:54:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-09-10T07:54:13.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:13.785 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T07:54:13.796 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T07:54:13.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:54:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.966+0000 7f02ce600640 1 Processor -- start 2024-09-10T07:54:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.966+0000 7f02ce600640 1 -- start start 2024-09-10T07:54:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02ce600640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02c8160710 0x7f02c80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02ce600640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02c8160af0 0x7f02c80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02ce600640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c80cfdd0 con 0x7f02c80cf800 2024-09-10T07:54:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02ce600640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f02c815cc90 con 0x7f02c8160710 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02ce600640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f02c815cf70 con 0x7f02c8160af0 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.967+0000 7f02c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02c8160710 0x7f02c80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02c8160710 0x7f02c80cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41600/0 (socket says 172.21.15.3:41600) 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 -- 172.21.15.3:0/1166579383 learned_addr learned my addr 172.21.15.3:0/1166579383 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02bfe00640 1 --2- 172.21.15.3:0/1166579383 >> v2:172.21.15.70:3300/0 conn(0x7f02c8160af0 0x7f02c80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 -- 172.21.15.3:0/1166579383 >> v2:172.21.15.70:3300/0 conn(0x7f02c8160af0 msgr2=0x7f02c80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 --2- 172.21.15.3:0/1166579383 >> v2:172.21.15.70:3300/0 conn(0x7f02c8160af0 0x7f02c80d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 -- 172.21.15.3:0/1166579383 >> v1:172.21.15.70:6789/0 conn(0x7f02c80cf800 legacy=0x7f02c806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.968+0000 7f02c7400640 1 -- 172.21.15.3:0/1166579383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c815d250 con 0x7f02c8160710 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.969+0000 7f02c7400640 1 --2- 172.21.15.3:0/1166579383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160710 0x7f02c80cf2c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f02c806df70 tx=0x7f02b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42643a837aeb7f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.969+0000 7f02c6a00640 1 -- 172.21.15.3:0/1166579383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037070 con 0x7f02c8160710 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.969+0000 7f02c6a00640 1 -- 172.21.15.3:0/1166579383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8032440 con 0x7f02c8160710 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.969+0000 7f02c6a00640 1 -- 172.21.15.3:0/1166579383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8032810 con 0x7f02c8160710 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 -- 172.21.15.3:0/1166579383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160710 msgr2=0x7f02c80cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 --2- 172.21.15.3:0/1166579383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160710 0x7f02c80cf2c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f02c806df70 tx=0x7f02b802a270 comp rx=0 tx=0).stop 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 -- 172.21.15.3:0/1166579383 shutdown_connections 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 --2- 172.21.15.3:0/1166579383 >> v2:172.21.15.70:3300/0 conn(0x7f02c8160af0 0x7f02c80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 --2- 172.21.15.3:0/1166579383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160710 0x7f02c80cf2c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 -- 172.21.15.3:0/1166579383 >> 172.21.15.3:0/1166579383 conn(0x7f02c806d000 msgr2=0x7f02c806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 -- 172.21.15.3:0/1166579383 shutdown_connections 2024-09-10T07:54:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.970+0000 7f02ce600640 1 -- 172.21.15.3:0/1166579383 wait complete. 2024-09-10T07:54:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.972+0000 7f02ce600640 1 Processor -- start 2024-09-10T07:54:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.972+0000 7f02ce600640 1 -- start start 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02c8160710 0x7f02c8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815d770 con 0x7f02c8160af0 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8073080 con 0x7f02c80cf800 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02ce600640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f02c814e9d0 con 0x7f02c8160710 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02c7400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02c7400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02c7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41608/0 (socket says 172.21.15.3:41608) 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02c7400640 1 -- 172.21.15.3:0/2199380028 learned_addr learned my addr 172.21.15.3:0/2199380028 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.973+0000 7f02bfe00640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02c8160710 0x7f02c8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 -- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02c8160710 msgr2=0x7f02c8153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02c8160710 0x7f02c8153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 -- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 msgr2=0x7f02c80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 -- 172.21.15.3:0/2199380028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c8150e10 con 0x7f02c80cf800 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7e00640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02c7400640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000c00 tx=0x7f02b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037040 con 0x7f02c80cf800 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8042070 con 0x7f02c80cf800 2024-09-10T07:54:13.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81502b0 con 0x7f02c80cf800 2024-09-10T07:54:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8030880 con 0x7f02c80cf800 2024-09-10T07:54:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.974+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c8150850 con 0x7f02c80cf800 2024-09-10T07:54:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.975+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c80afd50 con 0x7f02c80cf800 2024-09-10T07:54:13.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.976+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02b8030a20 con 0x7f02c80cf800 2024-09-10T07:54:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.977+0000 7f02bf400640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 0x7f02a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.977+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f02b80ab300 con 0x7f02c80cf800 2024-09-10T07:54:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.977+0000 7f02bfe00640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 0x7f02a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:13.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.978+0000 7f02bfe00640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 0x7f02a406e340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f02c80af8d0 tx=0x7f02b0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:13.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:13.983+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02b8079920 con 0x7f02c80cf800 2024-09-10T07:54:14.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.163+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02c80b2bf0 con 0x7f02a406be80 2024-09-10T07:54:14.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.164+0000 7f02bf400640 1 -- 172.21.15.3:0/2199380028 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f02c80b2bf0 con 0x7f02a406be80 2024-09-10T07:54:14.164 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:14.164 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 msgr2=0x7f02a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 0x7f02a406e340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f02c80af8d0 tx=0x7f02b0009190 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 msgr2=0x7f02c814e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000c00 tx=0x7f02b80046c0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 shutdown_connections 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f02a406be80 0x7f02a406e340 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02c8160af0 0x7f02c80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f02c8160710 0x7f02c8153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 --2- 172.21.15.3:0/2199380028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c80cf800 0x7f02c814e490 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 >> 172.21.15.3:0/2199380028 conn(0x7f02c806d000 msgr2=0x7f02c8073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:14.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 shutdown_connections 2024-09-10T07:54:14.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.166+0000 7f02ce600640 1 -- 172.21.15.3:0/2199380028 wait complete. 2024-09-10T07:54:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T07:54:13.594856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":264952,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709768,"statfs":{"total":773094113280,"available":772822802432,"internally_reserved":0,"allocated":11546624,"data_stored":5707069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122114,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001364"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-10T07:54:10.148067+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:54:10.148067+0000","last_peered":"2024-09-10T07:54:10.148067+0000","last_clean":"2024-09-10T07:54:10.148067+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:54:10.148067+0000","last_undegraded":"2024-09-10T07:54:10.148067+0000","last_fullsized":"2024-09-10T07:54:10.148067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476834,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476834,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476832,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":1145152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476833,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":549371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1961984,"data_stored":1221415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:54:14.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:54:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.339+0000 7f9428e00640 1 Processor -- start 2024-09-10T07:54:14.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.339+0000 7f9428e00640 1 -- start start 2024-09-10T07:54:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.340+0000 7f9428e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 0x7f94240d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.340+0000 7f9428e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94240d05f0 0x7f9424072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.340+0000 7f9428e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94240cfdd0 con 0x7f9424160710 2024-09-10T07:54:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.340+0000 7f9428e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f942415cc90 con 0x7f94240d05f0 2024-09-10T07:54:14.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.340+0000 7f9428e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f942415cf70 con 0x7f9424160af0 2024-09-10T07:54:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f9421e00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9424160710 0x7f94240cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42404/0 (socket says 172.21.15.3:42404) 2024-09-10T07:54:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f9421e00640 1 -- 172.21.15.3:0/2815753320 learned_addr learned my addr 172.21.15.3:0/2815753320 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f941be00640 1 --2- 172.21.15.3:0/2815753320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 0x7f94240d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f9422800640 1 --2- 172.21.15.3:0/2815753320 >> v2:172.21.15.70:3300/0 conn(0x7f94240d05f0 0x7f9424072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f9421400640 1 -- 172.21.15.3:0/2815753320 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457085382 0 0) 0x7f94240cfdd0 con 0x7f9424160710 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f9421400640 1 -- 172.21.15.3:0/2815753320 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9408003520 con 0x7f9424160710 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f941be00640 1 -- 172.21.15.3:0/2815753320 >> v2:172.21.15.70:3300/0 conn(0x7f94240d05f0 msgr2=0x7f9424072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f941be00640 1 --2- 172.21.15.3:0/2815753320 >> v2:172.21.15.70:3300/0 conn(0x7f94240d05f0 0x7f9424072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f941be00640 1 -- 172.21.15.3:0/2815753320 >> v1:172.21.15.3:6789/0 conn(0x7f9424160710 legacy=0x7f94240cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.341+0000 7f941be00640 1 -- 172.21.15.3:0/2815753320 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942415d250 con 0x7f9424160af0 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.342+0000 7f941be00640 1 --2- 172.21.15.3:0/2815753320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 0x7f94240d00b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f940c0097a0 tx=0x7f940c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2ffb539f7e1294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9421400640 1 -- 172.21.15.3:0/2815753320 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c037070 con 0x7f9424160af0 2024-09-10T07:54:14.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9421400640 1 -- 172.21.15.3:0/2815753320 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c032440 con 0x7f9424160af0 2024-09-10T07:54:14.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9421400640 1 -- 172.21.15.3:0/2815753320 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0327f0 con 0x7f9424160af0 2024-09-10T07:54:14.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 -- 172.21.15.3:0/2815753320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 msgr2=0x7f94240d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 --2- 172.21.15.3:0/2815753320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 0x7f94240d00b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f940c0097a0 tx=0x7f940c02a180 comp rx=0 tx=0).stop 2024-09-10T07:54:14.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 -- 172.21.15.3:0/2815753320 shutdown_connections 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 --2- 172.21.15.3:0/2815753320 >> v2:172.21.15.70:3300/0 conn(0x7f94240d05f0 0x7f9424072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 --2- 172.21.15.3:0/2815753320 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160af0 0x7f94240d00b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 -- 172.21.15.3:0/2815753320 >> 172.21.15.3:0/2815753320 conn(0x7f942406d000 msgr2=0x7f942406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 -- 172.21.15.3:0/2815753320 shutdown_connections 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.343+0000 7f9428e00640 1 -- 172.21.15.3:0/2815753320 wait complete. 2024-09-10T07:54:14.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.345+0000 7f9428e00640 1 Processor -- start 2024-09-10T07:54:14.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.345+0000 7f9428e00640 1 -- start start 2024-09-10T07:54:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94240d05f0 0x7f942415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 0x7f9424157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f942415d770 con 0x7f9424160af0 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94240d0a10 con 0x7f94240d05f0 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9428e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f942415b0e0 con 0x7f9424160710 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9422800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9422800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41616/0 (socket says 172.21.15.3:41616) 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9422800640 1 -- 172.21.15.3:0/528827869 learned_addr learned my addr 172.21.15.3:0/528827869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f941be00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 0x7f9424157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.346+0000 7f9421e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94240d05f0 0x7f942415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 -- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 msgr2=0x7f9424157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 0x7f9424157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 -- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94240d05f0 msgr2=0x7f942415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94240d05f0 0x7f942415aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 -- 172.21.15.3:0/528827869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942415b310 con 0x7f9424160af0 2024-09-10T07:54:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f941be00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 0x7f9424157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:14.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9422800640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f941c00efc0 tx=0x7f941c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c018020 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c014480 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c014870 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.347+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942415b5f0 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.348+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9424153840 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.348+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94240cfa60 con 0x7f9424160af0 2024-09-10T07:54:14.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.350+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f941c005d10 con 0x7f9424160af0 2024-09-10T07:54:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.351+0000 7f941a000640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 0x7f940006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.351+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f941c08b6d0 con 0x7f9424160af0 2024-09-10T07:54:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.351+0000 7f9421e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 0x7f940006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.352+0000 7f9421e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 0x7f940006e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9414005880 tx=0x7f9414009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.356+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f941c022070 con 0x7f9424160af0 2024-09-10T07:54:14.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.537+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94240afd50 con 0x7f940006be30 2024-09-10T07:54:14.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.538+0000 7f941a000640 1 -- 172.21.15.3:0/528827869 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f94240afd50 con 0x7f940006be30 2024-09-10T07:54:14.538 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:14.538 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 msgr2=0x7f940006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 0x7f940006e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9414005880 tx=0x7f9414009040 comp rx=0 tx=0).stop 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 msgr2=0x7f942414aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f941c00efc0 tx=0x7f941c00c490 comp rx=0 tx=0).stop 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 shutdown_connections 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f940006be30 0x7f940006e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9424160af0 0x7f942414aea0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9424160710 0x7f9424157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 --2- 172.21.15.3:0/528827869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94240d05f0 0x7f942415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 >> 172.21.15.3:0/528827869 conn(0x7f942406d000 msgr2=0x7f94240b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:14.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 shutdown_connections 2024-09-10T07:54:14.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.540+0000 7f9428e00640 1 -- 172.21.15.3:0/528827869 wait complete. 2024-09-10T07:54:14.556 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T07:54:13.594856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":264952,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709768,"statfs":{"total":773094113280,"available":772822802432,"internally_reserved":0,"allocated":11546624,"data_stored":5707069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122114,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001364"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-10T07:54:10.148067+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:54:10.148067+0000","last_peered":"2024-09-10T07:54:10.148067+0000","last_clean":"2024-09-10T07:54:10.148067+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:54:10.148067+0000","last_undegraded":"2024-09-10T07:54:10.148067+0000","last_fullsized":"2024-09-10T07:54:10.148067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476834,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476834,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476832,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":1145152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476833,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":549371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1961984,"data_stored":1221415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:54:14.556 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T07:54:14.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:54:14.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.713+0000 7fbabd200640 1 Processor -- start 2024-09-10T07:54:14.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.713+0000 7fbabd200640 1 -- start start 2024-09-10T07:54:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbabd200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbab815b4f0 0x7fbab80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbabd200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab80ccd80 con 0x7fbab8073d10 2024-09-10T07:54:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbabd200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab80cd060 con 0x7fbab815b110 2024-09-10T07:54:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbabd200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbab80b44f0 con 0x7fbab815b4f0 2024-09-10T07:54:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbab6800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbab8073d10 0x7fbab80cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42416/0 (socket says 172.21.15.3:42416) 2024-09-10T07:54:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbab6800640 1 -- 172.21.15.3:0/1946257572 learned_addr learned my addr 172.21.15.3:0/1946257572 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbaad400640 1 --2- 172.21.15.3:0/1946257572 >> v2:172.21.15.70:3300/0 conn(0x7fbab815b4f0 0x7fbab80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 455356845 0 0) 0x7fbab80ccd80 con 0x7fbab8073d10 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.715+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba9c003520 con 0x7fbab8073d10 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95728114 0 0) 0x7fbab80cd060 con 0x7fbab815b110 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbab80ccd80 con 0x7fbab815b110 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbaad400640 1 -- 172.21.15.3:0/1946257572 >> v1:172.21.15.70:6789/0 conn(0x7fbab815b110 legacy=0x7fbab80b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbaad400640 1 -- 172.21.15.3:0/1946257572 >> v1:172.21.15.3:6789/0 conn(0x7fbab8073d10 legacy=0x7fbab80cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059264006 0 0) 0x7fba9c003520 con 0x7fbab8073d10 2024-09-10T07:54:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.716+0000 7fbaad400640 1 -- 172.21.15.3:0/1946257572 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab80cd340 con 0x7fbab815b4f0 2024-09-10T07:54:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbaad400640 1 --2- 172.21.15.3:0/1946257572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b4f0 0x7fbab80d01e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbab80b4670 tx=0x7fbaa002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b6578c37859cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0037070 con 0x7fbab815b4f0 2024-09-10T07:54:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa002ac30 con 0x7fbab815b4f0 2024-09-10T07:54:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbab5400640 1 -- 172.21.15.3:0/1946257572 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00019f0 con 0x7fbab815b4f0 2024-09-10T07:54:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbabd200640 1 -- 172.21.15.3:0/1946257572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b4f0 msgr2=0x7fbab80d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.717+0000 7fbabd200640 1 --2- 172.21.15.3:0/1946257572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b4f0 0x7fbab80d01e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbab80b4670 tx=0x7fbaa002a1a0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.718+0000 7fbabd200640 1 -- 172.21.15.3:0/1946257572 shutdown_connections 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.718+0000 7fbabd200640 1 --2- 172.21.15.3:0/1946257572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b4f0 0x7fbab80d01e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.718+0000 7fbabd200640 1 -- 172.21.15.3:0/1946257572 >> 172.21.15.3:0/1946257572 conn(0x7fbab806d000 msgr2=0x7fbab806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.718+0000 7fbabd200640 1 -- 172.21.15.3:0/1946257572 shutdown_connections 2024-09-10T07:54:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.718+0000 7fbabd200640 1 -- 172.21.15.3:0/1946257572 wait complete. 2024-09-10T07:54:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.719+0000 7fbabd200640 1 Processor -- start 2024-09-10T07:54:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.719+0000 7fbabd200640 1 -- start start 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbab8073d10 0x7fbab8147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b110 0x7fbab815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbab80d08a0 con 0x7fbab815b4f0 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbab80d0720 con 0x7fbab815b110 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.720+0000 7fbabd200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbab80d0a20 con 0x7fbab8073d10 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41624/0 (socket says 172.21.15.3:41624) 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 -- 172.21.15.3:0/4118224226 learned_addr learned my addr 172.21.15.3:0/4118224226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbaad400640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b110 0x7fbab815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab5e00640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbab8073d10 0x7fbab8147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 -- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbab8073d10 msgr2=0x7fbab8147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbab8073d10 0x7fbab8147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 -- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b110 msgr2=0x7fbab815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b110 0x7fbab815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 -- 172.21.15.3:0/4118224226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab80ab0e0 con 0x7fbab815b4f0 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.721+0000 7fbab6800640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbab000cc80 tx=0x7fbab000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.722+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0017070 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.722+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0012400 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.722+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00127f0 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.722+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab80ab3c0 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.722+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab8148740 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.723+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab80d01e0 con 0x7fbab815b4f0 2024-09-10T07:54:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.724+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbab001a070 con 0x7fbab815b4f0 2024-09-10T07:54:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.725+0000 7fbaaea00640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 0x7fba9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.725+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fbab008ac50 con 0x7fbab815b4f0 2024-09-10T07:54:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.725+0000 7fbab5e00640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 0x7fba9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:14.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.726+0000 7fbab5e00640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 0x7fba9406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80011e0 tx=0x7fbaa8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:14.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.729+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbab0059160 con 0x7fbab815b4f0 2024-09-10T07:54:14.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.911+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbab80b52b0 con 0x7fba9406be30 2024-09-10T07:54:14.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.912+0000 7fbaaea00640 1 -- 172.21.15.3:0/4118224226 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7fbab80b52b0 con 0x7fba9406be30 2024-09-10T07:54:14.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:14.912 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:54:14.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 msgr2=0x7fba9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 0x7fba9406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80011e0 tx=0x7fbaa8009040 comp rx=0 tx=0).stop 2024-09-10T07:54:14.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 msgr2=0x7fbab815d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbab000cc80 tx=0x7fbab000a430 comp rx=0 tx=0).stop 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 shutdown_connections 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fba9406be30 0x7fba9406e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbab815b4f0 0x7fbab815d500 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbab815b110 0x7fbab815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.914+0000 7fbabd200640 1 --2- 172.21.15.3:0/4118224226 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbab8073d10 0x7fbab8147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.915+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 >> 172.21.15.3:0/4118224226 conn(0x7fbab806d000 msgr2=0x7fbab806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.915+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 shutdown_connections 2024-09-10T07:54:14.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:14.915+0000 7fbabd200640 1 -- 172.21.15.3:0/4118224226 wait complete. 2024-09-10T07:54:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T07:54:13.594856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":264952,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709768,"statfs":{"total":773094113280,"available":772822802432,"internally_reserved":0,"allocated":11546624,"data_stored":5707069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122114,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-256,"num_write_kb":-207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001364"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-10T07:54:10.148067+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:54:10.148067+0000","last_peered":"2024-09-10T07:54:10.148067+0000","last_clean":"2024-09-10T07:54:10.148067+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:54:10.148067+0000","last_undegraded":"2024-09-10T07:54:10.148067+0000","last_fullsized":"2024-09-10T07:54:10.148067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476834,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476834,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476832,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":1145152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476833,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":549371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":554784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1961984,"data_stored":1221415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:54:14.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T07:54:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.086+0000 7fb81f200640 1 Processor -- start 2024-09-10T07:54:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.086+0000 7fb81f200640 1 -- start start 2024-09-10T07:54:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81f200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb818161de0 0x7fb8180d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81f200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8180d10f0 0x7fb8180d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81f200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8180d1a80 con 0x7fb818161a00 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81f200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb8180d3060 con 0x7fb818161de0 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81f200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb8180d3340 con 0x7fb8180d10f0 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81d400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb818161de0 0x7fb8180d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81d400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb818161de0 0x7fb8180d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41630/0 (socket says 172.21.15.3:41630) 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81d400640 1 -- 172.21.15.3:0/2070830778 learned_addr learned my addr 172.21.15.3:0/2070830778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.087+0000 7fb81de00640 1 --1- 172.21.15.3:0/2070830778 >> v1:172.21.15.3:6789/0 conn(0x7fb818161a00 0x7fb8180cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42422/0 (socket says 172.21.15.3:42422) 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.088+0000 7fb81e800640 1 --2- 172.21.15.3:0/2070830778 >> v2:172.21.15.70:3300/0 conn(0x7fb8180d10f0 0x7fb8180d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.088+0000 7fb81d400640 1 -- 172.21.15.3:0/2070830778 >> v2:172.21.15.70:3300/0 conn(0x7fb8180d10f0 msgr2=0x7fb8180d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.088+0000 7fb81d400640 1 --2- 172.21.15.3:0/2070830778 >> v2:172.21.15.70:3300/0 conn(0x7fb8180d10f0 0x7fb8180d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.088+0000 7fb81d400640 1 -- 172.21.15.3:0/2070830778 >> v1:172.21.15.3:6789/0 conn(0x7fb818161a00 legacy=0x7fb8180cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.088+0000 7fb81d400640 1 -- 172.21.15.3:0/2070830778 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180d3620 con 0x7fb818161de0 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.089+0000 7fb81d400640 1 --2- 172.21.15.3:0/2070830778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb818161de0 0x7fb8180d2a80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb81808db10 tx=0x7fb80802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7a93bb7f0b7cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.089+0000 7fb81ca00640 1 -- 172.21.15.3:0/2070830778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808037070 con 0x7fb818161de0 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.089+0000 7fb81ca00640 1 -- 172.21.15.3:0/2070830778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb808032440 con 0x7fb818161de0 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.089+0000 7fb81ca00640 1 -- 172.21.15.3:0/2070830778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808032810 con 0x7fb818161de0 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 -- 172.21.15.3:0/2070830778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb818161de0 msgr2=0x7fb8180d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 --2- 172.21.15.3:0/2070830778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb818161de0 0x7fb8180d2a80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb81808db10 tx=0x7fb80802a270 comp rx=0 tx=0).stop 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 -- 172.21.15.3:0/2070830778 shutdown_connections 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 --2- 172.21.15.3:0/2070830778 >> v2:172.21.15.70:3300/0 conn(0x7fb8180d10f0 0x7fb8180d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 --2- 172.21.15.3:0/2070830778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb818161de0 0x7fb8180d2a80 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 -- 172.21.15.3:0/2070830778 >> 172.21.15.3:0/2070830778 conn(0x7fb818072770 msgr2=0x7fb81806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 -- 172.21.15.3:0/2070830778 shutdown_connections 2024-09-10T07:54:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.090+0000 7fb81f200640 1 -- 172.21.15.3:0/2070830778 wait complete. 2024-09-10T07:54:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.091+0000 7fb81f200640 1 Processor -- start 2024-09-10T07:54:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.092+0000 7fb81f200640 1 -- start start 2024-09-10T07:54:15.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.092+0000 7fb81f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.092+0000 7fb81f200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb818161a00 0x7fb81815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81f200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb818161de0 0x7fb81815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180b79f0 con 0x7fb8180d10f0 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81f200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb8180d3be0 con 0x7fb818161a00 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81f200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb8180b7b70 con 0x7fb818161de0 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41632/0 (socket says 172.21.15.3:41632) 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 -- 172.21.15.3:0/4054452347 learned_addr learned my addr 172.21.15.3:0/4054452347 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81d400640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb818161a00 0x7fb81815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81e800640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb818161de0 0x7fb81815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 -- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb818161de0 msgr2=0x7fb81815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb818161de0 0x7fb81815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 -- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb818161a00 msgr2=0x7fb81815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb818161a00 0x7fb81815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.093+0000 7fb81de00640 1 -- 172.21.15.3:0/4054452347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180ca290 con 0x7fb8180d10f0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb81de00640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c400 tx=0x7fb80c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0074e0 con 0x7fb8180d10f0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c007680 con 0x7fb8180d10f0 2024-09-10T07:54:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004a00 con 0x7fb8180d10f0 2024-09-10T07:54:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8180ca570 con 0x7fb8180d10f0 2024-09-10T07:54:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.094+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81814f1f0 con 0x7fb8180d10f0 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.095+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8180d1540 con 0x7fb8180d10f0 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.096+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb80c01a020 con 0x7fb8180d10f0 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.097+0000 7fb812000640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 0x7fb7fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.097+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fb80c08a2c0 con 0x7fb8180d10f0 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.097+0000 7fb81d400640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 0x7fb7fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.098+0000 7fb81d400640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 0x7fb7fc06e320 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb808034000 tx=0x7fb80802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:15.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.101+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb80c058860 con 0x7fb8180d10f0 2024-09-10T07:54:15.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.289+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8180cfef0 con 0x7fb8180d10f0 2024-09-10T07:54:15.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.290+0000 7fb812000640 1 -- 172.21.15.3:0/4054452347 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13226 (secure 0 0 0) 0x7fb80c08a0c0 con 0x7fb8180d10f0 2024-09-10T07:54:15.290 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:15.290 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":105,"fsid":"46339922-75e0-4945-8bb6-1acaef196937","created":"2024-09-10T07:46:06.428833+0000","modified":"2024-09-10T07:54:10.796699+0000","last_up_change":"2024-09-10T07:46:21.058718+0000","last_in_change":"2024-09-10T07:46:09.256869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T07:46:21.545953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37828b5f-b518-4707-8c43-fd1331a1b122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6809","nonce":1494406974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6811","nonce":1494406974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6815","nonce":1494406974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1494406974},{"type":"v1","addr":"172.21.15.3:6813","nonce":1494406974}]},"public_addr":"172.21.15.3:6809/1494406974","cluster_addr":"172.21.15.3:6811/1494406974","heartbeat_back_addr":"172.21.15.3:6815/1494406974","heartbeat_front_addr":"172.21.15.3:6813/1494406974","state":["exists","up"]},{"osd":1,"uuid":"c36cec6f-bb73-44cd-8a1e-1da4e53fe991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6801","nonce":2429800591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6803","nonce":2429800591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6807","nonce":2429800591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2429800591},{"type":"v1","addr":"172.21.15.3:6805","nonce":2429800591}]},"public_addr":"172.21.15.3:6801/2429800591","cluster_addr":"172.21.15.3:6803/2429800591","heartbeat_back_addr":"172.21.15.3:6807/2429800591","heartbeat_front_addr":"172.21.15.3:6805/2429800591","state":["exists","up"]},{"osd":2,"uuid":"072ff2e0-964b-4d8a-b818-99256d7ed638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6825","nonce":2150697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6827","nonce":2150697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6831","nonce":2150697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2150697764},{"type":"v1","addr":"172.21.15.3:6829","nonce":2150697764}]},"public_addr":"172.21.15.3:6825/2150697764","cluster_addr":"172.21.15.3:6827/2150697764","heartbeat_back_addr":"172.21.15.3:6831/2150697764","heartbeat_front_addr":"172.21.15.3:6829/2150697764","state":["exists","up"]},{"osd":3,"uuid":"08b5f076-289a-4f6c-a220-42a739ef896c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6817","nonce":1816157518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6819","nonce":1816157518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6823","nonce":1816157518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1816157518},{"type":"v1","addr":"172.21.15.3:6821","nonce":1816157518}]},"public_addr":"172.21.15.3:6817/1816157518","cluster_addr":"172.21.15.3:6819/1816157518","heartbeat_back_addr":"172.21.15.3:6823/1816157518","heartbeat_front_addr":"172.21.15.3:6821/1816157518","state":["exists","up"]},{"osd":4,"uuid":"5c0cd914-4953-4bf1-88d0-e198e9de98c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6801","nonce":2990268368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6803","nonce":2990268368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6807","nonce":2990268368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2990268368},{"type":"v1","addr":"172.21.15.70:6805","nonce":2990268368}]},"public_addr":"172.21.15.70:6801/2990268368","cluster_addr":"172.21.15.70:6803/2990268368","heartbeat_back_addr":"172.21.15.70:6807/2990268368","heartbeat_front_addr":"172.21.15.70:6805/2990268368","state":["exists","up"]},{"osd":5,"uuid":"976356b7-2f04-4f78-9e66-2ab3d912dd7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6825","nonce":2669649466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6827","nonce":2669649466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6831","nonce":2669649466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2669649466},{"type":"v1","addr":"172.21.15.70:6829","nonce":2669649466}]},"public_addr":"172.21.15.70:6825/2669649466","cluster_addr":"172.21.15.70:6827/2669649466","heartbeat_back_addr":"172.21.15.70:6831/2669649466","heartbeat_front_addr":"172.21.15.70:6829/2669649466","state":["exists","up"]},{"osd":6,"uuid":"0c1ed442-a009-4274-9150-7c91a33f76f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3978225537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6819","nonce":3978225537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6823","nonce":3978225537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3978225537},{"type":"v1","addr":"172.21.15.70:6821","nonce":3978225537}]},"public_addr":"172.21.15.70:6817/3978225537","cluster_addr":"172.21.15.70:6819/3978225537","heartbeat_back_addr":"172.21.15.70:6823/3978225537","heartbeat_front_addr":"172.21.15.70:6821/3978225537","state":["exists","up"]},{"osd":7,"uuid":"a7f357dd-1260-444b-ac01-702f14cab909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6809","nonce":3218842598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6811","nonce":3218842598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6815","nonce":3218842598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3218842598},{"type":"v1","addr":"172.21.15.70:6813","nonce":3218842598}]},"public_addr":"172.21.15.70:6809/3218842598","cluster_addr":"172.21.15.70:6811/3218842598","heartbeat_back_addr":"172.21.15.70:6815/3218842598","heartbeat_front_addr":"172.21.15.70:6813/3218842598","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.093511+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.012825+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.099688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.203501+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.674563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.857981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:17.095132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:46:16.737181+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6836/3004970849":"2024-09-11T07:54:07.712495+0000","172.21.15.3:6834/3004970849":"2024-09-11T07:54:07.712495+0000","172.21.15.3:6837/3773991953":"2024-09-11T07:54:07.712500+0000","172.21.15.70:6835/3268462742":"2024-09-11T07:54:07.712482+0000","172.21.15.70:6833/3268462742":"2024-09-11T07:54:07.712482+0000","172.21.15.3:6835/3773991953":"2024-09-11T07:54:07.712500+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 msgr2=0x7fb7fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 0x7fb7fc06e320 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb808034000 tx=0x7fb80802e000 comp rx=0 tx=0).stop 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 msgr2=0x7fb81814ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c400 tx=0x7fb80c00c8c0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 shutdown_connections 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb7fc06be60 0x7fb7fc06e320 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb818161de0 0x7fb81815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb818161a00 0x7fb81815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 --2- 172.21.15.3:0/4054452347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8180d10f0 0x7fb81814ecb0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:15.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 >> 172.21.15.3:0/4054452347 conn(0x7fb818072770 msgr2=0x7fb8180b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:15.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 shutdown_connections 2024-09-10T07:54:15.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:15.292+0000 7fb81f200640 1 -- 172.21.15.3:0/4054452347 wait complete. 2024-09-10T07:54:16.308 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T07:54:16.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.480+0000 7fcdc1e00640 1 Processor -- start 2024-09-10T07:54:16.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.480+0000 7fcdc1e00640 1 -- start start 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 0x7fcdbc15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdc1e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdbc0cd050 0x7fcdbc0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdc1e00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdbc0ce9a0 con 0x7fcdbc0b1f90 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdc1e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcdbc0cec80 con 0x7fcdbc0cd050 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdc1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc15efb0 con 0x7fcdbc158bf0 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdbc0cd050 0x7fcdbc0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdbc0cd050 0x7fcdbc0ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41648/0 (socket says 172.21.15.3:41648) 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 -- 172.21.15.3:0/253034680 learned_addr learned my addr 172.21.15.3:0/253034680 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdbb400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcdbc0b1f90 0x7fcdbc162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42424/0 (socket says 172.21.15.3:42424) 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdbaa00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 0x7fcdbc15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 -- 172.21.15.3:0/253034680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 msgr2=0x7fcdbc15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 0x7fcdbc15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdb3e00640 1 -- 172.21.15.3:0/253034680 >> v1:172.21.15.3:6789/0 conn(0x7fcdbc0b1f90 legacy=0x7fcdbc162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.482+0000 7fcdba000640 1 -- 172.21.15.3:0/253034680 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835963392 0 0) 0x7fcdbc0ce9a0 con 0x7fcdbc0b1f90 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.483+0000 7fcdb3e00640 1 -- 172.21.15.3:0/253034680 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc15f290 con 0x7fcdbc0cd050 2024-09-10T07:54:16.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.483+0000 7fcdbaa00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 0x7fcdbc15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:16.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdb3e00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0ce150 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009870 tx=0x7fcda402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142f54ec8e23e838 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdba000640 1 -- 172.21.15.3:0/253034680 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4037070 con 0x7fcdbc0cd050 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdba000640 1 -- 172.21.15.3:0/253034680 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda402ae50 con 0x7fcdbc0cd050 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdba000640 1 -- 172.21.15.3:0/253034680 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4032860 con 0x7fcdbc0cd050 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 -- 172.21.15.3:0/253034680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0cd050 msgr2=0x7fcdbc0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0ce150 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009870 tx=0x7fcda402a300 comp rx=0 tx=0).stop 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 -- 172.21.15.3:0/253034680 shutdown_connections 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0ce150 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/253034680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc158bf0 0x7fcdbc15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.484+0000 7fcdc1e00640 1 -- 172.21.15.3:0/253034680 >> 172.21.15.3:0/253034680 conn(0x7fcdbc062cc0 msgr2=0x7fcdbc06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:16.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.485+0000 7fcdc1e00640 1 -- 172.21.15.3:0/253034680 shutdown_connections 2024-09-10T07:54:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.485+0000 7fcdc1e00640 1 -- 172.21.15.3:0/253034680 wait complete. 2024-09-10T07:54:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.486+0000 7fcdc1e00640 1 Processor -- start 2024-09-10T07:54:16.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.486+0000 7fcdc1e00640 1 -- start start 2024-09-10T07:54:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc0b1f90 0x7fcdbc153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 0x7fcdbc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc158fd0 con 0x7fcdbc158bf0 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0ce690 con 0x7fcdbc0cd050 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.487+0000 7fcdc1e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc159150 con 0x7fcdbc0b1f90 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 0x7fcdbc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 0x7fcdbc15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41664/0 (socket says 172.21.15.3:41664) 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 -- 172.21.15.3:0/278937768 learned_addr learned my addr 172.21.15.3:0/278937768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbaa00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc0b1f90 0x7fcdbc153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdb3e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 -- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc0b1f90 msgr2=0x7fcdbc153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcdbc0b1f90 0x7fcdbc153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 -- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdbc0cd050 msgr2=0x7fcdbc0728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdbc0cd050 0x7fcdbc0728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc159760 con 0x7fcdbc158bf0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.488+0000 7fcdbb400640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 0x7fcdbc15a800 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c7d0 tx=0x7fcdb400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4018070 con 0x7fcdbc158bf0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb400a640 con 0x7fcdbc158bf0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400aa70 con 0x7fcdbc158bf0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc159a40 con 0x7fcdbc158bf0 2024-09-10T07:54:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc15a2e0 con 0x7fcdbc158bf0 2024-09-10T07:54:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.489+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcdbc159e40 con 0x7fcdbc158bf0 2024-09-10T07:54:16.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.491+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdb401b070 con 0x7fcdbc158bf0 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.491+0000 7fcdb3400640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.491+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fcdb408a250 con 0x7fcdbc158bf0 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdbaa00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdb3400640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7fcd98071e30 0x7fcd98074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd98074900 con 0x7fcd98071e30 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7fcdb4091050 con 0x7fcdbc158bf0 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdb3e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7fcd98071e30 0x7fcd98074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.492+0000 7fcdbaa00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006fd0 tx=0x7fcdac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.493+0000 7fcdb3e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7fcd98071e30 0x7fcd98074250 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.493+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== osd.0 v2:172.21.15.3:6808/1494406974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd98074900 con 0x7fcd98071e30 2024-09-10T07:54:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.507+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 --> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcdbc15cd90 con 0x7fcd98071e30 2024-09-10T07:54:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.507+0000 7fcdb3400640 1 -- 172.21.15.3:0/278937768 <== osd.0 v2:172.21.15.3:6808/1494406974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcdbc15cd90 con 0x7fcd98071e30 2024-09-10T07:54:16.507 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:16.507 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.507+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7fcd98071e30 msgr2=0x7fcd98074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.507+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6808/1494406974,v1:172.21.15.3:6809/1494406974] conn(0x7fcd98071e30 0x7fcd98074250 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 msgr2=0x7fcd9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fcd9806be30 0x7fcd9806e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006fd0 tx=0x7fcdac008040 comp rx=0 tx=0).stop 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 msgr2=0x7fcdbc15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 --2- 172.21.15.3:0/278937768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc158bf0 0x7fcdbc15a800 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c7d0 tx=0x7fcdb400cc90 comp rx=0 tx=0).stop 2024-09-10T07:54:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdbb400640 1 -- 172.21.15.3:0/278937768 reap_dead start 2024-09-10T07:54:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 shutdown_connections 2024-09-10T07:54:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 >> 172.21.15.3:0/278937768 conn(0x7fcdbc062cc0 msgr2=0x7fcdbc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 shutdown_connections 2024-09-10T07:54:16.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.508+0000 7fcdc1e00640 1 -- 172.21.15.3:0/278937768 wait complete. 2024-09-10T07:54:16.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T07:54:16.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.679+0000 7f29ff800640 1 Processor -- start 2024-09-10T07:54:16.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.679+0000 7f29ff800640 1 -- start start 2024-09-10T07:54:16.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29ff800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29f8161dd0 0x7f29f806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29ff800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f806fe10 con 0x7f29f806e390 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29ff800640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f80b47d0 con 0x7f29f80b5ea0 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29ff800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f29f806c870 con 0x7f29f8161dd0 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29fce00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29f8161dd0 0x7f29f806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29fce00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29f8161dd0 0x7f29f806f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41666/0 (socket says 172.21.15.3:41666) 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29fce00640 1 -- 172.21.15.3:0/1686600977 learned_addr learned my addr 172.21.15.3:0/1686600977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29fce00640 1 -- 172.21.15.3:0/1686600977 >> v1:172.21.15.70:6789/0 conn(0x7f29f80b5ea0 legacy=0x7f29f80b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29fce00640 1 -- 172.21.15.3:0/1686600977 >> v1:172.21.15.3:6789/0 conn(0x7f29f806e390 legacy=0x7f29f806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.682+0000 7f29fce00640 1 -- 172.21.15.3:0/1686600977 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f806cb50 con 0x7f29f8161dd0 2024-09-10T07:54:16.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.681+0000 7f29f7400640 1 -- 172.21.15.3:0/1686600977 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925493849 0 0) 0x7f29f806fe10 con 0x7f29f806e390 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.682+0000 7f29fce00640 1 --2- 172.21.15.3:0/1686600977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f806f8d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009870 tx=0x7f29ec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d05ed7fcc418333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29f7400640 1 -- 172.21.15.3:0/1686600977 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec037070 con 0x7f29f8161dd0 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29f7400640 1 -- 172.21.15.3:0/1686600977 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ec02ae50 con 0x7f29f8161dd0 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29f7400640 1 -- 172.21.15.3:0/1686600977 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec032860 con 0x7f29f8161dd0 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 -- 172.21.15.3:0/1686600977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 msgr2=0x7f29f806f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 --2- 172.21.15.3:0/1686600977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f806f8d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009870 tx=0x7f29ec02a300 comp rx=0 tx=0).stop 2024-09-10T07:54:16.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 -- 172.21.15.3:0/1686600977 shutdown_connections 2024-09-10T07:54:16.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 --2- 172.21.15.3:0/1686600977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f806f8d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 -- 172.21.15.3:0/1686600977 >> 172.21.15.3:0/1686600977 conn(0x7f29f80b3540 msgr2=0x7f29f80b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:16.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 -- 172.21.15.3:0/1686600977 shutdown_connections 2024-09-10T07:54:16.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.683+0000 7f29ff800640 1 -- 172.21.15.3:0/1686600977 wait complete. 2024-09-10T07:54:16.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.685+0000 7f29ff800640 1 Processor -- start 2024-09-10T07:54:16.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.685+0000 7f29ff800640 1 -- start start 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29f806e390 0x7f29f8156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f80b5ea0 0x7f29f8142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29f80b43f0 con 0x7f29f8161dd0 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29f80b06c0 con 0x7f29f80b5ea0 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29ff800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29f80b4570 con 0x7f29f806e390 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.686+0000 7f29fd800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fce00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29f806e390 0x7f29f8156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41670/0 (socket says 172.21.15.3:41670) 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 -- 172.21.15.3:0/1447864047 learned_addr learned my addr 172.21.15.3:0/1447864047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29f7e00640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f80b5ea0 0x7f29f8142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 -- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29f806e390 msgr2=0x7f29f8156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29f806e390 0x7f29f8156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 -- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f80b5ea0 msgr2=0x7f29f8142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f80b5ea0 0x7f29f8142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 -- 172.21.15.3:0/1447864047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f81567a0 con 0x7f29f8161dd0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29fd800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f29f000c760 tx=0x7f29f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0003da0 con 0x7f29f8161dd0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f0005070 con 0x7f29f8161dd0 2024-09-10T07:54:16.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.687+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0005460 con 0x7f29f8161dd0 2024-09-10T07:54:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.688+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f8144950 con 0x7f29f8161dd0 2024-09-10T07:54:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.688+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f8144f10 con 0x7f29f8161dd0 2024-09-10T07:54:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.689+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f80b1ef0 con 0x7f29f8161dd0 2024-09-10T07:54:16.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.690+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29f0005600 con 0x7f29f8161dd0 2024-09-10T07:54:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.690+0000 7f29f4c00640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 0x7f29d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.690+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f29f008aab0 con 0x7f29f8161dd0 2024-09-10T07:54:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.691+0000 7f29fce00640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 0x7f29d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:16.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.691+0000 7f29fce00640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 0x7f29d806e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f29ec034000 tx=0x7f29ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.695+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f29f0058fc0 con 0x7f29f8161dd0 2024-09-10T07:54:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.879+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f29f806f850 con 0x7f29d806be30 2024-09-10T07:54:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.879+0000 7f29f4c00640 1 -- 172.21.15.3:0/1447864047 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f29f806f850 con 0x7f29d806be30 2024-09-10T07:54:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 msgr2=0x7f29d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 0x7f29d806e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f29ec034000 tx=0x7f29ec02e000 comp rx=0 tx=0).stop 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 msgr2=0x7f29f81459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f29f000c760 tx=0x7f29f000cc20 comp rx=0 tx=0).stop 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 shutdown_connections 2024-09-10T07:54:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f29d806be30 0x7f29d806e2f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f8161dd0 0x7f29f81459f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f80b5ea0 0x7f29f8142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 --2- 172.21.15.3:0/1447864047 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29f806e390 0x7f29f8156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 >> 172.21.15.3:0/1447864047 conn(0x7f29f80b3540 msgr2=0x7f29f80afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 shutdown_connections 2024-09-10T07:54:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:16.881+0000 7f29ff800640 1 -- 172.21.15.3:0/1447864047 wait complete. 2024-09-10T07:54:16.897 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T07:54:16.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.059+0000 7f3b3a800640 1 Processor -- start 2024-09-10T07:54:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.059+0000 7f3b3a800640 1 -- start start 2024-09-10T07:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b3a800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b341591b0 0x7f3b3415c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b3a800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd360 0x7f3b340b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b3a800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340cd8d0 con 0x7f3b340b3e10 2024-09-10T07:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b3a800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3415b900 con 0x7f3b341591b0 2024-09-10T07:54:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b3a800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3b340cdbb0 con 0x7f3b340cd360 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b33400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b341591b0 0x7f3b3415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b33400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3b341591b0 0x7f3b3415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41676/0 (socket says 172.21.15.3:41676) 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b2be00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3b340b3e10 0x7f3b340ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42436/0 (socket says 172.21.15.3:42436) 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.060+0000 7f3b33400640 1 -- 172.21.15.3:0/967333174 learned_addr learned my addr 172.21.15.3:0/967333174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b33e00640 1 --2- 172.21.15.3:0/967333174 >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd360 0x7f3b340b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b33400640 1 -- 172.21.15.3:0/967333174 >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd360 msgr2=0x7f3b340b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b33400640 1 --2- 172.21.15.3:0/967333174 >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd360 0x7f3b340b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b33400640 1 -- 172.21.15.3:0/967333174 >> v1:172.21.15.3:6789/0 conn(0x7f3b340b3e10 legacy=0x7f3b340ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b33400640 1 -- 172.21.15.3:0/967333174 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3415bb20 con 0x7f3b341591b0 2024-09-10T07:54:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.061+0000 7f3b32a00640 1 -- 172.21.15.3:0/967333174 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435197975 0 0) 0x7f3b340cd8d0 con 0x7f3b340b3e10 2024-09-10T07:54:17.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.062+0000 7f3b33400640 1 --2- 172.21.15.3:0/967333174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341591b0 0x7f3b3415c600 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b340afd90 tx=0x7f3b2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9b2c96b8aea331 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.062+0000 7f3b32a00640 1 -- 172.21.15.3:0/967333174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24037070 con 0x7f3b341591b0 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.062+0000 7f3b32a00640 1 -- 172.21.15.3:0/967333174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24032440 con 0x7f3b341591b0 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.062+0000 7f3b32a00640 1 -- 172.21.15.3:0/967333174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24032810 con 0x7f3b341591b0 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 -- 172.21.15.3:0/967333174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341591b0 msgr2=0x7f3b3415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 --2- 172.21.15.3:0/967333174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341591b0 0x7f3b3415c600 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b340afd90 tx=0x7f3b2402a270 comp rx=0 tx=0).stop 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 -- 172.21.15.3:0/967333174 shutdown_connections 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 --2- 172.21.15.3:0/967333174 >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd360 0x7f3b340b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 --2- 172.21.15.3:0/967333174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341591b0 0x7f3b3415c600 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 -- 172.21.15.3:0/967333174 >> 172.21.15.3:0/967333174 conn(0x7f3b34062cc0 msgr2=0x7f3b3406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 -- 172.21.15.3:0/967333174 shutdown_connections 2024-09-10T07:54:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.063+0000 7f3b3a800640 1 -- 172.21.15.3:0/967333174 wait complete. 2024-09-10T07:54:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.065+0000 7f3b3a800640 1 Processor -- start 2024-09-10T07:54:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.065+0000 7f3b3a800640 1 -- start start 2024-09-10T07:54:17.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3b340b3e10 0x7f3b341566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 0x7f3b340ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b3415e5a0 0x7f3b340aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340ce410 con 0x7f3b340cd360 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340ce290 con 0x7f3b3415e5a0 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b3a800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340ce590 con 0x7f3b340b3e10 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 0x7f3b340ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 0x7f3b340ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41688/0 (socket says 172.21.15.3:41688) 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 -- 172.21.15.3:0/1510949587 learned_addr learned my addr 172.21.15.3:0/1510949587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b33e00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b3415e5a0 0x7f3b340aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b33400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3b340b3e10 0x7f3b341566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 -- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3b340b3e10 msgr2=0x7f3b341566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3b340b3e10 0x7f3b341566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 -- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b3415e5a0 msgr2=0x7f3b340aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b3415e5a0 0x7f3b340aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b2be00640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3415d500 con 0x7f3b340cd360 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b33e00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b3415e5a0 0x7f3b340aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.066+0000 7f3b33400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3b340b3e10 0x7f3b341566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b2be00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 0x7f3b340ae8a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c500 tx=0x7f3b1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c018070 con 0x7f3b340cd360 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c007580 con 0x7f3b340cd360 2024-09-10T07:54:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004a20 con 0x7f3b340cd360 2024-09-10T07:54:17.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3415d7e0 con 0x7f3b340cd360 2024-09-10T07:54:17.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.067+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3415dda0 con 0x7f3b340cd360 2024-09-10T07:54:17.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.068+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b3415e180 con 0x7f3b340cd360 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.069+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b1c004bc0 con 0x7f3b340cd360 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b2b400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b1006be30 0x7f3b1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f3b1c089f10 con 0x7f3b340cd360 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b2b400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f3b10071e30 0x7f3b10074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b33400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b1006be30 0x7f3b1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b1c01de50 con 0x7f3b10071e30 2024-09-10T07:54:17.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c08c530 con 0x7f3b340cd360 2024-09-10T07:54:17.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.070+0000 7f3b33e00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f3b10071e30 0x7f3b10074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.071+0000 7f3b33400640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b1006be30 0x7f3b1006e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034000 tx=0x7f3b2402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.071+0000 7f3b33e00640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f3b10071e30 0x7f3b10074250 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.072+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== osd.1 v2:172.21.15.3:6800/2429800591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b1c01de50 con 0x7f3b10071e30 2024-09-10T07:54:17.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.085+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 --> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3b3415b220 con 0x7f3b10071e30 2024-09-10T07:54:17.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.085+0000 7f3b2b400640 1 -- 172.21.15.3:0/1510949587 <== osd.1 v2:172.21.15.3:6800/2429800591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3b3415b220 con 0x7f3b10071e30 2024-09-10T07:54:17.085 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:17.086 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f3b10071e30 msgr2=0x7f3b10074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6800/2429800591,v1:172.21.15.3:6801/2429800591] conn(0x7f3b10071e30 0x7f3b10074250 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b1006be30 msgr2=0x7f3b1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f3b1006be30 0x7f3b1006e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034000 tx=0x7f3b2402e000 comp rx=0 tx=0).stop 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 msgr2=0x7f3b340ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 --2- 172.21.15.3:0/1510949587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b340cd360 0x7f3b340ae8a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c500 tx=0x7f3b1c00c9c0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b33e00640 1 -- 172.21.15.3:0/1510949587 reap_dead start 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 shutdown_connections 2024-09-10T07:54:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 >> 172.21.15.3:0/1510949587 conn(0x7f3b34062cc0 msgr2=0x7f3b340d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 shutdown_connections 2024-09-10T07:54:17.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.086+0000 7f3b3a800640 1 -- 172.21.15.3:0/1510949587 wait complete. 2024-09-10T07:54:17.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T07:54:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.259+0000 7f639be00640 1 Processor -- start 2024-09-10T07:54:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.259+0000 7f639be00640 1 -- start start 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f639c15e860 0x7f639c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639be00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f639c0cee40 con 0x7f639c0cfbf0 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639be00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f639c0cf120 con 0x7f639c073150 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639be00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f639c0d3780 con 0x7f639c15e860 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f639c15e860 0x7f639c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639aa00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f639c15e860 0x7f639c0cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41694/0 (socket says 172.21.15.3:41694) 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639a000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f639c0cfbf0 0x7f639c0d3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42444/0 (socket says 172.21.15.3:42444) 2024-09-10T07:54:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.260+0000 7f639aa00640 1 -- 172.21.15.3:0/1174034624 learned_addr learned my addr 172.21.15.3:0/1174034624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.261+0000 7f639aa00640 1 -- 172.21.15.3:0/1174034624 >> v1:172.21.15.70:6789/0 conn(0x7f639c073150 legacy=0x7f639c0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.261+0000 7f639aa00640 1 -- 172.21.15.3:0/1174034624 >> v1:172.21.15.3:6789/0 conn(0x7f639c0cfbf0 legacy=0x7f639c0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.261+0000 7f639aa00640 1 -- 172.21.15.3:0/1174034624 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c0d3410 con 0x7f639c15e860 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.262+0000 7f639aa00640 1 --2- 172.21.15.3:0/1174034624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0cf6b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f638c009870 tx=0x7f638c02a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5c91ce116243b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.262+0000 7f6399600640 1 -- 172.21.15.3:0/1174034624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c037070 con 0x7f639c15e860 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.262+0000 7f6399600640 1 -- 172.21.15.3:0/1174034624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c032640 con 0x7f639c15e860 2024-09-10T07:54:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.262+0000 7f6399600640 1 -- 172.21.15.3:0/1174034624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c032a10 con 0x7f639c15e860 2024-09-10T07:54:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 -- 172.21.15.3:0/1174034624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 msgr2=0x7f639c0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 --2- 172.21.15.3:0/1174034624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0cf6b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f638c009870 tx=0x7f638c02a2f0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 -- 172.21.15.3:0/1174034624 shutdown_connections 2024-09-10T07:54:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 --2- 172.21.15.3:0/1174034624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0cf6b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 -- 172.21.15.3:0/1174034624 >> 172.21.15.3:0/1174034624 conn(0x7f639c0d4650 msgr2=0x7f639c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 -- 172.21.15.3:0/1174034624 shutdown_connections 2024-09-10T07:54:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.263+0000 7f639be00640 1 -- 172.21.15.3:0/1174034624 wait complete. 2024-09-10T07:54:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.264+0000 7f639be00640 1 Processor -- start 2024-09-10T07:54:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.265+0000 7f639be00640 1 -- start start 2024-09-10T07:54:17.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.265+0000 7f639be00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639c073150 0x7f639c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.265+0000 7f639be00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f639c0cfbf0 0x7f639c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f639c0d2460 con 0x7f639c15e860 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639be00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f639c0cf400 con 0x7f639c0cfbf0 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639be00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f639c0b7f90 con 0x7f639c073150 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41696/0 (socket says 172.21.15.3:41696) 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 -- 172.21.15.3:0/147808566 learned_addr learned my addr 172.21.15.3:0/147808566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639aa00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639c073150 0x7f639c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639a000640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f639c0cfbf0 0x7f639c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 -- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639c073150 msgr2=0x7f639c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639c073150 0x7f639c0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 -- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f639c0cfbf0 msgr2=0x7f639c154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f639c0cfbf0 0x7f639c154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.266+0000 7f639b400640 1 -- 172.21.15.3:0/147808566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c0ac820 con 0x7f639c15e860 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f639b400640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f639400c7d0 tx=0x7f639400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394003b70 con 0x7f639c15e860 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6394003d30 con 0x7f639c15e860 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400aa90 con 0x7f639c15e860 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c0acb00 con 0x7f639c15e860 2024-09-10T07:54:17.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.267+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c0ad0c0 con 0x7f639c15e860 2024-09-10T07:54:17.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.268+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c0cf6b0 con 0x7f639c15e860 2024-09-10T07:54:17.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.269+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f639400ac30 con 0x7f639c15e860 2024-09-10T07:54:17.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.270+0000 7f6392a00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.270+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f639408ab30 con 0x7f639c15e860 2024-09-10T07:54:17.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.270+0000 7f639aa00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.271+0000 7f639aa00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 0x7f637c06e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f638c034000 tx=0x7f638c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.274+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6394059720 con 0x7f639c15e860 2024-09-10T07:54:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.460+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f639c0d2bb0 con 0x7f637c06be60 2024-09-10T07:54:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.461+0000 7f6392a00640 1 -- 172.21.15.3:0/147808566 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f639c0d2bb0 con 0x7f637c06be60 2024-09-10T07:54:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T07:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 msgr2=0x7f637c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 0x7f637c06e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f638c034000 tx=0x7f638c02e000 comp rx=0 tx=0).stop 2024-09-10T07:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 msgr2=0x7f639c0abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f639400c7d0 tx=0x7f639400cc90 comp rx=0 tx=0).stop 2024-09-10T07:54:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 shutdown_connections 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f639c15e860 0x7f639c0abeb0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f639c0cfbf0 0x7f639c154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 --2- 172.21.15.3:0/147808566 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f639c073150 0x7f639c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 >> 172.21.15.3:0/147808566 conn(0x7f639c0d4650 msgr2=0x7f639c0b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 shutdown_connections 2024-09-10T07:54:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.463+0000 7f639be00640 1 -- 172.21.15.3:0/147808566 wait complete. 2024-09-10T07:54:17.478 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T07:54:17.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.638+0000 7ffaafe00640 1 Processor -- start 2024-09-10T07:54:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.638+0000 7ffaafe00640 1 -- start start 2024-09-10T07:54:17.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.639+0000 7ffaafe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffaa8158bf0 0x7ffaa815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaafe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffaa80cd050 0x7ffaa80ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaafe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa80ce9a0 con 0x7ffaa80b1f90 2024-09-10T07:54:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaafe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa80cec80 con 0x7ffaa80cd050 2024-09-10T07:54:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaafe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa815efb0 con 0x7ffaa8158bf0 2024-09-10T07:54:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaaca00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffaa80cd050 0x7ffaa80ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaade00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffaa80b1f90 0x7ffaa8162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42456/0 (socket says 172.21.15.3:42456) 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaade00640 1 -- 172.21.15.3:0/362580452 learned_addr learned my addr 172.21.15.3:0/362580452 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaad400640 1 --2- 172.21.15.3:0/362580452 >> v2:172.21.15.70:3300/0 conn(0x7ffaa8158bf0 0x7ffaa815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaaca00640 1 -- 172.21.15.3:0/362580452 >> v2:172.21.15.70:3300/0 conn(0x7ffaa8158bf0 msgr2=0x7ffaa815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaaca00640 1 --2- 172.21.15.3:0/362580452 >> v2:172.21.15.70:3300/0 conn(0x7ffaa8158bf0 0x7ffaa815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaaca00640 1 -- 172.21.15.3:0/362580452 >> v1:172.21.15.3:6789/0 conn(0x7ffaa80b1f90 legacy=0x7ffaa8162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaaca00640 1 -- 172.21.15.3:0/362580452 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa815f290 con 0x7ffaa80cd050 2024-09-10T07:54:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.640+0000 7ffaa7e00640 1 -- 172.21.15.3:0/362580452 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860548407 0 0) 0x7ffaa80ce9a0 con 0x7ffaa80b1f90 2024-09-10T07:54:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.641+0000 7ffaaca00640 1 --2- 172.21.15.3:0/362580452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa80cd050 0x7ffaa80ce150 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa806cb30 tx=0x7ffa9402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3a218a7df8e54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaa7e00640 1 -- 172.21.15.3:0/362580452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94037070 con 0x7ffaa80cd050 2024-09-10T07:54:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaa7e00640 1 -- 172.21.15.3:0/362580452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa94032440 con 0x7ffaa80cd050 2024-09-10T07:54:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaa7e00640 1 -- 172.21.15.3:0/362580452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94032810 con 0x7ffaa80cd050 2024-09-10T07:54:17.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 -- 172.21.15.3:0/362580452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa80cd050 msgr2=0x7ffaa80ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 --2- 172.21.15.3:0/362580452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa80cd050 0x7ffaa80ce150 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa806cb30 tx=0x7ffa9402a230 comp rx=0 tx=0).stop 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 -- 172.21.15.3:0/362580452 shutdown_connections 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 --2- 172.21.15.3:0/362580452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa80cd050 0x7ffaa80ce150 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 --2- 172.21.15.3:0/362580452 >> v2:172.21.15.70:3300/0 conn(0x7ffaa8158bf0 0x7ffaa815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 -- 172.21.15.3:0/362580452 >> 172.21.15.3:0/362580452 conn(0x7ffaa8062cc0 msgr2=0x7ffaa806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 -- 172.21.15.3:0/362580452 shutdown_connections 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.642+0000 7ffaafe00640 1 -- 172.21.15.3:0/362580452 wait complete. 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.644+0000 7ffaafe00640 1 Processor -- start 2024-09-10T07:54:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.644+0000 7ffaafe00640 1 -- start start 2024-09-10T07:54:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80b1f90 0x7ffaa81508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa80cd050 0x7ffaa81f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 0x7ffaa81f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa8158fd0 con 0x7ffaa8158bf0 2024-09-10T07:54:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80ce690 con 0x7ffaa80b1f90 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaafe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa8159150 con 0x7ffaa80cd050 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaade00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 0x7ffaa81f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaade00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 0x7ffaa81f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41714/0 (socket says 172.21.15.3:41714) 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.645+0000 7ffaade00640 1 -- 172.21.15.3:0/1501150259 learned_addr learned my addr 172.21.15.3:0/1501150259 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaad400640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80b1f90 0x7ffaa81508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaaca00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa80cd050 0x7ffaa81f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 -- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa80cd050 msgr2=0x7ffaa81f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffaa80cd050 0x7ffaa81f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 -- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80b1f90 msgr2=0x7ffaa81508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80b1f90 0x7ffaa81508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa81f8470 con 0x7ffaa8158bf0 2024-09-10T07:54:17.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaad400640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80b1f90 0x7ffaa81508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:17.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.646+0000 7ffaade00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 0x7ffaa81f4940 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c7a0 tx=0x7ffaa000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.648+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0018070 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.648+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa000a640 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.648+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000aab0 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.648+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa81547d0 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.648+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa8162a20 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.649+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaa80ce150 con 0x7ffaa8158bf0 2024-09-10T07:54:17.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.650+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaa001b070 con 0x7ffaa8158bf0 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.651+0000 7ffaa5600640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa8806be30 0x7ffa8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.651+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7ffaa008a1b0 con 0x7ffaa8158bf0 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaa5600640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7ffa88071e30 0x7ffa88074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffaa001be50 con 0x7ffa88071e30 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7ffaa0091050 con 0x7ffaa8158bf0 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaad400640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa8806be30 0x7ffa8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaaca00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7ffa88071e30 0x7ffa88074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaad400640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa8806be30 0x7ffa8806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8161ae0 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.652+0000 7ffaaca00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7ffa88071e30 0x7ffa88074250 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.653+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== osd.2 v2:172.21.15.3:6824/2150697764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffaa001be50 con 0x7ffa88071e30 2024-09-10T07:54:17.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.667+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 --> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffaa8158200 con 0x7ffa88071e30 2024-09-10T07:54:17.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.667+0000 7ffaa5600640 1 -- 172.21.15.3:0/1501150259 <== osd.2 v2:172.21.15.3:6824/2150697764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffaa8158200 con 0x7ffa88071e30 2024-09-10T07:54:17.667 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:17.667 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:17.667 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7ffa88071e30 msgr2=0x7ffa88074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6824/2150697764,v1:172.21.15.3:6825/2150697764] conn(0x7ffa88071e30 0x7ffa88074250 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa8806be30 msgr2=0x7ffa8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7ffa8806be30 0x7ffa8806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8161ae0 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 msgr2=0x7ffaa81f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 --2- 172.21.15.3:0/1501150259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffaa8158bf0 0x7ffaa81f4940 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c7a0 tx=0x7ffaa000cc60 comp rx=0 tx=0).stop 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaade00640 1 -- 172.21.15.3:0/1501150259 reap_dead start 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 shutdown_connections 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 >> 172.21.15.3:0/1501150259 conn(0x7ffaa8062cc0 msgr2=0x7ffaa80cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 shutdown_connections 2024-09-10T07:54:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.668+0000 7ffaafe00640 1 -- 172.21.15.3:0/1501150259 wait complete. 2024-09-10T07:54:17.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T07:54:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.841+0000 7f7912e00640 1 Processor -- start 2024-09-10T07:54:17.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.841+0000 7f7912e00640 1 -- start start 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7912e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f790c15e860 0x7f790c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7912e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c0cfbf0 0x7f790c0d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7912e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f790c0cee40 con 0x7f790c0d4150 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7912e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f790c0d3780 con 0x7f790c0cfbf0 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7912e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f790c0cf120 con 0x7f790c15e860 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7911000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c0cfbf0 0x7f790c0d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7911000640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f790c0cfbf0 0x7f790c0d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41722/0 (socket says 172.21.15.3:41722) 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7911000640 1 -- 172.21.15.3:0/2018518896 learned_addr learned my addr 172.21.15.3:0/2018518896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.842+0000 7f7911a00640 1 --2- 172.21.15.3:0/2018518896 >> v2:172.21.15.70:3300/0 conn(0x7f790c15e860 0x7f790c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.843+0000 7f7911000640 1 -- 172.21.15.3:0/2018518896 >> v2:172.21.15.70:3300/0 conn(0x7f790c15e860 msgr2=0x7f790c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.843+0000 7f7911000640 1 --2- 172.21.15.3:0/2018518896 >> v2:172.21.15.70:3300/0 conn(0x7f790c15e860 0x7f790c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.843+0000 7f7911000640 1 -- 172.21.15.3:0/2018518896 >> v1:172.21.15.70:6789/0 conn(0x7f790c0d4150 legacy=0x7f790c0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.843+0000 7f7911000640 1 -- 172.21.15.3:0/2018518896 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c0d3410 con 0x7f790c0cfbf0 2024-09-10T07:54:17.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f7911000640 1 --2- 172.21.15.3:0/2018518896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c0cfbf0 0x7f790c0d3c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f790c0ba7c0 tx=0x7f78f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242eb70edd2dfe1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f790be00640 1 -- 172.21.15.3:0/2018518896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8037070 con 0x7f790c0cfbf0 2024-09-10T07:54:17.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f790be00640 1 -- 172.21.15.3:0/2018518896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f8032440 con 0x7f790c0cfbf0 2024-09-10T07:54:17.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f790be00640 1 -- 172.21.15.3:0/2018518896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8032810 con 0x7f790c0cfbf0 2024-09-10T07:54:17.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f7912e00640 1 -- 172.21.15.3:0/2018518896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c0cfbf0 msgr2=0x7f790c0d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.844+0000 7f7912e00640 1 --2- 172.21.15.3:0/2018518896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c0cfbf0 0x7f790c0d3c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f790c0ba7c0 tx=0x7f78f802a270 comp rx=0 tx=0).stop 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 -- 172.21.15.3:0/2018518896 shutdown_connections 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 --2- 172.21.15.3:0/2018518896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c0cfbf0 0x7f790c0d3c10 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 --2- 172.21.15.3:0/2018518896 >> v2:172.21.15.70:3300/0 conn(0x7f790c15e860 0x7f790c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 -- 172.21.15.3:0/2018518896 >> 172.21.15.3:0/2018518896 conn(0x7f790c0d4650 msgr2=0x7f790c0d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 -- 172.21.15.3:0/2018518896 shutdown_connections 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.845+0000 7f7912e00640 1 -- 172.21.15.3:0/2018518896 wait complete. 2024-09-10T07:54:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.846+0000 7f7912e00640 1 Processor -- start 2024-09-10T07:54:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.846+0000 7f7912e00640 1 -- start start 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 0x7f790c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 0x7f790c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f790c0b7f90 con 0x7f790c15e860 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f790c0d2460 con 0x7f790c0d4150 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f790c0b8110 con 0x7f790c0cfbf0 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.847+0000 7f7912400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41732/0 (socket says 172.21.15.3:41732) 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 -- 172.21.15.3:0/553260774 learned_addr learned my addr 172.21.15.3:0/553260774 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7911a00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 0x7f790c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7911000640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 0x7f790c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 -- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 msgr2=0x7f790c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 0x7f790c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 -- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 msgr2=0x7f790c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 0x7f790c1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 -- 172.21.15.3:0/553260774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c0afdd0 con 0x7f790c15e860 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7911000640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 0x7f790c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7911a00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 0x7f790c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912400640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f790400c6c0 tx=0x7f790400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904003b70 con 0x7f790c15e860 2024-09-10T07:54:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7904003d10 con 0x7f790c15e860 2024-09-10T07:54:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c15d4d0 con 0x7f790c15e860 2024-09-10T07:54:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.848+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79040052f0 con 0x7f790c15e860 2024-09-10T07:54:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.849+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c160eb0 con 0x7f790c15e860 2024-09-10T07:54:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.849+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c0d3c10 con 0x7f790c15e860 2024-09-10T07:54:17.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.850+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7904005490 con 0x7f790c15e860 2024-09-10T07:54:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.852+0000 7f7909600640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 0x7f78ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.852+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f790408ab40 con 0x7f790c15e860 2024-09-10T07:54:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.853+0000 7f7911a00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 0x7f78ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.853+0000 7f7911a00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 0x7f78ec06e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7900009a20 tx=0x7f7900008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:17.857+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f790c0afdd0 con 0x7f790c15e860 2024-09-10T07:54:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.044+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f790c0b7d70 con 0x7f78ec06be30 2024-09-10T07:54:18.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.045+0000 7f7909600640 1 -- 172.21.15.3:0/553260774 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f790c0b7d70 con 0x7f78ec06be30 2024-09-10T07:54:18.045 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 msgr2=0x7f78ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 0x7f78ec06e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7900009a20 tx=0x7f7900008040 comp rx=0 tx=0).stop 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 msgr2=0x7f790c0adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f790400c6c0 tx=0x7f790400cb80 comp rx=0 tx=0).stop 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 shutdown_connections 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f78ec06be30 0x7f78ec06e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f790c15e860 0x7f790c0adde0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f790c0d4150 0x7f790c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 --2- 172.21.15.3:0/553260774 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f790c0cfbf0 0x7f790c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 >> 172.21.15.3:0/553260774 conn(0x7f790c0d4650 msgr2=0x7f790c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 shutdown_connections 2024-09-10T07:54:18.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.047+0000 7f7912e00640 1 -- 172.21.15.3:0/553260774 wait complete. 2024-09-10T07:54:18.063 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T07:54:18.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:18.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.224+0000 7f8f83a00640 1 Processor -- start 2024-09-10T07:54:18.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.224+0000 7f8f83a00640 1 -- start start 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f83a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f7c0b4520 0x7f8f7c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f83a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0cd8d0 con 0x7f8f7c1591b0 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f83a00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c15bb20 con 0x7f8f7c0ccc50 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f83a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c15b900 con 0x7f8f7c0b4520 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f7be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f7c0b4520 0x7f8f7c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f81000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8f7c1591b0 0x7f8f7c0b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42464/0 (socket says 172.21.15.3:42464) 2024-09-10T07:54:18.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.225+0000 7f8f81000640 1 -- 172.21.15.3:0/1451449280 learned_addr learned my addr 172.21.15.3:0/1451449280 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.226+0000 7f8f7be00640 1 -- 172.21.15.3:0/1451449280 >> v1:172.21.15.70:6789/0 conn(0x7f8f7c0ccc50 legacy=0x7f8f7c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.226+0000 7f8f7be00640 1 -- 172.21.15.3:0/1451449280 >> v1:172.21.15.3:6789/0 conn(0x7f8f7c1591b0 legacy=0x7f8f7c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.226+0000 7f8f7be00640 1 -- 172.21.15.3:0/1451449280 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c15be00 con 0x7f8f7c0b4520 2024-09-10T07:54:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.226+0000 7f8f7b400640 1 -- 172.21.15.3:0/1451449280 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246380170 0 0) 0x7f8f7c0cd8d0 con 0x7f8f7c1591b0 2024-09-10T07:54:18.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.227+0000 7f8f7be00640 1 --2- 172.21.15.3:0/1451449280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c0b4520 0x7f8f7c15c600 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097a0 tx=0x7f8f6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d13d8db6fb75e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.227+0000 7f8f7b400640 1 -- 172.21.15.3:0/1451449280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037070 con 0x7f8f7c0b4520 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.227+0000 7f8f7b400640 1 -- 172.21.15.3:0/1451449280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68032440 con 0x7f8f7c0b4520 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.227+0000 7f8f7b400640 1 -- 172.21.15.3:0/1451449280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68032810 con 0x7f8f7c0b4520 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 -- 172.21.15.3:0/1451449280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c0b4520 msgr2=0x7f8f7c15c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 --2- 172.21.15.3:0/1451449280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c0b4520 0x7f8f7c15c600 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097a0 tx=0x7f8f6802a230 comp rx=0 tx=0).stop 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 -- 172.21.15.3:0/1451449280 shutdown_connections 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 --2- 172.21.15.3:0/1451449280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c0b4520 0x7f8f7c15c600 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 -- 172.21.15.3:0/1451449280 >> 172.21.15.3:0/1451449280 conn(0x7f8f7c062cc0 msgr2=0x7f8f7c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 -- 172.21.15.3:0/1451449280 shutdown_connections 2024-09-10T07:54:18.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.228+0000 7f8f83a00640 1 -- 172.21.15.3:0/1451449280 wait complete. 2024-09-10T07:54:18.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.230+0000 7f8f83a00640 1 Processor -- start 2024-09-10T07:54:18.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.230+0000 7f8f83a00640 1 -- start start 2024-09-10T07:54:18.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f7c0b4520 0x7f8f7c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f7c0ccc50 0x7f8f7c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 0x7f8f7c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0ce6a0 con 0x7f8f7c1503a0 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0ce520 con 0x7f8f7c0ccc50 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f83a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0ce820 con 0x7f8f7c0b4520 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 0x7f8f7c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 0x7f8f7c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41764/0 (socket says 172.21.15.3:41764) 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 -- 172.21.15.3:0/2809114294 learned_addr learned my addr 172.21.15.3:0/2809114294 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81000640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f7c0b4520 0x7f8f7c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f7be00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f7c0ccc50 0x7f8f7c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 -- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f7c0b4520 msgr2=0x7f8f7c151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8f7c0b4520 0x7f8f7c151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 -- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f7c0ccc50 msgr2=0x7f8f7c14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f7c0ccc50 0x7f8f7c14fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.231+0000 7f8f81a00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1521e0 con 0x7f8f7c1503a0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f81a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 0x7f8f7c0ae8a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7d0 tx=0x7f8f7400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003b70 con 0x7f8f7c1503a0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f74003d30 con 0x7f8f7c1503a0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1524c0 con 0x7f8f7c1503a0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400aa90 con 0x7f8f7c1503a0 2024-09-10T07:54:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.232+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c152a80 con 0x7f8f7c1503a0 2024-09-10T07:54:18.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.233+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f7c15c600 con 0x7f8f7c1503a0 2024-09-10T07:54:18.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.234+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f7400ac30 con 0x7f8f7c1503a0 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f78c00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5c06be30 0x7f8f5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f8f7408aa10 con 0x7f8f7c1503a0 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f78c00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f8f5c071e30 0x7f8f5c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f74010e50 con 0x7f8f5c071e30 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f8f7408f2a0 con 0x7f8f7c1503a0 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f81000640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5c06be30 0x7f8f5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.235+0000 7f8f7be00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f8f5c071e30 0x7f8f5c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.236+0000 7f8f81000640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5c06be30 0x7f8f5c06e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f70000f60 tx=0x7f8f70007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.236+0000 7f8f7be00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f8f5c071e30 0x7f8f5c074250 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.237+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== osd.3 v2:172.21.15.3:6816/1816157518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f74010e50 con 0x7f8f5c071e30 2024-09-10T07:54:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.250+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 --> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8f7c15d3f0 con 0x7f8f5c071e30 2024-09-10T07:54:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.250+0000 7f8f78c00640 1 -- 172.21.15.3:0/2809114294 <== osd.3 v2:172.21.15.3:6816/1816157518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8f7c15d3f0 con 0x7f8f5c071e30 2024-09-10T07:54:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:18.250 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f8f5c071e30 msgr2=0x7f8f5c074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6816/1816157518,v1:172.21.15.3:6817/1816157518] conn(0x7f8f5c071e30 0x7f8f5c074250 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5c06be30 msgr2=0x7f8f5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8f5c06be30 0x7f8f5c06e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f70000f60 tx=0x7f8f70007e40 comp rx=0 tx=0).stop 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 msgr2=0x7f8f7c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 --2- 172.21.15.3:0/2809114294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f7c1503a0 0x7f8f7c0ae8a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7d0 tx=0x7f8f7400cc90 comp rx=0 tx=0).stop 2024-09-10T07:54:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f81a00640 1 -- 172.21.15.3:0/2809114294 reap_dead start 2024-09-10T07:54:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 shutdown_connections 2024-09-10T07:54:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 >> 172.21.15.3:0/2809114294 conn(0x7f8f7c062cc0 msgr2=0x7f8f7c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 shutdown_connections 2024-09-10T07:54:18.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.251+0000 7f8f83a00640 1 -- 172.21.15.3:0/2809114294 wait complete. 2024-09-10T07:54:18.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T07:54:18.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.423+0000 7fb75ca00640 1 Processor -- start 2024-09-10T07:54:18.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.423+0000 7fb75ca00640 1 -- start start 2024-09-10T07:54:18.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.424+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb7580cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.424+0000 7fb75ca00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 0x7fb758072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.424+0000 7fb75ca00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7580cfdd0 con 0x7fb758160af0 2024-09-10T07:54:18.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.424+0000 7fb75ca00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb75815cc90 con 0x7fb7580d07c0 2024-09-10T07:54:18.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.424+0000 7fb75ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb75815cf70 con 0x7fb758160710 2024-09-10T07:54:18.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb74d400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb758160af0 0x7fb7580d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42480/0 (socket says 172.21.15.3:42480) 2024-09-10T07:54:18.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb74d400640 1 -- 172.21.15.3:0/3622273899 learned_addr learned my addr 172.21.15.3:0/3622273899 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:18.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755e00640 1 --2- 172.21.15.3:0/3622273899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb7580cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb756800640 1 --2- 172.21.15.3:0/3622273899 >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 0x7fb758072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755400640 1 -- 172.21.15.3:0/3622273899 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191163107 0 0) 0x7fb7580cfdd0 con 0x7fb758160af0 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755400640 1 -- 172.21.15.3:0/3622273899 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb73c003520 con 0x7fb758160af0 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755e00640 1 -- 172.21.15.3:0/3622273899 >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 msgr2=0x7fb758072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755e00640 1 --2- 172.21.15.3:0/3622273899 >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 0x7fb758072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755e00640 1 -- 172.21.15.3:0/3622273899 >> v1:172.21.15.3:6789/0 conn(0x7fb758160af0 legacy=0x7fb7580d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb755e00640 1 -- 172.21.15.3:0/3622273899 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75815d250 con 0x7fb758160710 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.425+0000 7fb756800640 1 --2- 172.21.15.3:0/3622273899 >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 0x7fb758072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:18.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.426+0000 7fb755e00640 1 --2- 172.21.15.3:0/3622273899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb7580cf2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7480097a0 tx=0x7fb74802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85441ae4df7507ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb755400640 1 -- 172.21.15.3:0/3622273899 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748037070 con 0x7fb758160710 2024-09-10T07:54:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb755400640 1 -- 172.21.15.3:0/3622273899 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb748032440 con 0x7fb758160710 2024-09-10T07:54:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb755400640 1 -- 172.21.15.3:0/3622273899 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748032810 con 0x7fb758160710 2024-09-10T07:54:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 -- 172.21.15.3:0/3622273899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 msgr2=0x7fb7580cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 --2- 172.21.15.3:0/3622273899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb7580cf2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7480097a0 tx=0x7fb74802a180 comp rx=0 tx=0).stop 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 -- 172.21.15.3:0/3622273899 shutdown_connections 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 --2- 172.21.15.3:0/3622273899 >> v2:172.21.15.70:3300/0 conn(0x7fb7580d07c0 0x7fb758072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 --2- 172.21.15.3:0/3622273899 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb7580cf2c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 -- 172.21.15.3:0/3622273899 >> 172.21.15.3:0/3622273899 conn(0x7fb75806d000 msgr2=0x7fb75806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 -- 172.21.15.3:0/3622273899 shutdown_connections 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.427+0000 7fb75ca00640 1 -- 172.21.15.3:0/3622273899 wait complete. 2024-09-10T07:54:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.429+0000 7fb75ca00640 1 Processor -- start 2024-09-10T07:54:18.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.429+0000 7fb75ca00640 1 -- start start 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb758153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 0x7fb7580aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb75815d770 con 0x7fb7580d07c0 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580cf800 con 0x7fb758160af0 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb75ca00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580ad9a0 con 0x7fb758160710 2024-09-10T07:54:18.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb755e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb755e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41768/0 (socket says 172.21.15.3:41768) 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.430+0000 7fb755e00640 1 -- 172.21.15.3:0/2670002918 learned_addr learned my addr 172.21.15.3:0/2670002918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb756800640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 0x7fb7580aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb74d400640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb758153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 -- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 msgr2=0x7fb758153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb758153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 -- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 msgr2=0x7fb7580aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 0x7fb7580aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 -- 172.21.15.3:0/2670002918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580adc80 con 0x7fb7580d07c0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb756800640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 0x7fb7580aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb74d400640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb758153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb755e00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb748035040 tx=0x7fb7480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748041030 con 0x7fb7580d07c0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74803d4e0 con 0x7fb7580d07c0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7580adf60 con 0x7fb7580d07c0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.431+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748037040 con 0x7fb7580d07c0 2024-09-10T07:54:18.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.432+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75814b320 con 0x7fb7580d07c0 2024-09-10T07:54:18.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.433+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7580afd50 con 0x7fb7580d07c0 2024-09-10T07:54:18.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.433+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb748044070 con 0x7fb7580d07c0 2024-09-10T07:54:18.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.435+0000 7fb74ea00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 0x7fb73406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.435+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fb7480ab320 con 0x7fb7580d07c0 2024-09-10T07:54:18.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.435+0000 7fb74d400640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 0x7fb73406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.436+0000 7fb74d400640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 0x7fb73406e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7580738a0 tx=0x7fb740009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.439+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb748079e90 con 0x7fb7580d07c0 2024-09-10T07:54:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.625+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb7580d0430 con 0x7fb73406be30 2024-09-10T07:54:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.626+0000 7fb74ea00640 1 -- 172.21.15.3:0/2670002918 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb7580d0430 con 0x7fb73406be30 2024-09-10T07:54:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 msgr2=0x7fb73406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 0x7fb73406e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7580738a0 tx=0x7fb740009040 comp rx=0 tx=0).stop 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 msgr2=0x7fb75814ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb748035040 tx=0x7fb7480046c0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 shutdown_connections 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fb73406be30 0x7fb73406e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb758160af0 0x7fb7580aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb758160710 0x7fb758153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 --2- 172.21.15.3:0/2670002918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7580d07c0 0x7fb75814ade0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 >> 172.21.15.3:0/2670002918 conn(0x7fb75806d000 msgr2=0x7fb7580b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 shutdown_connections 2024-09-10T07:54:18.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.628+0000 7fb75ca00640 1 -- 172.21.15.3:0/2670002918 wait complete. 2024-09-10T07:54:18.643 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T07:54:18.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.805+0000 7f7a9cc00640 1 Processor -- start 2024-09-10T07:54:18.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.805+0000 7f7a9cc00640 1 -- start start 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.806+0000 7f7a9cc00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a9815e7c0 0x7f7a980d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.806+0000 7f7a9cc00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a980cf090 con 0x7f7a980cead0 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.806+0000 7f7a9cc00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a980cf370 con 0x7f7a9806bb20 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.806+0000 7f7a9cc00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7a980cfca0 con 0x7f7a9815e7c0 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.807+0000 7f7a95400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7a980cead0 0x7f7a98072d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42492/0 (socket says 172.21.15.3:42492) 2024-09-10T07:54:18.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.807+0000 7f7a95400640 1 -- 172.21.15.3:0/3110237437 learned_addr learned my addr 172.21.15.3:0/3110237437 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.807+0000 7f7a95e00640 1 --2- 172.21.15.3:0/3110237437 >> v2:172.21.15.70:3300/0 conn(0x7f7a9815e7c0 0x7f7a980d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.807+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783543004 0 0) 0x7f7a980cf090 con 0x7f7a980cead0 2024-09-10T07:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.807+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a7c003520 con 0x7f7a980cead0 2024-09-10T07:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3181078524 0 0) 0x7f7a7c003520 con 0x7f7a980cead0 2024-09-10T07:54:18.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 >> v2:172.21.15.70:3300/0 conn(0x7f7a9815e7c0 msgr2=0x7f7a980d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 --2- 172.21.15.3:0/3110237437 >> v2:172.21.15.70:3300/0 conn(0x7f7a9815e7c0 0x7f7a980d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 >> v1:172.21.15.70:6789/0 conn(0x7f7a9806bb20 legacy=0x7f7a9806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a980cf650 con 0x7f7a980cead0 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f7a84001670 con 0x7f7a980cead0 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 >> v1:172.21.15.3:6789/0 conn(0x7f7a980cead0 legacy=0x7f7a98072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a7c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a7c0043d0 0x7f7a7c0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a7c005140 0x7f7a7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7a980cfca0 con 0x7f7a7c0039b0 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00c090 con 0x7f7a7c0043d0 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7c00c290 con 0x7f7a7c005140 2024-09-10T07:54:18.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a7c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95400640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a7c0043d0 0x7f7a7c0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a96800640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a7c005140 0x7f7a7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 -- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a7c005140 msgr2=0x7f7a7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a7c005140 0x7f7a7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 -- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a7c0043d0 msgr2=0x7f7a7c0048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a7c0043d0 0x7f7a7c0048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a95e00640 1 -- 172.21.15.3:0/3110237437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c003520 con 0x7f7a7c0039b0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.808+0000 7f7a96800640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a7c005140 0x7f7a7c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.809+0000 7f7a96800640 1 -- 172.21.15.3:0/3110237437 reap_dead start 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.809+0000 7f7a95e00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a7c003db0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009ac0 tx=0x7f7a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf28b1e214cd243f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88036070 con 0x7f7a7c0039b0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a88031400 con 0x7f7a7c0039b0 2024-09-10T07:54:18.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a94a00640 1 -- 172.21.15.3:0/3110237437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88009690 con 0x7f7a7c0039b0 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 -- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 msgr2=0x7f7a7c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a7c003db0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009ac0 tx=0x7f7a7c005830 comp rx=0 tx=0).stop 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 -- 172.21.15.3:0/3110237437 shutdown_connections 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 --2- 172.21.15.3:0/3110237437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a7c003db0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 -- 172.21.15.3:0/3110237437 >> 172.21.15.3:0/3110237437 conn(0x7f7a980b28a0 msgr2=0x7f7a980b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 -- 172.21.15.3:0/3110237437 shutdown_connections 2024-09-10T07:54:18.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.810+0000 7f7a9cc00640 1 -- 172.21.15.3:0/3110237437 wait complete. 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.812+0000 7f7a9cc00640 1 Processor -- start 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.812+0000 7f7a9cc00640 1 -- start start 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a981536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a980b42d0 0x7f7a980b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9815fd00 0x7f7a98160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7a980d0130 con 0x7f7a7c0039b0 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9815eba0 con 0x7f7a9815fd00 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a9cc00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7a980d02b0 con 0x7f7a980b42d0 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a95e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a981536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a95e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a981536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41776/0 (socket says 172.21.15.3:41776) 2024-09-10T07:54:18.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a95e00640 1 -- 172.21.15.3:0/806983167 learned_addr learned my addr 172.21.15.3:0/806983167 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a95400640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a980b42d0 0x7f7a980b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.813+0000 7f7a96800640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9815fd00 0x7f7a98160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 -- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a980b42d0 msgr2=0x7f7a980b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a980b42d0 0x7f7a980b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 -- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9815fd00 msgr2=0x7f7a98160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9815fd00 0x7f7a98160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a981606c0 con 0x7f7a7c0039b0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a95e00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a981536a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009be0 tx=0x7f7a88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88036050 con 0x7f7a7c0039b0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a88037040 con 0x7f7a7c0039b0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003df0 con 0x7f7a7c0039b0 2024-09-10T07:54:18.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a98156c90 con 0x7f7a7c0039b0 2024-09-10T07:54:18.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.814+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a98157250 con 0x7f7a7c0039b0 2024-09-10T07:54:18.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.815+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a980ce420 con 0x7f7a7c0039b0 2024-09-10T07:54:18.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.817+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a88044020 con 0x7f7a7c0039b0 2024-09-10T07:54:18.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.817+0000 7f7a8e000640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a7806be30 0x7f7a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f7a880aba20 con 0x7f7a7c0039b0 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a8e000640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7f7a78071e30 0x7f7a78074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a88051ea0 con 0x7f7a78071e30 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f7a880abeb0 con 0x7f7a7c0039b0 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a95400640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a7806be30 0x7f7a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a95400640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a7806be30 0x7f7a7806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7a9815f200 tx=0x7f7a84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.818+0000 7f7a96800640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7f7a78071e30 0x7f7a78074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.819+0000 7f7a96800640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7f7a78071e30 0x7f7a78074250 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.820+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== osd.4 v2:172.21.15.70:6800/2990268368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a88051ea0 con 0x7f7a78071e30 2024-09-10T07:54:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.833+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 --> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a980b2300 con 0x7f7a78071e30 2024-09-10T07:54:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.834+0000 7f7a8e000640 1 -- 172.21.15.3:0/806983167 <== osd.4 v2:172.21.15.70:6800/2990268368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a980b2300 con 0x7f7a78071e30 2024-09-10T07:54:18.834 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:18.834 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7f7a78071e30 msgr2=0x7f7a78074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.70:6800/2990268368,v1:172.21.15.70:6801/2990268368] conn(0x7f7a78071e30 0x7f7a78074250 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a7806be30 msgr2=0x7f7a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f7a7806be30 0x7f7a7806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7a9815f200 tx=0x7f7a84009040 comp rx=0 tx=0).stop 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 msgr2=0x7f7a981536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 --2- 172.21.15.3:0/806983167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a7c0039b0 0x7f7a981536a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009be0 tx=0x7f7a88004730 comp rx=0 tx=0).stop 2024-09-10T07:54:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a96800640 1 -- 172.21.15.3:0/806983167 reap_dead start 2024-09-10T07:54:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 shutdown_connections 2024-09-10T07:54:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 >> 172.21.15.3:0/806983167 conn(0x7f7a980b28a0 msgr2=0x7f7a980b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 shutdown_connections 2024-09-10T07:54:18.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:18.835+0000 7f7a9cc00640 1 -- 172.21.15.3:0/806983167 wait complete. 2024-09-10T07:54:18.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T07:54:19.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.006+0000 7fc4f6200640 1 Processor -- start 2024-09-10T07:54:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.006+0000 7fc4f6200640 1 -- start start 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.007+0000 7fc4f6200640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 0x7fc4f00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.007+0000 7fc4f6200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.007+0000 7fc4f6200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00cfdd0 con 0x7fc4f0160af0 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.007+0000 7fc4f6200640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f015cc90 con 0x7fc4f0160710 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.007+0000 7fc4f6200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f015cf70 con 0x7fc4f00d07c0 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4eea00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc4f0160af0 0x7fc4f00d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42494/0 (socket says 172.21.15.3:42494) 2024-09-10T07:54:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4eea00640 1 -- 172.21.15.3:0/3357969626 learned_addr learned my addr 172.21.15.3:0/3357969626 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4ef400640 1 --2- 172.21.15.3:0/3357969626 >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 0x7fc4f00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4efe00640 1 --2- 172.21.15.3:0/3357969626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4ee000640 1 -- 172.21.15.3:0/3357969626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101394647 0 0) 0x7fc4f00cfdd0 con 0x7fc4f0160af0 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4ee000640 1 -- 172.21.15.3:0/3357969626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d4003520 con 0x7fc4f0160af0 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4efe00640 1 -- 172.21.15.3:0/3357969626 >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 msgr2=0x7fc4f00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4efe00640 1 --2- 172.21.15.3:0/3357969626 >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 0x7fc4f00cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4efe00640 1 -- 172.21.15.3:0/3357969626 >> v1:172.21.15.3:6789/0 conn(0x7fc4f0160af0 legacy=0x7fc4f00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4efe00640 1 -- 172.21.15.3:0/3357969626 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f015d250 con 0x7fc4f00d07c0 2024-09-10T07:54:19.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.008+0000 7fc4ef400640 1 --2- 172.21.15.3:0/3357969626 >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 0x7fc4f00cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.009+0000 7fc4efe00640 1 --2- 172.21.15.3:0/3357969626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0072070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009d90 tx=0x7fc4e802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216c03eeef3cd405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4ee000640 1 -- 172.21.15.3:0/3357969626 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8007740 con 0x7fc4f00d07c0 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4ee000640 1 -- 172.21.15.3:0/3357969626 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e80078e0 con 0x7fc4f00d07c0 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4ee000640 1 -- 172.21.15.3:0/3357969626 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8007cb0 con 0x7fc4f00d07c0 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 -- 172.21.15.3:0/3357969626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 msgr2=0x7fc4f0072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 --2- 172.21.15.3:0/3357969626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0072070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009d90 tx=0x7fc4e802a660 comp rx=0 tx=0).stop 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 -- 172.21.15.3:0/3357969626 shutdown_connections 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 --2- 172.21.15.3:0/3357969626 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0072070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 --2- 172.21.15.3:0/3357969626 >> v2:172.21.15.70:3300/0 conn(0x7fc4f0160710 0x7fc4f00cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 -- 172.21.15.3:0/3357969626 >> 172.21.15.3:0/3357969626 conn(0x7fc4f006d000 msgr2=0x7fc4f006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 -- 172.21.15.3:0/3357969626 shutdown_connections 2024-09-10T07:54:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.010+0000 7fc4f6200640 1 -- 172.21.15.3:0/3357969626 wait complete. 2024-09-10T07:54:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.012+0000 7fc4f6200640 1 Processor -- start 2024-09-10T07:54:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.012+0000 7fc4f6200640 1 -- start start 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4f0160710 0x7fc4f01525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f015d770 con 0x7fc4f0160af0 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00cf800 con 0x7fc4f0160710 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4f6200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f0152af0 con 0x7fc4f00d07c0 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4efe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41786/0 (socket says 172.21.15.3:41786) 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4efe00640 1 -- 172.21.15.3:0/1901871589 learned_addr learned my addr 172.21.15.3:0/1901871589 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4eea00640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4f0160710 0x7fc4f01525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.013+0000 7fc4ef400640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 -- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 msgr2=0x7fc4f0152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 -- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4f0160710 msgr2=0x7fc4f01525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4f0160710 0x7fc4f01525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 -- 172.21.15.3:0/1901871589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f0155580 con 0x7fc4f0160af0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4efe00640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009eb0 tx=0x7fc4e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8040070 con 0x7fc4f0160af0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8031b40 con 0x7fc4f0160af0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8045460 con 0x7fc4f0160af0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.014+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0155860 con 0x7fc4f0160af0 2024-09-10T07:54:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.015+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f0155e20 con 0x7fc4f0160af0 2024-09-10T07:54:19.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.015+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f00afd50 con 0x7fc4f0160af0 2024-09-10T07:54:19.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.016+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4e8039040 con 0x7fc4f0160af0 2024-09-10T07:54:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.018+0000 7fc4e7400640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 0x7fc4d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.018+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fc4e80ac1d0 con 0x7fc4f0160af0 2024-09-10T07:54:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.018+0000 7fc4ef400640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 0x7fc4d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.019+0000 7fc4ef400640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 0x7fc4d006e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00aa690 tx=0x7fc4e0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.022+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc4e807a890 con 0x7fc4f0160af0 2024-09-10T07:54:19.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.205+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc4f00ce420 con 0x7fc4d006be30 2024-09-10T07:54:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.205+0000 7fc4e7400640 1 -- 172.21.15.3:0/1901871589 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc4f00ce420 con 0x7fc4d006be30 2024-09-10T07:54:19.205 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 msgr2=0x7fc4d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 0x7fc4d006e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00aa690 tx=0x7fc4e0009190 comp rx=0 tx=0).stop 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 msgr2=0x7fc4f015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009eb0 tx=0x7fc4e8004a40 comp rx=0 tx=0).stop 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 shutdown_connections 2024-09-10T07:54:19.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc4d006be30 0x7fc4d006e2f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4f0160af0 0x7fc4f015eb70 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4f0160710 0x7fc4f01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 --2- 172.21.15.3:0/1901871589 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4f00d07c0 0x7fc4f0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 >> 172.21.15.3:0/1901871589 conn(0x7fc4f006d000 msgr2=0x7fc4f0074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 shutdown_connections 2024-09-10T07:54:19.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.207+0000 7fc4f6200640 1 -- 172.21.15.3:0/1901871589 wait complete. 2024-09-10T07:54:19.223 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T07:54:19.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.383+0000 7effeb800640 1 Processor -- start 2024-09-10T07:54:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.383+0000 7effeb800640 1 -- start start 2024-09-10T07:54:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effeb800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7effe415c530 0x7effe40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effeb800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe40b26a0 0x7effe40722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effeb800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe40cf1f0 con 0x7effe40cd740 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effeb800640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7effe40cdbb0 con 0x7effe40b26a0 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effeb800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7effe40d00c0 con 0x7effe415c530 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe40b26a0 0x7effe40722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7effe40b26a0 0x7effe40722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41790/0 (socket says 172.21.15.3:41790) 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 -- 172.21.15.3:0/764865319 learned_addr learned my addr 172.21.15.3:0/764865319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe3e00640 1 --1- 172.21.15.3:0/764865319 >> v1:172.21.15.3:6789/0 conn(0x7effe40cd740 0x7effe40b1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42510/0 (socket says 172.21.15.3:42510) 2024-09-10T07:54:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe8e00640 1 --2- 172.21.15.3:0/764865319 >> v2:172.21.15.70:3300/0 conn(0x7effe415c530 0x7effe40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 -- 172.21.15.3:0/764865319 >> v2:172.21.15.70:3300/0 conn(0x7effe415c530 msgr2=0x7effe40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 --2- 172.21.15.3:0/764865319 >> v2:172.21.15.70:3300/0 conn(0x7effe415c530 0x7effe40cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 -- 172.21.15.3:0/764865319 >> v1:172.21.15.3:6789/0 conn(0x7effe40cd740 legacy=0x7effe40b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.385+0000 7effe9800640 1 -- 172.21.15.3:0/764865319 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe40ce150 con 0x7effe40b26a0 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.386+0000 7effe9800640 1 --2- 172.21.15.3:0/764865319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe40722e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7effe4063170 tx=0x7effdc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca14e5f11be54c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effe3400640 1 -- 172.21.15.3:0/764865319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc036070 con 0x7effe40b26a0 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effe3400640 1 -- 172.21.15.3:0/764865319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effdc031600 con 0x7effe40b26a0 2024-09-10T07:54:19.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effe3400640 1 -- 172.21.15.3:0/764865319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc0319d0 con 0x7effe40b26a0 2024-09-10T07:54:19.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 -- 172.21.15.3:0/764865319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 msgr2=0x7effe40722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 --2- 172.21.15.3:0/764865319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe40722e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7effe4063170 tx=0x7effdc02a730 comp rx=0 tx=0).stop 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 -- 172.21.15.3:0/764865319 shutdown_connections 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 --2- 172.21.15.3:0/764865319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe40722e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 --2- 172.21.15.3:0/764865319 >> v2:172.21.15.70:3300/0 conn(0x7effe415c530 0x7effe40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 -- 172.21.15.3:0/764865319 >> 172.21.15.3:0/764865319 conn(0x7effe4062cc0 msgr2=0x7effe406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 -- 172.21.15.3:0/764865319 shutdown_connections 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.387+0000 7effeb800640 1 -- 172.21.15.3:0/764865319 wait complete. 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.389+0000 7effeb800640 1 Processor -- start 2024-09-10T07:54:19.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.389+0000 7effeb800640 1 -- start start 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe414f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effe40cd740 0x7effe4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe415c530 0x7effe4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effe40ce9f0 con 0x7effe40b26a0 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7effe40ce750 con 0x7effe415c530 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effeb800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7effe40cecd0 con 0x7effe40cd740 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effe8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe414f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effe8e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe414f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41798/0 (socket says 172.21.15.3:41798) 2024-09-10T07:54:19.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effe8e00640 1 -- 172.21.15.3:0/4145659354 learned_addr learned my addr 172.21.15.3:0/4145659354 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.390+0000 7effe9800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe415c530 0x7effe4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe3e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effe40cd740 0x7effe4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 -- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effe40cd740 msgr2=0x7effe4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7effe40cd740 0x7effe4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 -- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe415c530 msgr2=0x7effe4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe415c530 0x7effe4150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe4152460 con 0x7effe40b26a0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe9800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe415c530 0x7effe4150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe8e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe414f3d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7effd800e870 tx=0x7effd800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8017070 con 0x7effe40b26a0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd800c940 con 0x7effe40b26a0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800cd30 con 0x7effe40b26a0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe4151b30 con 0x7effe40b26a0 2024-09-10T07:54:19.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.391+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41520f0 con 0x7effe40b26a0 2024-09-10T07:54:19.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.392+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effe40cd200 con 0x7effe40b26a0 2024-09-10T07:54:19.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.393+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effd801a070 con 0x7effe40b26a0 2024-09-10T07:54:19.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe0c00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7effc406be30 0x7effc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7effd808ab60 con 0x7effe40b26a0 2024-09-10T07:54:19.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe3e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7effc406be30 0x7effc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe0c00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7effc4071e30 0x7effc4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effd8015e50 con 0x7effc4071e30 2024-09-10T07:54:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.394+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7effd808f2a0 con 0x7effe40b26a0 2024-09-10T07:54:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.395+0000 7effe9800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7effc4071e30 0x7effc4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.395+0000 7effe3e00640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7effc406be30 0x7effc406e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7effe41f58e0 tx=0x7effd0007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.395+0000 7effe9800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7effc4071e30 0x7effc4074250 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.397+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== osd.5 v2:172.21.15.70:6824/2669649466 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effd8015e50 con 0x7effc4071e30 2024-09-10T07:54:19.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.410+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 --> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7effe41515b0 con 0x7effc4071e30 2024-09-10T07:54:19.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effe0c00640 1 -- 172.21.15.3:0/4145659354 <== osd.5 v2:172.21.15.70:6824/2669649466 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7effe41515b0 con 0x7effc4071e30 2024-09-10T07:54:19.410 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:19.411 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:19.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7effc4071e30 msgr2=0x7effc4074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.70:6824/2669649466,v1:172.21.15.70:6825/2669649466] conn(0x7effc4071e30 0x7effc4074250 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7effc406be30 msgr2=0x7effc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7effc406be30 0x7effc406e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7effe41f58e0 tx=0x7effd0007cc0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 msgr2=0x7effe414f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 --2- 172.21.15.3:0/4145659354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe40b26a0 0x7effe414f3d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7effd800e870 tx=0x7effd800ed30 comp rx=0 tx=0).stop 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effe9800640 1 -- 172.21.15.3:0/4145659354 reap_dead start 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 shutdown_connections 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 >> 172.21.15.3:0/4145659354 conn(0x7effe4062cc0 msgr2=0x7effe40730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 shutdown_connections 2024-09-10T07:54:19.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.411+0000 7effeb800640 1 -- 172.21.15.3:0/4145659354 wait complete. 2024-09-10T07:54:19.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T07:54:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.583+0000 7f9145a00640 1 Processor -- start 2024-09-10T07:54:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.583+0000 7f9145a00640 1 -- start start 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f9145a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9140160710 0x7f91400cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f9145a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91400d07c0 0x7f9140072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f9145a00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91400cfdd0 con 0x7f9140160af0 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f9145a00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f914015cc90 con 0x7f9140160710 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f9145a00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f914015cf70 con 0x7f91400d07c0 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f913ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9140160710 0x7f91400cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f913e000640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9140160af0 0x7f91400d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42518/0 (socket says 172.21.15.3:42518) 2024-09-10T07:54:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f913ea00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9140160710 0x7f91400cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41804/0 (socket says 172.21.15.3:41804) 2024-09-10T07:54:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.584+0000 7f913e000640 1 -- 172.21.15.3:0/94007569 learned_addr learned my addr 172.21.15.3:0/94007569 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913f400640 1 --2- 172.21.15.3:0/94007569 >> v2:172.21.15.70:3300/0 conn(0x7f91400d07c0 0x7f9140072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913ea00640 1 -- 172.21.15.3:0/94007569 >> v2:172.21.15.70:3300/0 conn(0x7f91400d07c0 msgr2=0x7f9140072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913ea00640 1 --2- 172.21.15.3:0/94007569 >> v2:172.21.15.70:3300/0 conn(0x7f91400d07c0 0x7f9140072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913ea00640 1 -- 172.21.15.3:0/94007569 >> v1:172.21.15.3:6789/0 conn(0x7f9140160af0 legacy=0x7f91400d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913ea00640 1 -- 172.21.15.3:0/94007569 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914015d250 con 0x7f9140160710 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.585+0000 7f913d600640 1 -- 172.21.15.3:0/94007569 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577072760 0 0) 0x7f91400cfdd0 con 0x7f9140160af0 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.586+0000 7f913ea00640 1 --2- 172.21.15.3:0/94007569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91400cf2c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f91400718d0 tx=0x7f913002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c19d1bf75cd56fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.586+0000 7f913d600640 1 -- 172.21.15.3:0/94007569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130037070 con 0x7f9140160710 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.586+0000 7f913d600640 1 -- 172.21.15.3:0/94007569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9130032440 con 0x7f9140160710 2024-09-10T07:54:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.586+0000 7f913d600640 1 -- 172.21.15.3:0/94007569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130032810 con 0x7f9140160710 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 -- 172.21.15.3:0/94007569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 msgr2=0x7f91400cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 --2- 172.21.15.3:0/94007569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91400cf2c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f91400718d0 tx=0x7f913002a1a0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 -- 172.21.15.3:0/94007569 shutdown_connections 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 --2- 172.21.15.3:0/94007569 >> v2:172.21.15.70:3300/0 conn(0x7f91400d07c0 0x7f9140072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 --2- 172.21.15.3:0/94007569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91400cf2c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 -- 172.21.15.3:0/94007569 >> 172.21.15.3:0/94007569 conn(0x7f914006d000 msgr2=0x7f914006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 -- 172.21.15.3:0/94007569 shutdown_connections 2024-09-10T07:54:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.587+0000 7f9145a00640 1 -- 172.21.15.3:0/94007569 wait complete. 2024-09-10T07:54:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.589+0000 7f9145a00640 1 Processor -- start 2024-09-10T07:54:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.589+0000 7f9145a00640 1 -- start start 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.589+0000 7f9145a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91400d07c0 0x7f9140147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.589+0000 7f9145a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f9145a00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9140160af0 0x7f91401e6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f9145a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f914015d770 con 0x7f9140160710 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f9145a00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91400cf800 con 0x7f91400d07c0 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f9145a00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91401e9250 con 0x7f9140160af0 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41812/0 (socket says 172.21.15.3:41812) 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 -- 172.21.15.3:0/2790003549 learned_addr learned my addr 172.21.15.3:0/2790003549 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913ea00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91400d07c0 0x7f9140147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913f400640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9140160af0 0x7f91401e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 -- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9140160af0 msgr2=0x7f91401e6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9140160af0 0x7f91401e6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 -- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91400d07c0 msgr2=0x7f9140147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91400d07c0 0x7f9140147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.590+0000 7f913e000640 1 -- 172.21.15.3:0/2790003549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401e6d60 con 0x7f9140160710 2024-09-10T07:54:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f913e000640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c430 tx=0x7f912c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c0074e0 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c0047e0 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c004bd0 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91401e9480 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.591+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401519b0 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.592+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914015c180 con 0x7f9140160710 2024-09-10T07:54:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.593+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f912c011070 con 0x7f9140160710 2024-09-10T07:54:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.594+0000 7f9136a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 0x7f912006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.594+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f912c08a760 con 0x7f9140160710 2024-09-10T07:54:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.594+0000 7f913ea00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 0x7f912006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.595+0000 7f913ea00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 0x7f912006e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9130034000 tx=0x7f913002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.598+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f912c058c70 con 0x7f9140160710 2024-09-10T07:54:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.784+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f9140069f70 con 0x7f912006be30 2024-09-10T07:54:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.785+0000 7f9136a00640 1 -- 172.21.15.3:0/2790003549 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9140069f70 con 0x7f912006be30 2024-09-10T07:54:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T07:54:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.786+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 msgr2=0x7f912006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.786+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 0x7f912006e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9130034000 tx=0x7f913002e000 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.786+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 msgr2=0x7f91401e8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.786+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c430 tx=0x7f912c00c8f0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 shutdown_connections 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f912006be30 0x7f912006e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9140160af0 0x7f91401e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140160710 0x7f91401e8d10 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 --2- 172.21.15.3:0/2790003549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91400d07c0 0x7f9140147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 >> 172.21.15.3:0/2790003549 conn(0x7f914006d000 msgr2=0x7f91400b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 shutdown_connections 2024-09-10T07:54:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.787+0000 7f9145a00640 1 -- 172.21.15.3:0/2790003549 wait complete. 2024-09-10T07:54:19.802 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T07:54:19.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:19.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.963+0000 7fbd2aa00640 1 Processor -- start 2024-09-10T07:54:19.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.963+0000 7fbd2aa00640 1 -- start start 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.964+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 0x7fbd241670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.964+0000 7fbd2aa00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240cf1f0 con 0x7fbd240cd910 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.964+0000 7fbd2aa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240ce150 con 0x7fbd2415c530 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.964+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbd240d00c0 con 0x7fbd240b26a0 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd1b400640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbd240cd910 0x7fbd240b1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42532/0 (socket says 172.21.15.3:42532) 2024-09-10T07:54:19.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd1b400640 1 -- 172.21.15.3:0/2305493248 learned_addr learned my addr 172.21.15.3:0/2305493248 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:19.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd28a00640 1 --2- 172.21.15.3:0/2305493248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 0x7fbd241670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987031119 0 0) 0x7fbd240cf1f0 con 0x7fbd240cd910 2024-09-10T07:54:19.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd08003520 con 0x7fbd240cd910 2024-09-10T07:54:19.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547118930 0 0) 0x7fbd240ce150 con 0x7fbd2415c530 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd240cf1f0 con 0x7fbd2415c530 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd28a00640 1 -- 172.21.15.3:0/2305493248 >> v1:172.21.15.70:6789/0 conn(0x7fbd2415c530 legacy=0x7fbd240cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.965+0000 7fbd28a00640 1 -- 172.21.15.3:0/2305493248 >> v1:172.21.15.3:6789/0 conn(0x7fbd240cd910 legacy=0x7fbd240b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.966+0000 7fbd28a00640 1 -- 172.21.15.3:0/2305493248 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd240ce430 con 0x7fbd240b26a0 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd28a00640 1 --2- 172.21.15.3:0/2305493248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 0x7fbd241670e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009dc0 tx=0x7fbd1c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57d7549894cc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c007740 con 0x7fbd240b26a0 2024-09-10T07:54:19.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c0078e0 con 0x7fbd240b26a0 2024-09-10T07:54:19.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd23400640 1 -- 172.21.15.3:0/2305493248 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c007cb0 con 0x7fbd240b26a0 2024-09-10T07:54:19.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2305493248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 msgr2=0x7fbd241670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.967+0000 7fbd2aa00640 1 --2- 172.21.15.3:0/2305493248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 0x7fbd241670e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009dc0 tx=0x7fbd1c02a6d0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.968+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2305493248 shutdown_connections 2024-09-10T07:54:19.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.968+0000 7fbd2aa00640 1 --2- 172.21.15.3:0/2305493248 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240b26a0 0x7fbd241670e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.968+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2305493248 >> 172.21.15.3:0/2305493248 conn(0x7fbd24062cc0 msgr2=0x7fbd2406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.968+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2305493248 shutdown_connections 2024-09-10T07:54:19.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.968+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2305493248 wait complete. 2024-09-10T07:54:19.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.969+0000 7fbd2aa00640 1 Processor -- start 2024-09-10T07:54:19.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.969+0000 7fbd2aa00640 1 -- start start 2024-09-10T07:54:19.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd240b26a0 0x7fbd24153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240cd910 0x7fbd24155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 0x7fbd241528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240cecd0 con 0x7fbd2415c530 2024-09-10T07:54:19.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240ce9f0 con 0x7fbd240b26a0 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd2aa00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2415c910 con 0x7fbd240cd910 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.970+0000 7fbd28a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 0x7fbd241528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 0x7fbd241528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41820/0 (socket says 172.21.15.3:41820) 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 -- 172.21.15.3:0/2752422838 learned_addr learned my addr 172.21.15.3:0/2752422838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd1b400640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240cd910 0x7fbd24155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd23e00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd240b26a0 0x7fbd24153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 -- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240cd910 msgr2=0x7fbd24155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240cd910 0x7fbd24155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 -- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd240b26a0 msgr2=0x7fbd24153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd240b26a0 0x7fbd24153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd241536e0 con 0x7fbd2415c530 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd1b400640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbd240cd910 0x7fbd24155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:19.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.971+0000 7fbd28a00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 0x7fbd241528d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c02acc0 tx=0x7fbd1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.972+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c040070 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.972+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c031ac0 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.972+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c045460 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.972+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd241565d0 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.972+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd24156b90 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.973+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd24073380 con 0x7fbd2415c530 2024-09-10T07:54:19.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.974+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd1c039040 con 0x7fbd2415c530 2024-09-10T07:54:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd20c00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbd0006be30 0x7fbd0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fbd1c0ac310 con 0x7fbd2415c530 2024-09-10T07:54:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd23e00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbd0006be30 0x7fbd0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd20c00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7fbd00071e30 0x7fbd00074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd1c03ee50 con 0x7fbd00071e30 2024-09-10T07:54:19.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7fbd1c0b3050 con 0x7fbd2415c530 2024-09-10T07:54:19.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd1b400640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7fbd00071e30 0x7fbd00074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:19.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.975+0000 7fbd23e00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbd0006be30 0x7fbd0006e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.976+0000 7fbd1b400640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7fbd00071e30 0x7fbd00074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:19.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.977+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== osd.6 v2:172.21.15.70:6816/3978225537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd1c03ee50 con 0x7fbd00071e30 2024-09-10T07:54:19.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.990+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 --> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbd24156fb0 con 0x7fbd00071e30 2024-09-10T07:54:19.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd20c00640 1 -- 172.21.15.3:0/2752422838 <== osd.6 v2:172.21.15.70:6816/3978225537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbd24156fb0 con 0x7fbd00071e30 2024-09-10T07:54:19.991 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:19.991 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7fbd00071e30 msgr2=0x7fbd00074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.70:6816/3978225537,v1:172.21.15.70:6817/3978225537] conn(0x7fbd00071e30 0x7fbd00074250 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbd0006be30 msgr2=0x7fbd0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbd0006be30 0x7fbd0006e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14009040 comp rx=0 tx=0).stop 2024-09-10T07:54:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 msgr2=0x7fbd241528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 --2- 172.21.15.3:0/2752422838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd2415c530 0x7fbd241528d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c02acc0 tx=0x7fbd1c004a40 comp rx=0 tx=0).stop 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd28a00640 1 -- 172.21.15.3:0/2752422838 reap_dead start 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 shutdown_connections 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.991+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 >> 172.21.15.3:0/2752422838 conn(0x7fbd24062cc0 msgr2=0x7fbd2415b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.992+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 shutdown_connections 2024-09-10T07:54:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:19.992+0000 7fbd2aa00640 1 -- 172.21.15.3:0/2752422838 wait complete. 2024-09-10T07:54:20.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T07:54:20.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.164+0000 7f578d200640 1 Processor -- start 2024-09-10T07:54:20.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.164+0000 7f578d200640 1 -- start start 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f578d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f578815e520 0x7f5788073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f578d200640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57880b7110 0x7f578806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f578d200640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5788071780 con 0x7f57880d1b70 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f578d200640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f57880b7860 con 0x7f57880b7110 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f578d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5788073e90 con 0x7f578815e520 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f5787e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57880b7110 0x7f578806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f5787e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57880b7110 0x7f578806bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41826/0 (socket says 172.21.15.3:41826) 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f5787e00640 1 -- 172.21.15.3:0/1265566624 learned_addr learned my addr 172.21.15.3:0/1265566624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.165+0000 7f5787400640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f578815e520 0x7f5788073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.166+0000 7f5787e00640 1 -- 172.21.15.3:0/1265566624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f578815e520 msgr2=0x7f5788073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.166+0000 7f5787e00640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f578815e520 0x7f5788073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.166+0000 7f5787e00640 1 -- 172.21.15.3:0/1265566624 >> v1:172.21.15.70:6789/0 conn(0x7f57880d1b70 legacy=0x7f57880b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.166+0000 7f5786a00640 1 -- 172.21.15.3:0/1265566624 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3960641369 0 0) 0x7f5788071780 con 0x7f57880d1b70 2024-09-10T07:54:20.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.166+0000 7f5787e00640 1 -- 172.21.15.3:0/1265566624 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57880709c0 con 0x7f57880b7110 2024-09-10T07:54:20.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.167+0000 7f5787e00640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880b7110 0x7f578806bb20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5780009e80 tx=0x7f578002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967a4640914db60a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.167+0000 7f5786a00640 1 -- 172.21.15.3:0/1265566624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780036070 con 0x7f57880b7110 2024-09-10T07:54:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.167+0000 7f5786a00640 1 -- 172.21.15.3:0/1265566624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5780031600 con 0x7f57880b7110 2024-09-10T07:54:20.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.167+0000 7f5786a00640 1 -- 172.21.15.3:0/1265566624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800319d0 con 0x7f57880b7110 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 -- 172.21.15.3:0/1265566624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880b7110 msgr2=0x7f578806bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880b7110 0x7f578806bb20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5780009e80 tx=0x7f578002a6c0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 -- 172.21.15.3:0/1265566624 shutdown_connections 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880b7110 0x7f578806bb20 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 --2- 172.21.15.3:0/1265566624 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f578815e520 0x7f5788073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 -- 172.21.15.3:0/1265566624 >> 172.21.15.3:0/1265566624 conn(0x7f57880b41e0 msgr2=0x7f57880b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 -- 172.21.15.3:0/1265566624 shutdown_connections 2024-09-10T07:54:20.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.168+0000 7f578d200640 1 -- 172.21.15.3:0/1265566624 wait complete. 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.169+0000 7f578d200640 1 Processor -- start 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 -- start start 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57880b7110 0x7f5788152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 0x7f578814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5788071260 con 0x7f57880d1b70 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5788070f80 con 0x7f578815e520 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.170+0000 7f578d200640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f578815e900 con 0x7f57880b7110 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41834/0 (socket says 172.21.15.3:41834) 2024-09-10T07:54:20.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 -- 172.21.15.3:0/4050911310 learned_addr learned my addr 172.21.15.3:0/4050911310 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f5787400640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57880b7110 0x7f5788152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f5787e00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 0x7f578814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 -- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57880b7110 msgr2=0x7f5788152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57880b7110 0x7f5788152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 -- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 msgr2=0x7f578814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 0x7f578814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f577ea00640 1 -- 172.21.15.3:0/4050911310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5788155c90 con 0x7f57880d1b70 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.171+0000 7f5787e00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 0x7f578814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f577ea00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f577000c4b0 tx=0x7f577000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770017070 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5770005480 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770003830 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57881e8180 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.172+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881e8740 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.173+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788073950 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.174+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f577001a070 con 0x7f57880d1b70 2024-09-10T07:54:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.175+0000 7f577fe00640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 0x7f576406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.175+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f5770089fe0 con 0x7f57880d1b70 2024-09-10T07:54:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.175+0000 7f5787400640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 0x7f576406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.176+0000 7f5787400640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 0x7f576406e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5778009aa0 tx=0x7f5778009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.179+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57700584f0 con 0x7f57880d1b70 2024-09-10T07:54:20.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.361+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5788069140 con 0x7f576406be30 2024-09-10T07:54:20.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.362+0000 7f577fe00640 1 -- 172.21.15.3:0/4050911310 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5788069140 con 0x7f576406be30 2024-09-10T07:54:20.362 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T07:54:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 msgr2=0x7f576406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 0x7f576406e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5778009aa0 tx=0x7f5778009190 comp rx=0 tx=0).stop 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 msgr2=0x7f57881555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f577000c4b0 tx=0x7f577000c970 comp rx=0 tx=0).stop 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 shutdown_connections 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f576406be30 0x7f576406e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f578815e520 0x7f578814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57880d1b70 0x7f57881555e0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 --2- 172.21.15.3:0/4050911310 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f57880b7110 0x7f5788152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 >> 172.21.15.3:0/4050911310 conn(0x7f57880b41e0 msgr2=0x7f578806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 shutdown_connections 2024-09-10T07:54:20.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.364+0000 7f578d200640 1 -- 172.21.15.3:0/4050911310 wait complete. 2024-09-10T07:54:20.381 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T07:54:20.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T07:54:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.542+0000 7fc853400640 1 Processor -- start 2024-09-10T07:54:20.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.543+0000 7fc853400640 1 -- start start 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc853400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc84c0cd910 0x7fc84c0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc853400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc853400640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc84c0cf1f0 con 0x7fc84c15c530 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc853400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc84c0d00c0 con 0x7fc84c0cd910 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc853400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c0b2960 con 0x7fc84c0b24d0 2024-09-10T07:54:20.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc851400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc84be00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc84c0cd910 0x7fc84c0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc851400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.3:34272/0 (socket says 172.21.15.3:34272) 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.544+0000 7fc851400640 1 -- 172.21.15.3:0/2980890612 learned_addr learned my addr 172.21.15.3:0/2980890612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.545+0000 7fc84be00640 1 -- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 msgr2=0x7fc84c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.545+0000 7fc84be00640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.545+0000 7fc84be00640 1 -- 172.21.15.3:0/2980890612 >> v1:172.21.15.70:6789/0 conn(0x7fc84c15c530 legacy=0x7fc84c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.545+0000 7fc84be00640 1 -- 172.21.15.3:0/2980890612 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c0ce150 con 0x7fc84c0cd910 2024-09-10T07:54:20.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.545+0000 7fc851400640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc84be00640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c0b1f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097d0 tx=0x7fc83802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2341e8e16b391b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc84b400640 1 -- 172.21.15.3:0/2980890612 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838037070 con 0x7fc84c0cd910 2024-09-10T07:54:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc84b400640 1 -- 172.21.15.3:0/2980890612 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc838032440 con 0x7fc84c0cd910 2024-09-10T07:54:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc84b400640 1 -- 172.21.15.3:0/2980890612 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838032810 con 0x7fc84c0cd910 2024-09-10T07:54:20.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc853400640 1 -- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 msgr2=0x7fc84c0b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.546+0000 7fc853400640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c0b1f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097d0 tx=0x7fc83802a1c0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 -- 172.21.15.3:0/2980890612 shutdown_connections 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 --2- 172.21.15.3:0/2980890612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c0b1f90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 -- 172.21.15.3:0/2980890612 >> 172.21.15.3:0/2980890612 conn(0x7fc84c062cc0 msgr2=0x7fc84c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 -- 172.21.15.3:0/2980890612 shutdown_connections 2024-09-10T07:54:20.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.547+0000 7fc853400640 1 -- 172.21.15.3:0/2980890612 wait complete. 2024-09-10T07:54:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.548+0000 7fc853400640 1 Processor -- start 2024-09-10T07:54:20.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.548+0000 7fc853400640 1 -- start start 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 0x7fc84c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c0ce9f0 con 0x7fc84c15c530 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c0ce750 con 0x7fc84c0cd910 2024-09-10T07:54:20.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.549+0000 7fc853400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c0cecd0 con 0x7fc84c0b24d0 2024-09-10T07:54:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 0x7fc84c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 0x7fc84c1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41850/0 (socket says 172.21.15.3:41850) 2024-09-10T07:54:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 -- 172.21.15.3:0/3763157066 learned_addr learned my addr 172.21.15.3:0/3763157066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc84be00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc850a00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 -- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 msgr2=0x7fc84c157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc84c0b24d0 0x7fc84c157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 -- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 msgr2=0x7fc84c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc84c0cd910 0x7fc84c15f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c1e7610 con 0x7fc84c15c530 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.550+0000 7fc851400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 0x7fc84c1574e0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc844009f10 tx=0x7fc84400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.551+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84400cc40 con 0x7fc84c15c530 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.551+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84400cde0 con 0x7fc84c15c530 2024-09-10T07:54:20.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.551+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844003bb0 con 0x7fc84c15c530 2024-09-10T07:54:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.551+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c1e9f20 con 0x7fc84c15c530 2024-09-10T07:54:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.551+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1e7330 con 0x7fc84c15c530 2024-09-10T07:54:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.552+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc84c1e6e00 con 0x7fc84c15c530 2024-09-10T07:54:20.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.553+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc84401b030 con 0x7fc84c15c530 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.553+0000 7fc848c00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc82c06be30 0x7fc82c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fc84408af30 con 0x7fc84c15c530 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc848c00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7fc82c071e30 0x7fc82c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc84400ae90 con 0x7fc82c071e30 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7fc84408d550 con 0x7fc84c15c530 2024-09-10T07:54:20.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc850a00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc82c06be30 0x7fc82c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc84be00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7fc82c071e30 0x7fc82c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.554+0000 7fc850a00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc82c06be30 0x7fc82c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0d0980 tx=0x7fc840009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.555+0000 7fc84be00640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7fc82c071e30 0x7fc82c074250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.556+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== osd.7 v2:172.21.15.70:6808/3218842598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc84400ae90 con 0x7fc82c071e30 2024-09-10T07:54:20.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.569+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 --> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc84c0b1f90 con 0x7fc82c071e30 2024-09-10T07:54:20.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc848c00640 1 -- 172.21.15.3:0/3763157066 <== osd.7 v2:172.21.15.70:6808/3218842598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc84c0b1f90 con 0x7fc82c071e30 2024-09-10T07:54:20.569 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-10T07:54:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T07:54:20.569 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-10T07:54:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7fc82c071e30 msgr2=0x7fc82c074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.70:6808/3218842598,v1:172.21.15.70:6809/3218842598] conn(0x7fc82c071e30 0x7fc82c074250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc82c06be30 msgr2=0x7fc82c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fc82c06be30 0x7fc82c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0d0980 tx=0x7fc840009040 comp rx=0 tx=0).stop 2024-09-10T07:54:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 msgr2=0x7fc84c1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 --2- 172.21.15.3:0/3763157066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc84c15c530 0x7fc84c1574e0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc844009f10 tx=0x7fc84400c490 comp rx=0 tx=0).stop 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc851400640 1 -- 172.21.15.3:0/3763157066 reap_dead start 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 shutdown_connections 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 >> 172.21.15.3:0/3763157066 conn(0x7fc84c062cc0 msgr2=0x7fc84c0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 shutdown_connections 2024-09-10T07:54:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.570+0000 7fc853400640 1 -- 172.21.15.3:0/3763157066 wait complete. 2024-09-10T07:54:20.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T07:54:20.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.741+0000 7f07abe00640 1 Processor -- start 2024-09-10T07:54:20.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.741+0000 7f07abe00640 1 -- start start 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.742+0000 7f07abe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a415f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07abe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07a406c230 0x7f07a40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07abe00640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a415fbb0 con 0x7f07a415fdb0 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07abe00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f07a40b36a0 con 0x7f07a406c230 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07abe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f07a40b3980 con 0x7f07a4161b30 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07a406c230 0x7f07a40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07a406c230 0x7f07a40b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41866/0 (socket says 172.21.15.3:41866) 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 -- 172.21.15.3:0/167229196 learned_addr learned my addr 172.21.15.3:0/167229196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a8a00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f07a415fdb0 0x7f07a406bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42538/0 (socket says 172.21.15.3:42538) 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9400640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 -- 172.21.15.3:0/167229196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 msgr2=0x7f07a415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 -- 172.21.15.3:0/167229196 >> v1:172.21.15.3:6789/0 conn(0x7f07a415fdb0 legacy=0x7f07a406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9e00640 1 -- 172.21.15.3:0/167229196 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a40b3c60 con 0x7f07a406c230 2024-09-10T07:54:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.743+0000 7f07a9400640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.744+0000 7f07a9e00640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40b4740 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f079c009e50 tx=0x7f079c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda9ae3ba737e115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07a3e00640 1 -- 172.21.15.3:0/167229196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c036070 con 0x7f07a406c230 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07a3e00640 1 -- 172.21.15.3:0/167229196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c031600 con 0x7f07a406c230 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07a3e00640 1 -- 172.21.15.3:0/167229196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c0319d0 con 0x7f07a406c230 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 -- 172.21.15.3:0/167229196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 msgr2=0x7f07a40b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40b4740 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f079c009e50 tx=0x7f079c02a710 comp rx=0 tx=0).stop 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 -- 172.21.15.3:0/167229196 shutdown_connections 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40b4740 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 --2- 172.21.15.3:0/167229196 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 -- 172.21.15.3:0/167229196 >> 172.21.15.3:0/167229196 conn(0x7f07a406d000 msgr2=0x7f07a406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 -- 172.21.15.3:0/167229196 shutdown_connections 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.745+0000 7f07abe00640 1 -- 172.21.15.3:0/167229196 wait complete. 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.747+0000 7f07abe00640 1 Processor -- start 2024-09-10T07:54:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.747+0000 7f07abe00640 1 -- start start 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 0x7f07a40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4161f10 con 0x7f07a406c230 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40b4220 con 0x7f07a415fdb0 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07abe00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f07a40aefa0 con 0x7f07a4161b30 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07a9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07a9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41868/0 (socket says 172.21.15.3:41868) 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07a9400640 1 -- 172.21.15.3:0/3954814688 learned_addr learned my addr 172.21.15.3:0/3954814688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07a9e00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.748+0000 7f07a8a00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 0x7f07a40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 -- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 msgr2=0x7f07a40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a40cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 -- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 msgr2=0x7f07a40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 0x7f07a40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 -- 172.21.15.3:0/3954814688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a40b1950 con 0x7f07a406c230 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a8a00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 0x7f07a40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a9400640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f079800e810 tx=0x7f079800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079800f040 con 0x7f07a406c230 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0798004030 con 0x7f07a406c230 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798004400 con 0x7f07a406c230 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a40b0d40 con 0x7f07a406c230 2024-09-10T07:54:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.749+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a40b1300 con 0x7f07a406c230 2024-09-10T07:54:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.750+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a40b3300 con 0x7f07a406c230 2024-09-10T07:54:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.751+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f079800c6d0 con 0x7f07a406c230 2024-09-10T07:54:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.752+0000 7f07a1600640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 0x7f078406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.752+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f079808a9a0 con 0x7f07a406c230 2024-09-10T07:54:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.752+0000 7f07a8a00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 0x7f078406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.753+0000 7f07a8a00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 0x7f078406e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0790007fd0 tx=0x7f0790007d60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.756+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0798017020 con 0x7f07a406c230 2024-09-10T07:54:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.938+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f07a4074200 con 0x7f078406be30 2024-09-10T07:54:20.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.939+0000 7f07a1600640 1 -- 172.21.15.3:0/3954814688 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f07a4074200 con 0x7f078406be30 2024-09-10T07:54:20.939 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 msgr2=0x7f078406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 0x7f078406e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0790007fd0 tx=0x7f0790007d60 comp rx=0 tx=0).stop 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 msgr2=0x7f07a40cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f079800e810 tx=0x7f079800ecd0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 shutdown_connections 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f078406be30 0x7f078406e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f07a4161b30 0x7f07a40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07a415fdb0 0x7f07a40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 --2- 172.21.15.3:0/3954814688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a406c230 0x7f07a40cf160 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:20.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 >> 172.21.15.3:0/3954814688 conn(0x7f07a406d000 msgr2=0x7f07a41592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:20.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 shutdown_connections 2024-09-10T07:54:20.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:20.941+0000 7f07abe00640 1 -- 172.21.15.3:0/3954814688 wait complete. 2024-09-10T07:54:20.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:54:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.113+0000 7fcea9400640 1 Processor -- start 2024-09-10T07:54:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.113+0000 7fcea9400640 1 -- start start 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.114+0000 7fcea9400640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea4160af0 0x7fcea40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.114+0000 7fcea9400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcea40d05f0 0x7fcea4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.114+0000 7fcea9400640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcea40cfdd0 con 0x7fcea4160710 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.114+0000 7fcea9400640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcea415cc90 con 0x7fcea4160af0 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.114+0000 7fcea9400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcea415cf70 con 0x7fcea40d05f0 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea4160af0 0x7fcea40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcea4160af0 0x7fcea40d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41878/0 (socket says 172.21.15.3:41878) 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fcea2800640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcea4160710 0x7fcea40cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42546/0 (socket says 172.21.15.3:42546) 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 -- 172.21.15.3:0/1282575113 learned_addr learned my addr 172.21.15.3:0/1282575113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fcea3200640 1 --2- 172.21.15.3:0/1282575113 >> v2:172.21.15.70:3300/0 conn(0x7fcea40d05f0 0x7fcea4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 -- 172.21.15.3:0/1282575113 >> v2:172.21.15.70:3300/0 conn(0x7fcea40d05f0 msgr2=0x7fcea4072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:21.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 --2- 172.21.15.3:0/1282575113 >> v2:172.21.15.70:3300/0 conn(0x7fcea40d05f0 0x7fcea4072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 -- 172.21.15.3:0/1282575113 >> v1:172.21.15.3:6789/0 conn(0x7fcea4160710 legacy=0x7fcea40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fce9be00640 1 -- 172.21.15.3:0/1282575113 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea415d250 con 0x7fcea4160af0 2024-09-10T07:54:21.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.115+0000 7fcea1e00640 1 -- 172.21.15.3:0/1282575113 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278609312 0 0) 0x7fcea40cfdd0 con 0x7fcea4160710 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.116+0000 7fce9be00640 1 --2- 172.21.15.3:0/1282575113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea40d00b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcea40718d0 tx=0x7fce8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c52be1287750c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea1e00640 1 -- 172.21.15.3:0/1282575113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c037070 con 0x7fcea4160af0 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea1e00640 1 -- 172.21.15.3:0/1282575113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c032440 con 0x7fcea4160af0 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea1e00640 1 -- 172.21.15.3:0/1282575113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c032810 con 0x7fcea4160af0 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 -- 172.21.15.3:0/1282575113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 msgr2=0x7fcea40d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 --2- 172.21.15.3:0/1282575113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea40d00b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcea40718d0 tx=0x7fce8c02a270 comp rx=0 tx=0).stop 2024-09-10T07:54:21.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 -- 172.21.15.3:0/1282575113 shutdown_connections 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 --2- 172.21.15.3:0/1282575113 >> v2:172.21.15.70:3300/0 conn(0x7fcea40d05f0 0x7fcea4072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 --2- 172.21.15.3:0/1282575113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea40d00b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 -- 172.21.15.3:0/1282575113 >> 172.21.15.3:0/1282575113 conn(0x7fcea406d000 msgr2=0x7fcea406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 -- 172.21.15.3:0/1282575113 shutdown_connections 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.117+0000 7fcea9400640 1 -- 172.21.15.3:0/1282575113 wait complete. 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.119+0000 7fcea9400640 1 Processor -- start 2024-09-10T07:54:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.119+0000 7fcea9400640 1 -- start start 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 0x7fcea4156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 0x7fcea41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcea415d770 con 0x7fcea4160af0 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcea415c960 con 0x7fcea40d05f0 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea9400640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcea4156070 con 0x7fcea4160710 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41884/0 (socket says 172.21.15.3:41884) 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.120+0000 7fcea3200640 1 -- 172.21.15.3:0/2546996364 learned_addr learned my addr 172.21.15.3:0/2546996364 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea2800640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 0x7fcea4156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 -- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 msgr2=0x7fcea41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fce9be00640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 0x7fcea41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 0x7fcea41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 -- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 msgr2=0x7fcea4156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 0x7fcea4156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 -- 172.21.15.3:0/2546996364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea4155180 con 0x7fcea4160af0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fce9be00640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 0x7fcea41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea2800640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 0x7fcea4156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea3200640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00efc0 tx=0x7fce9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018020 con 0x7fcea4160af0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9c014480 con 0x7fcea4160af0 2024-09-10T07:54:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.121+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4155460 con 0x7fcea4160af0 2024-09-10T07:54:21.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.122+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c014870 con 0x7fcea4160af0 2024-09-10T07:54:21.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.122+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea4155a20 con 0x7fcea4160af0 2024-09-10T07:54:21.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.123+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea40cfa60 con 0x7fcea4160af0 2024-09-10T07:54:21.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.124+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce9c005d10 con 0x7fcea4160af0 2024-09-10T07:54:21.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.124+0000 7fce9aa00640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:21.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.125+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fce9c08baf0 con 0x7fcea4160af0 2024-09-10T07:54:21.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.125+0000 7fcea2800640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:21.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.125+0000 7fcea2800640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce94000f60 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:21.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.129+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce9c022030 con 0x7fcea4160af0 2024-09-10T07:54:21.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.310+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcea40afd50 con 0x7fce8006be30 2024-09-10T07:54:21.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.310+0000 7fce9aa00640 1 -- 172.21.15.3:0/2546996364 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7fcea40afd50 con 0x7fce8006be30 2024-09-10T07:54:21.310 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:21.310 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:54:21.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.312+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 msgr2=0x7fce8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.312+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce94000f60 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 msgr2=0x7fcea4154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00efc0 tx=0x7fce9c00c490 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 shutdown_connections 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcea4160af0 0x7fcea4154970 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcea4160710 0x7fcea41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 --2- 172.21.15.3:0/2546996364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea40d05f0 0x7fcea4156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 >> 172.21.15.3:0/2546996364 conn(0x7fcea406d000 msgr2=0x7fcea406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 shutdown_connections 2024-09-10T07:54:21.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:21.313+0000 7fcea9400640 1 -- 172.21.15.3:0/2546996364 wait complete. 2024-09-10T07:54:21.327 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-10T07:54:19.595926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":264940,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709780,"statfs":{"total":773094113280,"available":772822814720,"internally_reserved":0,"allocated":11542528,"data_stored":5664904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.796439"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":322,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-10T07:54:12.770263+0000","last_change":"2024-09-10T07:46:23.663159+0000","last_active":"2024-09-10T07:54:12.770263+0000","last_peered":"2024-09-10T07:54:12.770263+0000","last_clean":"2024-09-10T07:54:12.770263+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:54:12.770263+0000","last_undegraded":"2024-09-10T07:54:12.770263+0000","last_fullsized":"2024-09-10T07:54:12.770263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T07:46:22.062699+0000","last_clean_scrub_stamp":"2024-09-10T07:46:22.062699+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.854997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476835,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476835,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476833,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1150889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1150889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:54:21.334 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T07:46:22.062699+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=7, tm_min=46, tm_sec=22, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=7, tm_min=54, tm_sec=15, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T07:54:21.334 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T07:54:23.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:23.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:41.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T07:54:41.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.506+0000 7fbdf8c00640 1 Processor -- start 2024-09-10T07:54:41.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.506+0000 7fbdf8c00640 1 -- start start 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 0x7fbdf40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbdf40b2540 0x7fbdf40cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40cfb00 0x7fbdf415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbdf415bac0 con 0x7fbdf40b2540 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbdf415bd30 con 0x7fbdf415b330 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf415c010 con 0x7fbdf40cfb00 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdebe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbdf40b2540 0x7fbdf40cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdebe00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbdf40b2540 0x7fbdf40cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50256/0 (socket says 172.21.15.3:50256) 2024-09-10T07:54:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.507+0000 7fbdebe00640 1 -- 172.21.15.3:0/2483591857 learned_addr learned my addr 172.21.15.3:0/2483591857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdf3e00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40cfb00 0x7fbdf415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdf3400640 1 --2- 172.21.15.3:0/2483591857 >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 0x7fbdf40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdebe00640 1 -- 172.21.15.3:0/2483591857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40cfb00 msgr2=0x7fbdf415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdebe00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40cfb00 0x7fbdf415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdebe00640 1 -- 172.21.15.3:0/2483591857 >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 msgr2=0x7fbdf40b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdebe00640 1 --2- 172.21.15.3:0/2483591857 >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 0x7fbdf40b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdebe00640 1 -- 172.21.15.3:0/2483591857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf415c2f0 con 0x7fbdf40b2540 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.508+0000 7fbdf3400640 1 --2- 172.21.15.3:0/2483591857 >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 0x7fbdf40b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.509+0000 7fbdebe00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf40b2540 0x7fbdf40cf5c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f920 tx=0x7fbddc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5a008b6c4f5dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.509+0000 7fbdf2a00640 1 -- 172.21.15.3:0/2483591857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc037070 con 0x7fbdf40b2540 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.509+0000 7fbdf2a00640 1 -- 172.21.15.3:0/2483591857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc032440 con 0x7fbdf40b2540 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.509+0000 7fbdf2a00640 1 -- 172.21.15.3:0/2483591857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc032810 con 0x7fbdf40b2540 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 -- 172.21.15.3:0/2483591857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf40b2540 msgr2=0x7fbdf40cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf40b2540 0x7fbdf40cf5c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f920 tx=0x7fbddc02a1e0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 -- 172.21.15.3:0/2483591857 shutdown_connections 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40cfb00 0x7fbdf415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/2483591857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf40b2540 0x7fbdf40cf5c0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/2483591857 >> v2:172.21.15.70:3300/0 conn(0x7fbdf415b330 0x7fbdf40b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 -- 172.21.15.3:0/2483591857 >> 172.21.15.3:0/2483591857 conn(0x7fbdf406bb20 msgr2=0x7fbdf406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 -- 172.21.15.3:0/2483591857 shutdown_connections 2024-09-10T07:54:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.510+0000 7fbdf8c00640 1 -- 172.21.15.3:0/2483591857 wait complete. 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.511+0000 7fbdf8c00640 1 Processor -- start 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 -- start start 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 0x7fbdf415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 0x7fbdf40affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf415b710 con 0x7fbdf415b330 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf40ce1c0 con 0x7fbdf40cfb00 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.512+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf415db80 con 0x7fbdf40b2540 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50262/0 (socket says 172.21.15.3:50262) 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 -- 172.21.15.3:0/43397712 learned_addr learned my addr 172.21.15.3:0/43397712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3400640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 0x7fbdf415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 -- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 msgr2=0x7fbdf415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 0x7fbdf415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdebe00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 0x7fbdf40affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 -- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 msgr2=0x7fbdf40affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 0x7fbdf40affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 -- 172.21.15.3:0/43397712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf415cf70 con 0x7fbdf415b330 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3400640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 0x7fbdf415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdebe00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 0x7fbdf40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.513+0000 7fbdf3e00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbdec007d50 tx=0x7fbdec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.514+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec014420 con 0x7fbdf415b330 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.514+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec019070 con 0x7fbdf415b330 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.514+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec014820 con 0x7fbdf415b330 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.514+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf415d250 con 0x7fbdf415b330 2024-09-10T07:54:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.514+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf415d810 con 0x7fbdf415b330 2024-09-10T07:54:41.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.515+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf4063610 con 0x7fbdf415b330 2024-09-10T07:54:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.516+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdec0149c0 con 0x7fbdf415b330 2024-09-10T07:54:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.517+0000 7fbdeb400640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 0x7fbdd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.517+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fbdec08b4a0 con 0x7fbdf415b330 2024-09-10T07:54:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.517+0000 7fbdf3400640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 0x7fbdd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.518+0000 7fbdf3400640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 0x7fbdd006e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006fd0 tx=0x7fbde4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.522+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbdec05a000 con 0x7fbdf415b330 2024-09-10T07:54:41.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.705+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 --> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdf41510a0 con 0x7fbdd006be30 2024-09-10T07:54:41.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.706+0000 7fbdeb400640 1 -- 172.21.15.3:0/43397712 <== mgr.4113 v2:172.21.15.3:6832/3286701713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fbde4009100 con 0x7fbdd006be30 2024-09-10T07:54:41.706 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:41.707 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-10T07:54:41.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 msgr2=0x7fbdd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 0x7fbdd006e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006fd0 tx=0x7fbde4008040 comp rx=0 tx=0).stop 2024-09-10T07:54:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 msgr2=0x7fbdf4149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbdec007d50 tx=0x7fbdec00e4b0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 shutdown_connections 2024-09-10T07:54:41.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fbdd006be30 0x7fbdd006e2f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbdf415b330 0x7fbdf4149bb0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf40cfb00 0x7fbdf40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 --2- 172.21.15.3:0/43397712 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbdf40b2540 0x7fbdf415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 >> 172.21.15.3:0/43397712 conn(0x7fbdf406bb20 msgr2=0x7fbdf40b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 shutdown_connections 2024-09-10T07:54:41.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.708+0000 7fbdf8c00640 1 -- 172.21.15.3:0/43397712 wait complete. 2024-09-10T07:54:41.723 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-10T07:54:41.600274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":264932,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709788,"statfs":{"total":773094113280,"available":772822822912,"internally_reserved":0,"allocated":11542528,"data_stored":5664904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002618"},"pg_stats":[{"pgid":"1.0","version":"105'119","reported_seq":334,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-10T07:54:18.178863+0000","last_change":"2024-09-10T07:54:18.178863+0000","last_active":"2024-09-10T07:54:18.178863+0000","last_peered":"2024-09-10T07:54:18.178863+0000","last_clean":"2024-09-10T07:54:18.178863+0000","last_became_active":"2024-09-10T07:46:23.660059+0000","last_became_peered":"2024-09-10T07:46:23.660059+0000","last_unstale":"2024-09-10T07:54:18.178863+0000","last_undegraded":"2024-09-10T07:54:18.178863+0000","last_fullsized":"2024-09-10T07:54:18.178863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'119","last_scrub_stamp":"2024-09-10T07:54:18.178733+0000","last_deep_scrub":"105'119","last_deep_scrub_stamp":"2024-09-10T07:54:18.178733+0000","last_clean_scrub_stamp":"2024-09-10T07:54:18.178733+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:05.870760+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1150889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1150889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":560521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T07:54:41.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T07:54:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.877+0000 7fa9b2e00640 1 Processor -- start 2024-09-10T07:54:41.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.877+0000 7fa9b2e00640 1 -- start start 2024-09-10T07:54:41.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2e00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac161b70 0x7fa9ac0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac0d3de0 con 0x7fa9ac0d3860 2024-09-10T07:54:41.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2e00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac161f50 con 0x7fa9ac161b70 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac0d40c0 con 0x7fa9ac0d4cb0 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac161b70 0x7fa9ac0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac161b70 0x7fa9ac0d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50272/0 (socket says 172.21.15.3:50272) 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 -- 172.21.15.3:0/3266999140 learned_addr learned my addr 172.21.15.3:0/3266999140 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b2400640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 -- 172.21.15.3:0/3266999140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 msgr2=0x7fa9ac0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac0cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 -- 172.21.15.3:0/3266999140 >> v1:172.21.15.70:6789/0 conn(0x7fa9ac0d3860 legacy=0x7fa9ac0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:41.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.879+0000 7fa9b1a00640 1 -- 172.21.15.3:0/3266999140 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac0cf690 con 0x7fa9ac161b70 2024-09-10T07:54:41.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.880+0000 7fa9b1a00640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0d3320 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009e50 tx=0x7fa99c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a148dbb4677f50a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.880+0000 7fa9abe00640 1 -- 172.21.15.3:0/3266999140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c036070 con 0x7fa9ac161b70 2024-09-10T07:54:41.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.880+0000 7fa9abe00640 1 -- 172.21.15.3:0/3266999140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99c031600 con 0x7fa9ac161b70 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.880+0000 7fa9abe00640 1 -- 172.21.15.3:0/3266999140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0319d0 con 0x7fa9ac161b70 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 -- 172.21.15.3:0/3266999140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 msgr2=0x7fa9ac0d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0d3320 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009e50 tx=0x7fa99c02a6c0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 -- 172.21.15.3:0/3266999140 shutdown_connections 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac0cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/3266999140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0d3320 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 -- 172.21.15.3:0/3266999140 >> 172.21.15.3:0/3266999140 conn(0x7fa9ac062630 msgr2=0x7fa9ac062a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 -- 172.21.15.3:0/3266999140 shutdown_connections 2024-09-10T07:54:41.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.881+0000 7fa9b2e00640 1 -- 172.21.15.3:0/3266999140 wait complete. 2024-09-10T07:54:41.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.882+0000 7fa9b2e00640 1 Processor -- start 2024-09-10T07:54:41.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.883+0000 7fa9b2e00640 1 -- start start 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.883+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 0x7fa9ac0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.883+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac1440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac0cf2d0 con 0x7fa9ac161b70 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac0c9fc0 con 0x7fa9ac0d3860 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac0cf450 con 0x7fa9ac0d4cb0 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b1a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 0x7fa9ac0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b1a00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 0x7fa9ac0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.3:36610/0 (socket says 172.21.15.3:36610) 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b1a00640 1 -- 172.21.15.3:0/1553344938 learned_addr learned my addr 172.21.15.3:0/1553344938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:41.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b1000640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac1440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 -- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 msgr2=0x7fa9ac1440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac1440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 -- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 msgr2=0x7fa9ac0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 0x7fa9ac0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.884+0000 7fa9b2400640 1 -- 172.21.15.3:0/1553344938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac0b6280 con 0x7fa9ac161b70 2024-09-10T07:54:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9b2400640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0b6a40 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b4180 tx=0x7fa9a400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40125f0 con 0x7fa9ac161b70 2024-09-10T07:54:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a4019070 con 0x7fa9ac161b70 2024-09-10T07:54:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40129b0 con 0x7fa9ac161b70 2024-09-10T07:54:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac0cddd0 con 0x7fa9ac161b70 2024-09-10T07:54:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.885+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac0ce390 con 0x7fa9ac161b70 2024-09-10T07:54:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.886+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac0b7540 con 0x7fa9ac161b70 2024-09-10T07:54:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.887+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9a4012b50 con 0x7fa9ac161b70 2024-09-10T07:54:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.888+0000 7fa9a9600640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 0x7fa98c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.888+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7fa9a408aea0 con 0x7fa9ac161b70 2024-09-10T07:54:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.888+0000 7fa9b1a00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 0x7fa98c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.889+0000 7fa9b1a00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 0x7fa98c06e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa99c000f80 tx=0x7fa99c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:41.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:41.892+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9a40593c0 con 0x7fa9ac161b70 2024-09-10T07:54:42.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.076+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fa9ac0b9950 con 0x7fa9ac161b70 2024-09-10T07:54:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.080+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fa9a405d6c0 con 0x7fa9ac161b70 2024-09-10T07:54:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.081+0000 7fa9a9600640 1 -- 172.21.15.3:0/1553344938 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa9a4061ef0 con 0x7fa9ac161b70 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 msgr2=0x7fa98c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 0x7fa98c06e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa99c000f80 tx=0x7fa99c009000 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 msgr2=0x7fa9ac0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0b6a40 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0b4180 tx=0x7fa9a400d9c0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 shutdown_connections 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7fa98c06be30 0x7fa98c06e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac161b70 0x7fa9ac0b6a40 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa9ac0d4cb0 0x7fa9ac1440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 --2- 172.21.15.3:0/1553344938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9ac0d3860 0x7fa9ac0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 >> 172.21.15.3:0/1553344938 conn(0x7fa9ac062630 msgr2=0x7fa9ac0b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 shutdown_connections 2024-09-10T07:54:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.083+0000 7fa9b2e00640 1 -- 172.21.15.3:0/1553344938 wait complete. 2024-09-10T07:54:42.099 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T07:54:42.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T07:54:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 Processor -- start 2024-09-10T07:54:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 -- start start 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ce815e860 0x7f8ce80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ce80d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ce80cee40 con 0x7f8ce80d4150 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce80d3780 con 0x7f8ce815e860 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8ced800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce80cf120 con 0x7f8ce80cfbf0 2024-09-10T07:54:42.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8cece00640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ce80d4150 0x7f8ce80b7880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53514/0 (socket says 172.21.15.3:53514) 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.256+0000 7f8cece00640 1 -- 172.21.15.3:0/308473689 learned_addr learned my addr 172.21.15.3:0/308473689 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061908871 0 0) 0x7f8ce80cee40 con 0x7f8ce80d4150 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/308473689 >> v2:172.21.15.70:3300/0 conn(0x7f8ce815e860 0x7f8ce80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ccc003520 con 0x7f8ce80d4150 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce7400640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ce80d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2254410568 0 0) 0x7f8ccc003520 con 0x7f8ce80d4150 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 msgr2=0x7f8ce80d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ce80d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 >> v2:172.21.15.70:3300/0 conn(0x7f8ce815e860 msgr2=0x7f8ce80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 --2- 172.21.15.3:0/308473689 >> v2:172.21.15.70:3300/0 conn(0x7f8ce815e860 0x7f8ce80cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce80d3410 con 0x7f8ce80d4150 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4143130971 0 0) 0x7f8ce00019f0 con 0x7f8ce80d4150 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 >> v1:172.21.15.3:6789/0 conn(0x7f8ce80d4150 legacy=0x7f8ce80b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ccc003a20 0x7f8ccc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ccc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ccc004d40 0x7f8ccc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce80d3780 con 0x7f8ccc004d40 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce80cf120 con 0x7f8ccc003a20 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.257+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc00b9f0 con 0x7f8ce80cfbf0 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ccc003a20 0x7f8ccc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8cece00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ccc004d40 0x7f8ccc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7400640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ccc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 -- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 msgr2=0x7f8ccc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ce80cfbf0 0x7f8ccc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 -- 172.21.15.3:0/308473689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ccc004d40 msgr2=0x7f8ccc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ccc004d40 0x7f8ccc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8ce7e00640 1 -- 172.21.15.3:0/308473689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc003520 con 0x7f8ccc003a20 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.258+0000 7f8cece00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ccc004d40 0x7f8ccc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:54:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.259+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ccc003a20 0x7f8ccc003e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ac0 tx=0x7f8ccc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da84967d03728ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8036070 con 0x7f8ccc003a20 2024-09-10T07:54:42.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8cd8031400 con 0x7f8ccc003a20 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ce6a00640 1 -- 172.21.15.3:0/308473689 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8009690 con 0x7f8ccc003a20 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 -- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ccc003a20 msgr2=0x7f8ccc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 --2- 172.21.15.3:0/308473689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ccc003a20 0x7f8ccc003e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ac0 tx=0x7f8ccc005830 comp rx=0 tx=0).stop 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8cece00640 1 -- 172.21.15.3:0/308473689 reap_dead start 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 -- 172.21.15.3:0/308473689 shutdown_connections 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 -- 172.21.15.3:0/308473689 >> 172.21.15.3:0/308473689 conn(0x7f8ce80d4650 msgr2=0x7f8ce80d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 -- 172.21.15.3:0/308473689 shutdown_connections 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.260+0000 7f8ced800640 1 -- 172.21.15.3:0/308473689 wait complete. 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.262+0000 7f8ced800640 1 Processor -- start 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.262+0000 7f8ced800640 1 -- start start 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ccc003a20 0x7f8ce80c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ce8153d60 0x7f8ce80cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce80d3c10 con 0x7f8ce80c8c70 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce80d2460 con 0x7f8ce8153d60 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ced800640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce80d3d90 con 0x7f8ccc003a20 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8cece00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8cece00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50300/0 (socket says 172.21.15.3:50300) 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8cece00640 1 -- 172.21.15.3:0/3456336834 learned_addr learned my addr 172.21.15.3:0/3456336834 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ccc003a20 0x7f8ce80c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.263+0000 7f8ce7400640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ce8153d60 0x7f8ce80cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 -- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ccc003a20 msgr2=0x7f8ce80c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ccc003a20 0x7f8ce80c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 -- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ce8153d60 msgr2=0x7f8ce80cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ce8153d60 0x7f8ce80cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 -- 172.21.15.3:0/3456336834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce81541e0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cece00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8ce808daa0 tx=0x7f8ce000af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000a3f0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8ce0018070 con 0x7f8ce80c8c70 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00133f0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce80c9630 con 0x7f8ce80c8c70 2024-09-10T07:54:42.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.264+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce80c98d0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.266+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce80d0660 con 0x7f8ce80c8c70 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.266+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ce0013590 con 0x7f8ce80c8c70 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.267+0000 7f8cdfe00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 0x7f8cc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.267+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5115+0+0 (secure 0 0 0) 0x7f8ce008b0d0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.268+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 0x7f8cc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:54:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.268+0000 7f8ce7e00640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 0x7f8cc806e320 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8033000 tx=0x7f8cd8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:54:42.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.272+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ce80d0660 con 0x7f8ce80c8c70 2024-09-10T07:54:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.492+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ce80cf6b0 con 0x7f8ce80c8c70 2024-09-10T07:54:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.493+0000 7f8cdfe00640 1 -- 172.21.15.3:0/3456336834 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3903 (secure 0 0 0) 0x7f8ce0059c30 con 0x7f8ce80c8c70 2024-09-10T07:54:42.492 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-10T07:54:42.493 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":116,"btime":"2024-09-10T07:54:08:777550+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1899547454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1899547454},{"type":"v1","addr":"172.21.15.70:6834","nonce":1899547454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":7430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/695440629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":695440629},{"type":"v1","addr":"172.21.15.70:6835","nonce":695440629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8886,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6836/546435152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":546435152},{"type":"v1","addr":"172.21.15.3:6836","nonce":546435152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3942846614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6835","nonce":3942846614},{"type":"v1","addr":"172.21.15.3:6837","nonce":3942846614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[]} 2024-09-10T07:54:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 116 2024-09-10T07:54:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 msgr2=0x7f8cc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 0x7f8cc806e320 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8033000 tx=0x7f8cd8008030 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 msgr2=0x7f8ce80c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f8ce808daa0 tx=0x7f8ce000af00 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 shutdown_connections 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:6832/3286701713,v1:172.21.15.3:6833/3286701713] conn(0x7f8cc806be60 0x7f8cc806e320 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce80c8c70 0x7f8ce80c90f0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ce8153d60 0x7f8ce80cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 --2- 172.21.15.3:0/3456336834 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8ccc003a20 0x7f8ce80c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 >> 172.21.15.3:0/3456336834 conn(0x7f8ce80d4650 msgr2=0x7f8ce8160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 shutdown_connections 2024-09-10T07:54:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-10T07:54:42.495+0000 7f8ced800640 1 -- 172.21.15.3:0/3456336834 wait complete. 2024-09-10T07:54:42.511 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T07:54:42.511 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T07:54:42.511 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T07:54:42.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.525 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T07:54:42.525 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T07:54:42.525 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.579 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T07:54:42.579 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T07:54:42.579 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.597 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T07:54:42.597 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T07:54:42.597 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.654 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T07:54:42.655 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T07:54:42.655 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T07:54:42.655 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.701 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T07:54:42.701 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T07:54:42.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.744 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T07:54:42.744 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T07:54:42.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.793 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T07:54:42.794 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T07:54:42.794 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.842 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T07:54:42.842 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T07:54:42.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.885 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T07:54:42.886 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T07:54:42.886 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.934 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T07:54:42.934 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T07:54:42.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:42.997 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T07:54:42.998 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T07:54:42.998 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.056 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T07:54:43.056 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T07:54:43.056 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T07:54:43.056 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.125 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T07:54:43.125 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T07:54:43.125 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.189 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T07:54:43.189 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T07:54:43.190 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T07:54:43.190 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.211 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T07:54:43.212 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T07:54:43.212 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.230 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T07:54:43.231 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T07:54:43.231 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T07:54:43.291 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T07:54:43.292 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T07:54:43.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Scrub error on inode' | head -n 1 2024-09-10T07:54:43.370 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:43.370 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T07:54:43.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:43.627 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T07:54:43.757 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:43.758 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T07:54:43.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:43.858 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T07:54:43.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:54:43.984 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T07:54:44.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:54:44.237 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T07:54:44.369 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:54:44.369 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T07:54:44.501 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:54:44.502 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T07:54:44.627 INFO:tasks.ceph:Archiving mon data... 2024-09-10T07:54:44.628 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/data/mon.a.tgz 2024-09-10T07:54:44.628 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-10T07:54:44.665 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.3BltK3axp6 2024-09-10T07:54:44.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3BltK3axp6 2024-09-10T07:54:44.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.3BltK3axp6 2024-09-10T07:54:44.965 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.3BltK3axp6 is 444KB 2024-09-10T07:54:45.057 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.3BltK3axp6 2024-09-10T07:54:45.091 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/data/mon.b.tgz 2024-09-10T07:54:45.092 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-10T07:54:45.128 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.9799OXzmWe 2024-09-10T07:54:45.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9799OXzmWe 2024-09-10T07:54:45.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.9799OXzmWe 2024-09-10T07:54:45.428 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.9799OXzmWe is 451KB 2024-09-10T07:54:45.518 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.9799OXzmWe 2024-09-10T07:54:45.553 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/data/mon.c.tgz 2024-09-10T07:54:45.553 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-10T07:54:45.627 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.HPfzHeasOT 2024-09-10T07:54:45.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HPfzHeasOT 2024-09-10T07:54:45.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.HPfzHeasOT 2024-09-10T07:54:45.908 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.HPfzHeasOT is 451KB 2024-09-10T07:54:45.997 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.HPfzHeasOT 2024-09-10T07:54:46.032 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T07:54:46.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T07:54:46.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T07:54:46.164 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T07:54:46.199 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T07:54:46.199 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T07:54:46.201 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi003/crash 2024-09-10T07:54:46.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T07:54:46.262 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi070/crash 2024-09-10T07:54:46.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T07:54:46.317 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T07:54:46.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:46.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:46.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:46.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T07:54:46.532 INFO:tasks.ceph:Compressing logs... 2024-09-10T07:54:46.532 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T07:54:46.536 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T07:54:46.587 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52972.log 2024-09-10T07:54:46.587 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46820.log 2024-09-10T07:54:46.588 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52972.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T07:54:46.589 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52972.log.gz 2024-09-10T07:54:46.589 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65661.log 2024-09-10T07:54:46.589 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46820.log.gz 2024-09-10T07:54:46.590 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40597.log 2024-09-10T07:54:46.590 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2024-09-10T07:54:46.591 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36110.log 2024-09-10T07:54:46.591 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44702.log 2024-09-10T07:54:46.591 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65661.log.gz 2024-09-10T07:54:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42580.log 2024-09-10T07:54:46.593 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2024-09-10T07:54:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.2.44702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44702.log.gz 2024-09-10T07:54:46.593 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65035.log 2024-09-10T07:54:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.1.40597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41015.log 2024-09-10T07:54:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40597.log.gz 2024-09-10T07:54:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T07:54:46.594 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36110.log.gz 2024-09-10T07:54:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.3.41015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41015.log.gz 2024-09-10T07:54:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T07:54:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35988.log 2024-09-10T07:54:46.597 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-10T07:54:46.597 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.35988.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T07:54:46.597 INFO:teuthology.orchestra.run.smithi070.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35988.log.gz 2024-09-10T07:54:46.597 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36159.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T07:54:46.597 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-10T07:54:46.598 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71423.log 2024-09-10T07:54:46.598 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42580.log.gz 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57080.log 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65035.log.gz 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71423.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71423.log.gz 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59667.log 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57080.log.gz 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67226.log 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.59667.log.gz 2024-09-10T07:54:46.599 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49431.log 2024-09-10T07:54:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36239.log 2024-09-10T07:54:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41011.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36239.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36239.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.3.41011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41011.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40616.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36199.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60634.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67226.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47414.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49431.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56730.log 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60634.log.gz 2024-09-10T07:54:46.603 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.47414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47414.log.gz 2024-09-10T07:54:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56730.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56730.log.gz 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68556.log 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43248.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz --verbose 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.44923.log 2024-09-10T07:54:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.1.40616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35948.log 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40616.log.gz 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36199.log.gz 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.35948.log: /var/log/ceph/ceph-osd.6.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35948.log.gz 2024-09-10T07:54:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T07:54:46.608 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36028.log 2024-09-10T07:54:46.608 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65245.log 2024-09-10T07:54:46.609 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.68556.log.gz 2024-09-10T07:54:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44923.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2024-09-10T07:54:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69828.log 2024-09-10T07:54:46.609 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65245.log: gzip -5 --verbose 87.6% -- /var/log/ceph/ceph-client.admin.53150.log -- replaced with /var/log/ceph/ceph-client.admin.65245.log.gz 2024-09-10T07:54:46.609 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36028.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50024.log 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69828.log.gz 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53150.log.gz 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46131.log 2024-09-10T07:54:46.611 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50024.log.gz 2024-09-10T07:54:46.613 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi070.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70395.log 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46131.log.gz 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70395.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70395.log.gz 2024-09-10T07:54:46.614 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69186.log 2024-09-10T07:54:46.615 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45615.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-09-10T07:54:46.615 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60676.log 2024-09-10T07:54:46.615 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66397.log 2024-09-10T07:54:46.616 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9%/var/log/ceph/ceph-client.admin.60676.log: -- replaced with /var/log/ceph/ceph-client.admin.69186.log.gz 2024-09-10T07:54:46.617 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41680.log 2024-09-10T07:54:46.617 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60676.log.gz 2024-09-10T07:54:46.618 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.36028.log.gz 2024-09-10T07:54:46.618 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T07:54:46.618 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-09-10T07:54:46.619 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66397.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.66397.log.gz /var/log/ceph/ceph-client.admin.71999.log 2024-09-10T07:54:46.619 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.619 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2024-09-10T07:54:46.620 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41680.log.gz 2024-09-10T07:54:46.620 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71999.log.gz 2024-09-10T07:54:46.620 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67400.log 2024-09-10T07:54:46.621 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50265.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2024-09-10T07:54:46.621 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61325.log 2024-09-10T07:54:46.622 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55720.log 2024-09-10T07:54:46.622 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67400.log.gz 2024-09-10T07:54:46.622 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61325.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.61325.log.gz /var/log/ceph/ceph-client.admin.57854.log 2024-09-10T07:54:46.622 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.622 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.36068.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-09-10T07:54:46.623 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59008.log 2024-09-10T07:54:46.623 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2024-09-10T07:54:46.624 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57854.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57854.log.gzgzip 2024-09-10T07:54:46.624 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47156.log 2024-09-10T07:54:46.625 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57164.log 2024-09-10T07:54:46.625 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47156.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59008.log.gz 2024-09-10T07:54:46.626 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66229.log 2024-09-10T07:54:46.626 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.47156.log.gz 2024-09-10T07:54:46.627 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41894.log 2024-09-10T07:54:46.627 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57164.log.gz 2024-09-10T07:54:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56916.log 2024-09-10T07:54:46.628 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66229.log.gz 2024-09-10T07:54:46.628 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70563.log 2024-09-10T07:54:46.629 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41894.log.gz 2024-09-10T07:54:46.629 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56916.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.42915.log -- replaced with /var/log/ceph/ceph-client.admin.56916.log.gz 2024-09-10T07:54:46.630 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.630 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66271.log 2024-09-10T07:54:46.631 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70563.log.gz 2024-09-10T07:54:46.631 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz --verbose 2024-09-10T07:54:46.631 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.50353.log 2024-09-10T07:54:46.632 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68199.log 2024-09-10T07:54:46.632 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66271.log.gz 2024-09-10T07:54:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46564.log 2024-09-10T07:54:46.633 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50353.log.gz 2024-09-10T07:54:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45007.log 2024-09-10T07:54:46.634 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68199.log.gz 2024-09-10T07:54:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51313.log 2024-09-10T07:54:46.635 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46564.log.gz 2024-09-10T07:54:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52048.log 2024-09-10T07:54:46.636 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45007.log.gz 2024-09-10T07:54:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69534.log 2024-09-10T07:54:46.637 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T07:54:46.638 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51313.log.gz 2024-09-10T07:54:46.638 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-09-10T07:54:46.638 INFO:teuthology.orchestra.run.smithi003.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52048.log.gz 2024-09-10T07:54:46.639 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61017.log 2024-09-10T07:54:46.639 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.69534.log.gz 2024-09-10T07:54:46.640 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54572.log 87.9% 2024-09-10T07:54:46.640 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-09-10T07:54:46.641 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61017.log.gz 2024-09-10T07:54:46.641 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62373.log 2024-09-10T07:54:46.641 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54572.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54572.log.gz 2024-09-10T07:54:46.642 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58855.log 2024-09-10T07:54:46.642 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-09-10T07:54:46.643 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.62373.log.gz 2024-09-10T07:54:46.643 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58855.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58855.log.gz 2024-09-10T07:54:46.643 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56193.log 2024-09-10T07:54:46.644 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71339.log 2024-09-10T07:54:46.644 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-09-10T07:54:46.645 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65798.log 2024-09-10T07:54:46.646 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.56193.log.gz 2024-09-10T07:54:46.646 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71339.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71339.log.gz 2024-09-10T07:54:46.646 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61623.log 2024-09-10T07:54:46.647 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65798.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.65798.log.gz /var/log/ceph/ceph-client.admin.64531.log 2024-09-10T07:54:46.647 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.648 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60855.log 2024-09-10T07:54:46.648 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61623.log.gz 2024-09-10T07:54:46.649 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64531.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64531.log.gz 2024-09-10T07:54:46.649 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53524.log 2024-09-10T07:54:46.650 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64363.log 2024-09-10T07:54:46.650 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60855.log.gz 2024-09-10T07:54:46.651 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53524.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53524.log.gz 2024-09-10T07:54:46.651 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-09-10T07:54:46.652 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64363.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64363.log.gz 2024-09-10T07:54:46.652 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T07:54:46.653 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57122.log 2024-09-10T07:54:46.653 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-09-10T07:54:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49778.log 2024-09-10T07:54:46.655 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57122.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.57122.log.gz 2024-09-10T07:54:46.655 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-09-10T07:54:46.656 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57631.log 2024-09-10T07:54:46.656 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49778.log.gz 2024-09-10T07:54:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42898.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-09-10T07:54:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-09-10T07:54:46.657 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42398.log 2024-09-10T07:54:46.658 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57631.log.gz 2024-09-10T07:54:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61665.log 2024-09-10T07:54:46.659 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-09-10T07:54:46.659 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61190.log 2024-09-10T07:54:46.660 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42398.log.gz 2024-09-10T07:54:46.661 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70479.log 2024-09-10T07:54:46.661 INFO:teuthology.orchestra.run.smithi003.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61665.log.gz 2024-09-10T07:54:46.662 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-10T07:54:46.662 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61190.log.gz 2024-09-10T07:54:46.662 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68125.log 2024-09-10T07:54:46.663 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70479.log.gz 2024-09-10T07:54:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70437.log 2024-09-10T07:54:46.664 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-10T07:54:46.664 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55886.log 2024-09-10T07:54:46.665 INFO:teuthology.orchestra.run.smithi003.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.68125.log.gz 2024-09-10T07:54:46.665 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-09-10T07:54:46.666 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70437.log.gz 2024-09-10T07:54:46.666 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71843.log 2024-09-10T07:54:46.667 INFO:teuthology.orchestra.run.smithi003.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55886.log.gz 2024-09-10T07:54:46.667 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63679.log 2024-09-10T07:54:46.668 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-09-10T07:54:46.668 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67753.log 2024-09-10T07:54:46.668 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71843.log.gz 2024-09-10T07:54:46.669 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65365.log 2024-09-10T07:54:46.669 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63679.log.gz 2024-09-10T07:54:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68088.log 2024-09-10T07:54:46.671 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67753.log.gz 2024-09-10T07:54:46.671 INFO:teuthology.orchestra.run.smithi070.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T07:54:46.672 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71801.log 2024-09-10T07:54:46.672 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65365.log.gz 2024-09-10T07:54:46.672 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69954.log 2024-09-10T07:54:46.672 INFO:teuthology.orchestra.run.smithi003.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.68088.log.gz 2024-09-10T07:54:46.673 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58722.log 2024-09-10T07:54:46.673 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71801.log.gz 2024-09-10T07:54:46.674 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-09-10T07:54:46.674 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69954.log.gz 2024-09-10T07:54:46.675 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.58722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58722.log.gz 2024-09-10T07:54:46.675 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56790.log 2024-09-10T07:54:46.676 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67268.log 2024-09-10T07:54:46.676 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-09-10T07:54:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60592.log 2024-09-10T07:54:46.677 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56790.log.gz 2024-09-10T07:54:46.678 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65493.log 2024-09-10T07:54:46.678 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.67268.log.gz 2024-09-10T07:54:46.679 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71087.log 2024-09-10T07:54:46.679 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60592.log.gz 2024-09-10T07:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65493.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65493.log.gz 2024-09-10T07:54:46.680 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69786.log 2024-09-10T07:54:46.681 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61497.log 2024-09-10T07:54:46.681 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71087.log.gz 2024-09-10T07:54:46.682 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64045.log 2024-09-10T07:54:46.682 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69786.log.gz 2024-09-10T07:54:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61497.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61497.log.gz 2024-09-10T07:54:46.683 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44839.log 2024-09-10T07:54:46.684 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70727.log 2024-09-10T07:54:46.684 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64045.log.gz 2024-09-10T07:54:46.685 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44839.log.gz 2024-09-10T07:54:46.685 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70521.log 2024-09-10T07:54:46.686 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67130.log 2024-09-10T07:54:46.686 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70727.log.gz 2024-09-10T07:54:46.687 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57673.log 2024-09-10T07:54:46.687 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70521.log.gz 2024-09-10T07:54:46.688 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53388.log 2024-09-10T07:54:46.688 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67130.log.gz 2024-09-10T07:54:46.689 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57673.log.gz 2024-09-10T07:54:46.689 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52835.log 2024-09-10T07:54:46.690 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43251.log 2024-09-10T07:54:46.690 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1%/var/log/ceph/ceph-client.admin.52835.log: -- replaced with /var/log/ceph/ceph-client.admin.53388.log.gz 2024-09-10T07:54:46.691 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.52835.log.gz /var/log/ceph/ceph-client.admin.45091.log 2024-09-10T07:54:46.691 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.692 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-09-10T07:54:46.692 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43251.log.gz 2024-09-10T07:54:46.693 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43658.log 89.1% 2024-09-10T07:54:46.693 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2024-09-10T07:54:46.694 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66313.log 2024-09-10T07:54:46.694 INFO:teuthology.orchestra.run.smithi003.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-09-10T07:54:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60284.log 2024-09-10T07:54:46.695 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43658.log.gz 2024-09-10T07:54:46.695 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58366.log 2024-09-10T07:54:46.696 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66313.log.gz 2024-09-10T07:54:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60284.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.60284.log.gz 2024-09-10T07:54:46.697 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.63349.log 2024-09-10T07:54:46.697 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69870.log 2024-09-10T07:54:46.698 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58366.log.gz 2024-09-10T07:54:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46862.log 2024-09-10T07:54:46.699 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63349.log.gz 2024-09-10T07:54:46.699 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2024-09-10T07:54:46.700 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69870.log.gz 2024-09-10T07:54:46.700 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46862.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46862.log.gzgzip 2024-09-10T07:54:46.700 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63415.log 2024-09-10T07:54:46.701 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61455.log 2024-09-10T07:54:46.701 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2024-09-10T07:54:46.702 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63481.log 2024-09-10T07:54:46.703 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63415.log.gz 2024-09-10T07:54:46.703 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54033.log 2024-09-10T07:54:46.704 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61455.log.gz 2024-09-10T07:54:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69600.log 2024-09-10T07:54:46.705 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.63481.log.gz 2024-09-10T07:54:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 2024-09-10T07:54:46.705 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54033.log.gz 2024-09-10T07:54:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69600.log.gz 2024-09-10T07:54:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2024-09-10T07:54:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68162.log 2024-09-10T07:54:46.707 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-09-10T07:54:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53650.log 2024-09-10T07:54:46.709 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2024-09-10T07:54:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56958.log 2024-09-10T07:54:46.709 INFO:teuthology.orchestra.run.smithi003.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68162.log.gz 2024-09-10T07:54:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53650.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53650.log.gz 2024-09-10T07:54:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65161.log 2024-09-10T07:54:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51027.log 2024-09-10T07:54:46.711 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56958.log.gz 2024-09-10T07:54:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-09-10T07:54:46.712 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65161.log.gz 2024-09-10T07:54:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.51027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51027.log.gz 2024-09-10T07:54:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57206.log 2024-09-10T07:54:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58068.log 2024-09-10T07:54:46.714 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-09-10T07:54:46.715 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58900.log 2024-09-10T07:54:46.715 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57206.log.gz 2024-09-10T07:54:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58068.log.gz 2024-09-10T07:54:46.716 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2024-09-10T07:54:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-09-10T07:54:46.717 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58900.log.gz 2024-09-10T07:54:46.718 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52793.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz /var/log/ceph/ceph-client.admin.71591.log 2024-09-10T07:54:46.718 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.719 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63256.log 2024-09-10T07:54:46.719 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-09-10T07:54:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68424.log 2024-09-10T07:54:46.720 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71591.log.gz 2024-09-10T07:54:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47492.log 2024-09-10T07:54:46.721 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63256.log.gz 2024-09-10T07:54:46.722 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60158.log 2024-09-10T07:54:46.722 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68424.log.gz 2024-09-10T07:54:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47492.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.58236.log -- replaced with /var/log/ceph/ceph-client.admin.47492.log.gz 2024-09-10T07:54:46.723 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.724 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60158.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.69744.log -- replaced with /var/log/ceph/ceph-client.admin.60158.log.gz 2024-09-10T07:54:46.724 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.725 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56874.log 2024-09-10T07:54:46.725 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58236.log.gz 2024-09-10T07:54:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69744.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.69744.log.gz 2024-09-10T07:54:46.726 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68748.log 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56874.log.gz 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64720.log 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gz 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60116.log 2024-09-10T07:54:46.729 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68748.log.gz 2024-09-10T07:54:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64720.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64720.log.gz 2024-09-10T07:54:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51750.log 2024-09-10T07:54:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68236.log 2024-09-10T07:54:46.731 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60116.log.gz 2024-09-10T07:54:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2024-09-10T07:54:46.732 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.51750.log.gz 2024-09-10T07:54:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-09-10T07:54:46.733 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68236.log.gz 2024-09-10T07:54:46.733 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55209.log 2024-09-10T07:54:46.733 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2024-09-10T07:54:46.734 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-09-10T07:54:46.735 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-09-10T07:54:46.736 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61283.log 2024-09-10T07:54:46.736 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55209.log.gz 2024-09-10T07:54:46.737 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70122.log 2024-09-10T07:54:46.737 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz 2024-09-10T07:54:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61283.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61283.log.gz 2024-09-10T07:54:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2024-09-10T07:54:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60718.log 2024-09-10T07:54:46.739 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70122.log.gz 2024-09-10T07:54:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35832.log 2024-09-10T07:54:46.740 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2024-09-10T07:54:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50437.log 2024-09-10T07:54:46.741 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60718.log.gz 2024-09-10T07:54:46.741 INFO:teuthology.orchestra.run.smithi070.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T07:54:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66565.log 2024-09-10T07:54:46.742 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35832.log.gz 2024-09-10T07:54:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57248.log 2024-09-10T07:54:46.742 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50437.log.gz 2024-09-10T07:54:46.743 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69294.log 2024-09-10T07:54:46.743 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66565.log.gz 2024-09-10T07:54:46.744 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65577.log 89.3% 2024-09-10T07:54:46.744 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57248.log.gz 2024-09-10T07:54:46.745 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41724.log 2024-09-10T07:54:46.745 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69294.log.gz 2024-09-10T07:54:46.746 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62087.log 2024-09-10T07:54:46.746 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65577.log.gz 2024-09-10T07:54:46.747 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49683.log 2024-09-10T07:54:46.747 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41724.log.gz 2024-09-10T07:54:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.62087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62087.log.gz 2024-09-10T07:54:46.748 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63805.log 2024-09-10T07:54:46.749 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49898.log 2024-09-10T07:54:46.749 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49683.log.gz 2024-09-10T07:54:46.750 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2024-09-10T07:54:46.750 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63805.log.gz 2024-09-10T07:54:46.751 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71381.log 2024-09-10T07:54:46.751 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49898.log.gz 2024-09-10T07:54:46.752 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50839.log 2024-09-10T07:54:46.752 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2024-09-10T07:54:46.753 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60935.log 2024-09-10T07:54:46.753 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71381.log.gz 2024-09-10T07:54:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50839.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.50839.log.gz /var/log/ceph/ceph-client.admin.42578.log 2024-09-10T07:54:46.754 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70164.log 2024-09-10T07:54:46.755 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60935.log.gz 2024-09-10T07:54:46.755 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42578.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-09-10T07:54:46.756 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68292.log 2024-09-10T07:54:46.756 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50605.log 2024-09-10T07:54:46.757 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70164.log.gz 2024-09-10T07:54:46.757 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65960.log 2024-09-10T07:54:46.758 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.68292.log.gz 2024-09-10T07:54:46.758 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61899.log 2024-09-10T07:54:46.759 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50605.log.gz 2024-09-10T07:54:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65960.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65960.log.gz 2024-09-10T07:54:46.759 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46736.log 2024-09-10T07:54:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66439.log 2024-09-10T07:54:46.760 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61899.log.gz 2024-09-10T07:54:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66355.log 2024-09-10T07:54:46.761 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46736.log.gz 2024-09-10T07:54:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59625.log 2024-09-10T07:54:46.762 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66439.log.gz 2024-09-10T07:54:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66355.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66355.log.gz 2024-09-10T07:54:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55960.log 2024-09-10T07:54:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68664.log 2024-09-10T07:54:46.764 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59625.log.gz 2024-09-10T07:54:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49557.log 2024-09-10T07:54:46.765 INFO:teuthology.orchestra.run.smithi003.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.55960.log.gz 2024-09-10T07:54:46.766 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48664.log 2024-09-10T07:54:46.766 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68664.log.gz 2024-09-10T07:54:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49557.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49557.log.gz 2024-09-10T07:54:46.767 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65119.log 2024-09-10T07:54:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65077.log 2024-09-10T07:54:46.768 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48664.log.gz 2024-09-10T07:54:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67334.log 2024-09-10T07:54:46.769 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65119.log.gz 2024-09-10T07:54:46.770 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46215.log 2024-09-10T07:54:46.770 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65077.log.gz 2024-09-10T07:54:46.770 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67334.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T07:54:46.771 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.67334.log.gz 2024-09-10T07:54:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46215.log.gz 2024-09-10T07:54:46.772 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-10T07:54:46.773 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-09-10T07:54:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43240.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-10T07:54:46.774 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60074.log 2024-09-10T07:54:46.775 INFO:teuthology.orchestra.run.smithi003.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T07:54:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-09-10T07:54:46.775 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-09-10T07:54:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42230.log 2024-09-10T07:54:46.777 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60074.log.gz 2024-09-10T07:54:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59427.log 2024-09-10T07:54:46.777 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-09-10T07:54:46.778 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63547.log 2024-09-10T07:54:46.778 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42230.log.gz 2024-09-10T07:54:46.779 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-09-10T07:54:46.779 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59427.log.gz 2024-09-10T07:54:46.780 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54274.log 2024-09-10T07:54:46.780 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.63547.log.gz 2024-09-10T07:54:46.781 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62265.log 2024-09-10T07:54:46.781 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-09-10T07:54:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-09-10T07:54:46.782 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54274.log.gz 2024-09-10T07:54:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62265.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62265.log.gz 2024-09-10T07:54:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71633.log 2024-09-10T07:54:46.784 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49982.log 2024-09-10T07:54:46.784 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-09-10T07:54:46.785 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71633.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.56034.log -- replaced with /var/log/ceph/ceph-client.admin.71633.log.gz 2024-09-10T07:54:46.785 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:46.786 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55101.log 2024-09-10T07:54:46.786 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49982.log.gz 2024-09-10T07:54:46.787 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61581.log 2024-09-10T07:54:46.787 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56034.log.gz 2024-09-10T07:54:46.788 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68988.log 2024-09-10T07:54:46.788 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55101.log.gz 2024-09-10T07:54:46.789 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48430.log 2024-09-10T07:54:46.789 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61581.log.gz 2024-09-10T07:54:46.789 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68490.log 2024-09-10T07:54:46.790 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68988.log.gz 2024-09-10T07:54:46.791 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71171.log 2024-09-10T07:54:46.791 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48430.log.gz 2024-09-10T07:54:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46257.log 2024-09-10T07:54:46.792 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68490.log.gz 2024-09-10T07:54:46.793 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71171.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71171.log.gz 2024-09-10T07:54:46.793 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62307.log 2024-09-10T07:54:46.794 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-09-10T07:54:46.794 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46257.log.gz 2024-09-10T07:54:46.794 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2024-09-10T07:54:46.795 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62307.log.gz 2024-09-10T07:54:46.795 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59493.log 2024-09-10T07:54:46.796 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-09-10T07:54:46.797 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42579.log 88.0% 2024-09-10T07:54:46.797 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2024-09-10T07:54:46.798 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61413.log 2024-09-10T07:54:46.798 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59493.log.gz 2024-09-10T07:54:46.799 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42579.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42579.log.gz 2024-09-10T07:54:46.799 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57984.log 2024-09-10T07:54:46.800 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71549.log 2024-09-10T07:54:46.800 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61413.log.gz 2024-09-10T07:54:46.800 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57984.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.57984.log.gz 2024-09-10T07:54:46.801 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42272.log 2024-09-10T07:54:46.802 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2024-09-10T07:54:46.802 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71549.log.gz 2024-09-10T07:54:46.802 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58468.log 88.2% 2024-09-10T07:54:46.802 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-09-10T07:54:46.803 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58426.log 2024-09-10T07:54:46.804 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2024-09-10T07:54:46.804 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61749.log 2024-09-10T07:54:46.805 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58468.log.gz 2024-09-10T07:54:46.805 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55045.log 2024-09-10T07:54:46.806 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58426.log.gz 2024-09-10T07:54:46.806 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-09-10T07:54:46.806 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61749.log.gz 2024-09-10T07:54:46.807 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64447.log 2024-09-10T07:54:46.808 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55045.log.gz 2024-09-10T07:54:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42903.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-09-10T07:54:46.808 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60200.log 2024-09-10T07:54:46.809 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64405.log 2024-09-10T07:54:46.809 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64447.log.gz 2024-09-10T07:54:46.810 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59980.log 2024-09-10T07:54:46.810 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60200.log.gz 2024-09-10T07:54:46.811 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68922.log 2024-09-10T07:54:46.811 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64405.log.gz 2024-09-10T07:54:46.812 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59980.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.59980.log.gz 2024-09-10T07:54:46.812 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54723.log 2024-09-10T07:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70811.log 2024-09-10T07:54:46.813 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68922.log.gz 2024-09-10T07:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46606.log 2024-09-10T07:54:46.814 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54723.log.gz 2024-09-10T07:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2024-09-10T07:54:46.815 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70811.log.gz 2024-09-10T07:54:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53566.log 2024-09-10T07:54:46.816 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46606.log.gz 2024-09-10T07:54:46.817 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53482.log 2024-09-10T07:54:46.817 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2024-09-10T07:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66776.log 2024-09-10T07:54:46.818 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53566.log.gz 2024-09-10T07:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68706.log 2024-09-10T07:54:46.819 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53482.log.gz 2024-09-10T07:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-09-10T07:54:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68706.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.66776.log.gz 2024-09-10T07:54:46.821 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68706.log.gz 2024-09-10T07:54:46.821 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65535.log 2024-09-10T07:54:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-09-10T07:54:46.822 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-09-10T07:54:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59140.log 2024-09-10T07:54:46.823 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65535.log.gz 2024-09-10T07:54:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51031.log 2024-09-10T07:54:46.824 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-09-10T07:54:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59140.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59140.log.gz 2024-09-10T07:54:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-09-10T07:54:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.51031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51031.log.gz 2024-09-10T07:54:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64941.log 2024-09-10T07:54:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58804.log 2024-09-10T07:54:46.826 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-09-10T07:54:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2024-09-10T07:54:46.827 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64941.log.gz 2024-09-10T07:54:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48580.log 2024-09-10T07:54:46.828 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58804.log.gz 2024-09-10T07:54:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54446.log 2024-09-10T07:54:46.829 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz 2024-09-10T07:54:46.830 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65287.log 2024-09-10T07:54:46.830 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48580.log.gz 2024-09-10T07:54:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50395.log 2024-09-10T07:54:46.831 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54446.log.gz 2024-09-10T07:54:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71507.log 2024-09-10T07:54:46.832 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65287.log.gz 2024-09-10T07:54:46.833 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46904.log 2024-09-10T07:54:46.833 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50395.log.gz 2024-09-10T07:54:46.834 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58942.log 2024-09-10T07:54:46.834 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71507.log.gz 2024-09-10T07:54:46.835 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69054.log 2024-09-10T07:54:46.835 INFO:teuthology.orchestra.run.smithi003.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46904.log.gz 2024-09-10T07:54:46.836 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44236.log 2024-09-10T07:54:46.836 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58942.log.gz 2024-09-10T07:54:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71959.log 2024-09-10T07:54:46.837 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69054.log.gz 2024-09-10T07:54:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.44236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44236.log.gz 2024-09-10T07:54:46.838 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71213.log 2024-09-10T07:54:46.839 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63913.log 2024-09-10T07:54:46.839 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71959.log.gz 2024-09-10T07:54:46.840 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57290.log 2024-09-10T07:54:46.840 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71213.log.gz 2024-09-10T07:54:46.841 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63847.log 2024-09-10T07:54:46.841 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.63913.log.gz 2024-09-10T07:54:46.842 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52751.log 2024-09-10T07:54:46.842 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57290.log.gz 2024-09-10T07:54:46.843 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63847.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T07:54:46.843 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63847.log.gz 2024-09-10T07:54:46.844 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52751.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52751.log.gz 2024-09-10T07:54:46.844 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-09-10T07:54:46.845 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66649.log 2024-09-10T07:54:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45244.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-09-10T07:54:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53991.log 2024-09-10T07:54:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-09-10T07:54:46.847 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66649.log.gz 2024-09-10T07:54:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66183.log 2024-09-10T07:54:46.848 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53991.log.gz 2024-09-10T07:54:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45412.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-09-10T07:54:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61707.log 2024-09-10T07:54:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-09-10T07:54:46.851 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66183.log.gz 2024-09-10T07:54:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60490.log 2024-09-10T07:54:46.852 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61707.log.gz 2024-09-10T07:54:46.853 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49183.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-09-10T07:54:46.853 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70038.log 2024-09-10T07:54:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64489.log 2024-09-10T07:54:46.854 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60490.log.gz 2024-09-10T07:54:46.855 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54911.log 2024-09-10T07:54:46.855 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70038.log.gz 2024-09-10T07:54:46.856 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64489.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64489.log.gz 2024-09-10T07:54:46.856 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46513.log 2024-09-10T07:54:46.857 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.54911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57332.log 2024-09-10T07:54:46.857 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54911.log.gz 2024-09-10T07:54:46.858 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-09-10T07:54:46.858 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46513.log.gz 2024-09-10T07:54:46.859 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57332.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57332.log.gz 2024-09-10T07:54:46.860 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64846.log 2024-09-10T07:54:46.860 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2024-09-10T07:54:46.861 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64846.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-09-10T07:54:46.862 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61371.log 2024-09-10T07:54:46.862 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64846.log.gz 2024-09-10T07:54:46.863 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47222.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2024-09-10T07:54:46.863 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50647.log 2024-09-10T07:54:46.864 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43237.log 2024-09-10T07:54:46.864 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61371.log.gz 2024-09-10T07:54:46.865 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58152.log 2024-09-10T07:54:46.865 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50647.log.gz 2024-09-10T07:54:46.866 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43237.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43237.log.gz 2024-09-10T07:54:46.866 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68790.log 2024-09-10T07:54:46.867 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51160.log 2024-09-10T07:54:46.868 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58152.log.gz 2024-09-10T07:54:46.869 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53608.log 2024-09-10T07:54:46.869 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68790.log.gz 2024-09-10T07:54:46.869 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60326.log 2024-09-10T07:54:46.870 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53608.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51160.log.gz 2024-09-10T07:54:46.871 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54137.log 2024-09-10T07:54:46.871 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53608.log.gz 2024-09-10T07:54:46.872 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60326.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60326.log.gz 2024-09-10T07:54:46.872 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56127.log 2024-09-10T07:54:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62505.log 2024-09-10T07:54:46.873 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54137.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54137.log.gz 2024-09-10T07:54:46.874 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66481.log 2024-09-10T07:54:46.874 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56127.log.gz 2024-09-10T07:54:46.875 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42104.log 2024-09-10T07:54:46.875 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62505.log.gz 2024-09-10T07:54:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66481.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66481.log.gz 2024-09-10T07:54:46.876 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58194.log 2024-09-10T07:54:46.877 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62942.log 2024-09-10T07:54:46.878 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42104.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42104.log.gz 2024-09-10T07:54:46.878 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58194.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58194.log.gz 2024-09-10T07:54:46.879 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56594.log 2024-09-10T07:54:46.879 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log/var/log/ceph/ceph-client.admin.62942.log: 2024-09-10T07:54:46.881 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56594.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56594.log.gz 2024-09-10T07:54:46.881 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62942.log.gz 2024-09-10T07:54:46.881 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55923.log 2024-09-10T07:54:46.881 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45049.log: -- /var/log/ceph/ceph-client.admin.43532.log 2024-09-10T07:54:46.882 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55923.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-09-10T07:54:46.883 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70667.log 2024-09-10T07:54:46.883 INFO:teuthology.orchestra.run.smithi003.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55923.log.gz 2024-09-10T07:54:46.884 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65878.log 2024-09-10T07:54:46.884 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gz 2024-09-10T07:54:46.885 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70667.log.gz 2024-09-10T07:54:46.885 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61791.log 2024-09-10T07:54:46.886 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64111.log 2024-09-10T07:54:46.886 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65878.log: /var/log/ceph/ceph-client.admin.61791.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65878.log.gz 2024-09-10T07:54:46.887 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61791.log.gz 2024-09-10T07:54:46.887 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52122.log 2024-09-10T07:54:46.888 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56343.log 2024-09-10T07:54:46.888 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64111.log.gz 2024-09-10T07:54:46.889 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63763.log 2024-09-10T07:54:46.889 INFO:teuthology.orchestra.run.smithi003.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52122.log.gz 2024-09-10T07:54:46.890 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59559.log 2024-09-10T07:54:46.891 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63763.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63763.log.gz 2024-09-10T07:54:46.891 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56343.log.gz 2024-09-10T07:54:46.891 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55275.log 2024-09-10T07:54:46.892 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-09-10T07:54:46.892 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T07:54:46.893 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55275.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59559.log.gz 2024-09-10T07:54:46.893 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71759.log 2024-09-10T07:54:46.894 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55275.log.gz 2024-09-10T07:54:46.894 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69912.log 2024-09-10T07:54:46.895 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-09-10T07:54:46.895 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.71759.log: /var/log/ceph/ceph-client.admin.45657.log 2024-09-10T07:54:46.896 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71759.log.gz 2024-09-10T07:54:46.897 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71129.log 2024-09-10T07:54:46.897 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69912.log.gz 2024-09-10T07:54:46.897 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T07:54:46.898 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-09-10T07:54:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71129.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71129.log.gz 2024-09-10T07:54:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57938.log 2024-09-10T07:54:46.900 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71255.log 2024-09-10T07:54:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57938.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57938.log.gz 2024-09-10T07:54:46.901 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2024-09-10T07:54:46.902 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42314.log 2024-09-10T07:54:46.902 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71255.log.gz 2024-09-10T07:54:46.903 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46089.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2024-09-10T07:54:46.903 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66691.log 2024-09-10T07:54:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-09-10T07:54:46.905 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42314.log.gz 2024-09-10T07:54:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66691.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66691.log.gz 2024-09-10T07:54:46.905 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58278.log 2024-09-10T07:54:46.906 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-09-10T07:54:46.907 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-09-10T07:54:46.907 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58278.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.58278.log.gz 2024-09-10T07:54:46.907 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69402.log 2024-09-10T07:54:46.908 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57469.log 2024-09-10T07:54:46.909 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-09-10T07:54:46.909 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-09-10T07:54:46.910 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69402.log.gz 2024-09-10T07:54:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57803.log 2024-09-10T07:54:46.911 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57469.log.gz 2024-09-10T07:54:46.911 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-09-10T07:54:46.912 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz 2024-09-10T07:54:46.913 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-10T07:54:46.913 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57803.log.gz 2024-09-10T07:54:46.914 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63613.log 2024-09-10T07:54:46.914 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-09-10T07:54:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53276.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-10T07:54:46.915 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63219.log 2024-09-10T07:54:46.916 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-09-10T07:54:46.917 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49940.log 89.2% 2024-09-10T07:54:46.917 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63219.log.gz 2024-09-10T07:54:46.917 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63613.log.gz 2024-09-10T07:54:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-09-10T07:54:46.919 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-09-10T07:54:46.919 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49940.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49940.log.gz -5 2024-09-10T07:54:46.919 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57715.log 2024-09-10T07:54:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58534.log 2024-09-10T07:54:46.921 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-09-10T07:54:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62169.log 2024-09-10T07:54:46.922 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57715.log.gz 2024-09-10T07:54:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-09-10T07:54:46.923 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58534.log.gz 2024-09-10T07:54:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62169.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62169.log.gz 2024-09-10T07:54:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48496.log 2024-09-10T07:54:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50563.log 2024-09-10T07:54:46.925 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-09-10T07:54:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-09-10T07:54:46.927 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48496.log.gz 2024-09-10T07:54:46.927 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63293.log 2024-09-10T07:54:46.927 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50563.log.gz 2024-09-10T07:54:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46694.log 2024-09-10T07:54:46.929 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-09-10T07:54:46.929 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-09-10T07:54:46.930 INFO:teuthology.orchestra.run.smithi003.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.63293.log.gz 2024-09-10T07:54:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46694.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T07:54:46.931 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1%/var/log/ceph/ceph-client.admin.43243.log: -- replaced with /var/log/ceph/ceph-client.admin.46694.log.gz 2024-09-10T07:54:46.932 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-09-10T07:54:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61059.log 2024-09-10T07:54:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56071.log 2024-09-10T07:54:46.934 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-09-10T07:54:46.934 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61059.log.gz 2024-09-10T07:54:46.935 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67181.log 2024-09-10T07:54:46.935 INFO:teuthology.orchestra.run.smithi003.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.56071.log.gz 2024-09-10T07:54:46.936 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50181.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-09-10T07:54:46.937 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-09-10T07:54:46.937 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67181.log.gz 2024-09-10T07:54:46.938 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-09-10T07:54:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.54657.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-09-10T07:54:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45573.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-09-10T07:54:46.940 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-09-10T07:54:46.941 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2024-09-10T07:54:46.942 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-09-10T07:54:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43574.log 2024-09-10T07:54:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55143.log 2024-09-10T07:54:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51816.log: /var/log/ceph/ceph-client.admin.43574.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz 2024-09-10T07:54:46.945 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43574.log.gz 2024-09-10T07:54:46.945 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47114.log 2024-09-10T07:54:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55143.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55143.log.gz 2024-09-10T07:54:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48812.log 2024-09-10T07:54:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58026.log 2024-09-10T07:54:46.948 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47114.log.gz 2024-09-10T07:54:46.949 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42062.log 2024-09-10T07:54:46.949 INFO:teuthology.orchestra.run.smithi003.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.48812.log.gz 2024-09-10T07:54:46.950 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58026.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58026.log.gz 2024-09-10T07:54:46.950 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41596.log 2024-09-10T07:54:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2024-09-10T07:54:46.951 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42062.log.gz 2024-09-10T07:54:46.953 INFO:teuthology.orchestra.run.smithi070.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T07:54:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-09-10T07:54:46.953 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41596.log.gz 2024-09-10T07:54:46.954 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47072.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz 2024-09-10T07:54:46.954 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42356.log 2024-09-10T07:54:46.955 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2024-09-10T07:54:46.955 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-09-10T07:54:46.957 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42356.log: gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42356.log.gz 2024-09-10T07:54:46.957 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.45531.log 2024-09-10T07:54:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2024-09-10T07:54:46.958 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42575.log 2024-09-10T07:54:46.959 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz --verbose 2024-09-10T07:54:46.960 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.36099.log 2024-09-10T07:54:46.961 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49515.log 2024-09-10T07:54:46.961 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42575.log.gz 2024-09-10T07:54:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36099.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36099.log.gz 2024-09-10T07:54:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64195.log 2024-09-10T07:54:46.963 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49515.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T07:54:46.964 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49515.log.gz 2024-09-10T07:54:46.965 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45322.log 2024-09-10T07:54:46.965 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64195.log.gz 2024-09-10T07:54:46.966 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52355.log 2024-09-10T07:54:46.967 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45322.log.gz 2024-09-10T07:54:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52355.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52355.log.gz 2024-09-10T07:54:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52085.log 2024-09-10T07:54:46.971 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53909.log 2024-09-10T07:54:46.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52085.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.52085.log.gz 2024-09-10T07:54:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53909.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53909.log.gz 2024-09-10T07:54:46.974 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56688.log 2024-09-10T07:54:46.976 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69252.log 2024-09-10T07:54:46.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56688.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56688.log.gz 2024-09-10T07:54:46.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69252.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69252.log.gz 2024-09-10T07:54:46.979 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55588.log 2024-09-10T07:54:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65703.log 2024-09-10T07:54:46.982 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55588.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55588.log.gz 2024-09-10T07:54:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65703.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65703.log.gz 2024-09-10T07:54:46.983 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-10T07:54:46.985 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61101.log 2024-09-10T07:54:46.986 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42912.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-10T07:54:46.988 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61101.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61101.log.gz 2024-09-10T07:54:46.988 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-09-10T07:54:46.990 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57896.log 2024-09-10T07:54:46.991 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49225.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-09-10T07:54:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57896.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57896.log.gz 2024-09-10T07:54:46.993 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59074.log 2024-09-10T07:54:46.994 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62091.log 2024-09-10T07:54:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59074.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59074.log.gz 2024-09-10T07:54:46.996 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.62091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62091.log.gz 2024-09-10T07:54:46.997 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64573.log 2024-09-10T07:54:46.998 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T07:54:46.999 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T07:54:47.000 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64573.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64573.log.gz 2024-09-10T07:54:47.002 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-09-10T07:54:47.005 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68622.log 2024-09-10T07:54:47.005 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64321.log 2024-09-10T07:54:47.005 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-09-10T07:54:47.006 INFO:teuthology.orchestra.run.smithi003.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T07:54:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68622.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68622.log.gz 2024-09-10T07:54:47.007 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57034.log 2024-09-10T07:54:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64658.log 2024-09-10T07:54:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64321.log: /var/log/ceph/ceph-client.admin.57034.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64321.log.gz 2024-09-10T07:54:47.010 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57034.log.gz 2024-09-10T07:54:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64658.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64658.log.gz 2024-09-10T07:54:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63108.log 2024-09-10T07:54:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59732.log 2024-09-10T07:54:47.016 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63108.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63108.log.gz 2024-09-10T07:54:47.016 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-09-10T07:54:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59732.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.59732.log.gz 2024-09-10T07:54:47.019 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66132.log 2024-09-10T07:54:47.019 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-09-10T07:54:47.020 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69468.log 2024-09-10T07:54:47.021 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66132.log.gz 2024-09-10T07:54:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69468.log: /var/log/ceph/ceph-client.admin.45783.log 2024-09-10T07:54:47.023 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T07:54:47.024 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.69468.log.gz 2024-09-10T07:54:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45783.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-09-10T07:54:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48921.log 2024-09-10T07:54:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-09-10T07:54:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48921.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48921.log.gz 2024-09-10T07:54:47.029 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60760.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-09-10T07:54:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62439.log 2024-09-10T07:54:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-09-10T07:54:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62439.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62439.log.gz 2024-09-10T07:54:47.034 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T07:54:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53318.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-09-10T07:54:47.035 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T07:54:47.035 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71717.log 2024-09-10T07:54:47.036 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49389.log 2024-09-10T07:54:47.037 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71717.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29741.log 2024-09-10T07:54:47.038 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71717.log.gz 2024-09-10T07:54:47.038 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49389.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49389.log.gz 2024-09-10T07:54:47.039 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.29741.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29741.log.gz 2024-09-10T07:54:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2024-09-10T07:54:47.041 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64279.log 2024-09-10T07:54:47.043 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2024-09-10T07:54:47.043 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2024-09-10T07:54:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53829.log 2024-09-10T07:54:47.044 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64279.log.gz 2024-09-10T07:54:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68051.log 2024-09-10T07:54:47.045 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2024-09-10T07:54:47.047 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53829.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53829.log.gz 2024-09-10T07:54:47.047 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-09-10T07:54:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68051.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67048.log 2024-09-10T07:54:47.048 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68051.log.gz 2024-09-10T07:54:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61833.log 2024-09-10T07:54:47.050 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-09-10T07:54:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.67048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67048.log.gz 2024-09-10T07:54:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52421.log 2024-09-10T07:54:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60532.log 2024-09-10T07:54:47.053 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61833.log.gz 2024-09-10T07:54:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59797.log 2024-09-10T07:54:47.054 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52421.log.gz 2024-09-10T07:54:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60532.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60532.log.gz 2024-09-10T07:54:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.59797.log: --verbose -- /var/log/ceph/ceph-client.admin.57569.log 2024-09-10T07:54:47.056 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59797.log.gz 2024-09-10T07:54:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65619.log 2024-09-10T07:54:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47762.log 2024-09-10T07:54:47.059 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57569.log.gz 2024-09-10T07:54:47.060 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2024-09-10T07:54:47.060 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65619.log.gz 2024-09-10T07:54:47.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70080.log 2024-09-10T07:54:47.062 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47762.log.gz 2024-09-10T07:54:47.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52487.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2024-09-10T07:54:47.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-09-10T07:54:47.063 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70080.log.gz 2024-09-10T07:54:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-09-10T07:54:47.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58726.log 2024-09-10T07:54:47.066 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-09-10T07:54:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63145.log 2024-09-10T07:54:47.067 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-09-10T07:54:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.58726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58726.log.gz 2024-09-10T07:54:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42913.log 2024-09-10T07:54:47.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43448.log 2024-09-10T07:54:47.070 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63145.log.gz 2024-09-10T07:54:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42913.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42913.log.gz 2024-09-10T07:54:47.071 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-09-10T07:54:47.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43448.log.gz 2024-09-10T07:54:47.073 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:47.073 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T07:54:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-09-10T07:54:47.074 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48179.log 2024-09-10T07:54:47.076 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-10T07:54:47.076 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m0.507s 2024-09-10T07:54:47.076 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m2.371s 2024-09-10T07:54:47.076 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m0.287s 2024-09-10T07:54:47.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56277.log 2024-09-10T07:54:47.078 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-09-10T07:54:47.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44706.log 2024-09-10T07:54:47.078 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48179.log.gz 2024-09-10T07:54:47.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56277.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56277.log.gz 2024-09-10T07:54:47.079 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 2024-09-10T07:54:47.079 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.2.44706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44706.log.gz 2024-09-10T07:54:47.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2024-09-10T07:54:47.082 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-09-10T07:54:47.082 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2024-09-10T07:54:47.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46946.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2024-09-10T07:54:47.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2024-09-10T07:54:47.084 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-09-10T07:54:47.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68358.log 2024-09-10T07:54:47.086 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T07:54:47.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-09-10T07:54:47.086 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-09-10T07:54:47.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47828.log 2024-09-10T07:54:47.088 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45825.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.68358.log.gz 2024-09-10T07:54:47.089 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2024-09-10T07:54:47.089 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-09-10T07:54:47.090 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47828.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47828.log.gz 2024-09-10T07:54:47.090 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70769.log 2024-09-10T07:54:47.091 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64153.log 2024-09-10T07:54:47.091 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2024-09-10T07:54:47.092 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42576.log 2024-09-10T07:54:47.092 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70769.log.gz 2024-09-10T07:54:47.093 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64153.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.64153.log.gz 2024-09-10T07:54:47.093 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71045.log 2024-09-10T07:54:47.094 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64804.log 2024-09-10T07:54:47.095 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-09-10T07:54:47.095 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71045.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71045.log.gz 2024-09-10T07:54:47.095 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66523.log 2024-09-10T07:54:47.096 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67466.log 2024-09-10T07:54:47.097 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64804.log: /var/log/ceph/ceph-client.admin.66523.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64804.log.gz 2024-09-10T07:54:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-09-10T07:54:47.098 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66523.log.gz 2024-09-10T07:54:47.098 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70259.log 2024-09-10T07:54:47.099 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T07:54:47.100 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67466.log.gz 2024-09-10T07:54:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42188.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-09-10T07:54:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2024-09-10T07:54:47.100 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60242.log 2024-09-10T07:54:47.101 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70259.log.gz 2024-09-10T07:54:47.101 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63182.log 2024-09-10T07:54:47.102 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2024-09-10T07:54:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60242.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60242.log.gz 2024-09-10T07:54:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 2024-09-10T07:54:47.103 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66044.log 2024-09-10T07:54:47.104 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63182.log.gz 2024-09-10T07:54:47.105 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-09-10T07:54:47.105 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2024-09-10T07:54:47.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-09-10T07:54:47.106 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66044.log.gz 2024-09-10T07:54:47.107 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49141.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-09-10T07:54:47.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62220.log 2024-09-10T07:54:47.107 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49641.log 2024-09-10T07:54:47.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62220.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-09-10T07:54:47.109 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41506.log 2024-09-10T07:54:47.109 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62220.log.gz 2024-09-10T07:54:47.110 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-09-10T07:54:47.110 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2024-09-10T07:54:47.111 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55997.log 2024-09-10T07:54:47.111 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41506.log.gz 2024-09-10T07:54:47.112 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46778.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-09-10T07:54:47.112 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54093.log 2024-09-10T07:54:47.113 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54232.log 2024-09-10T07:54:47.113 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55997.log.gz 2024-09-10T07:54:47.114 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54093.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54093.log.gz 2024-09-10T07:54:47.114 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53440.log 2024-09-10T07:54:47.115 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57374.log 2024-09-10T07:54:47.115 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54232.log.gz 2024-09-10T07:54:47.116 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53440.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53440.log.gz 2024-09-10T07:54:47.116 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69120.log 2024-09-10T07:54:47.117 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-09-10T07:54:47.117 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57374.log.gz 2024-09-10T07:54:47.118 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41978.log 2024-09-10T07:54:47.118 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69120.log.gz 2024-09-10T07:54:47.119 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69336.log 2024-09-10T07:54:47.119 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41978.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-09-10T07:54:47.120 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66842.log 2024-09-10T07:54:47.120 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41978.log.gz 2024-09-10T07:54:47.121 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55654.log 2024-09-10T07:54:47.122 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66842.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.69336.log.gz 2024-09-10T07:54:47.122 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66842.log.gz 2024-09-10T07:54:47.122 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54915.log 2024-09-10T07:54:47.123 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.55654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56832.log 2024-09-10T07:54:47.123 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55654.log.gz 2024-09-10T07:54:47.124 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.54915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54915.log.gz 2024-09-10T07:54:47.124 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48738.log 2024-09-10T07:54:47.125 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.56832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-09-10T07:54:47.125 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56832.log.gz 2024-09-10T07:54:47.126 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66002.log 2024-09-10T07:54:47.126 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48738.log.gz 2024-09-10T07:54:47.127 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45909.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-09-10T07:54:47.127 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49473.log 2024-09-10T07:54:47.128 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54530.log 2024-09-10T07:54:47.128 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66002.log.gz 2024-09-10T07:54:47.129 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49473.log.gz 2024-09-10T07:54:47.129 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63979.log 2024-09-10T07:54:47.130 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66607.log 2024-09-10T07:54:47.130 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54530.log.gz 2024-09-10T07:54:47.131 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68856.log 2024-09-10T07:54:47.131 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63979.log.gz 2024-09-10T07:54:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64615.log 2024-09-10T07:54:47.132 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66607.log.gz 2024-09-10T07:54:47.133 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.68856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-09-10T07:54:47.134 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68856.log.gz 2024-09-10T07:54:47.134 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54404.log 2024-09-10T07:54:47.134 INFO:teuthology.orchestra.run.smithi003.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T07:54:47.135 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64615.log.gz 2024-09-10T07:54:47.135 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42146.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-09-10T07:54:47.135 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-09-10T07:54:47.136 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2024-09-10T07:54:47.137 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54404.log.gz 2024-09-10T07:54:47.138 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50223.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-09-10T07:54:47.138 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42909.log 2024-09-10T07:54:47.138 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70353.log 2024-09-10T07:54:47.139 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42909.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2024-09-10T07:54:47.140 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42909.log.gz 2024-09-10T07:54:47.140 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2024-09-10T07:54:47.141 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-09-10T07:54:47.141 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70353.log.gz 2024-09-10T07:54:47.142 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41852.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.46341.log -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2024-09-10T07:54:47.142 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:47.142 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59862.log 2024-09-10T07:54:47.143 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-09-10T07:54:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46341.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46341.log.gz 2024-09-10T07:54:47.144 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54362.log 2024-09-10T07:54:47.145 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.59862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66088.log 2024-09-10T07:54:47.145 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54362.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59862.log.gz 2024-09-10T07:54:47.146 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54362.log.gz 2024-09-10T07:54:47.146 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-09-10T07:54:47.147 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2024-09-10T07:54:47.147 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66088.log.gz 2024-09-10T07:54:47.148 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62810.log 2024-09-10T07:54:47.148 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-09-10T07:54:47.149 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2024-09-10T07:54:47.149 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36196.log 2024-09-10T07:54:47.150 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-09-10T07:54:47.150 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36196.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.62810.log.gz 2024-09-10T07:54:47.151 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36196.log.gz 2024-09-10T07:54:47.151 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46173.log 2024-09-10T07:54:47.152 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58324.log 2024-09-10T07:54:47.152 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-09-10T07:54:47.153 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60448.log 2024-09-10T07:54:47.153 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46173.log.gz 2024-09-10T07:54:47.154 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58324.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58324.log.gz 2024-09-10T07:54:47.154 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57759.log 2024-09-10T07:54:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.60448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46299.log 2024-09-10T07:54:47.156 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60448.log.gz 2024-09-10T07:54:47.156 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.57759.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-09-10T07:54:47.156 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44264.log 2024-09-10T07:54:47.157 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46383.log 2024-09-10T07:54:47.158 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46299.log.gz 2024-09-10T07:54:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.44264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44264.log.gz 2024-09-10T07:54:47.158 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64237.log 2024-09-10T07:54:47.159 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72041.log 2024-09-10T07:54:47.160 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46383.log.gz 2024-09-10T07:54:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64237.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64237.log.gz 2024-09-10T07:54:47.160 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2024-09-10T07:54:47.161 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.72041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65203.log 2024-09-10T07:54:47.162 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72041.log.gz 2024-09-10T07:54:47.162 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46988.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2024-09-10T07:54:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42916.log 2024-09-10T07:54:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48245.log 2024-09-10T07:54:47.164 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65203.log.gz 2024-09-10T07:54:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42916.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.42916.log.gz --verbose 2024-09-10T07:54:47.165 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.0.47410.log 2024-09-10T07:54:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62876.log 2024-09-10T07:54:47.166 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48245.log.gz 2024-09-10T07:54:47.166 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.47410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47410.log.gz 2024-09-10T07:54:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66733.log 2024-09-10T07:54:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.62876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69642.log 2024-09-10T07:54:47.168 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.62876.log.gz 2024-09-10T07:54:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.66733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61145.log 2024-09-10T07:54:47.169 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66733.log.gz 2024-09-10T07:54:47.169 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.69642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2024-09-10T07:54:47.169 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69642.log.gz 2024-09-10T07:54:47.170 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.61145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54994.log 2024-09-10T07:54:47.171 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61145.log.gz 2024-09-10T07:54:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.53734.log: gzip -5 87.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz -- 2024-09-10T07:54:47.171 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.43700.log 2024-09-10T07:54:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54994.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T07:54:47.172 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54994.log.gz 2024-09-10T07:54:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58110.log 2024-09-10T07:54:47.173 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-09-10T07:54:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-09-10T07:54:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.58110.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58110.log.gzgzip 2024-09-10T07:54:47.175 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41810.log 2024-09-10T07:54:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-09-10T07:54:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46425.log 2024-09-10T07:54:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65409.log 2024-09-10T07:54:47.178 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41810.log.gz 2024-09-10T07:54:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46425.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T07:54:47.179 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46425.log.gz 2024-09-10T07:54:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65409.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65409.log.gz 2024-09-10T07:54:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65451.log 2024-09-10T07:54:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48775.log 2024-09-10T07:54:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.65451.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65451.log.gz 2024-09-10T07:54:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71297.log 2024-09-10T07:54:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54488.log 2024-09-10T07:54:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48775.log: /var/log/ceph/ceph-client.admin.71297.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48775.log.gz 2024-09-10T07:54:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47030.log 2024-09-10T07:54:47.184 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71297.log.gz 2024-09-10T07:54:47.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52619.log 2024-09-10T07:54:47.185 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54488.log.gz 2024-09-10T07:54:47.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50521.log 2024-09-10T07:54:47.187 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47030.log.gz 2024-09-10T07:54:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71885.log 2024-09-10T07:54:47.188 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52619.log.gz 2024-09-10T07:54:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67052.log 2024-09-10T07:54:47.189 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50521.log.gz 2024-09-10T07:54:47.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71885.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71885.log.gz 2024-09-10T07:54:47.190 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71675.log 2024-09-10T07:54:47.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.67052.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.0.67052.log.gz 2024-09-10T07:54:47.191 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.63721.log 2024-09-10T07:54:47.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71675.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71675.log.gz 2024-09-10T07:54:47.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69702.log 2024-09-10T07:54:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70853.log 2024-09-10T07:54:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.63721.log: /var/log/ceph/ceph-client.admin.69702.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63721.log.gz 2024-09-10T07:54:47.195 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.69702.log.gz /var/log/ceph/ceph-client.admin.48113.log 2024-09-10T07:54:47.195 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:47.196 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54316.log 2024-09-10T07:54:47.196 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70853.log.gz 2024-09-10T07:54:47.197 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-09-10T07:54:47.197 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-09-10T07:54:47.198 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.54316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64762.log 2024-09-10T07:54:47.198 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54316.log.gz 2024-09-10T07:54:47.199 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67885.log 2024-09-10T07:54:47.199 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-09-10T07:54:47.200 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.64762.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64762.log.gzgzip 2024-09-10T07:54:47.200 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44398.log 2024-09-10T07:54:47.201 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67819.log 2024-09-10T07:54:47.202 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44398.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.67885.log.gz 2024-09-10T07:54:47.202 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-09-10T07:54:47.203 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44398.log.gz 2024-09-10T07:54:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.67819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71465.log 2024-09-10T07:54:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39643.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67819.log.gz 2024-09-10T07:54:47.205 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70605.log 2024-09-10T07:54:47.205 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-09-10T07:54:47.206 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.71465.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71465.log.gz 2024-09-10T07:54:47.207 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.70605.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70605.log.gz 2024-09-10T07:54:47.367 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T07:54:47.466 INFO:teuthology.orchestra.run.smithi003.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T07:54:47.514 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T07:54:47.577 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T07:54:47.581 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-10T07:54:47.581 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m1.015s 2024-09-10T07:54:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m3.011s 2024-09-10T07:54:47.582 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m1.497s 2024-09-10T07:54:47.582 INFO:tasks.ceph:Archiving logs... 2024-09-10T07:54:47.582 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi003/log 2024-09-10T07:54:47.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T07:54:48.435 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi070/log 2024-09-10T07:54:48.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T07:54:48.807 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T07:54:48.816 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T07:54:48.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T07:54:48.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T07:54:48.941 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T07:54:48.950 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T07:54:48.950 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:54:48.955 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:54:48.985 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:54:48.988 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:54:48.992 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:54:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-10T07:54:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 26 +2234us[+2234us] +/- 37ms 2024-09-10T07:54:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 +2446us[+2467us] +/- 62ms 2024-09-10T07:54:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -7817us[-7817us] +/- 75ms 2024-09-10T07:54:48.993 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 27 +2349us[+2349us] +/- 37ms 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 +2557us[+2552us] +/- 62ms 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -7702us[-7702us] +/- 75ms 2024-09-10T07:54:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:48.997 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T07:54:49.006 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T07:54:49.007 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T07:54:49.015 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T07:54:49.168 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T07:54:49.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:54:49.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:49.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:54:49.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:49.402 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T07:54:49.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T07:54:49.421 INFO:teuthology.task.internal:Duration was 965.139458 seconds 2024-09-10T07:54:49.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T07:54:49.431 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T07:54:49.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T07:54:49.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T07:54:49.519 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:54:49.523 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:54:50.032 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T07:54:50.032 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-10T07:54:50.032 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T07:54:50.084 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T07:54:50.084 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T07:54:50.145 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T07:54:50.145 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T07:54:50.150 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T07:54:50.209 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T07:54:50.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T07:54:50.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T07:54:50.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T07:54:50.384 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T07:54:50.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T07:54:50.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T07:54:50.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T07:54:50.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T07:54:50.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T07:54:50.577 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-10T07:54:50.581 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-10T07:54:50.610 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T07:54:50.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:50.643 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T07:54:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:54:50.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T07:54:50.687 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T07:54:50.688 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi003 2024-09-10T07:54:50.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T07:54:50.753 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898880/remote/smithi070 2024-09-10T07:54:50.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T07:54:50.816 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T07:54:50.816 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T07:54:50.821 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T07:54:50.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T07:54:50.868 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T07:54:50.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T07:54:50.948 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T07:54:50.948 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T07:54:50.952 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T07:54:50.983 INFO:teuthology.orchestra.run.smithi003.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 07:54 /home/ubuntu/cephtest 2024-09-10T07:54:50.987 INFO:teuthology.orchestra.run.smithi070.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 07:54 /home/ubuntu/cephtest 2024-09-10T07:54:50.989 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T07:54:50.999 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T07:54:51.023 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} duration: 965.1394584178925 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T07:54:51.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T07:54:51.099 INFO:teuthology.run:pass